Suppose that we're building an app that recommends movies. We've scraped a large set of reviews off the web, but (for obvious reasons) we would like to recommend only movies with positive reviews. In this assignment, you will use the Naive Bayes algorithm to train a binary sentiment classifier with a dataset of movie reviews. The task is to learn a bag of words (unigram, bigram) model that will classify a review as positive or negative based on the words it contains.

Contents

• General Guidelines
• Problem Statement
• Dataset
• Unigram Model
• Bigram Model
• Part0: Using Virtual Environment
• Part1: Training and Development
• Part2: Improve Your Accuracy
• Provided Code Skeleton
• What to submit

• For general instructions, see the main MP page and the course policies.
• In addition to the standard python libraries, you should import nltk, numpy, and scikit-learn.
• Code will be submitted on gradescope.

You are given a dataset consisting of positive and negative reviews. Using the training set, you will learn a Naive Bayes classifier that will predict the right class label given an unseen review. Use the development set to test the accuracy of your learned model. We will have a separate (unseen) test set that we will use to run your code after you turn it in.

Dataset

Then combine the bigram model and the unigram model into a mixture model defined with parameter $\lambda$: $$(1-\lambda) \log \left[ P(Y) \prod_{i=1}^n P(w_i|Y) \right] + \lambda \log \left[ P(Y) \prod_{i=1}^m P(b_i|Y) \right]$$ You will need to find the best parameter $\lambda$ that gives the highest classification accuracy.

Part 0: Using Virtual Environment

• Training: To train the algorithm you are going to need to build a bag of words model using the emails. After you build the model, you will need to estimate the log-likelihoods $\log P(\mathrm{Word}|\mathrm{Type}=\mathrm{Positive})$. The variable $\mathrm{Type}$ can only take on two values, $\mathrm{Positive}$ or $\mathrm{Negative}$. Additionally, you will need to make sure you smooth the likelihoods to prevent zero probabilities. In order to accomplish this task, use Laplace smoothing.

• Development: After you have computed the log-likelihoods, you will have your model predict whether or not the emails from the development set are positive. In order to do this, you will do maximum a posteriori (MAP) classification using the sum of log probabilities shown above.

The whole process (training and testing on the development set) should take no more than 1 minute and 1.1G of memory. With these restrictions, you may be able to use the entire vocabulary in the training data. However, if your implementation is less efficient, you may need to cut your vocabulary size (i.e. only maintain top 5000 words).

Part 2: Improve your Accuracy

We have provided (tar, zip) all the code to get you started on your MP, which means you will only have to implement the logic behind naive bayes.

• reader.py - This file is responsible for reading in the data set. It takes in all of the emails, splits each of those emails into a list of words, stems them if you used the --stemming flag, and then stores all of the lists of words into a larger list (so that each individual list of words corresponds with a single email).

• mp3.py - This is the main file that starts the program, and computes the accuracy, precision, recall, and F1-score using your implementation of naive Bayes.

• naive_bayes.py This is the file where you will be doing all of your work. Each function takes as input the training data, training labels, development set, and other parameters. The training data provided is the output from reader.py. The training labels is the list of labels corresponding to each email in the training data. The development set is the list of emails that you are going to test your implementation on. You will have naiveBayes() and bigramBayes() to output the predicted labels for the development set from your naive Bayes model.

To understand more about how to run the MP, run python3 mp3.py -h in your terminal.

Code Submission