Building a Sentiment Analysis Model with TensorFlow

Introduction

Sentiment analysis is a fascinating application of natural language processing (NLP) that allows us to determine the emotional tone of a given piece of text. Whether it's analyzing customer reviews, monitoring social media sentiment, or improving chatbots, sentiment analysis is a valuable tool across multiple industries.

In this post, we'll go through a straightforward approach to building a sentiment analysis model using TensorFlow with the IMDb movie reviews dataset. This dataset consists of positive and negative movie reviews, making it a great dataset for classifying sentiment in text.

Why Sentiment Analysis Matters

Sentiment analysis has a wide range of applications, from business intelligence to personal projects. Companies use it to monitor brand perception, automate customer support responses, and analyze market trends. Even in casual use, sentiment analysis can help gauge the mood of discussions on social media or filter toxic comments from online forums.

Now, let's dive into the process of building a simple yet effective sentiment analysis model using TensorFlow.

Step 1: Installing the Required Libraries

Before we start, we need to install the necessary dependencies. If you haven't already installed TensorFlow and TensorFlow Datasets, you can do so using the following command:

pip install tensorflow tensorflow-datasets numpy

This will ensure that you have everything you need to load the dataset, preprocess the data, and build the model.

Step 2: Importing the Libraries

Once the libraries are installed, the next step is to import them into our script. This allows us to access essential functionalities like loading datasets, tokenizing text, and building the model.

import tensorflow as tf
import tensorflow_datasets as tfds
from tensorflow import keras
import numpy as np

pad_sequences = tf.keras.preprocessing.sequence.pad_sequences

Step 3: Loading the IMDb Movie Reviews Dataset

We'll be using the IMDb movie reviews dataset, which is available through TensorFlow Datasets (TFDS). This dataset contains thousands of movie reviews labeled as either positive or negative. By using TFDS, we can easily load and preprocess the data.

dataset, info = tfds.load('imdb_reviews', with_info=True, as_supervised=True)

train_data, test_data = dataset['train'], dataset['test']

Step 4: Setting Hyperparameters

To control various aspects of our model, we define key hyperparameters such as vocabulary size, embedding dimensions, and sequence length.

vocab_size = 1000
embedding_dim = 16
max_length = 10
oov_token = "<OOV>"

The vocab_size determines how many unique words will be considered, while max_length ensures that all sequences have a fixed size for training.

Step 5: Tokenizing the Text

Before feeding data into the model, we need to convert text into numerical representations. We start by creating a vocabulary of the most frequently occurring words in the dataset.

word_counts = {}
for sentence,  in traindata:
	for word in sentence.numpy().decode('utf-8').lower().split():
        word_counts[word] = word_counts.get(word, 0) + 1

sorted_words = sorted(word_counts, key=word_counts.get, reverse=True)

vocab = {word: idx+1 for idx, word in enumerate(sorted_words[:vocab_size-1])}

vocab[oov_token] = 0

Next, we define a function to convert sentences into sequences of numerical values based on our vocabulary.

def text_to_sequence(text):
	text = text.numpy().decode('utf-8').lower()
	return [vocab.get(word, vocab[oov_token]) for word in text.split()]

Step 6: Preprocessing the Data

With our tokenizer in place, we can now convert the dataset into numerical sequences and pad them to a fixed length.

def preprocess_data(dataset):
	sequences = []
    labels = []
    for sentence, label in dataset:
        seq = text_to_sequence(sentence)
        sequences.append(seq)
        labels.append(label.numpy())
    return sequences, np.array(labels)

train_sequences, train_labels = preprocess_data(train_data)
test_sequences, test_labels = preprocess_data(test_data)

train_sequences = pad_sequences(train_sequences, maxlen=max_length, padding='post', truncating='post')

test_sequences = pad_sequences(test_sequences, maxlen=max_length, padding='post', truncating='post')

Step 7: Defining the Model

Now, we build a simple neural network using an embedding layer, LSTM layers for sequential processing, and dense layers for classification.

model = keras.Sequential([
	keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),
    keras.layers.LSTM(64, return_sequences=True),
    keras.layers.LSTM(32),
    keras.layers.Dense(16, activation='relu'),
    keras.layers.Dense(1, activation='sigmoid')
])

The LSTM layers help the model understand patterns in sequential data, while the final dense layer outputs a probability score for classification.

Step 8: Compiling and Training the Model

Before training, we compile the model using binary cross-entropy as the loss function and Adam as the optimizer.

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

Training is performed over 15 epochs with a batch size of 24.

model.fit(train_sequences, train_labels, epochs=15, batch_size=24, verbose=1)

(Optional) Step 9: Saving the Model

After training, we save the model so it can be used later for making predictions.

model.save("sentiment_analysis_bot.keras")

Evaluating Model Performance

To check how well the model performs on unseen data, we evaluate it on the test set.

test_loss, test_accuracy = model.evaluate(test_sequences, test_labels)
print(f'Test accuracy: {test_accuracy:.4f}')

Predicting Sentiment

With the trained model, we can now predict the sentiment of new text inputs.

def predict_sentiment(text):
    seq = text_to_sequence(tf.constant(text))
    padded = pad_sequences([seq], maxlen=max_length, padding='post', truncating='post')
    prediction = model.predict(padded)[0][0]
    return "Positive" if prediction > 0.5 else "Negative"

Example usage:

print(predict_sentiment("I really enjoyed the movie, it was amazing!"))
print(predict_sentiment("This was the worst film I've ever seen."))

Conclusion

Building a sentiment analysis model with TensorFlow is a great way to get started with natural language processing. This approach allows us to process text data, train a model on real-world reviews, and make predictions on new inputs. While this model is relatively simple, more advanced techniques, such as transformer-based architectures like BERT, can significantly improve accuracy and handle more complex structures.

If you're interested in taking this further, consider experimenting with different hyperparameters, trying out larger vocabularies, or even fine-tuning a pre-trained model.

GitHub Link: https://github.com/PowerSpy/Sentient-Analysis-AI