This article was published as part of the Data Science Blogathon.
Overview
- Every basic and fundamental component that is required for sentiment analysis.
- I have used a simple approach to explain all the basics, so that even a beginning reader can get a complete understanding of all the concepts.
- Topics: Text preprocessing, Vocabulary corpus, Feature extraction (Sparse representation and frequency dictionary), Logistic regression model for sentiment analysis.
Sentiment analysis is a supervised machine learning technique used to analyze and predict the polarity of sentiments within a text (either positive or negative).
It is often used by businesses and companies to understand the experience of their users, emotions, answers, etc. so that they can improve the quality and flexibility of their products and services.
Now, Let's dive deeper into understanding how machine learning engineers use this sentiment analysis technique to examine sentiments from various texts.
Gathering data
"MORE DATA, BETTER! “
There are so many open data sources that can be used to train ML models, so it is a personal choice to collect data yourself or use open data sets to train our algorithm.
Text-based data sets are generally distributed as JSONJSON, o JavaScript Object Notation, It is a lightweight data exchange format that is easy for humans to read and write, and easy for machines to analyze and generate. It is commonly used in web applications to send and receive information between a server and a client. Its structure is based on key-value pairs, making it versatile and widely adopted in software development.. the CSV formats, so to use them, we can retrieve the data in a python list or in a dictionary / data frame object.
The data must be divided into the train, validation and test sets in a common way of 60% 20% 20% O 70% 15% 15%.
The popular Twitter dataset can be downloaded from here.
Pipeline
Each Machine Learning task must have a PipelinePipeline is a term that is used in a variety of contexts, mainly in technology and project management. It refers to a set of processes or stages that allow the continuous flow of work from the conception of an idea to its final implementation. In the field of software development, for instance, A pipeline can include scheduling, Testing and deployment, thus guaranteeing greater efficiency and quality in the.... Pipelines are used to divide your machine learning workflows into independent modular parts, reusable that can then be piped together to continually improve model accuracy and achieve a successful algorithm.
We will follow a basic pipeline structure for our problem, so that a reader can easily understand every part of the pipeline used in our workflow. Our pipeline will include the following steps:
- Pre-processing of text and construction of vocabulary: Elimination of unwanted texts (empty words), punctuation, URLs, identifiers, etc. that have no sentimental value. And then add preprocessed unique words to a vocabulary.
- Feature extraction: Iterating through each data example to extract characteristics using a frequency dictionary and finally create an array of characteristics.
- Model of trainingTraining is a systematic process designed to improve skills, physical knowledge or abilities. It is applied in various areas, like sport, Education and professional development. An effective training program includes goal planning, regular practice and evaluation of progress. Adaptation to individual needs and motivation are key factors in achieving successful and sustainable results in any discipline....: We will then use our feature matrix to train a logistic regression model to use that model to predict sentiments..
- Test pattern: Using our trained model to get the predictions from data you never saw.
Data pre-processing
It is an important step in our project portfolio. Text preprocessing can be used to remove words and scores from text data that have no sentimental value, as text preprocessing can significantly improve our training time, since the size of our data will be reduced and will be limited to words that have some sentimental value. Preprocessing includes handling of
-
For words
Words that have no value / semantic or sentimental weight in a sentence. for instance: Y, it is, the, you, etc.
How to process them? We will create a list that includes all possible stopwords like
[‘ourselves’, ‘hers’, ‘between’, ‘yourself’, ‘but’, ‘again’, ‘there’, ‘about’, ‘once’, ‘during’, ‘out’, ‘very’, ‘having’, ‘with’, ‘they’, ‘own’, ‘an’, ‘be’, ‘some’, ‘for’, ‘do’, ‘its’, ‘yours’, ‘such’, ‘into’, ‘of’, ‘most’, ‘itself’, ‘other’, ‘off’, ‘is’, ‘s’, ‘am’, ‘or’, ‘who’, ‘as’, ‘from’, ‘him’, ‘each’, ‘the’, ‘themselves’, ‘until’, ‘below’, ‘are’, ‘we’, ‘these’, ‘your’, ‘his’, ‘through’, ‘don’, ‘nor’, ‘me’, ‘were’, ‘her’, ‘more’, ‘himself’, ‘this’, ‘down’, ‘should’, ‘our’, ‘their’, ‘while’, ‘above’, ‘both’, ‘up’, ‘to’, ‘ours’, ‘had’, ‘she’, ‘all’, ‘no’, ‘when’, ‘at’, ‘any’, ‘before’, ‘them’, ‘same’, ‘and’, ‘been’, ‘have’, ‘in’, ‘will’, ‘on’, ‘does’, ‘yourselves’, ‘then’, ‘that’, ‘because’, ‘what’, ‘over’, ‘why’, ‘so’, ‘can’, ‘did’, ‘not’, ‘now’, ‘under’, ‘he’, ‘you’, ‘herself’, ‘has’, ‘just’, ‘where’, ‘too’, ‘only’, ‘myself’, ‘which’, ‘those’, ‘i’, ‘after’, ‘few’, ‘whom’, ‘t’, ‘being’, ‘if’, ‘theirs’, ‘my’, ‘against’, ‘a’, ‘by’, ‘doing’, ‘it’, ‘how’, ‘further’, ‘was’, ‘here’, ‘than’]
We will now iterate through each example in our data and remove every word from our data that is present in the stopword list.
-
Scores
Punctuation marks are symbols that we use to emphasize our text. for instance:! , @, #, $, etc.
How to process them? We will process them in a similar way to how we have processed stopwords, we will create a list of them and process each example with that list.
-
URLs and identifiers
URLs are the links that start with the Http protocol declaration, for instance. ‘https: //….’ and identifiers are used to mention people on social media, for instance. '@Username’ both share a null sentimental meaning.
How to process them? Process them by creating some function ‘process_handles_urls ()’ which will take the data from our train and eliminate the words that start with ‘https: //’ O '@’ of each example.
-
Derivative
Derivation is a process of reducing a word to its base root word. For instance, ‘turn’ is a root word of turn, turn, turn, etc. Since the root word offers the same sentimental value for all its suffixed words, we can reduce each word to its base root, which can reduce the size of our vocabulary and training time. as well as.
How to process them? Process them by creating some ‘do_stemming function ()’ which will take the data and derive the words from each example.
-
Lower casing
We must use similar upper and lower case for each word in the data to represent 'Word', ‘WORD’, ‘word’ there should only be one case to follow, namely, lowercase, this can also help reduce vocabulary size and eliminate word repetition.
How to process them? iterate through each example, use the .lower method () to convert each chunk of text to lowercase.
Vocabulary corpus
After preprocessing the data, it's time to create a vocabulary that stores each unique word and assigns some numerical value to each different word (this is also called tokenization).
We will use this vocabulary dictionary for feature extraction.
Feature extraction
One of the problems, when working with language processing, is that machine learning algorithms cannot work directly on raw text. Then, we need some feature extraction techniques to convert text to an array (o vector) numerical characteristics.
Let's take some examples of positive and negative tweets:
NOTE: The above example is not processed so we'll process it first before moving on to further steps.
Poor representation
It is a naive approach to extract characteristics from a text. According to the sparse representation, we can create an array of characteristics by iterating through full data, and for each word, in the text example we will assign 1 in the position of that word in the vocabulary list and for words that do not occur, U.S ‘ I will assign 0. Then, our feature matrix will have rows = total sentences in our data and columns = total words in vocabulary.
Disadvantages:
- Great training time
- Great prediction time
Frequency dictionary
A frequency dictionary keeps track of the positive and negative frequencies of each word in our data.
Positive frequency: the Number of times a word appears in sentences with positive sentiment.
Negative frequency: the Number of times a word appears in sentences with negative feeling.
Feature extraction with frequency dictionary:
Using the Frequency Dictionary for Feature Extraction, we can reduce the dimensions of each row that represents each sentence of a characteristics matrix (namely, equal to the number of words in the vocabulary in case of underrepresentation) in three dimensions.
The characteristics of the data of a text are extracted with the dictionary of characteristics using the following formulas:
The process almost looks like:
We now have a three-dimensional feature vector for our tweet that looks like this:
Xm = [1,8,11]
Now we will iterate through each example to extract characteristics from each example and then we will use those characteristics to create the matrix of characteristics that we can use for training.. In the end, we have an array of characteristics like:
Logistic regression for sentiment analysis
Logistic regression models the probabilities of classification problems with two possible outcomes. It is an extension of the linear regression model for classification problems.
Uses of logistic regression a sigmoid function to map the output of our linear function (θTx) Come in 0 Y 1 with some threshold (generally 0.5) to differentiate between two classes, so if h> 0.5 it's a positive class, and if h <0.5 it's a negative class. (Explaining full logistic regression is beyond the scope of this article.)
Training sentiment analysis model
The training of our model will follow the following steps:
We initialize our parameter θ, that we can use in our sigmoid, then we calculate the gradientGradient is a term used in various fields, such as mathematics and computer science, to describe a continuous variation of values. In mathematics, refers to the rate of change of a function, while in graphic design, Applies to color transition. This concept is essential to understand phenomena such as optimization in algorithms and visual representation of data, allowing a better interpretation and analysis in... which we will use to update θ and then calculate the cost. We will keep repeating the steps until the cost is minimized / converge.
Testing our model
To test our model we will use our validation set and follow the following steps:
- Split the data in X_validation (text) and Y_validation (feeling).
- Use feature extraction for X_validation to transform texts into numeric features.
- Find the vector h (= sigmoide (θTX)) for each text in the validation set.
- Assign some function to get the actual classes while comparing against a threshold.
- Find the accuracy of our predictions.
Summary
I'm glad you made it this far! If you are a beginner in natural language processing, I hope I can give you an idea of how things work under the hood and make you able to cover more complex and advanced topics and, if you are a professional, I hope I was able to brush up on your basics.
Natural language processing is a vast domain of artificial intelligence, its applications are used in various paradigms, like chatbots, sentiment analysis, automatic translation, autocorrect, etc. There are several e-learning platforms and articles, works, etc. free distribution that can be helpful to further advance the journey.
References: Natural Language Processing Specialization
