This article was published as part of the Data Science Blogathon
Introduction
This article is part of an ongoing blog series on natural language processing (PNL). In the previous article, we discuss semantic analysis, what is a NLP task level. In that article, we discuss semantic analysis techniques in which we discuss a technique called entity extraction, that is very important to understand in NLP.
Therefore, in this article, We will delve into the entity extraction technique called Named Entity Recognition, which is a very useful component in the NLP pipeline.
This is the part 10 from the blog series on the Step-by-Step Guide to Natural Language Processing.
Table of Contents
1. What is Named Entity Recognition (DOWN)?
2. Different blocks present in a typical NER model
3. Deep understanding of named entity recognition with an example
4. How Does Named Entity Recognition Work?
5. Named entity recognition use cases
6. How can I use NER?
What is Named Entity Recognition (DOWN)?
Let's first analyze what the entities mean.
Entities are the most important fragments of a particular sentence, as nominal phrases, verbal phrases or both. Generally, entity detection algorithms are joint models of:
- Rules-based analysis, Python
- Dictionary searches,
- Labeled POS,
- Dependency analysis.
For instance,
In the previous sentence, the entities are:
Date: Thursday, Time: night, Location: Chateau Marmont, Person: Cate Blanchett
Now, we can start our discussion on the recognition of named entities (DOWN),
1. Named entity recognition is one of the key entity detection methods in NLP.
2. Named entity recognition is a natural language processing technique that can automatically scan entire articles and extract some fundamental entities in a text and classify them into predefined categories. Entities can be,
- Organizations,
- Quantities,
- Monetary values,
- Percentages and more.
- Names of people
- Company names
- Geographical locations (both physical and political)
- Product names
- Dates and times
- Amounts of money
- Event names
3. In simple words, named entity recognition is the process of detecting named entities, as people's names, location names, company names, etc. of the text.
4. Also known as entity identification or entity extraction or entity fragmentation.
For instance,
5. With the help of named entity recognition, we can extract key information to understand the text, or just use it to extract important information and store it in a database.
6. The applicability of entity detection can be seen in many applications, What
- Automated chatbots,
- Content parsers,
- Consumer insights, etc.
Commonly used named entity types:
Image source: Google images
Different blocks present in a typical named entity recognition model
A typical NER model consists of the following three blocks:
Noun Phrase Identification
This step tries to extract all noun phrases from a text with the help of dependency analysis and tagging of part of speech.
Phrase classification
In this classification step, We classify all the nominal phrases extracted from the previous step in their respective categories. To disambiguate locations, API de Google Maps can provide a very good path. and to identify personal names or company names, the open databases of DBpedia, Wikipedia can be used. Apart of this, we can also make lookup tables and dictionaries combining information with the help of different sources.
Entity disambiguation
Sometimes, what happens is that the entities are classified incorrectly, so creating a validation layer over the results becomes useful. The use of knowledge charts can be leveraged for this purpose. Some of the most popular knowledge charts are:
Deep understanding of NER with an example
Consider the following sentence:
The blue cells represent the nouns. Some of these nouns describe real things present in the world.
For instance, From the above, the following nouns represent physical places on a map.
“London”, “England”, “United Kingdom”
It would be great if we could detect that! With that amount of information, we could automatically extract a list of real world places mentioned in a document with the help of NLP.
Therefore, the goal of NER is to detect and label these nouns with the real-world concepts they represent.
Then, when we execute each token present in the sentence through a NER tagging model, our prayer looks like this,
Let's analyze what exactly the NER system does.
NER systems don't just do a simple dictionary lookup. However, they're using the context of how a word appears in the sentence and used a statistical model to guess what kind of noun that particular word represents.
Since NER makes it easy to extract structured data from text, it has many uses. It is one of the easiest methods to quickly get insightful value from an NLP pipeline..
If you want to try NER yourself, see the Link.
How Does Named Entity Recognition Work?
How can we simply observe, after reading a particular text, we can naturally recognize named entities as people, values, locations, etc.
For instance, Consider the following sentence:
Sentence: Sundar Pichai, the CEO of Google Inc. is walking in the streets of California.
From the previous sentence, we can identify three types of entities: (Named entities)
- (“Person”: “Sundar Pichai”),
- (“Org”: “Google Inc.”),
- (“Location”: “California”).
But to do the same with the help of computers, we must first help them recognize entities so that they can categorize them. Then, to do it, we can count on the help of machine learning and natural language processing (NLP).
Let's discuss the role of both when implementing NER using computers:
- PNL: That studies the structure and rules of language and forms intelligent systems that are capable of deriving meaning from text and speech.
- Machine learning: Help machines learn and improve over time.
To know what an entity is, a NER model needs to be able to detect a word or string of words that make up an entity (for instance, California) and decide to which category of entity it belongs.
Then, as a final step, we can say that the heart of any NER model is a two-step process:
- Detect a named entity
- Categorize the entity
Then, first, we need to create categories of entities, As name, Location, Event, Organization, etc., and feed a NER model with relevant training data.
Later, when labeling some word and phrase samples with their corresponding entities, eventually we will teach our NER model to detect entities and categorize them.
Named entity recognition use cases
As we have commented in the previous section, the named entity recognition (DOWN) it will help us to easily identify the key components in a text, as people's names, places, trademarks, monetary values and more.
And extracting the main entities from a text helps us sort unstructured data and detect important information, which is crucial if you have to deal with large data sets.
Then, Let's take a look at some of the interesting use cases of Named Entity Recognition:
Customer Support
Image source: Google images
Let's analyze the use case of customer support tickets where we deal with an increasing number of tickets, there we can use named entity recognition techniques to handle customer requests faster.
From a business perspective, if we automate repetitive customer service tasks, how to categorize customer issues and inquiries, will save you valuable time. As a result, helps improve your resolution rates and increases customer satisfaction.
Here, we can also use entity extraction to extract the relevant information, such as product names or serial numbers, making it easy to send tickets to the most appropriate agent or team to handle that problem.
Gain insight from customer feedback
Image source: Google images
For almost all product-based businesses, online reviews are a great source of customer feedback, as they can provide valuable information on what customers like and dislike about your products and the aspects of your business that need improvement for business growth.
Then, here we can use NER systems to organize all customer feedback and detect recurring problems.
For instance, We can use the NER system to detect locations that are most frequently mentioned in negative customer reviews, which could lead you to focus on a particular office branch.
Recommender system
Image source: Google images
Many modern apps like Netflix, YouTube, Facebook, etc. rely on recommendation systems to produce optimal customer experiences. Many of these systems are based on the recognition of named entities, that can provide suggestions based on the user's search history.
For instance, If you watch a lot of educational videos on YouTube, you will get more recommendations that have been classified as entity education.
Summarizing resumes
Image source: Google images
When recruiting new people, recruiters spend many hours of their day reviewing resumes and searching for the right candidate. Every resume contains almost the same type of information, but its organized form and its format are different, so it becomes a classic example of unstructured data.
Then, here with the help of an entity extractor, recruiting teams can instantly extract the most relevant information about candidates, from personal information like name, address, phone number, date of birth and email, etc., to information related to their training and experience such as certifications, Titles, company names, skills, etc.
Some more use cases of NER are:
- Search engine algorithm optimization,
- Content classification for news channels, etc.
How can I use NER?
If you are working on a business problem statement and think your business could benefit from NER, you can use it quite easily with the help of the following excellent open source libraries:
Each one has its pros and cons, which you can explore by referring to the links mentioned above.
This ends our Part 10 from the blog series on natural language processing!
Other blog posts of mine
You can also check out my previous blog posts.
Past Data Science Blog Posts.
Here it is my Linkedin profile in case you want to connect with me. I will be happy to be connected with you.
For any query, you can email me at Gmail.
Final notes
Thank you for reading!
I hope you liked the article. If you like, share it with your friends too. Anything not mentioned or do you want to share your thoughts? Feel free to comment below and I'll get back to you. 😉
