Introduction
Did you know that every time you upload a photo to Facebook, the platform uses facial recognition algorithms to identify the people in that image? Or that certain governments around the world use facial recognition technology to identify and catch criminals?? I don't need to tell you that you can now unlock smartphones with your face!!
The applications of this subdomain of computer vision are vast and companies around the world are already reaping the benefits.. The use of facial recognition models will only increase in the coming years, so why not learn how to build one from scratch?
In this article, we are going to do just that. We will first understand the inner workings of facial recognition and then we will take a simple case study and implement it in Python. At the end of the article, you will have built your first facial recognition model!
Table of Contents
- Understand how facial recognition works
- Case study
- Python implementation
- Understanding Python code
- Applications of facial recognition algorithms
Understand how facial recognition works
To understand how facial recognition works, first let's get an idea of the concept of feature vector.
Each machine learning algorithm takes a set of data as input and learns from this data. The algorithm goes through the data and identifies patterns in the data. For instance, suppose we want to identify which face is present in a certain image, there are several things that we can consider as a pattern:
- Alto / face width.
- Height and width may not be reliable, since the image can be changed to a smaller face. But nevertheless, even after scale change, what remains unchanged are the relationships: the relationship between the height of the face and the width of the face will not change.
- Face color.
- Width of other parts of the face such as lips, nose, etc.
Clearly, there is a pattern here: different faces have different dimensions like above. Similar faces have similar dimensions. The challenging part is turning a particular face into numbers: machine learning algorithms only understand numbers. This numerical representation of a “method” (or an item in the training set) is called as feature vector. A feature vector is made up of several numbers in a specific order.
As a simple example, we can map one “method” in a feature vector that can comprise several features such as:
- Face height (cm)
- Face width (cm)
- Medium face color (R, G, B)
- Lip width (cm)
- Nose height (cm)
Essentially, given a picture, we can plot various features and convert them to a feature vector like:
| Face height (cm) | Face width (cm) | Medium face color (RGB) | Lip width (cm) | Nose height (cm) |
| 23,1 | 15,8 | (255, 224, 189) | 5.2 | 4.4 |
Then, our image is now a vector that could be represented as (23.1, 15.8, 255, 224, 189, 5.2, 4.4). Of course, there could be countless other features that could be derived from the image (for instance, hair color, facial hair, glasses, etc.). But nevertheless, for example, let's consider only these 5 simple features.
Now, once we have encoded each image into a feature vector, the problem becomes much simpler. Clearly, when we have 2 faces (images) that represent the same person, the derived feature vectors will be quite similar. In other words, the “distance” between the 2 feature vectors will be quite small.
Machine learning can help us here with 2 stuff:
- Deriving the feature vector: it is difficult to manually list all functions because there are so many. A machine learning algorithm can intelligently label many of these features. For instance, a complex feature could be: ratio of nose height to forehead width. Now it will be quite difficult for a human being to list all those characteristics of “Second order”.
- Matching algorithms: Once the feature vectors have been obtained, a machine learning algorithm must match a new image with the set of feature vectors present in the corpus.
Now that we have a basic understanding of how facial recognition works, let's create our own facial recognition algorithm using some of the most popular python libraries.
Case study
They give us a lot of faces, possibly from celebrities like Mark Zuckerberg, Warren Buffett, Bill Gates, Shah Rukh Khan, etc. Call this bunch of faces our “corpus”. Now, we are given the image of another celebrity (“new celebrity”). The task is simple: identify if this “new celebrity” is among those present in the “corpus”.
These are some of the images of the corpus:
As you can see, we have celebrities like Barack Obama, Bill Gates, Jeff Bezos, Mark Zuckerberg, Ray Dalio and Shah Rukh Khan.
Now, here is the “new celebrity”:
Note: all the above images are taken from google images.
This is obviously Shah Rukh Khan. But nevertheless, for a computer this is a challenging task. The challenge stems from the fact that for us, the humans, it's easy to combine so many features of the images to see which is which celebrity. But nevertheless, for a computer, it is not easy to learn to recognize these faces.
There is an incredibly simple Python library that encapsulates everything we learned earlier: create feature vectors from faces and know how to differentiate between faces. This Python library is called like Facial recognition and deep down, employs dlib – a set of C tools ++ Modern that contains various machine learning algorithms that help to write sophisticated C-based applications ++.
Facial recognition The library in Python can perform a large number of tasks:
- Find all the faces in a given image
- Find and manipulate facial features in an image.
- Identify faces in images
- Real-time facial recognition
Here, we will talk about the third use case: identify faces in images.
You can find the source code of Facial recognition library here on Github: https://github.com/ageitgey/face_recognition.
In fact, there is also a tutorial on how to install Facial recognition Library: https://github.com/ageitgey/face_recognition#installation-options. Before install Facial recognition, you need to install dlib as well as. You can find the instructions to install dlib here: https://gist.github.com/ageitgey/629d75c1baac34dfa5ca2a1928a7aeaf.
Python implementation
This section contains the code for building a simple facial recognition system using the Facial recognition Library. This is the implementation part, we will go through the code to understand it in more detail in the next section.
# import the libraries import import face_recognition # make a list of all the available images images = os.listdir('images') # load your image image_to_be_matched = face_recognition.load_image_file('my_image.jpg') # encoded the loaded image into a feature vector image_to_be_matched_encoded = face_recognition.face_encodings( image_to_be_matched)[0] # iterate over each image for image in images: # load the image current_image = face_recognition.load_image_file("images/" + image) # encode the loaded image into a feature vector current_image_encoded = face_recognition.face_encodings(current_image)[0] # match your image with the image and check if it matches result = face_recognition.compare_faces( [image_to_be_matched_encoded], current_image_encoded) # check if it was a match if result[0] == True: print "Matched: " + image else: print "Not matched: " + image
The folder structure is as follows:
facial recognition:
- fr.py
- my_image.jpg
- images /
- barack_obama.jpg
- bill_gates.jpg
- jeff_bezos.jpg
- mark_zuckerberg.jpg
- ray_dalio.jpg
- shah_rukh_khan.jpg
- warren_buffett.jpg
Our root directory, facial recognition contains:
- Our previous facial recognition code in the form of fr.py.
- my_image.jpg: the image to be recognized (“new celebrity”).
- images / – the “corpus”.
When you create the folder structure as above and run the above code, this is what you get as a result:
Matched: shah_rukh_khan.jpg Not matched: warren_buffett.jpg Not matched: barack_obama.jpg Not matched: ray_dalio.jpg Not matched: bill_gates.jpg Not matched: jeff_bezos.jpg Not matched: mark_zuckerberg.jpg
Clearly, the “new celebrity” is Shah Rukh Khan and our facial recognition system is capable of detecting it.
Understanding Python code
Now, let's go through the code to understand how it works:
# import the libraries import import face_recognition
These are simply imports. We will use the built-in you library to read all the images in our corpus and we will use Facial recognition in order to write the algorithm.
# make a list of all the available images images = os.listdir('images')
This simple code helps us to identify the path of all the images in the corpus. Once this line is executed, we will have:
images = ['shah_rukh_khan.jpg', 'warren_buffett.jpg', 'barack_obama.jpg', 'ray_dalio.jpg', 'bill_gates.jpg', 'jeff_bezos.jpg', 'mark_zuckerberg.jpg']
Now, the following code loads the image of the new celebrity:
# load your image image_to_be_matched = face_recognition.load_image_file('my_image.jpg')
To ensure that the algorithms can interpret the image, we convert the image into a feature vector:
# encoded the loaded image into a feature vector image_to_be_matched_encoded = face_recognition.face_encodings( image_to_be_matched)[0]
The rest of the code is now quite easy:
# iterate over each image for image in images: # load the image current_image = face_recognition.load_image_file("images/" + image) # encode the loaded image into a feature vector current_image_encoded = face_recognition.face_encodings(current_image)[0] # match your image with the image and check if it matches result = face_recognition.compare_faces( [image_to_be_matched_encoded], current_image_encoded) # check if it was a match if result[0] == True: print "Matched: " + image else: print "Not matched: " + image
Here we are:
- Loop over each image.
- Encode the image into a feature vector.
- Comparing the loaded image with the image to be recognized.
- If it's a coincidence, we print it. If it doesn't match, we also print it.
The result shown above clearly suggests that this simple facial recognition algorithm works incredibly well.. Let's try to replace My image with another image:
When I run the algorithm again, you will see the following result:
Not matched: shah_rukh_khan.jpg Not matched: warren_buffett.jpg Not matched: barack_obama.jpg Not matched: ray_dalio.jpg Not matched: bill_gates.jpg Not matched: jeff_bezos.jpg Not matched: mark_zuckerberg.jpg
Clearly, the system did not identify Jack Ma as any of the aforementioned celebrities. This indicates that our algorithm is quite good at both:
- Correctly identify those that are present in the corpus
- Mark a mismatch for those not present in the corpus
Facial recognition apps
Facial recognition is a well-researched problem and is widely used in both industry and academia. As an example, a criminal in China was caught because a facial recognition system in a shopping mall detected his face and raised the alarm. Clearly, facial recognition can be used to mitigate crime. There are many other interesting use cases of facial recognition:
- Face authentication: Apple has introduced Face ID for facial authentication on iPhones. Some of the major banks are trying to use face authentication for lockers.
- Customer service: Some of the banks in Malaysia have installed systems that use facial recognition to detect valuable customers of the bank so that the bank can provide personalized service. This way, banks can generate more revenue by retaining those customers and keeping them happy.
- Insurance underwriting: Many insurance companies use facial recognition to match the person's face to the one provided on the proof of photo identification.. Thus, the subscription process becomes much faster.
Final notes
In summary, facial recognition is an interesting problem with many powerful use cases that can significantly help society in various dimensions. Although there will always be an ethical risk associated with the commercialization of such techniques, that is a debate that we will put aside for another time.
I hope this article has been useful to you. Submit your comments and suggestions in the comment section below!!
About the Author
Safe Goel, co-founder and CEO of AllinCall Research & Solutions
