Distance metrics are a key part of various machine learning algorithms. Estas métricas de distancia se utilizan tanto en el supervised learningSupervised learning is a machine learning approach where a model is trained using a set of labeled data. Each input in the dataset is associated with a known output, allowing the model to learn to predict outcomes for new inputs. This method is widely used in applications such as image classification, speech recognition and trend prediction, highlighting its importance in... como no supervisado, generally to calculate the similarity between data points.
An effective distance metric improves the performance of our machine learning model, either for sorting or grouping tasks.
Let's say we want to create clusters using the K-Means Clustering or Nearest Neighbor algorithm to solve a classification or regression problem.. How would you define the similarity between different observations here? How can we tell that two points are similar to each other?
This will happen if their characteristics are similar, truth? When we plot these points, will be closer to each other in the distance.
Therefore, we can calculate the distance between points and then define the similarity between them. Here's the million dollar question: How do we calculate this distance and what are the different distance metrics in machine learning?
That is what we intend to answer in this article. We will analyze 4 types of distance metrics in machine learning and understand how they work in Piton.
4 types of distance metrics in machine learning
- Euclidean distance
- Distance from Manhattan
- Minkowski distance
- Hamming distance
Let's start with the most used distance metric: the Euclidean distance.
1. Euclidean distance
The Euclidean distance represents the shortest distance between two points.
Most machine learning algorithms, including K-Means, use this distance metric to measure the similarity between observations. Let's say we have two points as shown below:
Then, the Euclidean distance between these two points A and B will be:
Here is the formula for the Euclidean distance:
We use this formula when it comes to 2 dimensions. We can generalize this for an n-dimensional space as:
Where,
- n = number of dimensions
- pi, qi = data points
Let's code the Euclidean distance in Piton. This will give you a better understanding of how this distance metric works..
First we will import the necessary libraries. I will use the SciPy library which contains prewritten codes for most of the distance functions used in Python:
These are the two sample points that we will use to calculate the different distance functions. Let us now calculate the Euclidean distance between these two points:
This is how we can calculate the Euclidean distance between two points in Python. Now let's understand the metric of the second distance, the distance from manhattan.
2. Distance from Manhattan
The Manhattan distance is the sum of the absolute differences between points in all dimensions.
We can represent the distance from Manhattan as:
Since the above representation is two-dimensional, to calculate the distance from Manhattan, we will take the sum of the absolute distances in the x and y directions. Then, the distance from Manhattan in two-dimensional space is given as:
And the generalized formula for an n-dimensional space is given as:
Where,
- n = number of dimensions
- pi, qi = data points
Now, we will calculate the Manhattan distance between the two points:
Note that The distance from Manhattan is also known as the city block distance. SciPy has a function called City block which returns the Manhattan distance between two points.
Let's now look at the following distance metric: the Minkowski distance.
3. Minkowski distance
The Minkowski distance is the generalized form of the Euclidean and Manhattan distance.
The formula for the Minkowski distance is given as:
Here, p represents the order of the norm. Let's calculate the Minkowski Distance of the order 3:
The parameter p of the SciPy Minkowski distance metric represents the order of the norm. When order (p) it is 1, will represent the Distance from Manhattan and when the order in the above formula is 2, will represent the Euclidean Distance.
Let's check that in Python:
Here, you can see that when the order is 1, both Minkowski and Manhattan Distance are the same. Let's also check the Euclidean distance:
When the order is 2, we can see that the Minkowski and Euclidean distances are the same.
Up to now, we've covered the distance metrics used when dealing with continuous or numeric variables. But What if we have categorical variables? How can we decide the similarity between categorical variables? This is where we can make use of another distance metric called Hamming Distance.
4. Hamming distance
Hamming distance measures the similarity between two strings of the same length. The Hamming distance between two strings of the same length is the number of positions where the corresponding characters are different.
Let's understand the concept with an example. Let's say we have two strings:
“Euclidiana” Y “Manhattan”
Since the length of these strings is equal, we can calculate the Hamming distance. We will go character by character and join the chains. The first character of both strings (e and m respectively) is different. Similarly, the second character of both strings (uya) is different. and so on.
Look carefully: seven characters are different, while two characters (the last two characters) they are similar:
Therefore, the Hamming Distance here will be 7. Note that the greater the Hamming Distance between two strings, the more different those strings will be (and vice versa).
Let's see how we can calculate Hamming distance of two strings in Python. First, we will define two strings that we will use:
These are the two chains “Euclidean” Y “manhattan” that we have also seen in the example. Let us now calculate the Hamming distance between these two strings:
As we saw in the previous example, the Hamming distance between "euclidean" and "manhattan" is 7. We also saw that the Hamming distance only works when we have strings of the same length.
Let's see what happens when we have chains of different lengths:
You can see that the lengths of both chains are different. Let's see what will happen when we try to calculate the Hamming distance between these two strings:
This throws an error saying that the lengths of the arrays must be the same. Therefore, Hamming distance only works when we have strings or arrays of the same length.
These are some of the similarity measures or distance matrices that are generally used in Machine Learning.
