One of the most common questions, that is done in several data science forums it is:
What is the difference between machine learning and statistical modeling?
I've been researching for the last 2 years. Generally, It doesn't take me more than a day to get a clear answer to the topic I'm researching. Despite this, this was definitely one of the hardest nuts to crack. When I came across this question at the beginning, I hardly found a clear answer that can determine how machine learning differs from statistical modeling. Given the similarity in terms of the objective that both try to solve, the only difference lies in the volume of data involved and human participation to build a model. Here's an interesting Venn diagram on the coverage of machine learning and statistical modeling in the data science universe (Reference: SAS Institute)
In this post, I will try to highlight the difference between the two as best I can.. I encourage the most experienced people in this industry to add to this post to highlight the difference.
Before starting, let's understand the objective behind what we are trying to solve to use any of these tools. The common goal behind using either tool is Learning from data. Both approaches aim to learn about the underlying phenomena through the use of data generated in the procedure..
Now that it is clear that the goal behind either approach is the same, let's review its definition and differences.
before continuing: Machine Learning Basics for Beginners
Definition:
Let's start with simple definitions:
Machine learning is …
an algorithm that can learn from data without relying on rule-based programming.
Statistical modeling is …
formalization of relationships between variables in the form of mathematical equations.
For people like me, who enjoy understanding concepts from practical applications, these definitions don't help much. Then, let's see a business case here.
A business case
Let's now look at an interesting example made public by McKinsey differentiating the two algorithms:
Case : Understand the risk level of customer churn over a period of time for a telecommunications company.
Available data : Two conductors: A and B
What McKinsey shows below is an absolute delight!! Just look at the graph below to understand the difference between a statistical model and a machine learning algorithm..
What did you observe in the previous graph? The statistical model is about obtaining a simple formulation of a boundary in a classification model obstacle. Here we see a nonlinear limit that, To some extent, separates people at risk from people without risk. But when we see the contours generated by the Machine Learning algorithm, we witness that statistical modeling is not comparable for the problem in question with the Machine Learning algorithm. Machine learning contours seem to capture all patterns beyond the limits of linearity or even the continuity of the limits. Here's what machine learning can do for you.
If this is not inspiration enough, machine learning algorithm is used in YouTube recommendation engines / Google, etc., that can generate trillions of observations in a second to arrive at a near perfect recommendation. Even with a laptop from 16 GB of RAM, I work daily with datasets containing millions of rows with hundreds of columns parametersThe "parameters" are variables or criteria that are used to define, measure or evaluate a phenomenon or system. In various fields such as statistics, Computer Science and Scientific Research, Parameters are critical to establishing norms and standards that guide data analysis and interpretation. Their proper selection and handling are crucial to obtain accurate and relevant results in any study or project.... and I build a complete model in no more than 30 minutes. A statistical model, Besides, you need a supercomputer to run a million observations with a thousand parameters.
Differences between machine learning and statistical modeling:
Given the taste of the difference in the production of these two approaches, let's understand the difference in the two paradigms, even though they both do almost equivalent work:
- Schools from which they come
- When did they arise?
- Assumptions in which they work
- Type of data they process
- Feature and Object Nomenclatures
- Techniques used
- Predictive power and human efforts involved to put it into practice
All the previously mentioned differences separate the two to some degree., but there is no hard line between machine learning and statistical modeling.
They belong to different schools
Machine learning is …
a subfield of computer science and artificial intelligence that is responsible for building systems that can learn from data, instead of explicitly programmed instructions.
Statistical modeling is …
a subfield of mathematics that is responsible for finding relationships between variables to predict an outcome
They arose at different times
Statistical modeling has been around for centuries. Despite this, machine learning is a very recent development. It emerged in the decade of 1990 when constant advances in digitization and cheap computing power allowed data scientists to stop building finished models and, However, train computers to do it. The unmanageable volume and complexity of the big data the world is swimming in has increased the potential and need for machine learning.
Scope of the assumptions involved
Statistical modeling works with a number of assumptions. As an example, a linear regression assumes:
- Linear relationship between variableIn statistics and mathematics, a "variable" is a symbol that represents a value that can change or vary. There are different types of variables, and qualitative, that describe non-numerical characteristics, and quantitative, representing numerical quantities. Variables are fundamental in experiments and studies, since they allow the analysis of relationships and patterns between different elements, facilitating the understanding of complex phenomena.... independent and dependent
- Homocedasticidad
- Mean error at zero for each dependent value
- Independence of observations
- The error must be distributed evenly for each value of the dependent variable.
Equivalently, logistic regressions come with their own set of assumptions. Even a nonlinear model has to meet a continuous segregation limit. Machine learning algorithms assume some of these things, but in general they are saved from most of these assumptions. The biggest advantage of using a machine learning algorithm is that there is feasible no boundary continuity as shown in the case study above.. At the same time, we don't need to specify the distribution of the dependent or independent variable in a machine learning algorithm.
Types of data with which they deal
Machine learning algorithms are wide-ranging tools. Online learning tools predict data on the fly. These tools are capable of learning from billions of observations one by one. They make predictions and learn simultaneously. Other algorithms like Random Forest and Gradient Boosting are also exceptionally fast with big data.. Machine learning works great with broad (large number of attributes) and deep (large number of observations). Despite this, statistical models are generally applied for smaller data with fewer attributes or end up overfitting.
Naming convention
Here are names that refer to almost the same things:
Formulation
Even when the end goal for both machine learning and statistical modeling is the same, the formulation of two is significantly different.
In a statistical model, we simply try to estimate the function f in
Dependent Variable ( Y ) = f(Independent Variable) + error function
Machine learning removes the deterministic function “f” of the equation. It just becomes
Output(Y) ----- > Input (X)
It will try to find bags of X in n dimensions (where n is the number of attributes), where the occurrence of Y is significantly different.
Predictive power and human effort
Nature assumes nothing before forcing an event to occur.
Then, the least assumptions in a predictive model, the greater the predictive power. Machine learning, as the name suggests, requires minimal human effort. Machine learning works in iterations in which the computer tries to discover hidden patterns in the data. Because the machine does this job with complete data and is independent of all assumptions, predictive power is generally very strong for these models. The statistical model is intensive in mathematics and is based on the estimation of coefficients. You need the modeler to understand the binding between variables before entering them.
Final notes
Despite this, it may seem that machine learning and statistical modeling are two different branches of predictive modeling, they are almost the same. The difference between these two has narrowed significantly over the last decade. Both branches have learned a lot from each other and will become closer in the future. I hope I have motivated you enough to gain skills in each of these two domains and then compare how they complement each other..
If you are interested in getting machine learning algorithms, we have what you need. We are in the process of building a learning path for machine learning that will be published soon..
Let us know what you think is the difference between machine learning and statistical modeling. Do you have any case studies to point out the differences between the two?
