Red neuronal artificial | ANN Beginner's Guide

This article was published as part of the Data Science Blogathon

Introduction

machine. It is inspired by the functioning of a human brain and, Thus, is a set of neural network algorithms that tries to imitate the functioning of a human brain and learn from experiences.

In this article, we are going to learn how a basic neural network works and how it improves itself to make the best predictions.

Table of Contents

1. Neural networks and their components
2. Perceptron and multilayer perceptron
3. Step-by-step work of the neural network
4. Back propagation and how it works
5. Brief about activation functions

Artificial neural networks and their components

Neural networks is a computational learning system that uses a network of functions to understand and translate a data input from one way into a desired output, normally in another form. The concept of artificial neural network was inspired by human biology and the way in which neurons of the human brain work together to understand the inputs of the human senses.

In simple words, neural networks are a set of algorithms that try to recognize patterns, data relationships and information through the process that is inspired and functions like the brain / human biology.

Components (edit) / Neural network architecture

A simple neural network consists of three components :

• Input layer
• Hidden cloak
• Output layer

Source: Wikipedia

Input layer: Also known as input nodes, are the entrances / information from the outside world that is provided to the model to learn and draw conclusions. The input nodes pass the information to the next layer, namely, hidden layer.

Hidden cloak: The hidden layer is the set of neurons where all the calculations are performed on the input data. There can be any number of hidden layers in a neural network. The simplest network consists of a single hidden layer.

Output layer: The output layer is the output / model conclusions derived from all calculations performed. There may be one or more nodes in the output layer. If we have a binary classification problem, the output node is 1, but in the case of multiple class classification, output nodes can be more than 1.

Perceptron and multilayer perceptron

Perceptron is a simple form of neural network and consists of a single layer where all the mathematical calculations are performed.

Source: kindsonthegenius.com

While, Multilayer Perceptron also know as Artificial neural networks It consists of more than one perception that is grouped to form a multi-layered neural network.

Source: Half

In the picture above, the artificial neural network consists of four interconnected layers:

• An input layer, with 6 input nodes.
• Front cover 1 hidden, with 4 hidden nodes / 4 perceptrons
• Hidden cloak 2, with 4 hidden nodes
• Output layer with 1 exit node

Step by step Working de la red neuronal artificial

Source: Xenonstack.com

1. In the first step The input units are passed, namely, the data is passed with some weights attached to the hidden layer.. We can have any number of hidden layers. In the picture above, the inputs x₁,X₂,X₃,….X_North is passed.

2. Each hidden layer consists of neurons. All inputs are connected to each neuron.

3. After transmitting the tickets, all calculation is done in the hidden layer (Blue oval in the picture)

The calculation carried out in hidden layers is carried out in two steps which are the following :

• First, all inputs are multiplied by their weights. Weight is the gradient or coefficient of each variable. Shows the strength of the particular input. After assigning the weights, a bias variable is added. Bias is a constant that helps the model to fit in the best possible way.

WITH₁ = W₁*In₁ + W₂*In₂ + W₃*In₃ + W₄*In₄ + W₅*In₅ + b

W₁, W₂, W₃, W₄, W5 are the weights assigned to the In inputs₁, In₂, In₃, In₄, In_5, and b is the bias.

• Later, in the second step, the The activation function is applied to the linear equation Z1. The activation function is a non-linear transformation that is applied to the input before sending it to the next layer of neurons. The importance of the activation function is to instill non-linearity in the model.

There are various activation functions which will be listed in the next section.

4. The whole process described in point 3 performed on each hidden layer. After going through every hidden layer, we go to the last layer, namely, our output layer that gives us the final output.

The process explained above is known as forward propagation.

5. After getting the predictions from the output layer, the error is calculated, namely, the difference between actual and expected output.

If the error is big, then steps are taken to minimize the error and for the same purpose, Backward propagation is performed.

What is backward propagation and how does it work?

Reverse propagation is the process of updating and finding the optimal values ​​of weights or coefficients that helps the model to minimize the error, namely, the difference between the actual and predicted values.

But here is the question: How are weights updated and new weights calculated??

Weights are updated with the help of optimizers.. Optimizers are the methods / mathematical formulations to change the attributes of neural networks, namely, the weights to minimize the error.

Downward sloping backward propagation

Gradient Descent is one of the optimizers that helps to calculate the new weights. Let's understand step by step how Gradient Descent optimizes the cost function.

In the image below, the curve is our cost function curve and our goal is to minimize the error such that J_min namely, global minimums are reached.

Source: Quora

Steps to reach global minimums:

1. First, weights are randomly initialized namely, the random value of the weight and the intersections are assigned to the model while the forward propagation and errors are calculated after all the calculation. (As discussed above)

2. So he the gradient is calculated, namely, derived from error with current weights

3. Later, the new weights are calculated using the following formula, where a is the learning rate which is the parameter also known as the step size to control the speed or steps of the back propagation. Provides additional control over how fast we want to move around the curve to reach global lows.

Source: hmkcode.com

4.This process of calculating the new weights, then the errors of the new weights and then the update of the weights. continues until we hit global lows and the loss is minimized.

One point to keep in mind here is that the learning rate, namely, a in our weight update The equation must be chosen wisely. The learning rate is the amount of change or the size of the step taken to reach the global minimums. It should not be too small since it will take time to converge, as well as it should not be very big that does not reach the global minimums at all. Therefore, the learning rate is the hyperparameter that we have to choose based on the model.

Source: Educative.io

To know the detailed mathematics and the backpropagation chain rule, see attachment tutorial.

Brief about activation functions

Trigger functions are attached to each neuron and are mathematical equations that determine whether a neuron should fire or not based on whether the neuron's input is relevant to the model's prediction or not. The purpose of the activation function is to introduce non-linearity in the data.

Various types of trigger functions are:

Related