Target
- When we talk about the Pre-trained model in the domain of Computer Vision, Alexnet is emerging as a leading architecture.
- Let's understand the architecture of Alexnet initiative by its authors.
Introduction
Alexnet won Imagenet's large-scale visual accreditation challenge in 2012. The model was proposed in 2012 in the research post called Imagenet Classification with Deep Convolution Neural Network by Alex Krizhevsky and colleagues.
In this model, network depth increased compared to Lenet-5. In case you want to know more about Lenet-5, I will recommend you to consult the following article-
Note: If you are more interested in learning concepts in an audiovisual format, we have this full post explained in the video below. If that is not the case, you can keep reading.
Alexnet has eight layers with parametersThe "parameters" are variables or criteria that are used to define, measure or evaluate a phenomenon or system. In various fields such as statistics, Computer Science and Scientific Research, Parameters are critical to establishing norms and standards that guide data analysis and interpretation. Their proper selection and handling are crucial to obtain accurate and relevant results in any study or project.... that can be learned. The model consists of five layers with a combination of maximum grouping followed by 3 absolutely connected layers and use activation resumeThe ReLU activation function (Rectified Linear Unit) It is widely used in neural networks due to its simplicity and effectiveness. Defined as ( f(x) = max(0, x) ), ReLU allows neurons to fire only when the input is positive, which helps mitigate the problem of gradient fading. Its use has been shown to improve performance in various deep learning tasks, making ReLU an option... in each of these layers, except in the Output layerThe "Output layer" is a concept used in the field of information technology and systems design. It refers to the last layer of a software model or architecture that is responsible for presenting the results to the end user. This layer is crucial for the user experience, since it allows direct interaction with the system and the visualization of processed data.....
They found that using relu as a wake functionThe activation function is a key component in neural networks, since it determines the output of a neuron based on its input. Its main purpose is to introduce nonlinearities into the model, allowing you to learn complex patterns in data. There are various activation functions, like the sigmoid, ReLU and tanh, each with particular characteristics that affect the performance of the model in different applications.... accelerated the speed of the trainingTraining is a systematic process designed to improve skills, physical knowledge or abilities. It is applied in various areas, like sport, Education and professional development. An effective training program includes goal planning, regular practice and evaluation of progress. Adaptation to individual needs and motivation are key factors in achieving successful and sustainable results in any discipline.... in almost six times. They also used the layers of abandonment, that prevented your model from overfitting. At the same time, the model is trained on the Imagenet dataset. The Imagenet dataset is almost 14 million images in a thousand classes.
Let's see the architectural details in this post.
Alexnet architecture
One thing to pay attention here, since Alexnet is deep architecture, the authors introduced padding to prevent the size of the feature maps from being drastically reduced. The input to this model are the images of size 227X227X3.
Layers of convolution and maximum grouping
Then we apply the first convolution layer with 96 11X11 size filters with stride 4. The activation function used in this layer is relu. The output characteristics map is 55X55X96.
In case you don't know how to calculate the output size of a convolution layer
output = ((Inlet filter size) / stride) +1
At the same time, the number of filters becomes the channel in the output characteristics map.
Then, we have the first layer of Maxpooling, 3X3 size and stride 2. After, we obtain the resulting feature map with the size 27X27X96.
After that, we apply the second convolution operation. This time, the filter size is reduced to 5X5 and we have 256 filters of this type. The stride is 1 and the padding 2. The activation function used is again relu. Now the output size we get is 27X27X256.
Again, we apply a 3X3 size maximum grouping coat with stride 2. The resulting feature map has the form 13X13X256.
Now we apply the third convolution operation with 384 3X3 stride size filters 1 and also padding 1. Again the activation function used is relu. The output feature map has the form 13X13X384.
Then we have the fourth convolution operation with 384 3X3 size filters. The stride along with the padding is 1. At the same time that activation function used is relu. Now the output size remains unchanged, In other words, 13X13X384.
After that, we have the final 3X3 size convolution layer with 256 filters of this type. Stride and padding are set to one and the trigger function is relu. The resulting feature map has the form 13X13X256.
Then, if you look at the architecture so far, the number of filters increases as we go deeper. Therefore, is extracting more features as we move into architecture. At the same time, filter size is shrinking, which means that the initial filter was bigger and, as we go, filter size decreases, resulting in a decrease in the shape of the feature map.
Then, we apply the third layer of maximum grouping of size 3X3 and stride 2. Resulting in the characteristics map of the 6X6X256 shape.
Absolutely connected and abandoned layers
After that, we have our first layer of abandonment. The churn rate is set to 0,5.
Then we have the first layer absolutely connected with a Relu activation functionThe ReLU activation function (Rectified Linear Unit) It is widely used in neural networks due to its simplicity and effectiveness. is defined as ( f(x) = max(0, x) ), meaning that it produces an output of zero for negative values and a linear increment for positive values. Its ability to mitigate the problem of gradient fading makes it a preferred choice in deep architectures..... The output size is 4096. Then comes another layer of churn with the churn rate set to 0,5.
This followed by a second layer absolutely connected with 4096 neurons and relu activation.
In conclusion, we have the last absolutely connected layer or output layer with 1000 neurons, since we have 10000 classes in the dataset. The activation function used in this layer is Softmax.
This is the architecture of the Alexnet model. Has a total of 62,3 millions of learnable parameters.
Final notes
To quickly summarize the architecture we have seen in this post.
- Has 8 layers with parameters that can be learned.
- Input to the model are RGB images.
- Has 5 convolution layers with a combination of maximum grouping layers.
- Then has 3 absolutely connected layers.
- The activation function used in all layers is Relu.
- Used two layers of abandon.
- The activation function used in the output layer is Softmax.
- The total number of parameters in this architecture is 62,3 millions.
So this was all about Alexnet. If you have any doubts, let me know in the comments below.
