This is the last article in this series. This article is about another pre-trained CNN known as the ResNet along with an output visualization parameter known as the confusion matrix.

ResNet

This is also known as a residual network. It has three variations 51,101,151. They used a simple technique to achieve this high number of layers.

Credit – Xiaozhu0429/ Wikimedia Commons / CC-BY-SA-4.0

The problem in using many layers is that the input information gets changed in accordance with each layer and subsequently, the information will become completely morphed. So to prevent this, the input information is sent in again like a recurrent for every two steps so that the layers don’t forget the original information. Using this simple technique they achieved about 100+ layers.

ResNet these are the three fundamentals used throughout the network.

  (conv1): Conv2d (3, 64, kernel_size= (7, 7), stride= (2, 2), padding= (3, 3))

  (relu): ReLU

  (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1)

These are the layers found within a single bottleneck of the ResNet.

    (0): Bottleneck

  1    (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1))

  2    (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))     

  3    (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1))    

      (relu): ReLU(inplace=True)

   Down sampling   

   Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1))

    (1): Bottleneck

  4    (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1))

  5    (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))     

  6   (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1))     

      (relu): ReLU(inplace=True)

    )

    (2): Bottleneck

  7    (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1))

  8    (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))

  9   (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1))

   (relu): ReLU

There are many bottlenecks like these throughout the network. Hence by this, the ResNet is able to perform well and produce good accuracy. As a matter of fact, the ResNet is the model which won the ImageNet task competition.

There are 4 layers in this architecture. Each layer has a bottleneck which comprises convolution followed by relu activation function. There are 46 convolutions, 2 pooling, 2 FC layers.

Type	No of layers
7*7 convolution	1
1*1, k=64 + 3*3, k=64+1*1, k=256 convolution	9
1*1, k=128+ 3*3, k=128+1*1, k=512  convolution	10
1*1, k=256+ 3*3, k=256 + 1*1, k=1024 convolution	16
1 * 1, k=512+3 * 3, k=512+1 * 1, k=2048 convolution	9
Pooling and FC	4
Total	50

There is a particular aspect apart from the accuracy which is used to evaluate the model, especially in research papers. That method is known as the confusion matrix. It is seen in a lot of places and in the medical field it can be seen in test results. The terms used in the confusion matrix have become popularized in the anti-PCR test for COVID.

The four terms used in a confusion matrix are True Positive, True Negative, and False positive, and false negative. This is known as the confusion matrix.

True positive- both the truth and prediction are positive

True negative- both the truth and prediction are negative

False-positive- the truth is negative but the prediction is positive

False-negative- the truth is positive but the prediction is false

Out of these the false positive is dangerous and has to be ensured that this value is minimal.

We have now come to the end of the series. Hope that you have got some knowledge in this field of science. Deep learning is a very interesting field since we can do a variety of projects using the artificial brain which we have with ourselves. Also, the technology present nowadays makes these implementations so easy. So I recommend all to study and do projects using these concepts. Till then,

HAPPY LEARNING!!!

This article is about one of the pre-trained CNN models known as the VGG-16. The process of using a pretrained CNN is known as transfer learning. In this case, we need not build a CNN instead we can use this with a modification. The modifications are:-

• Removing the top (input) and bottom (output) layers
• Adding input layer with size equal to the dimension of the image
• Adding output layer with size equal to number of classes
• Adding additional layers (if needed)

The pre-trained model explained in this article is called the VGGNet. This model was developed by the Oxford University researchers as a solution to the ImageNet task. The ImageNet data consists of 10 classes with 1000 images each leading to 10000 images in total.

VGGNet

I/p 1     2   3     4     5        6       7         8      9          10     11            12       13   o/p

Credit: – Nshafiei neural network in Machine learning  Creative Commons Attribution-ShareAlike 4.0 License.

This is the architecture for VGGNet. This has been found for the CIFAR-10 dataset, a standard dataset containing 1000 classes. This was used for multiclass classification. Some modifications are made before using it for detecting OA. The output dimension is changed into 1*1*2 and the given images must be reshaped to 224*224 since this dimension is compatible with VGGNet. The dimensions and other terms like padding, stride, number of filters, dimension of filter are chosen by researchers and found optimal. In general, any number can be used in this place.

The numbers given below the figure correspond to the layer number. So the VGGNet is 13 layered and is CNN till layer 10 and the rest are FNN.

Colour index	Name
Grey	Convolution
Red	Pooling
Blue	FFN

Computations and parameters for each layer

Input

224*224 images are converted into a vector whose dimension is 224*224*3 based on the RGB value.

Layer 1-C1

This is the first convolutional layer. Here 64 filters are used.

Wi =224, P=1, S=1, K=64, f=3*3

Wo =224 (this is the input Wi for the next layer)

Dim= 224*224*64

Parameter= 64*3*3= 576

Layer 2-P1

This is the first pooling layer

 Wi =224, S=2, P=1, f=3

Wo=112 (this is the input Wi for the next layer)

Dim= 112*112*3

Parameter= 0

Layer 3-C2C3

Here two convolutions are applied. 128 filters are used.

Wi =112, P=1, S=1, K=64, f=3

Wo=112 (this is the input Wi for the next layer)

Dim= 112*112*128

Parameter= 128*3*3=1152

Layer 4- P2

Second pooling layer

Wi =112, P=1, S=2, f=3*3

Wo =56 (this is the input Wi for the next layer)

Dim= 56*56*3

Parameter= 0

Layer 5- C4C5C6

Combination of three convolutions

Wi =56, P=1, S=1, K=256, f=3*3

Wo = 56 (this is the input Wi for the next layer)

Dim= 224*224*64

Parameter= 64*3*3= 576

Layer 6-P3

Third pooling layer

Wi =56, P=1, S=2, f=3*3

Wo =28 (this is the input Wi for the next layer)

Dim= 28*28*3

Parameter= 0

Layer 7-C7C8C9

Combination of three convolutions

Wi =28, P=1, S=1, K=512, f=3*3

Wo =28 (this is the input Wi for the next layer)

Dim= 28*28*512

Parameter= 512*3*3= 4608

Layer 8-P4

Fourth pooling layer

Wi =28, P=1, S=2, f=3*3

Wo =14 (this is the input Wi for the next layer)

Dim= 14*14*3

Parameter= 0

Layer 9-C10C11C12

Last convolution layer, Combination of three convolutions

Wi =14, P=1, S=1, K=512, f=3*3

Wo =14 (this is the input Wi for the next layer)

Dim= 14*14*512

Parameter= 512*3*3= 4608

Layer 10-P5

Last pooling layer and last layer in CNN

Wi =14, P=1, S=2, f=3*3

Wo =7 (this is the input Wi for the next layer)

Dim= 7*7*3

Parameter= 512*3*3= 4608

With here the CNN gets over. So a complex 224*224*3 boil down to 7*7*3

Trends in CNN

As the layer number increases,

1. The dimension decreases.
2. The filter number increases.
3. Filter dimension is constant.

In convolution

Padding of 1 and stride of 1 to transfer original dimensions to output

In pooling

Padding of 1 and stride of 2 are used in order to half the dimensions.

Layer 11- FF1

4096 neurons

Parameter= 512*7*7*4096=102M

Wo= 4096

Layer 12- FF2

4096 neurons

Wo= 4096

Parameter= 4096*4096= 16M

Output layer

2 classes

• non-osteoarthritic
• osteoarthritic

Parameter= 4096*2= 8192

Parameters

Layer	Value of parameters
Convolution	16M
FF1	102M
FF2	16M
Total	134M

It takes a very large amount of time nearly hours for a machine on CPU to learn all the parameters. Hence they came with speed enhancers like faster processors known as GPU Graphic Processing Unit which may finish the work up to 85% faster than CPU.

HAPPY LEARNING!!

The previous article was about the padding, stride, and parameters of CNN. This article is about the pooling and the procedure to build an image classifier.

Pooling

This is another aspect of CNN. There are different types of pooling like min pooling, max pooling, avg pooling, etc. the process is the same as before i.e. the kernel vector slides over the input vector and does computations on the dot product. If a 3*3 kernel is considered then it is applied over a 3*3 region inside the vector, it finds the dot product in the case of convolution. The same in pooling finds a particular value and substitutes that value in the output vector. The kernel value decides the type of pooling. The following table shows the operation done by the pooling.

Type of pooling	The value seen in the output layer
Max pooling	Maximum of all considered cells
Min pooling	Minimum of all considered cells
Avg pooling	Average of all considered cells

The considered cells are bounded within the kernel dimensions.

The pictorial representation of average pooling is shown above. The number of parameters in pooling is zero.

Convolution and pooling are the basis for feature extraction. The vector obtained from this step is fed into an FFN which then does the required task on the image.

Features of CNN

1. Sparse connectivity
2. Weight sharing.

    

    Feature extraction-CNN              classifier-FNN

In general, CNN is first then FFN is later. But the order or number or types of convolution and pooling can vary based on the complexity and choice of the user.

Already there are a lot of models like VGGNet, AlexNet, GoogleNet, and ResNet. These models are made standard and their architecture has been already defined by researchers. We have to reshape our images in accordance with the dimensions of the model.

General procedure to build an image classifier using CNN

1. Obtain the data in the form of image datasets.
2. Set the output classes for the model to perform the classification on.
3. Transform or in specific reshape the dimension of the images compatible to the model. The image size maybe 20*20 but the model accepts only 200*200 images; then we must reshape them to that size.
4. Split the given data into training data and evaluation data. This is done by creating new datasets for both training and validation. More images are required for training.
5. Define the model used for this task.
6. Roughly sketch the architecture of the network.
7. Determine the number of convolutions, pooling etc. and their order
8. Determine the dimensions for the first layer, padding, stride, number of filters and dimensions of filter.
9. Apply the formula and find the output dimensions for the next layer.
10. Repeat 5d till the last layer in CNN.
11. Determine the number of layers and number of neurons per layer and parameters in FNN.
12. Sketch the architecture with the parameters and dimension.
13. Incorporate these details into the machine.
14. Or import a predefined model.  In that case the classes in the last layer in the FNN must be replaced with ‘1’ for binary classification or with the number of classes. This is known as transfer learning.
15. Train the model using the training dataset and calculate the loss function for periodic steps in the training.
16. Check if the machine has performed correctly by comparing the true output with model prediction and hence compute the training accuracy.
17. Test the machine with the evaluation data and verify the performance on that data and compute the validation accuracy.
18.   If both the accuracies are satisfactory then the machine is complete.

HAPPY LEARNING!!