knowledge Transfer

As we go deeper in the neural network typically you start off with larger images [32x32x3] then the height and width will gradually trend down as you go deeper in the neural network. Whereas the number of channels generally increases. You see this general trend in a lot of other convolutional neural networks.

A custom collate_fn can be used to customize collation, e.g., padding sequential data to a max length of a batch.collate_fn is called with a list of data samples at each time. It is expected to collate the input samples into a batch for yielding from the data loader iterator. 

The pre-trained model is “frozen” and only the weights of the classifier get updated during training. In this case, the convolutional base extracted all the features associated with each image

In this tutorial, we’ll discuss what regularization is and when and why it may be helpful to add it to our model.

PyTorch provides many classes to make data loading easy and code more readable. In this tutorial, we will see how to load and preprocess/augment custom datasets.

Network would already be able to extract generic features from your dataset. The network will not have to learn to extract generic features from scratch.

It is sometimes desirable to use a separate penalty with a different coefficient for each layer of the network.

You need to reset the weights of the model so that each cross-validation fold starts from some random initial state and not learning from the previous folds. You could call reset_parameters() on all child modules.

A macro-average will compute the metric independently for each class and then take the average hence treating all classes equally, whereas a micro-average will aggregate the contributions of all classes to compute the average metric.

Global Average Pooling does something different. It applies average pooling on the spatial dimensions until each spatial dimension is one, and leaves other dimensions unchanged.