Knowledge Transfer

torch.gather() creates a new tensor from the input tensor by taking the values from each row or column along the input dimension. The index values are passed as tensors, specifying which value to take from each ‘row’ or ‘column’.

The parameters method to ask any nn.Module for a list of parameters owned by it or any of its submodules. Calling model.parameters() will collect weight and bias from modules. It’s instructive to inspect the parameters in this case by printing their shapes.

All images for training are located in one folder and the target labels are in a CSV file. It should be possible to use a list of labels instead of inferring the classes from the directory structure. We have a list of labels corresponding number of files in the directory.

we will focus on writing the inference code for the single sample image. This will involve two parts, one where we prepare the image so that it can be fed to ResNet, and next, we will write the code to get the actual prediction from the model.

The batch size defines the number of samples in the batch. That batch propagated through the network before updating the model parameters.

Confusion matrix lets us see not only where the model was wrong, but also how it was wrong. That is, we can look at patterns of misclassification. For example, our model had an easy time differentiating ‘truck and dog’, but a much more difficult time classifying ‘dog and cat’. 

Keras library provides a way to calculate standard metrics when training and evaluating deep learning models. In Keras, metrics are passed during the compile stage as shown below. You can pass several metrics by comma separating them

When creating an np.array from torch.tensor or vice versa, both object reference the same underlying storage in memory. Since np.ndarray does not store the computational graph associated with the array, this graph should be explicitly removed using detach() when sharing both numpy and torch wish to reference the same tensor.

If backward is called again as in any training loop then the loss is evaluated again, and the gradient at each leaf is accumulated on top.

Using the designated settings for training model.train() and evaluation model.eval() will automatically set the mode for the dropout layer and batch normalization layers and rescale appropriately so that we do not have to worry about that at all.