Knowledge Transfer

L2 regularization is also referred to as weight decay. The reason for this name is that thinking about SGD and backpropagation, the negative gradient of the L2 regularization term with respect to a parameter w_i is – 2 * lambda * w_i, where lambda is the aforementioned hyperparameter, simply named weight decay in PyTorch. 

When we call loss.backward(), PyTorch traverses this graph in the reverse direction to compute the gradients and accumulate their values in the grad attribute of those tensors (the leaf nodes of the graph).

In working with images, it is good practice to compute the mean and standard deviation on all the training data in advance and then subtract and divide by these fixed, precomputed quantities.

PyTorch offers a package called Torchvision that includes many commonly used transforms for image processing. In PyTorch, we can achieve this using the Normalize Transform.

If you observe that the model is overfitting, you can increase the dropout rate. Conversely, you should try decreasing the dropout rate if the model underfits the training set.

Inside your data are subsets of data that represent different groups  If you plot the price values for each subset, you might get a bunch of different shapes.

torch.gather() creates a new tensor from the input tensor by taking the values from each row or column along the input dimension. The index values are passed as tensors, specifying which value to take from each ‘row’ or ‘column’.

The parameters method to ask any nn.Module for a list of parameters owned by it or any of its submodules. Calling model.parameters() will collect weight and bias from modules. It’s instructive to inspect the parameters in this case by printing their shapes.

All images for training are located in one folder and the target labels are in a CSV file. It should be possible to use a list of labels instead of inferring the classes from the directory structure. We have a list of labels corresponding number of files in the directory.

we will focus on writing the inference code for the single sample image. This will involve two parts, one where we prepare the image so that it can be fed to ResNet, and next, we will write the code to get the actual prediction from the model.