Category Archives: PyTorch

Forcing your network to learn redundant representations might sound very inefficient. But in practice, it makes things more robust and prevents overfitting. It also makes your network act as if taking the consensus over an ensemble of networks.

A custom collate_fn can be used to customize collation, e.g., padding sequential data to a max length of a batch.collate_fn is called with a list of data samples at each time. It is expected to collate the input samples into a batch for yielding from the data loader iterator. 

The pre-trained model is “frozen” and only the weights of the classifier get updated during training. In this case, the convolutional base extracted all the features associated with each image

PyTorch provides many classes to make data loading easy and code more readable. In this tutorial, we will see how to load and preprocess/augment custom datasets.

Network would already be able to extract generic features from your dataset. The network will not have to learn to extract generic features from scratch.

You need to reset the weights of the model so that each cross-validation fold starts from some random initial state and not learning from the previous folds. You could call reset_parameters() on all child modules.

A macro-average will compute the metric independently for each class and then take the average hence treating all classes equally, whereas a micro-average will aggregate the contributions of all classes to compute the average metric.

The optimal learning rate will be dependent on both your model architecture and your dataset. While using a default learning rate may provide decent results, you can often improve the performance or speed up training by searching for an optimal learning rate.

We use something called samplers for OverSampling. Though we did not use samplers exclusively, PyTorch used it for us internally. When we say shuffle=False, PyTorch ended up using SequentialSampler it gives an index from zero to the length of the dataset. When shuffle=True it ends up using a RandomSampler.

classification layer of the pre-trained model is specific to the original classification task, and subsequently specific to the set of classes on which the model was trained. You simply add a new classifier layer, which will be trained from scratch.