Category Archives: PyTorch

This process is callable nominalization with attributes having a rescaled range of 0 and 1. It ensures the existence of an optimization algorithm that forms the core of gradient descent -an exam of the learning algorithm.

The transformers API makes it possible to save all of these pieces to disk at once, saving everything into a single archive in the PyTorch or TensorFlow saved model format.

PyTorch would compute the derivatives of the loss throughout the chain of functions (the computation graph) and accumulate their values in the grad attribute of those tensors.

You can be fairly sure that the model is using two-node binary classification because multi-class classification would have three or more output nodes and one-node binary classification would have one output node

In multi-label classification, we use a binary classifier where each neuron(y_train.shape[1]) in the output layer is responsible for one vs all class classification. binary_crossentropy is suited for binary classification and thus used for multi-label classification.

Stratified train_test_split to maintain the imbalance so that the test and train dataset have the same distribution, then never touch the test set again. Stratified ensure that each dataset split has the same proportion of observations with a given label.

Shuffle the list before splitting else you won’t get all the classes in the three splits since these indices would be used by the Subset class to sample from the original dataset. Shuffling the elements of a tensor amounts to finding a permutation of its indices. The random_split function does exactly this:

Pinned memory is used as a staging area for transfers from the device to the host. We can avoid the cost of the transfer between pageable and pinned host arrays by directly allocating our host arrays in pinned memory.

It’s quite a bit faster due to the with torch.no_grad() context manager explicitly informing PyTorch that no gradients need to be computed. Context managers like with torch.no_grad(): can be used to control auto-grad’s behavior.

Using the related set_grad_enabled context, we can also condition the code to run with autograd enabled or disabled, according to a Boolean expression—typically indicating whether we are running in training or inference mode.