Skip to content

Knowledge Transfer

  • PyTorch
  • Keras
  • Flutter
  • TensorFlow
  • Pandas
  • Android
  • Contact Us
January 21, 2023

Activation function for Output Layer in Regression, Binary, Multi-Class, and Multi-Label Classification

Kerasadmin

The ReLU activation function is a default choice for the hidden layers. For the output layer, in general, you will want the logistics activation function for binary classification, the softmax activation function for multiclass classification, and no activation function for regression.

January 13, 2023

Adam optimizer with learning rate weight decay using AdamW in keras

Kerasadmin

Common deep learning libraries only implement L2 regularization, not the original weight decay. Therefore, on datasets where the use of L2 regularization is beneficial for SGD on many popular image classification datasets, Adam leads to worse results than SGD with momentum for which L2 regularization behaves as expected.

January 6, 2023

Split data set into Train and Test set Using Keras image_dataset_from_directory/folder.

Kerasadmin

Fraction of the training data to be used as validation data. The Keras will set apart this fraction of the training data. The validation data is selected from the last samples in the x and y data provided.

January 1, 2023

Split Imbalanced dataset using sklearn Stratified train_test_split().

KerasPyTorchadmin

Stratified train_test_split to maintain the imbalance so that the test and train dataset have the same distribution, then never touch the test set again. Stratified ensure that each dataset split has the same proportion of observations with a given label.

December 26, 2022

Split Custom PyTorch DataSet into Training, Testing and Validation set using random_split

PyTorchadmin

Shuffle the list before splitting else you won’t get all the classes in the three splits since these indices would be used by the Subset class to sample from the original dataset. Shuffling the elements of a tensor amounts to finding a permutation of its indices. The random_split function does exactly this:

December 13, 2022

PyTorch DataLoader set pin_memory to True

PyTorchadmin

Pinned memory is used as a staging area for transfers from the device to the host. We can avoid the cost of the transfer between pageable and pinned host arrays by directly allocating our host arrays in pinned memory.

December 6, 2022

PyTorch:Difference between “tensor.detach()” vs “with torch.nograd()”

PyTorchadmin

It’s quite a bit faster due to the with torch.no_grad() context manager explicitly informing PyTorch that no gradients need to be computed. Context managers like with torch.no_grad(): can be used to control auto-grad’s behavior.

December 2, 2022

What does require_grad=false or true in PyTorch?

PyTorchadmin

Using the related set_grad_enabled context, we can also condition the code to run with autograd enabled or disabled, according to a Boolean expression—typically indicating whether we are running in training or inference mode.

November 26, 2022

PyTorch: What does model.train()?

PyTorchadmin

When the user specifies model.eval() and the model contains a batch normalization module, the running estimates are frozen and used for normalization. To unfreeze running estimates and return to using the minibatch statistics, we call model.train(), just as we did for dropout.

November 19, 2022

How to save and load PyTorch Tensor to file?

PyTorchadmin

We can save tensors quickly this way but if we want to load them with the file format itself is not interoperable. We can’t read the tensor with software other than PyTorch. Depending on the use case, this may or may not be a limitation, but we should learn how to save tensors interoperably.

Posts navigation

1 2 … 24 Next

Recent Posts

  • Activation function for Output Layer in Regression, Binary, Multi-Class, and Multi-Label Classification
  • Adam optimizer with learning rate weight decay using AdamW in keras
  • Split data set into Train and Test set Using Keras image_dataset_from_directory/folder.
  • Split Imbalanced dataset using sklearn Stratified train_test_split().
  • Split Custom PyTorch DataSet into Training, Testing and Validation set using random_split
  • Privacy Policy
Copyright © 2023 Knowledge Transfer All Rights Reserved.