Category Archives: Keras
Advantages of ReLU vs Tanh vs Sigmoid activation function in deep neural networks.
The saturated neurons can kill gradients if we’re too positive or too negative of an input. They’re also not zero-centered and so we get these, this inefficient kind of gradient update. The third problem is an exponential function. This is a little bit computationally expensive.
How to deactivate dropout layers while evaluation and prediction mode in Keras?
Forcing your network to learn redundant representations might sound very inefficient. But in practice, it makes things more robust and prevents overfitting. It also makes your network act as if taking the consensus over an ensemble of networks.
Calculate Output Size of Convolutional and Pooling layers in CNN.
As we go deeper in the neural network typically you start off with larger images [32x32x3] then the height and width will gradually trend down as you go deeper in the neural network. Whereas the number of channels generally increases. You see this general trend in a lot of other convolutional neural networks.
Micro and Macro Averages for imbalance multiclass classification
A macro-average will compute the metric independently for each class and then take the average hence treating all classes equally, whereas a micro-average will aggregate the contributions of all classes to compute the average metric.
Filters, kernel size, input shape in Conv2d layer
We don’t explicitly define the filters that our convolutional layer will use, instead parameterize the filters and let the network learn the best filters to use during training. We need to define “how many filters we’ll use at each layer”.
How to save Keras training History object to File using Callback?
You can learn a lot about Keras models by observing their History objects after training. In this post, you will discover how you can save the history object into a CSV file of deep learning models training metrics over time during training.