knowledge Transfer

standardize the inputs to your network as much as possible, so that learning is more stable by reducing variability across the training data. In terms of normalization of the data, that all features are in the same range so that they contribute equally.

Leaky ReLU function is nearly identical to the standard ReLU function. The Leaky ReLU sacrifices hard-zero sparsity for a gradient which is potentially more robust during optimization. Alpha is a fixed parameter(float >= 0.).

sample_weights functionality adds more importance to some samples than others during training.so that the loss function used by the model is weighted per-sample not per class. It changes the way the loss is calculated

You can do this whether you’re building Sequential models, Functional API models, or subclassed models. Using this custom training algorithm, you still get the benefit from the convenient features of fit(), such as callbacks, built-in distribution support, or step

Every FormField has it’s own validate function and within that validate function it will invoke the validator. The error message is displayed in the TextField.

By default, the progress indicators will use the color from your theme’s accentColor. You can also set a custom color with the valueColor parameter.

The sigmoid was not zero-centered tanh fixed this and now ReLU has this problem again and that’s one of the issues of the ReLU. ReLU doesn’t activate for negative inputs, it’s possible to end up with “dead neurons” that never fire.

In multi-classes classification last layer use “softmax” activation, which means it will return an array of 10 probability scores (summing to 1) for 10 class. Each score will be the probability that the current class belongs to one of our 10 classes.

All category is not represented in the same quantity. Some of them can be very infrequent which may represent a hard challenge for any ML algorithm.

Implementations of pre-trained BERT models already exist in TensorFlow due to its popularity. I leveraged the popular transformers library while building out this project.