The weight initialization process can be critical to the model’s performance, and it requires the right method. In this tutorial, we’ll talk about how to initialize custom weights in an artificial neural network using NumPy array.
Default Weight initialization
Whenever we build and compile a model, the values for the weights will be set with random numbers. One random number per weight. Typically, these random numbers will be normally distributed such that the distribution of these numbers has a mean of 0 and a standard deviation of 1. Let’s see how we can specify a weight initializer for our own model in code using Keras.
input_shape=(28,28,1) model = tf.keras.Sequential( [ tf.keras.Input(shape=input_shape,name='input'), tf.keras.layers.Conv2D(32, kernel_size=(3, 3),kernel_initializer=tf.keras.initializers.zeros(), activation="relu",use_bias=True,name='conv1'), tf.keras.layers.Flatten(), tf.keras.layers.Dense(10, activation="relu",use_bias=True,bias_initializer='zeros',name='dense1'), tf.keras.layers.Dense(1, activation="softmax",name='output'), ] )
We’re going to use this arbitrary model that has two hidden one Convolutional and one Dense layer and an output layer with one node. First, we print its original weights that are initialized with zero.
The list has 2 elements, of shape
(input_dim, output_dim) and
(output_dim,) for weights and biases respectively. Every layer has parameter weights that you can set with a NumPy array for example
layer.set_weights(weights)(with the same shapes as the output of
Set weights for the Convolution Layer
You can pass a NumPy array to your convolutional layer through the
set_weights method. Remember that the weights of a convolutional layer are not only the weights but also the bias(if
use_bias is true). So if you want to set your weights you need to add an extra dimension for bias.
You can get the expected length of the list and its array shapes using the method
get_weights() from the same instances above.
Generate Random Weight
numpy.random.rand(shape) create an array of the given shape and populate it with random samples from a uniform distribution over [0, 1] Let’s create a (3,3,1,32).
Now you can set weights these ways:
set_weights() method of keras accepts a list of NumPy arrays. The shape of this should be the same as the shape of the output of
get_weights() on the same layer. Here’s the code:
If you call
get_weights ()again after you set the weights, it returns the list of NumPy arrays we passed to