The weight initialization process can be critical to the model’s performance, and it requires the right method. In this tutorial, we’ll talk about how to initialize custom weights in an artificial neural network using NumPy array.

Default Weight initialization

Whenever we build and compile a model, the values for the weights will be set with random numbers. One random number per weight. Typically, these random numbers will be normally distributed such that the distribution of these numbers has a mean of 0 and a standard deviation of 1. Let’s see how we can specify a weight initializer for our own model in code using Keras.

input_shape=(28,28,1)
model = tf.keras.Sequential(
    [
        tf.keras.Input(shape=input_shape,name='input'),
        tf.keras.layers.Conv2D(32, kernel_size=(3, 3),kernel_initializer=tf.keras.initializers.zeros(),
                               activation="relu",use_bias=True,name='conv1'),
        tf.keras.layers.Flatten(),
        tf.keras.layers.Dense(10, activation="relu",use_bias=True,bias_initializer='zeros',name='dense1'),
        tf.keras.layers.Dense(1, activation="softmax",name='output'),
    ]
)

We’re going to use this arbitrary model that has two hidden one Convolutional and one Dense layer and an output layer with one node. First, we print its original weights that are initialized with zero.

Keras Model Weight

The list has 2 elements, of shape (input_dim, output_dim) and (output_dim,) for weights and biases respectively. Every layer has parameter weights that you can set with a NumPy array for example layer.set_weights(weights)(with the same shapes as the output of get_weights()).

Set weights for the Convolution Layer

You can pass a NumPy array to your convolutional layer through the set_weights method. Remember that the weights of a convolutional layer are not only the weights but also the bias(if use_bias is true). So if you want to set your weights you need to add an extra dimension for bias.

You can get the expected length of the list and its array shapes using the method get_weights() from the same instances above.

Keras layer output shape

Generate Random Weight

numpy.random.rand(shape) create an array of the given shape and populate it with random samples from a uniform distribution over [0, 1] Let’s create a (3,3,1,32).

Now you can set weights these ways:

model.layers[0].set_weights([weights,bias])

The set_weights() method of keras accepts a list of NumPy arrays. The shape of this should be the same as the shape of the output of get_weights() on the same layer. Here’s the code:

Keras get weight

If you call get_weights ()again after you set the weights, it returns the list of NumPy arrays we passed to set_weight()

Related Post