Keras model can be saved during and after training. Using a saved model you can resume training where it left off and avoid long training times or you can share the model so others can recreate your work.

Sometimes, you need only model weights and not the entire model. In this case, you can retrieve the values of the weights as a list of Numpy arrays via save_weights(), and set the state of the model via load_weights.

In this tutorial, we will learn how to save and load weight in Keras. This tutorial uses tf.keras, a high-level API to build and train models in TensorFlow 2.0.

To demonstrate save and load weights, you’ll use the CIFAR10. To speed up these runs, use the first 2000 examples

(x_train, y_train), (x_val, y_val) = tf.keras.datasets.cifar10.load_data()

x_train = x_train.astype('float32')
x_val = x_val.astype('float32')

x_train /= 255
x_val /= 255

IMG_SIZE=32

BATCH_SIZE=32

Create a Model

First building a simple sequential model:

def create_model():
  model = tf.keras.Sequential([
      tf.keras.layers.Conv2D(kernel_size=3, filters=32, padding='same', activation='relu', input_shape=[IMG_SIZE,IMG_SIZE, 3]),
      
      tf.keras.layers.Conv2D(kernel_size=3, filters=64, padding='same', activation='relu'),
      tf.keras.layers.MaxPooling2D(pool_size=2),
    
      tf.keras.layers.Conv2D(kernel_size=3, filters=128, padding='same', activation='relu'),
      tf.keras.layers.Conv2D(kernel_size=1, filters=256, padding='same', activation='relu'),
      tf.keras.layers.GlobalAveragePooling2D(),
    
      tf.keras.layers.Dense(10,'softmax')])

  model.compile(optimizer=tf.keras.optimizers.RMSprop(),
              loss=tf.keras.losses.SparseCategoricalCrossentropy(),
              metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])
  return model

Let’s train this model, just so it has weight values to save, as well as an optimizer state.

Save model weights at the end of epochs

If you like to save the model weights at the end epochs then you need to create tf.keras.callbacks.ModelCheckpoint callback. It allows us to continually save weight both at the end of epochs. You can also adjust the frequency of the weight using period arguments.

model=create_model()

model_checkpoint=tf.keras.callbacks.ModelCheckpoint('CIFAR10{epoch:02d}.h5',period=3,save_weights_only=True)

history = model.fit(x_train[:2000], y_train[:2000],
              batch_size=BATCH_SIZE,
              epochs=6,
              callbacks=[model_checkpoint],
              validation_data=(x_val[:100], y_val[:100]),
              shuffle=True)

This code creates weight files at every 3 epochs. The weight stores as the collection of HDF5 files that contain only the trained weights in a binary format.

Manually saving weight is just as simple with the Model.save_weights method.

model.save_weights(filepath='final_weight.h5')

Load weight into the model

When restoring a model from weights-only, you must have a model with the same architecture as the original model. So First Create a new, untrained model and evaluate it on the test set.

new_model=create_model()

new_model.evaluate(x_val,y_val)

Since it’s the same model architecture, you can share weights despite that it’s a different instance of the model. Let’s load the weights from the .h5 file and re-evaluate.

new_model.load_weights('CIFAR1006.h5')
new_model.evaluate(x_val,y_val)

Load model from .h5 weight file

save_model=tf.keras.models.load_model('CIFAR1006.h5')

ValueError: No model found in config file.

You can’t load a model from weights only. In this case, you can’t use load_model method. You have to set and define the architecture of your model and then use model.load_weights('CIFAR1006.h5'). Take a look at this for example for Load mode from hdf5 file in keras.

Related Post

Run this code in Google colab