Keras Image Augmentation using TensorFlow tf.image

The primary thing with CNN model is data which plays an important role during training. The data has to good diversity. The object of interest needs to be present in varying sizes, lighting conditions and poses if we desire that our CNN model generalizes well during the testing phase. To overcome limited quantity and limited diversity of data, we generate our own data with the existing data which we have. This generating of data is called data augmentation. It is a cheap way to improve the quality of your model.

The tf.image provides image augmentation functions that all the computation is done on GPU. In this tutorial, we use TensorFlow eager_execution so that we can see the augment Image directly.

1.Resize Image

Images gathered from the internet will be of different sizes. The images being fed to CNN model will be required of a fixed size. Let first preprocess the images to the resize which CNN needs.

2.Crop Image

tf.image provide a various cropping function for crop image.tf.image.central_crop remove the outer parts of an image but retain the central region of the image along each dimension.

3.Crop and Resize

Extracts crops from the input image at positions defined at the bounding box locations in boxes and resizes to a common output size specified by crop_size.

4.Crop to Bounding Box

Extracts a fixed size slice from the input image and does not allow resizing or aspect ratio change.

5.Flipping

Flipping is important for CNN to remove certain features of the object is available in only a particular side. For example, you don’t want CNN model to learn that apple leaf happens only on the right side as observed in the base image. Flipping produces a different set of images from the rotation at multiple of 90 degrees.

1.random_flip_left_right() for Randomly flip an image horizontally (left to right).2.random_flip_up_down() for randomly flips an image vertically (upside down). 3.flip_up_down() for flip an image vertically (upside down).

6.Rotate Image

Rotate image counterclockwise by the passed angle in radians.

7.Translation

We would like our model to recognize the object present in any part of the image. Also, the object can be present partially in the corner or edges of the image. For this reason, we shift the object to various parts of the image.

8.Transform

Applies the given transform to the image.

9.Adding Salt and Pepper noise

Over-fitting happens when your neural network tries to learn high- frequency features that may not be useful. Gaussian noise effectively distorting the high-frequency features.

A toned down version of this is the salt and pepper noise, which presents itself as random black and white pixels spread through the image.

10.Convert RGB to Grayscale

Following function Converts one or more images from RGB to Grayscale.

11.Change brightness

Following function adjust the brightness of RGB or Grayscale images.

12.Adjust Contrast

Following function adjust contrast of RGB or grayscale images.

13.Adjust Hue

Following function adjust hue of an RGB image.

The image hue is adjusted by converting the image to HSV and rotating the hue channel (H) by delta. The image is then converted back to RGB. The delta must be in the interval [-1, 1].

14. Adjust Saturation

The image saturation is adjusted by converting the image to HSV and multiplying the saturation (S) channel by saturation_factor and clipping. The image is then converted back to RGB.

15.Upload and Display image in Colab

In this tutorial, we use google colab and eager execution for image augmentation.

def upload_files():
  from google.colab import files
  uploaded = files.upload()
  for k, v in uploaded.items():
    open(k, 'wb').write(v)
  return list(uploaded.keys())
upload_files()

import tensorflow as tf
import matplotlib.image as mpimg
import matplotlib.pyplot as plt

tf.enable_eager_execution()


image_path = 'apple_256.jpg'

image_string=tf.read_file(image_path)

image=tf.image.decode_jpeg(image_string,channels=3)
# image=tf.image.convert_image_dtype(image,dtype=tf.float32)


def show_image(original_image,augmented_image,title):
  fig=plt.figure()
  fig.suptitle(title)
  
  original_plt=fig.add_subplot(1,2,1)
  
  original_plt.set_title('original image')
  original_plt.imshow(original_image)

  augmented_plt=fig.add_subplot(1,2,2) 
  augmented_plt.set_title('augmented image')
  augmented_plt.imshow(augmented_image)
  plt.show(block=True)