The Sequential class allows us to build neural networks on the fly without having to define an explicit class. This makes it much easier for us to rapidly build neural networks and skip over the part where we have to implement their forward() function this is because the sequential class implements the forward() function for us. In this tutorial, we’re gonna learn “How to use PyTorch Sequential class to build ConvNet”.

The First thing we’re gonna do is to start with our imports. We’re gonna import all the stuff that we need.

import torch
import torchvision
import torchvision.transforms as transforms

import numpy as np
import matplotlib.pyplot as plt

Dataset

The next thing we’re gonna do is create a dataset to train because we are not only constructing a sequential model but we also train a sequential model.

For this tutorial, we will use the CIFAR10 dataset. It has 10 classes. The images in CIFAR-10 are of size 3x32x32, i.e. 3-channel color images of 32×32 pixels in size.

Pytorch has created a package called torchvision, that has data loaders for common datasets such as Imagenet, CIFAR10, MNIST, 

The output of torchvision datasets are PIL images of range [0, 1]. We transform them to Tensors of normalized range [-1, 1].

transform=transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5,0.5,0.5),(0.5,0.5,0.5))])

Using torchvision, it’s extremely easy to load CIFAR10.

train_ds=torchvision.datasets.CIFAR10(root='./data',train=True,download=True,transform=transform)

train_ds_loader=torch.utils.data.DataLoader(train_ds,batch_size=32,shuffle=True,num_workers=2)

test_ds=torchvision.datasets.CIFAR10(root='./data',train=False,download=True,transform=transform)
test_ds_loader=torch.utils.data.DataLoader(test_ds,batch_size=32,shuffle=False,num_workers=2)

From the trainset, we’re gonna access the first element which is going to be a tuple that contains our image tensor and our labels tensor. We’ll inspect the shape of the image which is 3x32x32 the height and width are 32 and this is a color image.

image,label=train_ds[0]
image.shape #torch.Size([3, 32, 32])

Let just use the pyplot to show this image. The only thing to do here is that normalize this particular image. So that was in order to allow the image to be plotted.

def imshow(img):
  img=img/2+0.5
  npimg=img.numpy()
  plt.imshow(np.transpose(npimg,(1,2,0)))
  plt.show()

dataiter=iter(train_ds_loader)
images,labels=dataiter.next()

imshow(torchvision.utils.make_grid(images))

Now ready to take this information and use it to build our first sequential model.

Create a Model

At this point, you may be wondering about operations like activation functions and pooling operations, and flattening operations. Most of the operations are in the neural network functional API. The cool thing is that Pytorch has wrapped inside a neural network module itself.

Sequential class constructs the forward method implicitly by sequentially building network architecture. PyTorch sequential model is a container class or also known as a wrapper class that allows us to compose the neural network models. We can compose any neural network model together using the Sequential model this means that we compose layers to make networks and we can even compose multiple networks together.

import torch.nn as nn
import torch.nn.functional as F

torch.nn.functional as F allows us to create sequential models and by having the ability to define our layers, our activation functions, our flatten operations are pulling operations. The main takeaway here is that having all of these functions wrapped up as neural network models allow us to use the sequential class to wrap multiple modules and compose them together. We can compose convolutions with maxpooling operations with ReLU activation functions with flatten operations and what this does is it allows us to build our models in a sequential.

model =nn.Sequential(
    nn.Conv2d(3,32,kernel_size=3,padding=1),
    nn.ReLU(),
    nn.Conv2d(32,64,kernel_size=3,stride=1,padding=1),
    nn.ReLU(),
    nn.MaxPool2d(2,2),

    nn.Conv2d(64,128,kernel_size=3,stride=1,padding=1),
    nn.ReLU(),
    nn.Conv2d(128,128,kernel_size=3,stride=1,padding=1),
    nn.ReLU(),
    nn.MaxPool2d(2,2),

    nn.Conv2d(128,256,kernel_size=3,stride=1,padding=1),
    nn.ReLU(),
    nn.Conv2d(256,256,kernel_size=3,stride=1,padding=1),
    nn.ReLU(),
    nn.MaxPool2d(2,2),

    nn.Flatten(),
    nn.Linear(256*4*4,1024),
    nn.ReLU(),
    nn.Linear(1024,512),
    nn.ReLU(),
    nn.Linear(512,10))

How easy this looks compared to when we had to construct a model through the class way of doing it. Sequential class lives in the neural network package and this is a class that we are building by or we’re building an instance of this class by passing in other modules in a sequential.

Loss function and optimizer

Let’s use a Classification Cross-Entropy loss and SGD with momentum.

import torch.optim as optim

loss_fn=nn.CrossEntropyLoss()
optimizer=optim.SGD(model.parameters(),lr=0.001,momentum=0.9)

Train the model

We simply have to loop over our data iterator, and feed the inputs to the network and optimize.

train_losses=[]
valid_losses=[]

for epoch in range(1,num_epoch+1):
  train_loss=0.0
  valid_loss=0.0

  model.train()
  for img,lbl in train_ds_loader:
    img=img.cuda()
    lbl=lbl.cuda()

    optimizer.zero_grad()
    predict=model(img)
    loss=loss_fn(predict,lbl)
    loss.backward()
    optimizer.step()
    train_loss+=loss.item()*img.size(0)

  model.eval()
  for img,lbl in test_ds_loader:
    img=img.cuda()
    lbl=lbl.cuda()

    predict=model(img)
    loss=loss_fn(predict,lbl)

    valid_loss+=loss.item()*img.size(0)

  train_loss=train_loss/len(train_ds_loader.sampler) 
  valid_loss=valid_loss/len(test_ds_loader.sampler)

  train_losses.append(train_loss)
  valid_losses.append(valid_loss)

  print('Epoch:{} Train Loss:{:.4f} valid Losss:{:.4f}'.format(epoch,train_loss,valid_loss))

In this case, we’re not going to evaluate or predict the train network because we’re just interested in seeing how to create a sequential model and train it and access the class and then create an instance of the class by passing in sequentially any number of neural network modules.

Related Post

Run this code in Google Colab.