Manipulating the shape of a tensor is expected in the field of deep learning. In a neural network, you often require tensors of a particular shape.

Since the required shape of a tensor can change between neurons in a given neural network, it is good to have a low-level understanding of our inputs and outputs in deep learning. 

In this tutorial, we’ll explore the similarities and differences between these tensor.reshape() and tensor.view() functions, and when you should use each one to reshape your tensors.

What does tensor.reshape() do?

tensor.reshape() allows us to restructure a tensor so that we maintain the same data but organize it as a different number of rows and columns. PyTorch reshape method is used to change the shape (number of rows and columns) of a tensor without changing the element values.

import torch

#Create 4x3 tensor
tensor = torch.tensor([[1, 2, 3],
                       [4, 5, 6],
                       [7, 8, 9],
                      [10, 11, 12]])
PyTorch reshape
# Reshape tensor into 1x6 tensor
tensor.reshape(1, 6)

RuntimeError: shape ‘[1, 6]’ is invalid for input of size 12

The only requirement is that the shape of the original and new matrix contains the same number of elements (i.e., are the same size).

One useful argument in reshape is -1, which effectively means “as many as needed,” so reshape(1, -1) means one row and as many columns as needed.

What does tensor.view() do?

Python lists are collections of Python objects that are individually allocated in memory, as shown on the left. PyTorch tensors or NumPy arrays, on the other hand, are views over (typically) contiguous memory blocks containing unboxed C numeric types rather than Python objects.

PyTorch tensor Memory

Values in tensors are allocated in contiguous chunks of memory managed by torch.Storage instances. A PyTorch Tensor instance is a view of such a Storage instance that is capable of indexing into that storage using offset and per-dimension strides.

Some tensor operations in PyTorch only work on contiguous tensors, such as view. In that case, PyTorch will throw an informative exception and require us to call contiguous explicitly. It’s worth noting that calling contiguous will do nothing (and will not hurt performance) if the tensor is already contiguous. 

Calling view on a tensor returns a new tensor that changes the number of dimensions and the striding information, without changing the storage.

Pytorch view

view is really important, it changes the way the tensor looks at the same data contained in the storage.

view() reshapes the tensor without copying memory, similar to NumPy’s reshape().the new tensor returned by “view” shares the underlying data with the original tensor, so it is really a view into the old tensor instead of creating a brand new one.

Here you see PyTorch makes a tensor by converting an underlying block of contiguous memory into a matrix-like object by adding a shape and stride attribute.

Pytorch View storage
The right-hand tensor (shape (3,2)) can be computed from the left-hand one with t2 = t1.view(3,2).

This means we can rearrange our tensor at zero cost because no data will be copied. Our call to view requires us to provide the new shape for the returned tensor.

We can use -1 as a placeholder for “however many indexes are left, given the other dimensions and the original number of elements.” 

Conclusion

view() and reshape() are essentially the same except for how they work behind the scenes. The view() function requires the tensor data to be contiguous in memory, but reshape() allows discontiguous data storage. Reshape can be a view OR a copy, when possible.

Related Post