Neural networks require data to be represented as multidimensional numerical tensors, often 32-bit floating-point.  PyTorch expects data to be laid out along specific dimensions according to the model architecture—for example, convolutional versus recurrent. 

A common type of tensor operation that’s essential to understand is tensor reshaping. We can reshape data effectively with the PyTorch tensor API. 

Reshaping a tensor means rearranging its rows and columns to match a target shape. Naturally, the reshaped tensor has the same total number of coefficients as the initial tensor.

We’ll cover many aspects of tensors—such as creating views of tensors; and indexing tensors with other tensors. There are multiple ways of reshaping a PyTorch tensor. You can apply one of these methods on a tensor of any dimensionality. Reshaping is best understood via simple examples.

1. torch.reshape(input, shape)

torch.reshape() returns a tensor with the same data and number of elements as input, but with the specified shape. When possible, the returned tensor will be a view of input. Otherwise, it will be a copy.

inputs = torch.tensor([[4.0, 1.0],
                       [5.0, 3.0], 
                       [2.0, 1.0]])

x = inputs.reshape(1,2,3)
print(x.shape) #torch.Size([1, 2, 3])

see also:

PyTorch Difference Between View and Reshape.

2: tensor.view(shape) 

tensor.view() changes how the tensor looks at the same data contained in storage. 

import torch

inputs = torch.tensor([[4.0, 1.0],
                       [5.0, 3.0], 
                       [2.0, 1.0]])

print(inputs.shape) #torch.Size([3, 2])

inputs = inputs.view(2,3)

print(inputs.shape) #torch.Size([2, 3])

As you learned in the previous post, calling view on a tensor returns a new tensor that changes the number of dimensions and the striding information, without changing the storage. This means we can rearrange our tensor at zero cost because no data will be copied.

Our call to view requires us to provide the new shape for the returned tensor. We use -1 as a placeholder for “however many indexes are left, given the other dimensions and the original number of elements.” 

see also:

What does tensor.view() do in PyTorch?

What does view(-1) in PyTorch?

Difference Between Contiguous and Non-Contiguous Array

3. tensor.resize_(*sizes)

tensor.resize_() resizes the self-tensor to the specified size. If the number of elements is larger than the current storage size, then the underlying storage is resized to fit the new number of elements.

inputs = torch.tensor([[4.0, 1.0],
                       [5.0, 3.0], 
                       [2.0, 1.0]])

print(inputs.resize_(3,3))

#tensor([[4.0000e+00, 1.0000e+00, 5.0000e+00],
        [3.0000e+00, 2.0000e+00, 1.0000e+00],
        [1.1210e-43, 0.0000e+00, 2.6053e-33]])

If the number of elements is smaller, the underlying storage is not changed. Existing elements are preserved but any new memory is uninitialized.

inputs = torch.tensor([[4.0, 1.0],
                       [5.0, 3.0], 
                       [2.0, 1.0]])

print(inputs.resize_(2,2))

#tensor([[4., 1.],
        #[5., 3.]])

4. tensor.unsqueeze()

torch.unsqueeze(tensor, i) adds a new dimension at the i’th dimension. The returned tensor shares the same data as the original tensor. In this example, we can use unqueeze() to add the new dimensions.

y = inputs.unsqueeze(0)
print(y.shape) #torch.Size([1, 3, 3])

see also:

What does Unsqueeze do in PyTorch?

PyTorch Contiguous Tensor

PyTorch changes image channel order between channel first and channel last

5. Transposing Tensor

Transpose tensors share the same storage and they differ only in shape and stride. We can use the t function, a shorthand alternative to transpose for two-dimensional tensors: 

inputs = torch.tensor([[4.0, 1.0],
                       [5.0, 3.0], 
                       [2.0, 1.0]])

inputs.t()

#tensor([[4., 5., 2.],
        [1., 3., 1.]])

No new memory is allocated: trans-posing is obtained only by creating a new Tensor instance with a different stride ordering than the original. 

We can transpose a multidimensional array by specifying the two dimensions along which transposing (flipping shape and stride) should occur: 

t = torch.ones(3, 4, 5)

transpose_t = t.transpose(0, 2)
print(t.shape) #torch.Size([3, 4, 5])
print(transpose_t.shape) #torch.Size([5, 4, 3])

6. Add dimension with None

You can use NumPy-style None dimension insertion to add dimensions anywhere you want.

import torch

inputs = torch.tensor([[4.0, 1.0],
                       [5.0, 3.0], 
                       [2.0, 1.0]])
x = inputs[None, :, None, :]

print(inputs.shape) #torch.Size([3, 2])
print(x.shape) #torch.Size([1, 3, 1, 2])

Conclusion

tensor.resize_() is a low-level method. The storage is reinterpreted as C-contiguous. it is ignoring the current strides unless the target size equals the current size, in which case the tensor is left unchanged. 

For most purposes, you will instead want to use view(), which checks for contiguity, or reshape(), which copies data if needed.