In PyTorch, element-wise matrix multiplication (the Hadamard product) is a binary operation that takes in two matrices of the same dimensions and returns a matrix of the multiplied corresponding elements.

Element Wise Matrix Multiplication
The element-wise matrix multiplication on identically shaped matrices produces and a third matrix of the same dimensions.

This operation can be thought of as a “naive matrix multiplication” and is different from the matrix product. The element-wise matrix multiplication is associative and distributive. Unlike the matrix product, it is also commutative.

For two matrices A and B of the same dimension m × n, the element-wise matrix multiplication A ⊙ B is a matrix of the same dimension as the operands, with elements given. For example, the element-wise matrix multiplication for two arbitrary 3 × 3 matrices is:

PyTorch Element Wise Matrix Multiplication

For matrices of different dimensions (m × n and p × q, where mp or q), the element-wise matrix multiplication is undefined. Given two tensors x and y, you can simply use x * y or torch.mul(x, y).

import torch

x = torch.tensor([[4,1,5],

y = torch.tensor([[2,3,7],

torch.mul(x, y)

x * y
tensor([[ 8,  3, 35],
        [20, 27,  7],
        [16, 28,  6]])

The @ operator performs matrix multiplication. If A and B are NumPy arrays, then A @ B is equivalent to np.matmul(A, B). Many other libraries, like TensorFlow, and PyTorch, support the @ operator as well. However, you cannot use @ on pure Python arrays (i.e., lists of lists). 

Related Post

PyTorch Matrix Multiplication from Scratch