Convolutional neural networks constitute a very useful tool for machine learning practitioners. A convolutional layer’s output shape is affected by the shape of its input as well as the choice of kernel shape, zero padding, and strides.
The bread and butter of neural networks are affine transformations: a vector is received as input and is multiplied with a matrix to produce an output. The bias vector is added before passing the result through a non-linearity.
A convolution layer is a linear transformation that preserves the notion of ordering. It is sparse, only a few input units contribute to a given output unit and reuse parameters (the same weights are applied to multiple locations in the input).
The convolution layer uses filters that perform convolution operations as it is scanning the input with respect to its dimensions. The resulting output is called a feature map or activation map.
How to calculate a feature map?
A feature map, or activation map, is the output activations for a given filter (a1 in your case) and the definition is the same regardless of what layer you are on.
The light blue grid is the input feature map or input image. To keep the simple, a single input feature map is represented, but it is common to have multiple feature maps stacked one onto another. A kernel (red border) of value slides across the input feature map.
At each location, the product between each element of the kernel and the input element it overlaps is computed and the results are summed up to obtain the output in the current location. The procedure can be repeated using different kernels to form as many output feature maps as desired.
The final outputs of this procedure are called output feature maps. If there are multiple input feature maps, the kernel will have to be 3-dimensional or, equivalently each one of the feature maps will be convolved with a distinct kernel. The resulting feature maps will be summed up elementwise to produce the output feature map.
A convolution mapping from two input feature maps to three output feature maps using a 3 × 2 × 3 × 3 collection of kernels w. In the left pathway, input feature map 1 is convolved with kernel w1,1, and input feature map 2 is convolved with kernel w1,2, and the results are summed together element-wise to form the first output feature map.
Difference between Feature Map and Activation Map
Feature map and activation map are exactly the same thing. It is called an activation map because it is a mapping that corresponds to the activation of different parts of the image, and also a feature map because it is also a mapping of where a certain kind of feature is found in the image.
Related Post
- Feature extraction from an image using pre-trained PyTorch model
- Extract Intermediate Layer Output from PyTorch CNN model.
- PyTorch Freeze Layer for fixed feature extractor in Transfer Learning
- How to modify pre-train PyTorch model for Finetuning and Feature Extraction?
- Visualize Feature Maps of PyTorch Convolutional Neural Networks
- How to remove last layer from PyTorch Pre-train model?