December 14, 2019
What is batch size, steps, iteration, and epoch in the neural network?
adminWhen you put m examples in a mini-batch, you need to do O(m) computation and use O(m) memory, and you reduce the amount of uncertainty in the gradient by a factor of only O(sqrt(m)).