Larger batches can improve hardware utilization and reduce gradient noise but require more memory. Smaller batches produce noisier updates that may affect optimization and generalization.
Batch size is the number of training examples used to estimate a gradient before a parameter update.
Larger batches can improve hardware utilization and reduce gradient noise but require more memory. Smaller batches produce noisier updates that may affect optimization and generalization.