Warm-up is gradually increasing the learning rate from near-zero at the start of training to prevent large, destabilizing gradient updates before the optimizer has seen enough data. This technique is especially important for large models and transformer architectures, where premature large updates can push the model into poor regions of the loss landscape.