Dropout is a regularization technique that randomly zeros a fraction of activations during training, forcing the network to distribute knowledge across neurons rather than relying on any single one. At inference time dropout is disabled and outputs are scaled accordingly. Interestingly, many modern large language models train with zero dropout, relying on data scale for regularization instead.