Weight decay is a regularization technique that shrinks parameter values toward zero each update, discouraging the model from relying too heavily on any single weight and reducing overfitting. It acts as a soft constraint on model complexity and is a standard component of modern optimizers like AdamW.