Pre-training is the initial, large-scale training phase where a model learns general knowledge from a broad corpus before any task-specific adaptation. For language models, this typically involves next-token prediction on trillions of tokens from the internet; for vision models, it may involve contrastive learning on image-text pairs. Pre-training is by far the most expensive phase in terms of compute and data, but the resulting foundation model can then be fine-tuned or prompted for countless downstream tasks at much lower cost.