The model's logits are divided by the temperature before probabilities are computed. Lower values concentrate probability on likely tokens, while higher values spread probability more broadly and usually increase variation.
Temperature is a decoding parameter that controls how sharply a model favors high-probability tokens.
The model's logits are divided by the temperature before probabilities are computed. Lower values concentrate probability on likely tokens, while higher values spread probability more broadly and usually increase variation.