Developers select it through prior knowledge or validation experiments rather than direct gradient updates on the training objective. Tuning on held-out data helps compare choices without using the final test set.
A hyperparameter is a configuration chosen outside ordinary parameter learning, such as a learning rate, depth, or regularization strength.
Developers select it through prior knowledge or validation experiments rather than direct gradient updates on the training objective. Tuning on held-out data helps compare choices without using the final test set.