Training & Fine-Tuningaka Kahneman-Tversky Optimization

KTO

A preference learning method needing only binary good/bad labels per output rather than paired comparisons.

Kahneman-Tversky Optimization (KTO) is a preference learning method that only needs binary good/bad labels per output rather than paired comparisons, making data collection simpler than DPO. Inspired by prospect theory from behavioral economics, KTO models the asymmetry between gains and losses in human preference judgments.

Related terms

Direct Preference Optimization Preference Learning Reinforcement Learning from Human Feedback Reward Model Mixed Precision Training Self-Scaffolding RL

Training & Fine-Tuningaka Kahneman-Tversky Optimization

KTO

A preference learning method needing only binary good/bad labels per output rather than paired comparisons.

Ask Melo about this ← all terms

Related terms

Direct Preference Optimization Preference Learning Reinforcement Learning from Human Feedback Reward Model Mixed Precision Training Self-Scaffolding RL