Model Training & Adaptation
Preference data
Examples showing which of two or more model outputs are considered better.
Example
Reviewers choose the better of two summaries, creating preference data for later training.
Why it matters
It turns people's judgments into examples that can guide later improvements.
You'll hear it
“Which answer is better for this reader, and why?”
Can you control it?
Set by the team
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Can both compared answers be poor?
- Yes. The preferred one may simply be less bad, rather than genuinely good.
- Can the order of options influence judgments?
- It can. Presentation choices may affect which answer people favor.
- Should disagreement always be removed?
- No. Disagreement can reveal unclear instructions or legitimate differences in what users need.