Model Training & Adaptation
Reward model
A model trained to score outputs according to human or system preferences.
Example
A reward model learns to predict which response a human reviewer would prefer.
Why it matters
It provides repeatable feedback when having a person judge every candidate would be impractical.
You'll hear it
“That answer sounds polished, but why did the scorer prefer it?”
Can you control it?
Set by the team
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Can the scoring system be fooled?
- Yes. A reply may exploit favored patterns without being more helpful or accurate.
- Does one score capture every quality?
- No. A single number can hide differences in truth, clarity, safety or usefulness.
- Can its judgments become less useful after changes?
- Yes. New tasks or response styles can differ from the examples it learned to judge.