Evaluation & Quality
Human evaluation
Assessment where people judge model outputs for quality, usefulness, or correctness.
Example
People score responses for correctness, usefulness, tone, and safety.
Why it matters
It brings practical judgment to qualities that a simple automatic score may miss.
You'll hear it
“The reviewers agreed on accuracy but disagreed about tone.”
Can you control it?
Set by the team
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Do human reviewers always agree?
- No. Experience, expectations and interpretation can lead to different judgments.
- Can knowing the AI's name influence a review?
- Yes. Reputation can affect expectations, so concealed labels may help some comparisons.
- Does using people eliminate evaluation mistakes?
- No. Reviewers can miss facts, become tired or apply criteria inconsistently.