Evaluation & Quality
Test set
Data held out from training and used to measure how well a model generalizes.
Example
The final test set remains untouched until the team is ready to estimate real performance.
Why it matters
It gives a more independent estimate of performance after development choices are finished.
You'll hear it
“Don't use those answers to choose the training settings.”
Can you control it?
Set by the team
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Can repeated peeking weaken the test?
- Yes. Adjusting the system in response effectively makes those examples part of development.
- Does a separate file guarantee a clean separation?
- No. Duplicates or closely related examples can still appear in both collections.
- Can a test set become outdated?
- Yes. Real users and conditions may change after it was assembled.