Data & Databases
Evaluation data contamination
Test material leaking into AI learning or development, making the test less independent.
Example
An AI test question also appeared in its teaching material.
Why people use it
It helps explain when a test is no longer a fair check of unfamiliar work.
What you'll hear
“Had the system already seen these test questions?”
What this means for you
Use fresh or protected test examples and check for overlap.
Can you control it?
No
No direct control. This describes a wider issue, concept or result rather than something you can simply switch on or off in a tool.
Common questions
- Does a high benchmark score then measure truly unseen performance?
- It may not. Prior exposure can inflate the score.
- Can small variations of a test question still matter?
- Yes. Closely related examples can provide clues even without an exact duplicate.
- Does public availability create a possible concern?
- Yes. Public test material may later appear in collections used for training or development.