AI Safety & Responsible AI
Adversarial testing
Testing a system with deliberately difficult or manipulative inputs.
Example
Evaluators test prompts designed to exploit known weaknesses rather than ordinary use.
Why it matters
It helps reveal fragile behavior before people depend on it.
You'll hear it
“This slightly altered request caused a very different decision.”
Can you control it?
Sometimes
Sometimes. Some products or workflows let you influence adversarial testing, while others keep it fixed or manage it automatically. Check the settings and documentation for the tool you are using.
Common questions
- Must the test involve a malicious person?
- No. Deliberately difficult cases can also model accidental misuse or unusual conditions.
- Does one failed test show every use is unsafe?
- No. The result needs interpretation in terms of the tested conditions and consequences.
- Can defenses that help one test fail elsewhere?
- Yes. A narrow fix may not address the wider weakness.