Security & Privacy
Evasion attack
Changing something an AI receives during use to trick it into making a mistake.
Example
Researchers alter a test image so an AI puts it in the wrong group.
Why people use it
It examines attempts to trick AI through the information given during use.
What you'll hear
“Could a changed image cause the detector to miss this?”
What this means for you
Test protections against changes an attacker could realistically make.
Can you control it?
No
No direct control. This describes a wider issue, concept or result rather than something you can simply switch on or off in a tool.
Common questions
- Does this attack change the original teaching material?
- Not necessarily. It can target what the finished system receives while leaving its earlier learning untouched.
- Does a successful attack have to look obvious to a person?
- No. Some changes can be hard to notice while still affecting the AI.
- Can a defense work in one setting and fail in another?
- Yes. Camera conditions, access and the attack method can change the result.