AI Safety & Responsible AI
Catastrophic AI risk
The possibility of AI contributing to extremely severe harm affecting large numbers of people.
Example
A risk analysis considers failure scenarios affecting critical services across many organizations.
Why people use it
It helps discuss rare but extremely severe possibilities instead of considering only routine mistakes.
What you'll hear
“How large could the harm become?”
What this means for you
Separate severity, likelihood and the evidence supporting each scenario.
Can you control it?
No
No direct control. This describes a wider issue, concept or result rather than something you can simply switch on or off in a tool.
Common questions
- Is catastrophic risk identical to existential risk?
- No. A catastrophe can be enormous without permanently eliminating humanity's future potential.
- Does a severe possible outcome mean it is likely?
- No. The size of a possible harm and its likelihood are separate questions.
- Can several systems contribute to the same catastrophe?
- Yes. Interactions or shared dependencies can matter, not just one system acting alone.