AI Foundations
Exploding gradient
An AI training problem where learning adjustments become too large and make learning unstable.
Example
Training becomes unstable after unusually large learning changes.
Why people use it
It names a training failure where changes become too large to stay stable.
What you'll hear
“The learning updates suddenly became enormous.”
What this means for you
Ask the people running training to investigate the unstable calculations before continuing.
Can you control it?
No
No direct control. This describes a wider issue, concept or result rather than something you can simply switch on or off in a tool.
Common questions
- Does this mean the computer is physically overheating?
- No. It describes unstable learning calculations, not the machine's temperature.
- Can it make training stop completely?
- Yes. Very large calculations can produce unusable numbers or prevent the process from continuing.
- Does it show that the teaching examples are wrong?
- Not necessarily. The learning setup can be unstable even when the examples are appropriate.