AI Foundations
Vanishing gradient
An AI training problem where learning signals become too small to help some parts improve.
Example
Early layers in a deep network receive very weak update signals.
Why people use it
It explains why some parts of a network may learn extremely slowly.
What you'll hear
“The early layers are barely changing.”
What this means for you
Use training diagnostics to distinguish learning instability from inadequate data.
Can you control it?
No
No direct control. This describes a wider issue, concept or result rather than something you can simply switch on or off in a tool.
Common questions
- Does a vanishing gradient mean the AI system has no learned settings?
- No. It concerns the strength of learning updates, not AI system size.
- Can later parts still learn while earlier parts struggle?
- Yes. The strength of the learning signal can differ across the network.
- Does running longer always fix it?
- No. The design or learning method may need to change for useful signals to reach those parts.