Model Training & Adaptation
Distillation
Training a smaller model to imitate the behavior of a larger or stronger model.
Example
A smaller model learns to imitate the useful behavior of a larger model.
Why it matters
It can reduce the computing cost of deploying a useful ability.
You'll hear it
“The smaller version is faster, but check the difficult cases.”
Can you control it?
Set by the team
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Does the smaller system copy every ability?
- No. Some skills or details may not transfer reliably.
- Can it inherit errors from the larger system?
- Yes. Learning from mistaken examples can reproduce the teacher's weaknesses.
- Does a smaller model always run faster?
- Not necessarily. Equipment, software and the task also affect practical speed.