Infrastructure & Compute
Quantization
Reducing the numerical precision used by a model to make it smaller or faster.
Example
Quantization stores model values with fewer bits so the model uses less memory.
Why it matters
It can make a model fit smaller memory budgets or run more efficiently.
You'll hear it
“The smaller version fits, but check whether its answers changed.”
Can you control it?
Set by the team
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Does quantization always preserve identical behavior?
- No. Rounding or reduced detail can change results.
- Is lower precision always faster in practice?
- No. Hardware and software need suitable support.
- Can some tasks be affected more than others?
- Yes. Sensitivity to small numerical changes can differ across tasks.