Model Serving & Economics
Compute efficiency
How much useful model performance is achieved for a given amount of computing resources.
Example
Quantization lets the same hardware answer more requests with less energy.
Why it matters
It can lower operating costs and make AI practical on less powerful equipment.
You'll hear it
“Are we saving power without making the answers worse?”
Can you control it?
Set by the team
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Is a faster model always more efficient?
- No. It may be using more powerful equipment to finish sooner.
- Can efficiency improvements change answers?
- Yes. Some shortcuts reduce detail or accuracy, so the revised setup needs testing.
- Can this matter on a phone?
- Yes. More efficient processing can help with battery use, heat and the device's limited memory.