Model Serving & Economics
Compute efficiency
How much useful model performance is achieved for a given amount of computing resources.
Example
Quantization lets the same hardware answer more requests with less energy.
Why people use it
Understanding “Compute efficiency” helps teams balance model quality, reliability, speed, and cost.
What you'll hear
“How does Compute efficiency affect latency, quality, or cost at scale?”