Infrastructure & Compute
Inference compute
The processing power consumed when a trained model generates outputs.
Example
A serving team measures the compute needed to answer 10,000 requests each minute.
Why people use it
Understanding “Inference compute” helps teams plan the hardware and services needed to run AI.
What you'll hear
“What does Inference compute mean for the infrastructure we need?”