APIs & Integration
Latency
The time between making a request and receiving a response or result.
Example
The team measures 800 milliseconds from sending a prompt to receiving the first output.
Why it matters
It helps teams judge whether a tool responds quickly enough for its intended use.
You'll hear it
“The answer is useful, but the wait interrupts the conversation.”
Can you control it?
Not directly
No, not directly. Latency describes a learned property, observed result, or broader concept rather than a normal user setting. You may influence it indirectly by changing the tool, data, prompt, or workflow.
Common questions
- Is latency the same as total capacity?
- No. A system can handle many requests while each individual request still takes time.
- Can a quick first word hide a long wait for completion?
- Yes. The reply may start quickly and then take much longer to finish.
- Can latency vary between similar requests?
- Yes. Demand, request size and service conditions can affect timing.