Infrastructure & Compute
Prefill
The stage when a language AI reads and processes your request before building its reply.
Example
AI takes time to read a long document before starting its answer.
Why people use it
It helps explain why a long request can take time before an answer starts.
What you'll hear
“It is still working through the material we sent.”
What this means for you
Consider the amount of material supplied when investigating a long initial wait.
Can you control it?
Developer-only
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Does prefill include generating the whole answer?
- No. Result generation happens in the decode stage.
- Can adding documents increase this stage's work?
- Yes. More material generally means more information to process before writing begins.
- Can repeated material sometimes be processed faster?
- Some services reuse earlier calculations, depending on their design and available features.