Infrastructure & Compute
VRAM
Video random-access memory
High-speed memory on a GPU used to store model data and intermediate calculations.
Example
A model must fit within the GPU's available VRAM during inference.
Why it matters
It keeps the information needed for fast GPU calculations close to the processor.
You'll hear it
“The model fits, but the longer request runs out of GPU memory.”
Can you control it?
Set by the team
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Does more VRAM automatically make every calculation faster?
- No. Capacity prevents some limits but does not determine all processing speed.
- Can response length affect memory needs?
- Yes. Some systems retain additional working information while generating longer replies.
- Can a model run when it does not fully fit?
- Some setups move parts elsewhere, but that can introduce delay and complexity.