Generative AI & LLMs
Sampling
The process of choosing the next generated token from the model's probability distribution.
Example
A writing tool samples among likely next tokens instead of always choosing the single most likely one.
Why it matters
It adds variety so a tool can explore different plausible continuations.
You'll hear it
“Run it again so we can compare another version.”
Can you control it?
Set by the team
The people building or running the AI choose this setup. An everyday user generally needs their help to change how this part works.
Common questions
- Can a less likely choice still make sense?
- Yes. Several possible continuations can be reasonable, even when their scores differ.
- Does randomness mean all choices are equally likely?
- No. The system usually favors options according to scores and the chosen settings.
- Can sampling make an answer less reliable?
- It can introduce unsuitable choices, but removing randomness does not guarantee truth either.