Skip to content
LATHICAI Glossary

Generative AI & LLMs

Top-k sampling

Choosing the next piece of AI-written text from a set number of the highest-scoring options.

Example

A generator allows selection from its top 40 candidate tokens.

Why people use it

It limits how many possible next pieces the AI may choose from.

What you'll hear

“Allow only the top few next-word choices.”

What this means for you

Tune the setting against actual quality and consistency needs.

Can you control it?

Sometimes

Sometimes. Your choices depend on the tool and your access. The settings available to an everyday user may differ from those available to the people running it.

Common questions

Does a larger k guarantee better writing?
No. It changes the range of choices and may increase variability.
What happens if k is one?
The system chooses only the highest-scoring option at each step, rather than sampling among several.
Does the same k mean the same amount of variety every time?
No. The likelihoods of the available choices can differ from one step to another.

Related terms

Still have questions?

Up to 500 characters.

Ask LATHIC about AI. Relevant glossary entries may be included.

Your question, the glossary entries it matches, and a rotating pseudonymous identifier go to Microsoft Azure’s OpenAI service through Vercel AI Gateway to generate an answer. Zero retention and no training are required of the provider, and LATHIC does not save your question or answer. Privacy Notice