Top-k Sampling

What is Top-k Sampling?

Top-k sampling limits the model's choices to the 'k' most probable next tokens, cutting off the long tail of low-probability words to improve coherence.

Where did the term "Top-k Sampling" come from?

Standard technique to prevent gibberish generation.

How is "Top-k Sampling" used today?

Default setting in many LLM APIs.

Related Terms