noun · AI setting
Top-p
also called nucleus sampling
Only let the AI choose from the best options. It throws away the long tail of unlikely guesses before picking.
A sampling setting that limits choices to the smallest group of top tokens whose probabilities add up to p.
The sampler sorts candidate tokens by probability and keeps them until their combined probability reaches p. The rest are dropped, the remaining probabilities are rescaled, and one token is drawn from that smaller set.
Real-life example
You set top-p to a low value for a customer reply bot. At each step the model considers only the few words that together cover most of the likelihood, and ignores the strange ones. The replies stay on track. Setting it to the maximum means nothing is cut.
Playground
Set a cut-off and the long tail of unlikely words disappears before the model picks. Slide it and draw words to see the difference.
Settings
My favorite pet is a …
Entropy of the choice: 2.03 bits. Each score is divided by the temperature, then turned into odds with softmax.
What comes after “My favorite pet is a”
The grey number is the raw score (logit) before temperature.
Press “Pick 100 times” to see how often each word wins.
Illustrative: made-up scores for one example sentence. A real model scores tens of thousands of possible tokens at every step.
Where the analogy breaks
-
Top-p is not a percentage of accuracy. A value of 0.9 means keep the most likely tokens covering 90 percent of the probability, not that 90 percent of the answer is right.
-
The number of tokens it keeps changes from step to step. When the model is sure it keeps very few, and when it is unsure it keeps many.
-
It overlaps with temperature. Some providers suggest adjusting one and leaving the other alone.