noun · AI concept
Logits
also called raw scores
The scoreboard before the winner is picked. For every possible next chunk of text, the AI first gives a score.
The unnormalized scores a model assigns to every token in its vocabulary before they are converted into probabilities.
The model's final layer outputs one number per token in the vocabulary. Those raw scores, called logits, are turned into probabilities by a softmax step, and only then does sampling pick the next token.
Real-life example
You ask a model to finish "The capital of France is". Internally it gives a high score to "Paris" and much lower scores to thousands of other tokens. Those scores are not odds yet. A conversion turns them into percentages, and temperature and sampling rules act on them before a token is chosen.
Where the analogy breaks
-
Logits are not probabilities. They can be negative or large, and only after conversion do they add up to one.
-
You often cannot see them. Many APIs expose log probabilities for some tokens instead of the full set of raw scores.
-
A high score is not a guarantee of truth. It means the token fits the text so far, not that the statement is correct.