noun · AI training
RLHF
also called reinforcement learning from human feedback
Teaching by thumbs up. People pick which answers they prefer, and the AI is nudged toward writing more like those.
Reinforcement learning from human feedback: a training stage where human preference rankings are used to tune a model toward preferred outputs.
Human annotators rank pairs of model outputs, and those rankings train a reward model that scores answers. The model's makers then tune the language model to produce outputs the reward model rates highly, which shapes tone, helpfulness and refusals.
Real-life example
You use an assistant that answers politely, admits when it is unsure, and declines clearly harmful requests. A raw model trained only on text would not reliably do that. People compared many pairs of its answers and chose the better ones, and the model was adjusted toward those choices.
Where the analogy breaks
-
It comes after the main training, not instead of it. First the model learns from a huge amount of text, and feedback tuning then shapes how it behaves.
-
People's preferences are not the same as truth. A model can learn that confident or agreeable answers get higher ratings, which feeds problems like sycophancy.
-
It is not the only method. Other ways of learning from preferences or feedback exist, and many products combine several.