noun · AI process
Training
also called pre-training
Studying before the exam. The model reads a huge pile of text, guesses what comes next, and gets nudged every time it is wrong.
The one-off phase in which a model's parameters are fitted to data, usually by predicting the next token on a very large text collection and then refined with further tuning.
An optimization loop: show the model text, compare its next-token guess to the real one, and adjust its numbers a tiny bit to shrink the error. Repeat over trillions of tokens, then add more targeted rounds of tuning.
Real-life example
You ask a model for a poem about your cat and it writes a good one. It never stored a poem about cats and copied it. During training it saw so many patterns of words that it can now build new ones. The work happened months before you typed anything, and your message only uses what training left behind.
Where the analogy breaks
-
Training does not store the text it read. It leaves behind numbers that capture patterns. That is why a model can write something new, and also why it cannot reliably quote a page back.
-
Chatting with a model does not train it. Training is a separate, expensive job run by the model's makers. Your conversation is only input unless the provider says it will be used later.
-
It is finished before you arrive, and it is not free. Training runs take huge amounts of computing power, so a model is rebuilt only every so often and its knowledge stops at a date.