noun · AI process
Inference
also called running the model
Using the trained brain. Every time you send a message and get a reply, that is inference.
Running a trained model on new input to produce output, which is what you pay for per request and what sets response speed.
A forward pass through the trained network for each token it writes, with the weights read and never changed. The input is processed in one go, then output tokens come one at a time.
Real-life example
You send a long document and ask for a summary. A short pause, then words start to appear, because the model is reading everything you sent. The words that follow arrive one after another because each one needs its own pass. None of this changed the model; it only used it.
Where the analogy breaks
-
Inference does not teach the model anything. The numbers stay exactly as they were before your request.
-
It is not one step. Reading your input happens in one big pass, but writing the reply is a loop, so long answers take noticeably longer than short ones.
-
It is where the running cost lives. Training is paid for up front by the maker, while inference is paid again on every request by whoever hosts the model.