Inference

Inference

The process of running a trained AI model on new input to produce an output, as distinct from training, where the model's parameters are adjusted. Every time a chatbot answers a question or an image generator produces a picture, it is performing inference; this is also the stage at which most of the compute and energy costs of AI use are incurred, spread across millions of everyday requests rather than concentrated in a single training run. (See also: Training data, Compute)