Quantisation

Quantisation

A technique for shrinking an AI model by reducing the numerical precision of its parameters, for example storing them with fewer decimal places, so the model takes up less memory and runs faster with only a small loss in accuracy. Quantisation is a key reason capable models can now run on ordinary laptops or phones rather than requiring data-centre-scale hardware. (See also: Parameters, Small language model)