Scaling laws

Scaling laws

Observed, fairly predictable relationships between an AI model's size (its parameters), the amount of training data and compute used, and its resulting performance. Scaling laws have driven much of the AI industry's strategy over the past decade, on the assumption that bigger models trained on more data will keep improving, though there is active debate about where this trend slows down. (See also: Compute, Emergent capabilities)