D7 · Neural Scaling Laws
Why Bigger Keeps Working
Give the leading explanations for scaling and mark which are still guesses.
The straight line has turned up again and again since 2017, in text, images, speech and video. Nobody agrees on why. This lesson walks through the serious candidates, and what it would take to believe each one.