R2 · Landmark Papers, Rebuilt
Scaling Laws (2020)
Fit a power law to a sweep of your own, and read what the exponent promises.
By 2019, people had noticed that bigger models did better. Nobody had a reliable rule for how much better, or for how long. So a training budget was spent on a guess, and you found out at the end.