F2 · The Slope of Everything
Every Model Is a Derivative Machine
Explain why one derivative of one number is all a model ever needs.
However large the model, the training step needs exactly one number at the end. The loss. Everything before it is arithmetic. Everything after it is one question: how does each knob move that number?