D2 · Gradient Descent
Give the Ball Some Weight
Explain momentum as a running average of gradients, and predict where it helps and where it overshoots.
Plain gradient descent forgets everything after each step. It feels the ground, steps, and starts over. So the first dip it meets is where it stops, even with a deeper one next door.