D2 · Gradient Descent
The Slope Tells You Which Way Is Down
Use the derivative at a point to decide which direction lowers the loss, and extend that to many weights at once.
Back to the fog. You cannot see the valley. You can only feel the tilt of the ground where you stand. That tilt is the slope. It is true only at your feet, and it is all a learning algorithm ever gets. The surprise of this chapter is how far local information can carry you.