D3 · Backpropagation
Checking Gradients Numerically
Compare analytic and numeric gradients and set a sensible tolerance.
A backward pass with a bug in it often still trains. The loss falls, just not as far as it should. No crash, no error message. People lose days blaming the learning rate.