D3 · Backpropagation
Reverse-Mode Autodiff
Explain why reverse mode suits one loss and many parameters.
Backprop was not invented for neural networks. It gets derivatives out of any program. The first person to write it down was tracking rounding errors in numerical code.