Stage 1 · Beginner · M1
The best straight line
One idea, published in 1805, still trains neural networks today.
12 lessons · 130 minEasy
About this chapter
Fit a line by eye, then let algebra find a better one, then let a walk downhill find the same one again. Those three routes to one answer are machine learning in miniature. Along the way the loss becomes a bowl you can see, extra features become extra columns, and a curve turns out to be a straight line in disguise. The chapter ends on the two things that go wrong: a model that memorizes its data, and a score of 0.98 that means nothing at all.
What you will be able to do
- 110 min
Fit it by eye first
Place a line through points by hand and say what makes one line better than another.
- 210 min
Why we square the error
Explain what squaring does to big and small misses, and what it assumes without saying so.
- 310 min
The loss surface is a bowl
Read a slope and an intercept as one point on a bowl-shaped loss surface.
- 411 min
The exact answer
Find the best line in one step with a formula, and say when the formula stops being practical.
- 511 min
The same answer, step by step
Fit the same line by walking downhill and compare it to the exact answer.
- 611 min
More than one input
Extend the model to several features and read each coefficient correctly.
- 710 min
Put the features on one scale
Standardize features, explain what it fixes, and say what it does not change.
- 811 min
Curves from a linear model
Add polynomial features to fit a curve, and explain why the model is still linear.
- 912 min
Trying too hard
Hold data back, read train and test error together, and name the two ways a model fails.
- 1012 min
Ridge and lasso as a budget
Add a penalty on weight size and predict which penalty drives a weight to exactly zero.
- 1111 min
R squared and its traps
Pick the right regression score, and say what a high R squared does not prove.
- 1211 min
Where this lives inside every network
Recognize the model you built as a linear layer, and decide when a linear baseline is enough.