Skip to content

Stage 1 · Beginner · M1

The Best Straight Line

One idea, published in 1805, still trains neural networks today.

12 lessons · 130 minEasy

Start chapter

About this chapter

Fit a line by eye, then let algebra find a better one, then let a walk downhill find the same one again. Those three routes to one answer are machine learning in miniature. Along the way the loss becomes a bowl you can see, extra features become extra columns, and a curve turns out to be a straight line in disguise. The chapter ends on the two things that go wrong: a model that memorizes its data, and a score of 0.98 that means nothing at all.

What you will be able to do

  1. 1

    Fit It by Eye First

    Place a line through points by hand and say what makes one line better than another.

    10 min
  2. 2

    Why We Square the Error

    Explain what squaring does to big and small misses, and what it assumes without saying so.

    10 min
  3. 3

    The Loss Surface Is a Bowl

    Read a slope and an intercept as one point on a bowl-shaped loss surface.

    10 min
  4. 4

    The Exact Answer

    Find the best line in one step with a formula, and say when the formula stops being practical.

    11 min
  5. 5

    The Same Answer, Step by Step

    Fit the same line by walking downhill and compare it to the exact answer.

    11 min
  6. 6

    More Than One Input

    Extend the model to several features and read each coefficient correctly.

    11 min
  7. 7

    Put the Features on One Scale

    Standardize features, explain what it fixes, and say what it does not change.

    10 min
  8. 8

    Curves From a Linear Model

    Add polynomial features to fit a curve, and explain why the model is still linear.

    11 min
  9. 9

    Trying Too Hard

    Hold data back, read train and test error together, and name the two ways a model fails.

    12 min
  10. 10

    Ridge and Lasso as a Budget

    Add a penalty on weight size and predict which penalty drives a weight to exactly zero.

    12 min
  11. 11

    R Squared and Its Traps

    Pick the right regression score, and say what a high R squared does not prove.

    11 min
  12. 12

    Where This Lives Inside Every Network

    Recognize the model you built as a linear layer, and decide when a linear baseline is enough.

    11 min

Before you start

Keep going

All chapters