Stage 2 · Intermediate · M3
Twenty Questions
The model that wins most tabular competitions is not a neural network.
12 lessons · 113 minSteady
About this chapter
A decision tree asks yes or no questions until it is sure, which is exactly how a person would sort the same data. One tree memorizes almost instantly, so this chapter shows the two great fixes: build many trees on random slices, or build them one after another to correct each other. That second idea, gradient boosting, is still the first thing to try on a table. You also learn to read feature importance without being fooled by it.
What you will be able to do
- 18 min
A Tree of Questions
Trace one example down a tree and read the prediction at the leaf.
- 210 min
Which Question to Ask First
Score a cut by how mixed its two buckets are, and pick the cut that leaves the least mess.
- 38 min
Why Tree Boundaries Are Staircases
Say why every edge of a tree's boundary is square, and what that costs it.
- 49 min
A Tree That Memorizes
Grow a tree until it memorizes the training set and see the test error rise.
- 59 min
Pruning and Depth Limits
Control tree size with depth, leaf size and cost-complexity pruning.
- 69 min
Many Trees, Many Samples
Explain bagging and why averaging independent errors helps.
- 710 min
Random Forests
Build a forest and say why it hides columns from every split.
- 810 min
Learning From Your Own Mistakes
Describe boosting as a chain of small models, each fixing what the last one got wrong.
- 911 min
Gradient Boosting
Tune the step size, the depth and the number of rounds, and say what each one does.
- 109 min
Feature Importance and Its Traps
Read an importance chart and name two ways it misleads you.
- 1110 min
Crediting Each Feature Fairly
Explain the SHAP idea and read a single-prediction explanation.
- 1210 min
When Trees Beat Deep Learning
Choose between trees and neural networks from the shape of the data.