The model that wins most tabular competitions is not a neural network.
M3 · Twenty Questions
A decision tree asks yes or no questions until it is sure, which is exactly how a person would sort the same data. One tree memorizes almost instantly, so this chapter shows the two great fixes: build many trees on random slices, or build them one after another to correct each other. That second idea, gradient boosting, is still the first thing to try on a table. You also learn to read feature importance without being fooled by it.
- A Tree of Questions
- Which Question to Ask First
- Why Tree Boundaries Are Staircases
- A Tree That Memorizes
- Pruning and Depth Limits
- Many Trees, Many Samples
- Random Forests
- Learning From Your Own Mistakes
- Gradient Boosting
- Feature Importance and Its Traps
- Crediting Each Feature Fairly
- When Trees Beat Deep Learning