Stage 3 · Expert · D8
Mechanistic interpretability
The dark matter of AI.
11 lessons · 126 minResearch
About this chapter
We can read every weight in a model and still not know what it is doing. Mechanistic interpretability tries to close that gap by finding features, then circuits, then a story about computation you can test. This chapter covers superposition, the reason a neuron can mean five things at once, and sparse autoencoders, the current best tool for pulling those meanings apart. You will patch activations to prove a claim rather than assert it, and you will see plainly where the field still fails.
What you will be able to do
- 110 min
What counts as a feature
Define a feature as a direction in activation space, not a neuron.
- 211 min
Neurons, features, circuits
Trace a small circuit that combines features into a decision.
- 310 min
The neuron that means five things
Recognize polysemantic neurons and say why they are the normal case.
- 412 min
Superposition
Explain how a model packs more features than it has dimensions.
- 513 min
Sparse autoencoders
Describe how an SAE is trained and what a learned feature dictionary contains.
- 611 min
Probing
Train a probe and say why a probe finding something is weak evidence of use.
- 713 min
Activation patching
Design a patching experiment that isolates one component's causal role.
- 812 min
Induction heads
Explain the copy-and-continue pattern and why it matters for in-context learning.
- 911 min
Attribution maps and their failures
Read a saliency map and describe a test that shows it can be meaningless.
- 1011 min
What we cannot read yet
State the current limits of interpretability honestly, with examples.
- 1112 min
Why safety needs this
Connect interpretability results to concrete safety decisions.