D8 · Mechanistic Interpretability
Superposition
Explain how a model packs more features than it has dimensions.
A plane has room for two directions at a right angle. Here are five. No pair meets at a right angle, so each one leaks a little into its neighbors.