L2 · Attention
Reading Attention Maps
Interpret a real attention map and list what it cannot prove.
Attention maps show up in papers, blogs and slides, because they are the one part of a transformer that looks readable. They are useful. They are also the most over-read picture in the field. This lesson reads a map, and marks where the reading stops.