F7 · Where AI Came From
Attention and Scale
Place the 2017 transformer and the bet on scale that followed it, and say what stayed the same underneath.
In 2017 a Google team published a paper with a bold title: Attention Is All You Need. Its design, the transformer, now sits under almost every chat model.