L3 · The Transformer
Anatomy of One Block
Name the parts of a transformer block and their order.
In 2017, eight researchers drew their whole model in one figure. At its heart sat one box. GPT-2 stacks that box 48 times, and GPT-3 stacks it 96 times. The box itself has barely changed.