L3 · The Transformer
The Feed-Forward Layer
Explain the widen-and-narrow shape and count its parameters.
Attention gets the famous name and most of the diagrams. It is not where most of the parameters are. Inside each block, the small feed-forward network holds twice as many numbers as attention.