L3 · The Transformer
From Vector to Next Token
Trace the last vector through the output head to a probability for every token.
After the last block, you have one vector per token. You need a probability for each of fifty thousand possible next tokens. The bridge is one matrix with a row per token. Score the vector against every row, and a softmax turns the scores into probabilities.