L3 · The Transformer
Three Shapes of Transformer
Choose an encoder, a decoder or both for a given task.
The 2017 transformer was built to translate. One stack read the whole English sentence. A second stack wrote the French or German one word by word, glancing back at the first. Within two years, each half was a famous model. GPT kept the writer, and BERT kept the reader.