D6 · Sequences & Memory
The Case for Attention
State the three problems attention solves that gates could not.
Nobody translates by reading a sentence, closing the book, and writing from memory. You look back. Write a few words, glance at the part of the source you are on, write a few more. Every model in this chapter forbids that glance. Attention allows it.