D6 · Sequences & Memory
GRU: Fewer Gates, Similar Result
Compare GRU to LSTM on parameters, speed and typical accuracy.
An LSTM makes two choices each step: how much old memory to keep, and how much new value to write. Often they move as a pair. Keeping means not writing, and writing means letting go. So why not turn one dial that sets both?