D6 · Sequences & Memory
Unrolling the Loop
Draw an RNN unrolled over time and point to the shared weights.
Textbooks draw a recurrent network as one box with an arrow curling back into itself. That picture hides what you need to see. Stretch it out. Draw one copy of the box per time step, each handing its state to the next. What you get is an ordinary network, as deep as the input is long.