L4 · Training a Language Model
The Loop at Scale
Walk one training step, and say what keeps time in a run that has no epochs.
The loop is the one from the deep learning chapter. Read a batch, guess, measure the error, send it back, nudge every weight. Nothing about it changes for a frontier model. Only the number of repeats does.