A4 · Capstones
Project 3: A Tiny Next-Word Model
Build a next-word model out of counts, sample from it, and score it in bits.
This model writes sentences it has never seen, with no grammar and no meaning inside it. All it keeps is a tally: after this word, which word came next, and how often.