L10 · Smaller, Faster, Cheaper
A Teacher With a Vocabulary
Train a student on a teacher's whole row of chances, and use heat to make the small ones teach.
Tell a small model only that the next word was coffee. It learns one fact. Show it the teacher's whole row: tea at 15 percent, sand at almost nothing. Now it learns how the teacher thinks about cups.