L1 · Words as Numbers
Text Has to Become Numbers
Explain why a model needs a fixed vocabulary of integer ids.
Ask a good chat model how many r's are in strawberry. Plenty of them say two. They are not bad at counting. They never got the letters. The word arrived as three numbers, and nothing else.