L1 · Words as Numbers
Meaning From Company
Explain the skip-gram training signal in one sentence and why it works.
In 1957 the linguist John Rupert Firth wrote: you shall know a word by the company it keeps. Arabic has a saying close to it: tell me who your friends are, and I will tell you who you are. Fifty-six years later, Firth's sentence became an algorithm.