NLP (2): Word Embeddings and Language Models

Mon, 06 Oct 2025 09:00:00 +0000

\vec{\text{king}} - \vec{\text{man}} + \vec{\text{woman}} \approx \vec{\text{queen}}

The entire trajectory of NLP shifted toward representation learning. This article walks through that shift—from the failure of one-hot vectors, to Word2Vec’s shallow networks, to the global statistics that GloVe exploits, to the subword n-grams that let FastText handle unseen words—and finally connects embeddings to the language models that gave rise to them.

Word Embeddings on Chen Kai Blog

NLP (2): Word Embeddings and Language Models