https://www.chenk.top/en/tags/language-models/Recent content in Language Models on Chen Kai BlogHugoenSun, 26 Oct 2025 09:00:00 +0000https://www.chenk.top/en/nlp/gpt-generative-models/Sun, 26 Oct 2025 09:00:00 +0000https://www.chenk.top/en/nlp/gpt-generative-models/<p>When you ask ChatGPT a question and a fluent multi-paragraph answer streams back token by token, you are watching a single deceptively simple loop: feed everything-so-far into a Transformer decoder, look at the probability distribution it produces over the vocabulary, pick one token, append it, repeat. That is <em>all</em> an autoregressive language model does. The miracle is not the loop — it is what happens when you scale the network behind the loop to hundreds of billions of parameters and train it on most of the internet.</p>