https://www.chenk.top/en/tags/autoscaling/Recent content in Autoscaling on Chen Kai BlogHugoenTue, 07 Apr 2026 09:00:00 +0000https://www.chenk.top/en/llm-engineering/12-production/Tue, 07 Apr 2026 09:00:00 +0000https://www.chenk.top/en/llm-engineering/12-production/<p>This is the last chapter. The previous ones covered building the model, the prompt, the retrieval, and the evaluation. This chapter focuses on maintaining it without breaking the bank. Production LLM serving is more like running a high-traffic web service than classical ML serving, except each web request costs money and can take up to two minutes.</p> <p>I&rsquo;ll focus more on numbers here than in earlier chapters. In production, the difference between a profitable feature and a money pit often boils down to a 2-5x cost factor that no one is tracking. The most useful skill to develop is back-of-the-envelope cost arithmetic for LLM workloads. The numbers below are accurate as of late 2025 / early 2026; verify them against current pricing before committing.</p>