System Design (4): Caching — Where to Cache, What to Evict, and When Caching Hurts

Thu, 17 Jul 2025 09:00:00 +0000

There is an old joke in computer science that the two hardest problems are cache invalidation, naming things, and off-by-one errors. The joke works because cache invalidation really is that hard. But caching is also the single most effective technique for improving system performance. A well-placed cache can reduce latency by 100x, cut database load by 90%, and save thousands of dollars in infrastructure costs per month.

The trick is knowing where to cache, what patterns to use, and — critically — when caching will make your system worse instead of better.

Caching on Chen Kai Blog

System Design (4): Caching — Where to Cache, What to Evict, and When Caching Hurts