Performance on Chen Kai Blog

System Design (4): Caching — Where to Cache, What to Evict, and When Caching Hurts

Thu, 17 Jul 2025 09:00:00 +0000

There is an old joke in computer science that the two hardest problems are cache invalidation, naming things, and off-by-one errors. The joke works because cache invalidation really is that hard. But caching is also the single most effective technique for improving system performance. A well-placed cache can reduce latency by 100x, cut database load by 90%, and save thousands of dollars in infrastructure costs per month.

The trick is knowing where to cache, what patterns to use, and — critically — when caching will make your system worse instead of better.

Databases (2): Indexing and Query Planning — How Databases Find Your Data

Fri, 19 Apr 2024 09:00:00 +0000

A query that returns in 2 milliseconds on your laptop with 1,000 rows will take 45 seconds on a production database with 50 million rows — unless you have the right indexes. Indexes are the single most impactful performance tool in your database toolkit, and understanding how they work changes the way you think about every schema and every query you write.

The Fundamental Problem: Finding a Row#

Imagine a table with 10 million rows, stored on disk as a heap file. Each row sits somewhere in a sequence of 8 KB pages. When you run:

Python Engineering (8): Performance — Profiling, Caching, and Knowing When to Stop

Wed, 27 Apr 2022 09:00:00 +0000

Donald Knuth’s famous quote is often half-remembered. The full version is: “We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.” The second sentence is the key. Performance work isn’t about making everything fast; it’s about finding the 3% that matters and making that fast.

This article is about finding that 3%. You’ll learn to profile first, optimize second, and measure the impact of each change.