System Design (5): Message Queues and Event-Driven Architecture

Sat, 19 Jul 2025 09:00:00 +0000

In 2011, LinkedIn’s engineering team was struggling with a problem that many growing companies face. Their monolithic application had become a web of tightly-coupled services, each making synchronous calls to half a dozen others. When any single service went down, cascading failures rippled through the entire system. Deploying a change to one service required coordinating with every team whose service it called.

Their solution was Apache Kafka — a distributed event log that decoupled producers from consumers. Instead of Service A calling Service B directly, Service A writes an event to Kafka, and Service B reads it when it is ready. If Service B is down, the events wait. If Service B is slow, it processes at its own pace. The producer does not need to know or care about the consumer.

Kafka on Chen Kai Blog

System Design (5): Message Queues and Event-Driven Architecture