System Design — the distributed-systems interview, from first principles

The classic backend system-design interview: design a URL shortener, a news feed, a chat service, a rate limiter. The questions look different but they all reduce to the same handful of levers — latency, throughput, consistency, fault tolerance, and scale — and the same handful of patterns for moving them. This track derives those patterns from one starting fact, so you can re-derive any design in the room instead of pattern-matching to a memorised diagram.

Two layers: 19 lessons + 44 cases

The 19 lessons below are the foundation layer — they derive the patterns. On top sits an applied layer of 44 case drills (URL shortener, news feed, chat, payments, stock exchange, …), drawn from the Alex Xu volumes and the archive, where you apply those patterns under realistic prompts. Foundation first, then cases. See the case syllabus and coverage map.

The one idea this whole track turns on

Start from the machine that needs no distributed systems at all: one server with one database. It is simple, it is strongly consistent (one copy of the truth), and an in-process call is nanoseconds. It has exactly two problems, and every technique in this track exists to solve one of them:

It can't scale. One box has a finite CPU, finite RAM, finite disk, and a finite network card. When load or data outgrows it, you must spread across many boxes.
It will fail. One box is a single point of failure — a disk dies, a kernel panics, a datacenter loses power — and when it's gone, you're at zero.

So you add more machines. The instant you do, you have a distributed system, and you have bought three new problems you did not have before: the machines must coordinate (agree on who does what and what's true), they are separated by a network that is slow and unreliable (the speed of light is a hard floor; packets drop), and their copies of state can disagree (consistency). The famous CAP theorem is just the formal statement that you cannot wish the network away.

The senior framing

System design is not "memorise the news-feed architecture." It is the disciplined trade of simplicity and consistency (which the single box gives you for free) for scale and availability (which only many boxes can give you), paying the bill in coordination, latency, and complexity. Every pattern below is one specific trade. An interviewer is testing whether you know which trade each pattern makes — and can say what it costs, not just what it buys.

How the track is structured

Strict linear. Each lesson assumes only the ones before it. We start with the measuring stick (what are we even optimising?), pin down the system's contract and its data, scale the easy tier (stateless), then spend the bulk of the track on the genuinely hard part — scaling state — before assembling everything into performance, reliability, an architecture, and a full design.

Stage	Lessons	What you can do after
The measuring stick	01–02: the method & napkin math, latency / throughput / queueing	Drive a design from requirements → estimates → bottleneck. Back-of-envelope QPS, storage, and bandwidth. Explain why P99 ≫ mean and why a queue at 80% load is fine but at 99% explodes.
The contract & the data	03–04: API design, data modeling & storage engines	Turn a prompt into the 3–5 endpoints that matter (REST / gRPC / GraphQL, idempotent writes, pagination, versioning) and a data model driven by access patterns — and know why a B-tree, an LSM-tree, and an index behave the way they do, because the rest of the track assumes it.
Scaling the stateless tier	05–06: horizontal scaling & load balancing, caching	Scale the cheap part. Choose an LB strategy, keep replicas stateless, and use a cache as the read-path lever — without getting bitten by stampedes and stale data.
Scaling state — the hard part	07–10: partitioning, replication, consistency & CAP, consensus	Shard data with consistent hashing, replicate it for durability and reads, reason precisely about which consistency model you're offering, and use consensus where you genuinely need agreement (and avoid it where you don't).
Decoupling & reliability	11–13: async messaging, distributed transactions, fault tolerance	Break the synchronous critical path with queues, get correctness across services without 2PC (sagas, outbox, idempotency), and make the system degrade-never-fail with redundancy, circuit breakers, and a fallback ladder.
Performance & operations	14–16: optimisation playbook, rate limiting & the API edge, observability	Make it fast on purpose — profile, then attack the dominant term (Amdahl) — protect the front door with a rate limiter and gateway, and run it in production with SLOs, error budgets, and the three pillars of telemetry.
Recognise & synthesise	17–19: system architecture, archetypes, capstone	Choose an architecture (monolith vs services, sync vs event-driven) and defend the boundaries, classify any prompt into one of the recurring system shapes, then walk a full design end-to-end under interview constraints — reaching for each pattern by name and defending the trade.

What an interviewer is actually testing

Skill	Strong	Weak
Drive from requirements, not buzzwords	Asks "read:write ratio? consistency requirement? scale?" first, then estimates QPS and storage, and lets those numbers pick the components.	Opens with "I'll use Kafka, Cassandra, and Redis" before knowing the load or the consistency need.
Name the trade, not just the pattern	"I'll shard by user_id — it co-locates a user's data so the timeline read is one shard, but it makes cross-user queries scatter-gather, and a celebrity is a hot shard."	"I'll add a cache / add a queue / shard it" with no statement of what it costs.
Estimate with numbers	"1M DAU × 50 reads/day ≈ 580 req/s average, ~5× peak ≈ 3k req/s; each row ~1 KB, 100 GB/yr — fits on one replicated Postgres, no sharding yet."	"It needs to be web-scale" with no figure, so over-engineers for traffic that doesn't exist.
Reason about failure	"What happens when this dependency is down, this region is gone, this message is delivered twice?" — and has an answer (degrade, failover, idempotency key) for each.	Designs only the happy path; treats the network as reliable and machines as immortal.

The lessons

The method & napkin math

The repeatable interview script: clarify requirements & SLOs → estimate (QPS, storage, bandwidth) → sketch the API → design the data model → scale & defend. The "latency numbers every engineer should know" and how an order-of-magnitude estimate picks your components before you name a single technology.

Latency, throughput & queueing

The two metrics that aren't the same thing. Why you report P99, not the mean. Little's law (L = λW) and why a queue at 80% utilisation is calm but at 99% blows up to infinity. Tail-at-scale: why one slow component poisons a fan-out, and how concurrency, batching, and pipelining each move a different number.

API design

The contract clients depend on, designed before the boxes — and the one thing that's expensive to change once they do. REST vs gRPC vs GraphQL and when each wins. Idempotency keys for safe retries, cursor vs offset pagination, versioning without breaking callers, machine-readable error contracts, and the async 202-+-poll/webhook pattern that takes long work off the request.

Data modeling & storage engines

The data layer the rest of the track quietly assumes. Model the queries, not just the entities; normalise vs denormalise; SQL vs NoSQL as a decision. The write-ahead log, and B-tree vs LSM-tree storage engines — the read/write-amplification trade that explains why Postgres, Cassandra, and Kafka behave so differently. What an index physically is, and what every index costs you on writes.

Horizontal scaling & load balancing

Vertical vs horizontal scaling and where each wall is. Why stateless is the magic word that makes a tier trivially scalable. L4 vs L7 load balancing, round-robin / least-connections / consistent-hashing strategies, health checks, and N+1 / N+2 capacity. Where session state has to go once the box can't hold it.

Caching — the read-path lever

Why caching is the highest-leverage trick in a read-heavy system, and the cost it hides: a second copy of the truth that can go stale. Cache-aside vs read/write-through vs write-back. Eviction (LRU/LFU/TTL), the hit-rate economics, the two hard problems (invalidation & naming), and the failure modes — stampede, penetration, avalanche — with their fixes.

Partitioning & sharding

When one box can't hold the data, you split it. Hash vs range vs directory partitioning and what each does to range scans and hot spots. Consistent hashing with virtual nodes — why it remaps only ~1/N of keys on a resize instead of nearly all of them. The choose-your-shard-key decision and how a celebrity becomes a hot shard.

Replication

Copies for durability, read-scaling, and availability — and the disagreement they introduce. Single-leader, multi-leader, leaderless (quorum) replication. Synchronous vs asynchronous and the replication-lag / read-your-writes problem. The quorum inequality W + R > N, and why a failover can silently lose the un-replicated tail.

Consistency models & CAP / PACELC

The ladder of guarantees: linearizable → sequential → causal → read-your-writes → eventual, and what each costs. CAP stated precisely (it's about the partition, not "pick 2 of 3") and why PACELC is the more useful version — because even with no partition you trade latency for consistency. How to choose per-operation, not per-system.

Consensus & coordination

When machines must agree — leader election, config, distributed locks, exactly-once decisions. Why consensus is expensive (a majority round-trip per decision) so you use it sparingly. Raft / Paxos at the level an interview needs, quorums and why clusters are odd-sized, split-brain and fencing, and the role of ZooKeeper / etcd.

Asynchronous messaging & streaming

Taking work off the synchronous critical path. Queues vs logs (RabbitMQ vs Kafka), pub/sub fan-out, the consumer-group scaling model. Delivery semantics — at-most / at-least / effectively-once — and why true exactly-once is a myth you fake with idempotency. Backpressure, dead-letter queues, and ordering guarantees per partition.

Distributed transactions & idempotency

ACID was free on one box; across services it isn't. Why two-phase commit blocks and doesn't scale. The saga pattern (a chain of local transactions with compensations), the transactional-outbox pattern for atomic "write DB + publish event," and idempotency keys as the workhorse that makes at-least-once delivery safe. Eventual consistency as a product decision.

Fault tolerance & graceful degradation

Availability as a budgeted number — each nine costs 10× for 10× less downtime. The series (∏Aᵢ) and parallel (1−∏(1−Aᵢ)) formulas that drive every redundancy decision. Timeouts, retries with backoff+jitter, circuit breakers, bulkheads, and load shedding. Failover, multi-region, RTO/RPO, and "degrade, never fail."

Optimisation playbook

The performance levers scattered across the track, collected into one menu. Measure first; Amdahl's law says optimise the dominant term or the ceiling defeats you. The ladder — do less work (cache, precompute), move work off the hot path (async, batch), cut round trips, make each unit faster (indexes, pooling), then parallelise — with the cost that makes each one backfire.

Rate limiting & the API edge

The system's front door: the API gateway, and the rate limiter that protects everything behind it. Token bucket vs leaky bucket vs sliding-window — the algorithm, the burst behaviour, and the memory cost. Where the limiter lives in a distributed fleet (and the shared-counter coordination problem), plus auth, TLS termination, and request coalescing at the edge.

Observability & operability

You cannot operate what you cannot see. The three pillars — metrics, logs, traces — and what each is for. SLI / SLO / SLA / error budget and how the budget turns "be reliable" into a quantity you trade against velocity. Why you alert on symptoms (SLO burn) not causes, the RED / USE methods, and distributed tracing across a request's fan-out.

System architecture & service decomposition

The architectural styles the archetypes quietly assume. Monolith-first, and the real cost of splitting: every boundary becomes a network call, every transaction a saga. Where service boundaries go (bounded contexts, database-per-service) and the distributed-monolith trap. Synchronous vs event-driven, CQRS and event sourcing, and the failure domains (AZ / region / cell) that bound the blast radius.

System design archetypes — recognising the shape

Almost every prompt is, underneath, one of a few recurring shapes — CRUD/transactional, feed/fanout, queue/async-workers, streaming/log, search/retrieval, real-time coordination, analytics/OLAP, blob/media. Each has a dominant force, a bottleneck that breaks first, and a default set of patterns from the lessons above. Classify in 60 seconds (read:write ratio, consistency need, payload size, interaction model), then design — and notice when a system is really a composite of several shapes.

Capstone — putting the patterns together

A full design walkthrough under interview conditions, reaching for each pattern by name. Requirements & estimates → API → data model → the read and write paths → where partitioning, caching, replication, queues, and consensus each land → the failure analysis. A reusable checklist and the senior moves that separate a hire from a strong-hire.

What this track does NOT cover

Out of scope on purpose — they live in other tracks:

ML-specific serving & training systems (inference replicas, KV cache, distributed training topologies). See ml_system_design.
Recommender / ads serving (candidate generation, ranking, the recsys reliability layer). See search_ads_recsys.
GPU kernels & low-level performance. See GPU Kernels for ML Engineers (CUDA interview coding in Part V) and Triton (Part VI).
Data-structures & algorithms coding. See python_coding.

This is the substrate underneath all of them. The recsys high-availability lesson, the ML serving lessons, and the inference-at-scale lessons all assume the partitioning, replication, consistency, and fault-tolerance vocabulary built here.

How to use this

Linearly the first time. Lessons 01–02 are the measuring stick; 03–04 fix the API contract and the data model the rest of the track leans on; lessons 07–10 (scaling state) are the load-bearing core. Skipping them makes the consistency and consensus reasoning feel like memorisation rather than derivation.
Re-derive the trade on paper. For each pattern, write the one sentence: "this buys ___, it costs ___." If you can't, you don't yet own it.
Touch the widget. Each lesson has an interactive calculator — queueing latency, quorum overlap, availability nines, cache hit-rate economics, B-tree vs LSM amplification, Amdahl's ceiling. They turn an abstract trade into a number you can feel.
Read the "interview prompts" box. Those are the questions you'll actually be asked. The parenthetical answer shows the depth that distinguishes a hire from a strong-hire.

A note on what makes system design "fundamental"

The brand-name technologies churn — today's Kafka / Cassandra / Redis / Envoy will be renamed in a decade. What does not change is the small set of forces underneath: the speed of light bounds latency, queues blow up near saturation, the network partitions, copies disagree, and machines die. Every product in the stack is one named implementation of one pattern for taming one of those forces. Learn the forces and the patterns, and each new tool is a one-page diff. Learn only the tools, and every new one is a thing to memorise. This track is about the forces.