Design YouTube

A video platform is two systems joined at a single byte of metadata: a write path that quietly turns one upload into many renditions, and a read path whose entire job is to make sure the origin almost never serves a byte.

1. Clarify the contract

Treat the prompt as a product contract before a box diagram. The system must:

• Accept large uploads reliably — multi-GB files over flaky consumer connections, resumable after a drop.
• Transcode each upload into a ladder of renditions — multiple resolutions/bitrates plus thumbnails and captions.
• Serve playback globally with low startup time and minimal rebuffering — adaptive bitrate over HTTP.
• Store metadata, comments, likes, and view counts.
• Survive viral spikes and honor copyright/safety takedowns.

And explicitly what it need not do, so we don't over-build: we are not designing the recommendation model (that's a separate offline pipeline that reads our metadata), we are not building a real-time chat/live-streaming path (different ingest), and we do not need strong consistency on social counters — a view count that is a few seconds or a few hundred counts stale harms no one. Naming these out loud is itself senior signal: it tells the interviewer you know where the consistency budget is allowed to be spent.

2. Put numbers on it — transcode amplification and CDN offload

Two numbers decide this entire architecture, and both come from arithmetic, not intuition.

Storage amplification: one upload becomes ~5×

A creator uploads one 1-hour 1080p file. Call it ~3 GB. We do not store just that — we transcode it into a rendition ladder so the player can adapt to bandwidth. Roughly five rungs (240p, 480p, 720p, 1080p, 4K). The lower rungs are tiny but the 4K rung is large, so as a planning rule the ladder costs on the order of the source again per major rung; a defensible back-of-envelope is:

stored per video ≈ 3 GB (source-class 1080p) × 5 renditions ≈ 15 GB

So transcoding amplifies storage ~5×. That single fact reorders the whole design: storage is dominated by renditions, not originals; transcode compute is a first-class cost center, not a detail; and "transcode eagerly for every rung on every video" is something you must justify, because the bottom rungs of a video nobody watches are pure waste (see the trade-off table). It also means the write path is fundamentally a fan-out batch job — DDIA Ch. 10's batch-processing model: one input record (the upload) deterministically produces many output records (the renditions), and like any batch job it is restartable and idempotent per output.

CDN offload: the origin must serve almost nothing

The read path's whole purpose is to keep bytes off the origin, because origin egress is the expensive, scarce resource. The lever is cache hit rate at the edge. If the edge serves a fraction h of bytes, the origin serves only 1−h:

origin bandwidth = total delivered × (1 − h)

At a realistic 95% edge hit rate the origin sees one-twentieth of delivered traffic. The curve is brutally nonlinear: the difference between 95% and 99% is not "4% better," it is a 5× reduction in origin egress (total/20 → total/100). This is the same hit-rate → backend-load relationship taught in lesson 06; here the "backend" is a fleet of origin storage nodes and the stakes are bandwidth dollars. Every cache-design decision below is in service of pushing h up and — just as important — keeping it from collapsing during a viral miss storm.

3. Data model & API

Keep the API expressing product operations; never leak the queue, shard, or cache layout. The interesting verbs are the upload session (because upload is multi-step and resumable) and the watch manifest.

The data model exposes the join point. Note that video_meta.status is the single flag the read path keys off:

4. Linearized design — two paths, one flag

Walk an upload through to a viewer. The first bottleneck — transcode fan-out — appears naturally at step 3, and the read path's defense appears at step 6.

1. 1. Resumable upload. Client opens a session and streams chunks straight into object storage (C16-class durable blob store). The API server never proxies the bytes.
2. 2. Complete = seal + enqueue. On complete, we verify the assembled object's checksum, write the source object, set status = processing, and push one transcode job per rung onto a queue.
3. 3. Transcode fan-out (the first bottleneck). A pool of GPU/CPU workers pulls jobs; each produces one rendition + segments + thumbnail. This is the ~5× amplification and the most expensive part of the system.
4. 4. Publish = flip metadata. When at least one playable rendition is ready, set status = published and write the manifest reference. This single write is the join between the two systems.
5. 5. Player fetches manifest from GET /watch, picks a starting rung, requests segments.
6. 6. Segments served by CDN edge. Edge hit → served locally (the 95%). Edge miss → goes to an origin shield that coalesces concurrent misses into one origin fetch, caches, and fans the bytes back out.

5. Deep dives

Deep dive 1 — Upload ≠ publish: processing is a state machine

The most common bug in a naive design is treating "upload completed" and "video is live" as the same event. They are separated by minutes of asynchronous, failure-prone work (transcoding a 4K rung can take a long time and can fail mid-way). If you flip the video to published on upload-complete, viewers hit a video with no playable rendition. If you wait for all renditions, a single stuck 4K job blocks the whole video from going live even though 720p has been ready for minutes. The fix is to model processing explicitly as a state machine and to make publish a function of rendition readiness, not of upload completion:

The key transition is processing → partially-playable: the moment the first rung finishes, we may flip status = published with a manifest that lists only the ready rungs, and let later rungs append to the manifest as they complete. This decouples "time to first watch" from "time to full ladder." The blocked state is a sink the read path must respect: takedowns and failed safety checks land here, and — crucially — the read path's playback-authorization check reads canonical status, not a cached copy, so a takedown is honored even while edges still hold segments (which we then purge). Because each transcode job is an idempotent batch output (DDIA Ch. 10), a crashed worker's job is simply re-run; partial output is overwritten, not duplicated, and the state machine never advances on a half-finished rung.

Deep dive 2 — CDN, origin shielding, and request coalescing for viral storms

Section 2 showed that 95% edge hit rate keeps origin at total/20. The danger is the transient when hit rate collapses: a video goes viral, and a segment that no edge has cached yet is requested by a million players in the same few seconds. Without protection, every one of those is an edge miss that forwards to origin — a cache-miss storm (a thundering herd) that can saturate origin egress even though steady-state hit rate is fine. Two mechanisms defend it:

• Origin shielding. Insert a designated mid-tier cache (the "shield") between the many edge POPs and the origin. Edge misses go to the shield, not directly to origin, so the origin sees misses from a handful of shields instead of thousands of edges. This collapses the fan-in.
• Request coalescing (single-flight). When N concurrent requests for the same uncached segment arrive at a shield, it issues one origin fetch and parks the other N−1 until that fetch returns, then serves them all from the freshly cached copy. So a million simultaneous first-requests for a viral segment become one origin read.

The arithmetic is the point: with coalescing, the origin load from a viral segment is O(number of shields), not O(number of viewers). This is the same admission-control instinct as lesson 15 applied to cache fills — collapse identical concurrent work into one unit. Because segments are immutable, content-addressed, and have long TTLs (a published segment never changes), the cache is safe to hold them aggressively; the only invalidation event is a takedown, which is a targeted purge, not a TTL.

Deep dive 3 — View counts and likes are eventually consistent

It is tempting to UPDATE videos SET views = views + 1 on each watch. At viral scale this is catastrophic: a single hot video would funnel millions of writes per second to one counter row — the canonical hot-shard problem — and worse, players request many segments per view, so a per-segment counter update would multiply that by the segment count. Synchronous global counters are simply the wrong model. Instead, treat each view/like as an append to a log of events and compute the displayed count asynchronously (DDIA Ch. 11's stream-processing view): edges emit lightweight, batched counter_event records (often sampled/aggregated at the edge first), a stream pipeline rolls them up, and the displayed number is updated every few seconds. The number is approximate and lagging — and that is fine, because the product contract (section 1) never required it to be exact. The senior framing: spend your consistency budget where correctness is user-visible (a video must not play after takedown) and refuse to spend it where it isn't (a view count being 3 seconds stale).

6. Trade-offs

The sharpest one is the eager-vs-lazy transcode trade, because it is where the ~5× amplification meets the long-tail viewing distribution. Most uploaded videos are watched by almost nobody, so eagerly producing a 4K rung for every upload burns GPU and storage on bytes that will never be served. But you cannot defer transcoding for a video that suddenly goes viral, because on-demand transcode-at-watch adds unacceptable startup latency. The senior answer is a hybrid keyed on a popularity signal: always produce a couple of safe mid rungs eagerly (so any video is instantly playable), and produce the expensive top/extra rungs lazily, triggered when watch rate crosses a threshold. This makes transcode spend track actual demand rather than upload volume.

7. Failure modes

8. Interview Q&A

Related lessons

The single most-leaned-on neighbor is C17 (Dropbox/file sync), the anchor for resumable multipart upload and idempotent commit — this case deliberately defers those mechanics there rather than re-deriving them. C16 (object storage) is the durable home for both source objects and renditions, and supplies the durability guarantees we assume. On the read path, lesson 06 gives the hit-rate → origin-load relationship that the whole CDN economics rests on, and lesson 15's coalescing/admission instinct is what we apply to cache fills. The transcode fan-out runs on lesson 11's async queue, scaled by queue age; lesson 13 covers the worker autoscaling and backoff. DDIA threads through: Ch. 10 (transcode as restartable batch fan-out), Ch. 11 (transcode jobs and counters as async streams), Ch. 1 (CDN/read-path load), Ch. 12 (idempotent resumable commit).