Design Twitter-style timelines
"Build Twitter's feed" is really three different feeds wearing one name. The senior move is to say which one you're building before you draw a single box — because each is built by a completely different machine.
Don't re-derive the crossover — point at C08
The classic interview trap is to spend ten minutes re-deriving fanout-on-write vs fanout-on-read. Don't. That derivation — the break-even where writes_per_post · followers overtakes reads · followees — lives in C08 (news feed), and a senior candidate references it in one sentence: "this is the C08 crossover; I'll use the hybrid push/pull split it lands on." What C08 does not stress, and what makes Twitter specifically hard, is two things:
- The graph is power-law, not uniform. A handful of accounts have 10–100M followers while the median user has a few hundred. A uniform crossover threshold is meaningless when the follower count spans seven orders of magnitude — you need a per-author decision, not a global one.
- "The feed" is not one product surface. Profile, home, and mentions are three timelines with three different build strategies, three different cost profiles, and three different consistency needs. Conflating them is the single most common way this design goes muddy.
The three timelines (the meta-signal)
Before any architecture, separate the three things the prompt calls "timeline." Saying this out loud is the strongest early signal you can send.
Profile is a single-author append log — the easy case, near-free at any scale. Mentions is notification plumbing: when a post is parsed, any @handle it contains gets the post ID appended to that handle's mentions list, completely independent of the follow graph. Home is the only one that needs the C08 hybrid, because it's the only one that joins a stream against an arbitrary, skewed set of follows. Designing all three with the same fanout machinery is the mistake; the rest of this case is about home.
Clarify the contract
Treat the prompt as a product contract before a box diagram. The system must:
- Post short messages + media references; serve the profile, home, and mentions timelines.
- Handle follow / unfollow / block / mute, and have those take effect promptly.
- Support viral accounts (tens of millions of followers) without write storms.
- Feel near-real-time on home reads without making every read an expensive graph fan-in.
What it need not do, and saying so buys credibility: home need not be perfectly chronological (ranking is allowed and expected), need not be strongly consistent (a post showing up 2s late is fine), and need not retain infinite scrollback (a bounded window — a few hundred recent IDs — covers nearly all reads; older reads fall back to the pull path).
Put numbers on it
Two numbers force every later decision: the read/write asymmetry and the IDs-vs-bodies storage gap.
Reads dominate writes. Take 200M daily actives each opening the app ~10×/day and refreshing a few times per open — call it ~50 home reads/user/day. That's 200M · 50 = 1010 reads/day ≈ 1010 / 86400 ≈ 116k reads/s, and several× that at peak. Writes are far rarer — if 10% of users post 2×/day that's 20M · 2 / 86400 ≈ 460 posts/s. Reads outrun writes by ~250×, which is the entire argument for fanout-on-write for ordinary authors: do the join once at write time so the 116k reads/s become cheap list lookups.
But the viral author breaks that. A push to 100M followers is 100M list writes per post. At even 460 posts/s, if a fraction come from mega-accounts the write amplification is catastrophic — a single celebrity tweet is more fanout work than millions of ordinary posts combined. So mega-accounts are flipped to fanout-on-read: their post is written once to their profile stream, and pulled in at each follower's hydration. The threshold is per-author, typically tens of thousands of followers.
Store IDs, not bodies — the ~100× number. A home timeline is a list of post IDs, not post contents. An ID (snowflake) is 8 bytes; a hydrated tweet body — text, author, media refs, counts — is easily 1–2 KB. Materialising 800 IDs per home timeline costs 800 · 8 B = 6.4 KB; materialising the bodies would cost 800 · ~1.6 KB ≈ 1.3 MB — roughly 200× more, multiplied across every follower of every author. That gap is why home timelines are stored as ID lists and bodies are hydrated on read from a shared tweet store/cache (one cached copy per tweet, not one per follower).
Data model & API
Keep the surface small; let it express the product op, not the shard/cache layout.
| Store / API | Shape & why |
|---|---|
tweets | Canonical, by (author_id, snowflake_id). Single source of truth for the body + a visibility flag (live / deleted / restricted). Hydration reads here. |
profile_timeline | Append-only per author: list of that author's tweet IDs by time. Built by the author's own write. No graph. |
home_timeline | Per user: bounded list (~800) of tweet IDs, written by the fanout workers for ordinary authors. Cache-resident (Redis-class), rebuildable. |
mentions | Per user: list of tweet IDs that @-mentioned them, written at parse time. |
graph | Follow edges + block/mute edges. followers(author) drives push; followees(user) drives the viral pull merge. |
POST /tweets · GET /home_timeline?cursor= · GET /users/{id}/timeline · GET /mentions · POST /follow · POST /block | |
Linearized design — follow a post, then a read
- Write. A post is written once, canonically, to
tweetsand appended to the author'sprofile_timeline. Cheap, single-partition, always done. - Classify the author. If followers < threshold (ordinary), enqueue a fanout job. If viral, do nothing more on write — readers will pull it.
- Fanout (async). Workers read
followers(author)and append the new ID to each follower'shome_timeline, trimming to the bounded window. This is C11's heaviest pipe; it rides the async queue (lesson 11) so a posting spike never blocks the writer. - Mention parse. In parallel, parse
@handlesand append the ID to each mentioned user'smentionslist. - Read (home). Fetch the user's pushed ID list; pull recent IDs from each viral followee's profile stream; merge by time into a candidate set.
- Hydration. Filter the candidate IDs against block/mute and the canonical visibility flag, then hydrate surviving IDs → bodies from the shared tweet cache. Rank the bounded set, return a stable cursor.
The first bottleneck is step 3 (viral fanout) — solved by the per-author push/pull split. The second, subtler bottleneck is step 6: hydration is where correctness lives, and the next two deep dives are about not corrupting it.
Deep dives
1. Name which timeline you're designing — the meta-signal
This is worth restating as its own move because it's the differentiator between a junior and senior answer. A junior hears "design Twitter" and starts drawing a fanout pipeline. A senior says: "There are three timelines. Profile is an append log — trivial. Mentions is a notification list — write-on-parse, graph-independent. Home is the hard one, and it's the C08 hybrid at skewed graph scale. I'll design home; the other two fall out of the same tweet store." Naming the surface scopes the problem, signals you know the product, and prevents you from accidentally applying expensive fanout to the two timelines that don't need it. The whole rest of the interview is then about home.
2. Follow / unfollow don't rewrite infinite history
The naive read of "home timeline = posts from everyone you follow" implies that following someone should inject all their history into your timeline, and unfollowing should rip it back out. At graph scale that's a non-starter: a follow could rewrite an unbounded list, and an unfollow could require scanning and editing millions of cached timelines. The discipline is to make graph mutations bounded and lazy:
- Follow: backfill only a recent window of the new followee's posts into your home timeline (e.g. their last few dozen, time-merged) — bounded work — and from now on you receive their fanout. You do not reconstruct their entire history; older posts surface via the pull path if you scroll far back.
- Unfollow / block / mute: stop fanning future posts to you, and rely on hydration-time filtering for anything already sitting in your cached timeline. You do not eagerly scrub the cache. The stale IDs are harmless — they get filtered on the next read and cleaned asynchronously.
So follow does bounded forward work; unfollow does no eager work and leans on the filter. The cost you accept is a few wasted IDs in the cache; the cost you refuse is rewriting infinite history on every graph edge change.
3. Cache invalidation — mark canonical, filter at hydration, clean async
Deletion exposes the same trap. A tweet pushed to 5M home timelines lives as 5M cached IDs. When the author deletes it, the wrong instinct is to delete the ID from all 5M lists synchronously — a second fanout storm, and one that must hit lists that may have aged out or live on cold shards. Lesson 06's invalidation problem at fanout scale.
The correct pattern flips where truth lives. The cached ID lists are not authoritative; the tweets store's visibility flag is. So:
- On delete/restrict: flip the canonical visibility flag in
tweets. One write. The 5M cached IDs are now stale-but-harmless pointers. - On read (hydration): every ID is checked against the canonical flag before its body is served. A deleted/restricted tweet's ID resolves to nothing and is dropped from the response. The user never sees it, even though the ID is still in their cached list.
- Async cleanup: a background sweep removes dead IDs from timelines lazily, on its own schedule, off the hot path.
This is the same invariant as the unfollow case, generalised: cached timelines are a hint; the canonical tweet store is the truth; correctness is enforced at hydration, never by rewriting fanned-out copies. This is DDIA Ch. 11's derived/materialised data in its purest form — the home timeline is a derived view, and you keep a derived view correct by re-deriving (filtering) from the source on read, not by trying to keep every copy perfectly in sync.
Trade-offs
| Choice | Buys | Costs | Choose when |
|---|---|---|---|
| Push (fanout-on-write) | O(1) home reads — just a list lookup | write storm ∝ followers; viral posts melt the queue | ordinary authors (followers below threshold) |
| Pull (fanout-on-read) | trivial writes; viral-safe | read-time graph fan-in; cost ∝ followees | viral authors; deep scrollback |
| Hybrid push/pull | bounds both write storm and read fan-in | merge complexity at hydration; per-author classification | real social graphs (the C08 answer, the default here) |
| Hydration-time filter vs timeline rewrite | O(1) deletes/blocks; no second fanout | a few wasted cached IDs; filter on every read | any large-fanout system (always, in practice) |
| Chronological vs ranked home | chrono: predictable, simple MVP | ranked: needs a model, lower predictability | chrono for the interview MVP; ranked for the product |
The sharpest trade-off is hydration-time filtering vs timeline rewrite. Rewrite gives you "clean" stored lists at the price of turning every delete, block, and visibility change into a fresh fanout storm against millions of cached copies — exactly the write amplification you adopted hybrid fanout to avoid. Filtering at hydration accepts that stored lists contain dead pointers, in exchange for making every visibility change a single canonical write. Because reads outrun writes ~250×, you might think doing work per-read is the expensive side — but the filter is a cheap flag check per ID, while the rewrite is millions of list edits per change. Filtering wins decisively, and it's the same call every large fanout system makes.
Failure modes
| Failure | Mitigation |
|---|---|
| Viral fanout storm | Per-author push/pull threshold; viral authors are pull-only, never fanned out. |
| Fanout queue backlog | Async queue absorbs spikes (lesson 11); home is eventually consistent — a few seconds of lag is acceptable, writes never block. |
| Blocked/deleted content leaks into a cached feed | Hydration-time visibility filter against the canonical flag — the cached ID is harmless because it's never served. |
| Hot graph shard (mega-account's followers) | That account is pull-path, so we never read its giant follower set on the hot path (lesson 07). |
| Home cache evicted / timeline gap after outage | Rebuild from profile streams (push side) + viral pull + graph snapshot — home is fully derived data, so it's reconstructible. |
| Ranking service down | Fall back to chronological merge of the candidate set — degraded, not broken. |
Interview Q&A
- "Design Twitter's feed" — where do you start? (senior answer) By naming that "feed" is three timelines: profile (append-by-author, trivial), mentions (write-on-parse, graph-independent), and home (the hard, personalised one). I design home; the other two fall out of the same canonical tweet store. Scoping to the right surface is the first signal.
- Push or pull? (senior answer) Neither alone — hybrid, and I won't re-derive the crossover because it's the C08 result. Ordinary authors push (reads outrun writes ~250×, so pre-compute the join); viral authors pull (a 100M-follower push is a write storm). The threshold is per-author followers, because the graph is power-law and a single global threshold is meaningless.
- Why store post IDs in home timelines instead of the posts themselves? (senior answer) An ID is 8 bytes, a hydrated body is 1–2 KB — about 100–200× larger, multiplied across every follower. Storing IDs and hydrating bodies from one shared tweet cache on read keeps a single copy per tweet instead of millions, and lets visibility/edits stay correct because the body is fetched fresh.
- What happens to my home timeline when I follow someone? (senior answer) I backfill only a bounded recent window of their posts and start receiving their fanout — I never reconstruct their full history. Following is bounded forward work; deep scrollback into their past comes via the pull path, not a rewrite.
- And when I unfollow or block? (senior answer) Stop fanning future posts, and do no eager cache scrub. Anything already cached is filtered at hydration against the block edge and the canonical visibility flag, then cleaned asynchronously. Eager scrubbing of millions of cached lists is the storm we're avoiding.
- A blocked user's reply is sitting in a cached timeline and leaks to the user — fix it. (senior answer) The leak means visibility is being trusted in the stored list instead of enforced at hydration. The fix is not to rewrite the timeline — it's to make hydration the gate: every candidate ID is checked against the canonical visibility flag and the block edge before its body is served, so the offending ID resolves to nothing and is dropped from the response. The cached ID stays put, harmlessly, and an async sweep removes it later. Timeline rewrite would "fix" this one case by re-introducing fanout-scale write amplification on every block — the wrong trade.
- How do you delete a tweet that's been fanned out to 5M timelines? (senior answer) Flip one canonical visibility flag in the tweet store. The 5M cached IDs become stale-but-harmless pointers that hydration drops; a background sweep cleans them lazily. One write, not 5M. This is DDIA Ch. 11's derived-data discipline: re-derive correctness from the source on read.
- Is home timeline strongly consistent? (senior answer) No, and it shouldn't be. Home is derived/materialised data delivered through an async fanout queue; a post arriving a couple of seconds late is fine, and that slack is exactly what lets the queue absorb posting spikes without blocking writers. The strong guarantees live in the canonical tweet store; the timeline is an eventually-consistent view of it.
Related lessons
This case is a satellite of C08 (news feed) — the push/pull crossover is derived there; here we run the same machinery at skewed graph scale and add the three-timelines framing and cache invalidation. The async fanout pipe is lesson 11 (async messaging)'s queue, sized so a posting spike never blocks writers. Hydration's "one cached body per tweet, filter on read" is lesson 06 (caching)'s invalidation problem at fanout scale. Keeping the mega-account's follower set off the hot read path is lesson 07 (partitioning)'s hot-shard avoidance. DDIA threads through: Ch. 1 frames this exact fanout example; Ch. 11 (derived/materialised data) is why hydration-time filtering is correct; Ch. 5 (replication/read replicas) underpins serving 116k reads/s off cached views.