Design a news feed
Reads vastly outnumber writes, so you precompute each user's timeline at write time — until one author has ten million followers and that single post becomes ten million writes. The whole design is the search for where to stop pushing and start pulling.
Kleppmann opens Designing Data-Intensive Applications (Ch. 1) with precisely this example — Twitter's home timeline — and frames it as the choice between two approaches: query the union of everyone you follow at read time, or maintain a per-user cache that gets a new entry written into it every time someone you follow posts. He notes Twitter shipped approach 1 first, switched to approach 2 for read performance, and then had to go hybrid because of celebrities. We are going to re-derive that whole arc from the numbers.
1. Clarify the contract
Treat the prompt as a product contract before drawing a single box. The system must:
- Publish a post from a user (text + media ref), durably and once.
- Read a personalized home timeline — posts from accounts you follow — with low p99 latency and stable cursor pagination.
- Mutate the follow graph (follow / unfollow), with the timeline reflecting the change within a bounded lag.
- Stay reasonably fresh — seconds-to-minutes lag is acceptable; we are not building a chat system (C09).
- Survive accounts with enormous follower counts without the publish path falling over.
And — just as important — what it need not do. It need not deliver posts in strict global order (a feed is a personalized view, not a ledger). It need not be transactional across users. It need not guarantee that two users see the same ranking. And it need not show every post — a feed is allowed to drop, dedupe, and rank; missing one post from three days ago is invisible, which is exactly the slack a hybrid design exploits.
2. Put numbers on it
The architecture is forced by arithmetic, so do the arithmetic first. Assume a large social network: 100M daily active users, each loads the feed ~10×/day, each posts ~0.2×/day.
Reads outnumber writes ~50:1. That ratio is the green light for fanout-on-write: do the expensive work on the rare event (the post), so the common event (the read) is a cheap cache lookup.
The fanout cost — where it all hinges
A fanout-on-write post costs one cache append per follower. So the cost of a single post is the author's follower count:
- Ordinary user, 200 followers: 200 cache writes/post. At 230 posts/s with a median of ~200 followers, that's ~46,000 timeline writes/s across the fleet — large but utterly routine for a sharded cache. Cheap.
- Celebrity, 10M followers: 10,000,000 cache writes for one post. At any realistic posting rate this is pathological: a single celebrity tweet generates more write traffic than the entire ordinary population, and it lands as a thundering-herd spike on the fanout workers and the cache shards.
- Push (fanout-on-write): cost ∝ followers F. One append per follower: F writes.
- Pull (fanout-on-read): the post is written once; readers merge it in. Cost ∝ how many of this author's followers actually read in the post's lifetime. If a follower reads ~10×/day and the post is "live" in the feed for ~1 day, that's ~10·F merge operations — but each merge is amortized across all the authors that reader follows (one read fetches one celebrity's recent posts alongside everyone else's), so the marginal per-author read cost is tiny and bounded.
The timeline cache budget
Each user's home timeline is a cache of recent post IDs (not bodies — hydrate bodies at read time from the post store, DDIA Ch. 3's separation of the index from the data). Size it:
400 GB of timeline cache, sharded by user ID across a Redis-style fleet, is entirely manageable — a few dozen nodes. Capping each timeline at ~500 IDs is what keeps it bounded: a feed nobody scrolls past entry 500 doesn't need entry 5,000 precomputed.
3. Data model & API
Keep the API expressing the product operation; never leak the queue, cache, or shard layout.
| API / operation | Contract |
|---|---|
POST /posts | Append a post for the author; returns a post ID. Triggers fanout (or doesn't, if author is high-fanout). |
GET /feed?cursor=… | Return the next page of the caller's home timeline, hydrated and ranked; returns a stable cursor. |
follow(user, target) / unfollow | Mutate the follow graph; schedule a timeline backfill so the change appears within the freshness budget. |
Four data objects, each with a clear owner:
| Object | Stored as | Why |
|---|---|---|
| Post | Canonical store, sharded by post ID: {post_id, author, body, media_ref, ts}. | The single source of truth. Everything else is derived from this + the graph. |
| Follow edge | Graph store, indexed both directions (followers(u) and following(u)). | Fanout needs followers; read-time merge needs following. |
| Home timeline | Cache of ~500 post IDs per user (push targets write here). | Derived data — the materialized feed. Rebuildable. |
| Author class | Flag/threshold on follower count (normal vs high-fanout). | Decides push vs pull per author at publish time. |
4. Linearized design
Walk a post and a read through the system in the order events actually occur; the bottleneck exposes itself.
- 1. Write the canonical post once. Durable, idempotent on a client-supplied key, in the post store. This step is identical for everyone and is the only step that must succeed for the post to exist.
- 2. Classify the author. Look up follower count. If below the threshold (~10k), enqueue a fanout job. If above, do nothing further at write time — the post stays in the post store and will be merged at read.
- 3. Fanout job (push path). A worker reads
followers(author)and appends the post ID to each follower's home-timeline cache, trimming to ~500. This runs async off a queue (C11 / lesson 11) so the publish API returns fast and spikes are absorbed by queue depth, not by the user. - 4. Feed read (pull merge). Fetch the reader's precomputed timeline IDs, then fetch recent posts from the small set of high-fanout authors this user follows, merge the two by timestamp/score, hydrate bodies from the post store, rank a bounded candidate set, and return with a cursor.
- 5. Backfill on graph change. A new follow schedules a backfill that pulls the followee's recent posts into the new follower's timeline; unfollow lazily filters at read time.
- 6. Observe (lesson 16). Track fanout queue depth, freshness lag (post-time → in-timeline), and read-merge fallback rate. Queue depth is the early-warning signal that a celebrity just posted.
5. Deep dives
Why hybrid is the senior answer, not pure push or pure pull
The two pure strategies each fail at one end of the follower distribution. Pure push (fanout-on-write for everyone) gives O(1) reads but punishes celebrity writes: one post = 10M cache writes, a queue spike that starves ordinary users' fanout and can take minutes to drain, all to update timelines most of whose owners will not read in that window. Pure pull (fanout-on-read for everyone) makes writes trivial but makes every read expensive: a user following 500 accounts must, on each load, query 500 authors' recent posts and merge them — 11,600 reads/s each doing a 500-way scatter-gather is the read-amplification mirror of the write problem. Neither extreme survives the follower distribution, which is heavy-tailed: most accounts are tiny, a handful are gigantic.
The hybrid routes each author to the cheap strategy for that author. Below the crossover, push (write cost is small because F is small). Above it, pull (read cost is amortized because only a few celebs are merged, alongside the precomputed bulk). The reader's feed is therefore the union of one cheap cache read (everyone normal they follow, precomputed) plus a tiny scatter (the handful of celebrities they follow). DDIA Ch. 1 names this exactly: most users are fanned out on write; the few with huge audiences are fetched and merged at read time. This is the answer that signals you understand the distribution, not just the mechanism.
The home timeline is derived data — design for rebuild
The single most stabilizing idea: the home timeline cache holds no ground truth. The truth is the post store plus the follow graph; the timeline is a materialized view of "recent posts from accounts I follow." DDIA Ch. 11 frames fanout-on-write as exactly this — materialized-view maintenance, where each write incrementally updates the view. The consequence is freeing: if a timeline is wrong (a fanout job was lost, a cache shard was wiped, a deploy bug double-appended), you do not need a transaction or a repair protocol — you rebuild it by re-running the query: take the user's following set, pull recent posts, merge, write. Because it is derived, it is disposable, and disposable data is recoverable data.
This is why the timeline can be a volatile cache rather than a durable store, why fanout can be best-effort async, and why losing a fanout job is an availability/freshness blip rather than data loss. It also dictates correctness at read time: timelines hold IDs, so a deleted post is handled by hydrating from the canonical store and dropping tombstones — the stale ID in the cache is harmless because the body is fetched fresh. Design the derived layer so any inconsistency is self-healing on the next rebuild.
The retrieval / ranking boundary
A common failure is to conflate "find candidate posts" with "rank the feed." Keep them separate. Retrieval collects candidates cheaply — read the precomputed timeline, merge the celeb pulls — producing maybe a few hundred candidates. Ranking then scores a bounded set (top N candidates by recency/affinity) with whatever model the latency budget affords. The critical line: you rank a bounded candidate set, never the universe. Synchronously fetching features and scoring every possible post for every reader at 11,600 reads/s blows the latency budget instantly; "rank everything" is not a design, it is a denial-of-service against yourself. A heavy ranker is a choice you make after retrieval has already bounded the work — and it is the first thing you shed under load (fall back to recency sort), because lesson 06's caching and a cheap chronological merge keep the feed serving even when the ranker is down.
6. Trade-offs
| Choice | Buys | Costs | Choose when |
|---|---|---|---|
| Fanout-on-write (push) | O(1) cache-lookup reads | write amplification ∝ followers | author below the ~10k crossover |
| Fanout-on-read (pull) | O(1) writes; no amplification | scatter-gather + merge on every read | author above the crossover (celebrity) |
| Timeline cache vs query-on-read | low p99 reads | derived-data staleness; cache memory | read:write ratio is high (it is, ~50:1) |
| Heavy ranker vs recency sort | better relevance/engagement | latency + a sheddable dependency | read budget allows; degrade to recency under load |
The sharpest trade-off is the push/pull split itself, and the subtlety interviewers probe is that the crossover is not a constant you set once. It depends on the author's follower count, the read:write ratio of that author's audience, and current queue pressure. A clean design exposes the threshold as a tunable and even lets a borderline author flip strategies dynamically: if the fanout queue is backing up, raise the threshold so more authors fall into the pull path and the write storm subsides. The hybrid is not "push for some, pull for others, decided forever" — it is a continuously-evaluated routing decision per author.
7. Failure modes
| Failure | Mitigation |
|---|---|
| Celebrity post → fanout storm | Classify high-fanout authors out of the push path entirely; merge their posts at read. The write that never happens cannot overload anything. |
| Fanout queue lag (freshness slips) | Expose freshness lag as an SLO (16); prioritize the queue (recent followers first) and shed/defer low-value fanout under pressure. |
| Deleted / edited post still referenced in timelines | Timelines hold IDs only; hydrate from canonical post store at read and drop tombstones. Stale IDs are harmless. |
| Lost fanout job / wiped cache shard | Timeline is derived — rebuild from following + post store. No repair protocol, no data loss. |
| Follow-graph race (new follow misses recent posts) | Schedule a backfill on follow; make cursors stable so concurrent fanout + backfill don't double-show or skip. |
8. Interview Q&A
- A user with 50M followers posts — design the write path. (senior answer) Do not fan out. The publish writes the canonical post once and stops, because the author is classified as high-fanout (far above the ~10k crossover). 50M cache appends per post is write amplification that would starve the queue and the cache fleet to update timelines most owners won't read. Instead the post lives in the post store and is merged at read time: when any of those 50M followers loads their feed, the reader's path pulls this author's recent posts alongside their precomputed timeline and merges by score. The cost moves from one giant synchronous storm to tiny amortized per-read merges — and only for followers who actually read.
- Where is the push/pull crossover and why? (senior answer) Around ~10k followers. Push costs F writes per post; pull costs roughly one bounded merge per reader-load, amortized across all authors a reader follows. Below ~10k, writing to every follower once is cheaper than making every follower pull; above it, write amplification dominates and the post should sit in place. It's a tunable, not a constant — raise it under queue pressure to push fewer authors.
- Reads outnumber writes 50:1 — why not pure fanout-on-write for everyone? (senior answer) The 50:1 ratio justifies precomputing in general, but the follower distribution is heavy-tailed: a few accounts have millions of followers, so their writes are 10M× an ordinary write. Pure push optimizes the average and dies on the tail. The hybrid optimizes per-author by routing celebrities to pull.
- The timeline cache for half your users gets wiped. How bad is it? (senior answer) A freshness/availability blip, not data loss. The timeline is derived data — a materialized view of posts ⋈ follow graph (DDIA Ch. 11). Rebuild each affected timeline by re-running the query (their
followingset + recent posts). No transaction, no repair protocol; that's the whole point of treating the feed as disposable. - How do you keep ranking from blowing the latency budget? (senior answer) Separate retrieval from ranking. Retrieval cheaply collects a few hundred candidates (cache read + celeb merge); ranking scores only that bounded set. Never rank the universe. The ranker is a sheddable dependency — degrade to recency sort under load, which keeps the feed serving.
- A user follows someone — when do their old posts appear? (senior answer) Schedule a backfill that pulls the followee's recent posts into the new follower's timeline within the freshness budget; if the followee is a celebrity, nothing to backfill since they're merged at read anyway. Unfollow is lazy — filter at read rather than scrubbing the cache. Stable cursors prevent the concurrent backfill from causing duplicates or gaps.
- How fresh does the feed need to be, and how does that buy you slack? (senior answer) Seconds-to-minutes lag is fine — a feed is not a chat (C09) or a ledger. That tolerance is exactly what lets fanout be async (lesson 11), the timeline be a volatile cache, and lost jobs be recoverable. Tight freshness would force synchronous fanout and remove every escape valve.
- Why store post IDs in the timeline rather than post bodies? (senior answer) Separation of index from data (DDIA Ch. 3). IDs keep the 400 GB cache small and let bodies be hydrated fresh — so deletes/edits are reflected automatically and a stale ID is harmless. Storing bodies would multiply cache size and require invalidating every copy on every edit.
Related lessons
This case leans on the foundations and anchors a concept others reuse: timelines are a read cache, so the hit-rate → origin-load reasoning of lesson 06 (caching) governs how often a read falls through to the post store; the fanout job is an async pipeline absorbing publish spikes, which is lesson 11 (async messaging)'s queue-as-shock-absorber — the same machinery C11 (messaging fanout) scales to a production social graph; and both the post store and the timeline cache are sharded by user/post ID, so the hot-shard pitfall of lesson 07 (partitioning) is exactly what a celebrity's followers would trigger if you pushed to them. Forward to C11, which treats this push/pull split as fanout at messaging scale.