Orientation · how to read this track
Orientation: the spine and "Where is truth?"
The neighboring tracks on this site assume the substrate: backend system design picks Postgres or Cassandra and moves on; the data-engineering track builds pipelines on top of a log it does not open; the ML-systems track trains on a feature store and serves from a vector index as if those were primitives. This track is the layer underneath all three. It opens the box. By the end you should be able to look at any of those products — a database, a queue, a cache, a search index, a feature store, a vector DB — and see the same handful of moves: which facts are authoritative, which reads are cheap, which writes are cheap, how much machines must agree, and what breaks when one of them dies.
New capability: You can place any storage or messaging product into one design space, name the single contract it offers, predict the bill it sends elsewhere, and answer one question about it — where is truth? — so the rest of the track reads as derivations, not a parade of products.
1 · What "data-intensive" means, and the one picture
An application is compute-intensive when its hardest problem is doing arithmetic fast — a physics simulation, a renderer, a training loop's matmuls. It is data-intensive when CPU is rarely the bottleneck and the hard problems are instead the amount of data, the complexity of the data, and the speed at which it changes. Almost every product you ship is the second kind. The difficulty is not computing an answer; it is storing facts durably, moving them between machines, interpreting them consistently as schemas drift, and getting many machines to agree on them when some are slow or dead.
Strip a data-intensive system to its skeleton and you always find the same shape. There is a system of record — the authoritative copy of the facts, the one you would rebuild everything else from after a disaster. Then there are derived views: copies shaped for a specific read pattern (a search index for full-text queries, a cache for hot keys, an analytics table for aggregation, an embedding index for similarity, precomputed features for a model). Derived views are never authoritative; they are functions of the system of record, kept in sync by a flow of changes. That flow — the change log — is the spine of the entire track.
Read that diagram as a promise and a hazard. The promise: each derived view gets exactly the layout its reads want, so each read is cheap. The hazard: the moment there is more than one copy, they can disagree — a view can lag, drop a change, or apply one twice — and now "what is true?" has more than one answer. Every later lesson is either (a) making one copy faster, safer, or more durable, or (b) keeping the copies honest with each other. Hold onto the rule on the last line: a derived view you cannot recompute from the log is not a view, it is a second source of truth you forgot you had.
2 · The Linear Spine: one truthful copy, pressured step by step
This track is one long argument, not a list of topics. It starts from the simplest correct system imaginable — one machine, one database, one copy of every fact, every operation a local transaction — where nothing is stale, nothing disagrees, and "what is true?" has exactly one answer. Then it applies pressure to that single truthful copy, one step at a time, and watches a region of the design space open up at each step. That ordered sequence of pressures is the spine. Each step is realized by one or two lessons; the order is the order of least new assumptions per step, because the dependencies are real — you cannot reason honestly about replication before you know what a durable log is, and you cannot reason about linearizability before you have felt replication lag.
The same eleven steps, named precisely, with the lessons that realize each:
Every named product is just a point — or a path — through this spine. A single-node Postgres lives at step 1 and pays nothing for distribution while buying clean transactions (L13). Cassandra trades those clean transactions for leaderless quorum replication that survives node loss (L09). Kafka is the change log itself, promoted to a first-class durable object (L19). A vector index for RAG is a derived view (step 9) whose freshness is a stream-processing question (L19) and whose recompute is a batch question (L18). You will not memorize these verdicts; you will be able to re-derive them from where each product sits on the spine.
If you only have time for the load-bearing steps: L03 (the durable ordered log, the object the whole track keeps reusing), L08 (replication lag — the first place "more than one copy" hurts), L16 (linearizability — the strongest single-copy illusion and what it costs), and L20 (how derived views stay correct end to end). The other lessons earn those four.
3 · The one reading habit: contract bought at a cost
Here is the single mental move that makes the rest of the track cheap. Every mechanism is a contract bought at a cost charged somewhere else. There is no free win; there are only relocations of cost. When you meet a new mechanism, ask three questions in this order — and refuse to name the tool until you have answered the first:
The trade-off table below is the same habit applied to the corners of the spine. Each row is a contract and its bill; the rest of the track fills in the mechanics behind each.
| Choice | Buys (contract) | Costs (the bill) |
|---|---|---|
| One database (no distribution) | One source of truth, clean local transactions, nothing can disagree | Bounded by one machine's capacity and one failure domain; one layout must serve every read |
| Replicate the data | Read throughput and availability — a copy survives a node loss | Replication lag: copies disagree for a window, so reads can be stale (L08) |
| Strong coordination (consensus, linearizability) | A single global order; the system behaves like one machine | Latency on every operation and reduced availability during a partition (L16–L17) |
| Asynchronous derived views | Fast writes, decoupled systems, each read pattern gets its own cheap layout | Readers see stale or partially-updated views; cross-system correctness becomes the hard problem (L18–L20) |
A worked instance to make the habit concrete. Suppose a feature store serves 50,000 reads/sec to an online ranker with a 10 ms p99 budget, and a single primary database tops out at 20,000 reads/sec. The constraint is read throughput (2.5× over capacity) under a tight latency tail. The tool is read replicas: add four followers and you have 5 × 20,000 = 100,000 reads/sec of headroom, well inside budget. The contract is "more read capacity, and a copy survives a primary crash." The bill arrives as replication lag: if a follower is 200 ms behind the primary, a user who just updated their profile and is re-ranked immediately may be scored on yesterday's feature. That single number — 200 ms of staleness — is the price of the throughput, and L08 is where you decide whether you can pay it or must route that one read to the primary (read-your-writes). You bought a contract; the bill landed on freshness. That is the whole track in one example.
4 · The recurring question: "Where is truth?"
The spine has exactly one organizing question, and it is concrete enough to answer for any mechanism you meet: where is truth, and what is merely a copy of it? Every lesson in this track ends with a small box that answers that question for the mechanism it just taught. The box names eight things, and naming them is the single most reliable way to find the bug before it ships.
What the box names, every time
- System of record — the one authoritative copy you would rebuild everything else from.
- Copies / derived views — every other copy of these facts, and whether each is recomputable from the log.
- Freshness budget — how stale a copy is allowed to be (a lag in ms, a window, "eventual").
- Owner — the team or service accountable for the record and for keeping the views honest.
- Deletion path — how a fact is truly removed everywhere, including every derived view (the GDPR question).
- Reconciliation / repair path — how a drifted copy is brought back into agreement (re-sync from log, anti-entropy, rebuild).
- Evidence it is correct — the check that proves the copies agree (checksums, row counts, audit, replay).
How to read every later lesson
- Name the constraint (workload number / invariant / failure model / freshness SLA) before naming any tool.
- State the contract the mechanism offers as a testable promise.
- Locate the bill: read latency, write latency, storage, availability under partition, freshness, or operational complexity.
- Then fill in the "Where is truth?" box — if you cannot name the system of record and the repair path, you do not yet understand the mechanism.
- Watch which step of the spine the lesson is answering, and which new pressure its answer exposes.
To anchor the artifact, here is the box for the orientation itself — the picture from §1, read as a "Where is truth?" answer. Every lesson's box has this exact shape, specialized to its mechanism.
Copies / derived views: the search index, cache, analytics table, feature store, and vector index — each a function of the record, each must be recomputable from the change log.
Freshness budget: set per view by its read pattern — a cache may tolerate seconds, an online feature may demand a tight lag, an analytics rollup may be hourly.
Owner: the team that owns the system of record owns its truth; each derived view has an owner accountable for keeping it honest with the record.
Deletion path: a delete on the record must propagate through the change log to every view, or a "deleted" fact survives in a cache or index — the failure that becomes the privacy lessons (L25).
Reconciliation / repair path: rebuild any view by replaying the change log from the record; a view that cannot be rebuilt this way is a hidden second source of truth.
Evidence it is correct: you can recompute each view from the log and get the same bytes — replay, checksum, or recount. If you cannot, you do not actually know it is correct.
Checkpoint exercise
Where this points next
We have the spine, a reading habit, and the one question that ties them together. But the very first step of the spine — "a single truthful copy" — hides a choice that quietly decides which later promises are cheap to keep: the shape you force your data into. Lesson 01 builds the three dominant shapes — relational, document, and graph — by modeling the same domain three ways and watching which queries get cheap and which become network-bound joins or fan-out writes. That is where the copy stops being an abstraction and acquires a structure you will live with for the rest of the track.
Interview prompts
- What makes an application "data-intensive" rather than "compute-intensive," and why does that change the hard problems? (§1 — the bottleneck is the amount, complexity, and rate-of-change of data, so the hard problems are durability, movement, interpretation, and agreement across machines and time — not CPU.)
- Sketch the universal shape of a data system. (§1 — a system of record (authoritative, durable) emits an ordered durable change log that feeds derived views (index, cache, analytics, features); every derived view is a cache of the record and must be recomputable from the log.)
- Walk the eleven steps of the spine and why the order is forced. (§2 — model it, let it age, meet demand, add redundancy, distribute, hold invariants, survive failure, then derive history, derive views, go cloud, answer society; the order minimizes new assumptions per step — you need the durable log (L03) before replication (L08), and replication lag (L08) before linearizability (L16).)
- State the three-question reading habit for any mechanism. (§3 — (1) what constraint forces it (name it before the tool); (2) what testable contract it offers; (3) where the bill lands — read/write latency, storage, availability under partition, freshness, or operational complexity. No mechanism is free.)
- What does the "Where is truth?" box make you name, and why does it catch bugs? (§4 — system of record, copies/derived views, freshness budget, owner, deletion path, reconciliation/repair path, evidence of correctness; if you cannot name the record or the repair path, you do not yet understand the mechanism.)
- A feature store needs 50k reads/sec but the primary serves 20k; you add read replicas. What did you buy and what is the bill? (§3 — bought read throughput + availability; bill is replication lag, so a just-written feature may be read stale until the follower catches up — possibly forcing a read-your-writes route to the primary, L08.)
- Why is it dangerous to treat a search index or vector DB as authoritative? (§1, §4 — it is a derived view; if it cannot be recomputed from the system of record it is a hidden second source of truth, so when it drifts there is no clean answer to "where is truth?")