all_lessons / data_intensive_systems / 01 · data models lesson 2 / 35 · ~16 min

Part 1 · A single truthful copy

Data Models: Relational, Document, and Graph

Lesson 00 laid out the spine and its organizing question — where is truth, and what is merely a copy of it? Before any of the machinery that question summons — replication, partitioning, transactions — there is an earlier choice that quietly decides which of those promises are cheap to keep. It is the shape you force your data into. A data model is the abstraction through which you describe the world to the machine, and it makes some questions a single fetch and others a sprawling join across the network. This lesson builds the three dominant shapes — relational, document, graph — by modeling the same domain three ways and watching which queries get cheap and which get expensive.

DDIA grounding

Arc drawn from Designing Data-Intensive Applications, Chapter 2 (Data Models and Query Languages — the relational/document/graph comparison and the object-relational impedance mismatch). The query-language half of that chapter — declarative vs imperative, the access-pattern contract — is the subject of lesson 02. Original synthesis, not a reproduction.

Linear position

Prerequisite: Lesson 00 (orientation) — you carry the track's spine and its central question, where is truth and what is a derived copy, and the instinct that every design choice sends a bill somewhere (latency, storage, ops complexity).
New capability: Given a domain and its hot queries, you can choose a data model — relational, document, or graph — by predicting which operations land on the cheap path and which become network-bound joins or fan-out writes, and you can defend the choice in trade-off terms.

The plan

Five moves. (1) Name the friction the model exists to resolve — the object-relational impedance mismatch — and the two levers that answer it: normalization and denormalization. (2) Build the relational model and the join, and show why many-to-many relationships are where it earns its keep. (3) Build the document model around the aggregate and locality, and show exactly where it breaks. (4) Build the graph model for relationships that are deep, recursive, and evolving. (5) Model one ML domain — a feature entity, its values, and its relationships — all three ways in a worked comparison, then collapse it to a trade-off table and the schema-on-write/schema-on-read axis.

1 · The impedance mismatch, and the two levers

Application code holds data as objects: nested, with lists and references — an Order has a Customer, a list of LineItems, a ShippingAddress. A relational database holds data as flat tables of rows. Bridging the two — flattening the object tree into rows on write, reassembling it on read — is enough recurring friction to have a name: the object-relational impedance mismatch (the term is borrowed from electronics, where mismatched impedance reflects power back instead of transmitting it). Object-relational mapping libraries hide the boilerplate but never fully erase the seam. Every data model is, in part, an answer to how much of that seam you accept.

The two levers you pull against the seam:

A foreign key is the mechanism normalization leans on: a column in one table whose value must match a key in another (orders.customer_id references customers.id), which the database can enforce so you never point at a customer that does not exist. A join is the read-time operation that follows those keys to stitch the rows back into one result. Normalization makes writes cheap and consistent and pushes work onto join-heavy reads; denormalization makes reads cheap and pushes work onto fan-out writes. Hold this tension — every model below is a different default position on it.

2 · Relational: facts split by key, rejoined on demand

The relational model stores the world as relations — tables of rows with named, typed columns — and keeps facts separated. One table per kind of thing; relationships expressed as foreign keys. Its superpower is that the relationships are not baked into the storage layout: the database joins facts at query time, so you can ask questions the schema author never anticipated, and it can enforce invariants (uniqueness, foreign-key integrity) as a property of the data rather than a hope about the application.

This pays off hardest for many-to-many relationships. A one-to-many relationship (one customer, many orders) nests cleanly — the orders "belong inside" the customer. A many-to-many does not nest: an order has many products, and each product appears in many orders; a feature belongs to many models, and each model consumes many features. There is no single tree. The relational answer is a small join table (also called an associative or junction table) holding just the pairs of keys:

RELATIONAL — model_feature_usage as a many-to-many join table models model_feature_usage features +----+--------+ +---------+-----------+ +----+-------------+ | id | name | | model_id| feature_id| | id | name | +----+--------+ +---------+-----------+ +----+-------------+ | 1 | ranker | +----| 1 | 10 |---+ | 10 | ctr_7d | | 2 | recs | | +-| 1 | 11 |-+ | | 11 | dwell_30d | +----+--------+ | | | 2 | 10 | | +->| 12 | embed_v3 | | | +---------+-----------+ | +----+-------------+ each row = one (model, feature) pair; both columns are foreign keys "which models use ctr_7d?" -> filter join table by feature_id=10 -> {1,2} "which features feed ranker?" -> filter by model_id=1 -> join features -> {ctr_7d, dwell_30d}

Both directions of the relationship are equally cheap because neither is privileged by the storage layout — the join table is symmetric. That symmetry is exactly what the document model will lose.

The bill the relational model sends: joins are work. On one machine a join is the query optimizer's problem and usually fine. But once the tables live on different partitions — different machines, a topic owned by lesson 11 — a join becomes data movement across the network, and a clean schema can be correct yet slow because the hot query gathers rows scattered across the cluster. Schema changes also need care: adding a column or splitting a table is a migration, not a no-op.

3 · Document: one aggregate, one fetch

A document model stores a nested aggregate — a self-contained tree of related data treated as a single unit — together in one record, typically as JSON or a binary equivalent. DDIA's canonical fit is a résumé or user profile: one person with a list of jobs, a list of schools, a list of contact methods — a tree of one-to-many relationships that is almost always fetched whole and rarely shared. Stored as one document, the whole profile is a single read with no joins. One order document is the same idea — it holds its customer snapshot, shipping address, and line-item array in one place. This is denormalization made the default: the object tree from your application maps almost directly onto the document, so the impedance mismatch from §1 nearly vanishes for entities you read and write whole.

DOCUMENT — one order is one aggregate, read in a single fetch order:48213 = { "id": 48213, "customer": { "id": 1, "name": "Ada", "tier": "gold" }, <- embedded copy "ship_to": { "city": "Berlin", "zip": "10115" }, "lines": [ { "sku": "embed_v3", "qty": 1, "price": 0.00 }, { "sku": "ctr_7d", "qty": 1, "price": 0.00 } ] } "show order 48213" -> ONE read, no join. Everything is already here (locality).

The key word is locality: because the aggregate is contiguous, rendering the whole order is a single read with no joins — the database touches one place on disk instead of chasing keys across tables. For workloads that read and write an entity as a whole — a user profile, a model card, a serving config, a raw event captured at ingestion — this is the lowest-latency shape there is, and the flexible schema means you can add a field to one document without migrating the others.

Now the break. Document models handle one-to-many well (the line-item array is the "many"), but they struggle precisely where relational shines: many-to-many and shared, mutable facts.

Where the aggregate cracks

Embed a shared fact and updates fan out. The customer name "Ada" is copied into every order document. Ada renames herself — now you must find and rewrite every order, or accept that old orders show a stale name. A relational schema changes it in one row.
Reference instead of embed and you have rebuilt joins by hand. Store "customer": 1 instead of the snapshot, and reading the order's customer name now needs a second lookup — a join, except your application code performs it, without the database's optimizer, indexes, or integrity checks. The locality you bought is gone, and you got none of the relational machinery in exchange.

So the document model's "awkward" operations are joins and fan-out updates. The design rule: a document model wins when the aggregate boundary matches your access pattern — when you almost always touch the whole tree and the cross-references are few. When many-to-many relationships dominate, you are fighting the model.

4 · Graph: relationships as the primary thing

Both models above treat entities as primary and relationships as secondary — keys to follow, or trees to nest. A graph model inverts this: edges are first-class data, as queryable and as richly attributed as the vertices. You reach for it when the interesting question is not "fetch this entity" but "walk relationships whose path and depth I cannot predict in advance" — friends of friends, packages transitively depending on a vulnerable library, the lineage of a feature back to its raw source tables, accounts linked by shared devices in a fraud ring.

Two common flavors:

Property graph
Vertices and edges, each with an id, a label, and arbitrary key/value properties. An edge stores its type, head, tail, and attributes (e.g. derived_from with a transform property). Neo4j and its Cypher language are the canonical example; you match path patterns like (f)-[:derived_from*1..5]->(src).
Triple-store
Everything is a (subject, predicate, object) triple: (ctr_7d, derived_from, clicks_table). Schema-light and uniform — facts and relationships look identical — which suits sparse, heterogeneous, evolving knowledge graphs. Queried with SPARQL; the data model behind RDF.
GRAPH — feature lineage, the recursive traversal that joins do clumsily clicks_table --derived_from--> ctr_7d --feeds--> ranker_model ^ | | +--feeds--> recs_model impressions --derived_from-------/ "what raw sources does ranker_model transitively depend on?" start at ranker_model, walk derived_from edges backward to depth N -> {ctr_7d} -> {clicks_table, impressions} ONE traversal In SQL this is a recursive self-join of unknown depth; in a graph it is the native op.

The thing a graph makes cheap is the variable-depth, recursive traversal. "Which raw tables does this model transitively depend on?" has no fixed number of joins — the chain could be two hops or seven — and a relational engine expresses it as a recursive query that is clumsy to write and often slow. A graph engine treats following edges as the primitive operation, so the traversal is native and the schema can grow new edge types without a migration, which is why graphs suit relationships that keep evolving.

The bill: traversals do not respect partition boundaries — an edge can point anywhere — so graphs are harder to shard and scale horizontally than tables you can split by key (lesson 11). The common production compromise is to keep the system of record relational and build a graph-shaped derived index for the specific traversals that need it — a pattern lesson 20 generalizes as derived data.

A short history — and why these models won

None of this was the first answer. Before the relational model, the dominant database designs of the 1970s were the hierarchical model (data as a single tree, like a document model with no escape) and the network model, standardized as CODASYL — a record could have multiple parents, but the only way to read it was to follow a chain of access paths hard-coded by the programmer, manually navigating from record to record like walking a linked list on disk. Change the query and you rewrote the traversal by hand; the application was the query plan, frozen into procedural code. The relational model won precisely by deleting that work: relations expose no access paths at all, so you state what you want and the query optimizer chooses how to get it — the same declarative-over-imperative shift lesson 02 builds out in full. You no longer navigate; you describe.

The graph model is, in one sense, the network model's idea returning — many-to-many links as first-class data — but this time with declarative traversal languages instead of hand-coded navigation. The three you will meet: Cypher (Neo4j's property-graph language, where you match path patterns), SPARQL (the query language for RDF triple-stores), and Datalog — the oldest and most foundational, a tiny rule language whose ideas underpin modern graph engines. Datalog states facts and rules that derive new facts; a rule can refer to itself, which is what makes recursive traversal natural:

DATALOG — facts plus a recursive rule facts: (these are the edges) derived_from(ctr_7d, clicks_table). derived_from(ranker, ctr_7d). rule: depends(X, Y) :- derived_from(X, Y). (base case) depends(X, Z) :- derived_from(X, Y), depends(Y, Z). (recursive case) \_________ body _________/ read ":-" as "is true IF". The rule says: X depends on Z if X is derived_from some Y that itself depends on Z. query: depends(ranker, ?) -> { ctr_7d, clicks_table } transitively

That recursive rule is the whole motivation for graph query languages: recursive and many-to-many queries — variable-depth lineage, friends-of-friends, transitive dependencies — are exactly what relational SQL expresses clumsily (the recursive self-join of §4) and what these languages make a one-liner. Datalog's rule-and-recursion model is the intellectual ancestor here; lesson 02 picks up the query-language thread in depth.

5 · One domain, three models — the worked comparison

Make it concrete. The domain: an ML feature platform. There are features (e.g. ctr_7d), entities the features describe (a user, an item), the feature values for each entity at a point in time, and relationships — which features feed which models, and which features are derived from which upstream features. Here are the four hot queries we care about:

SAME DOMAIN, THREE SHAPES RELATIONAL entities | features | feature_values(entity_id,feature_id,value,ts) | model_feature_usage(model_id,feature_id) | feature_lineage(child,parent) DOCUMENT user:1 = { id, features: { ctr_7d: 0.21, dwell_30d: 42, ... } } <- one aggregate per entity GRAPH (user)-[:has_value]->(reading) (feature)-[:feeds]->(model) (feature)-[:derived_from]->(feature|source)

Now run the four queries through each model and watch the cost flip:

QueryRelationalDocument (per-entity aggregate)Graph
Q1 serve — all values for one userFilter feature_values by entity_id; fast with the right index, but a row-per-feature scanCheap — one document fetch, all values contiguous (locality)Traverse one vertex's has_value edges — fine, but heavier than a single read
Q2 reverse — models using ctr_7dCheap — filter the join table by feature_id; symmetricExpensive — the relationship is not in the per-user docs; scan every model doc or keep a separate indexCheap — follow feeds edges out of the feature
Q3 update — feature renamedCheap — one row in features; everyone references by idExpensive — if the value docs embed the feature name, rewrite every user document (fan-out)Cheap — one vertex property
Q4 lineage — transitive sourcesRecursive self-join of unknown depth — correct but clumsy and often slowNot expressible without rebuilding joins in app codeCheap — native variable-depth traversal of derived_from

No model wins all four. The document model owns the serving read (Q1) and loses the relationship queries (Q2, Q4). The relational model owns the many-to-many and update queries (Q2, Q3) and is clumsy on recursion (Q4). The graph owns recursion and relationships and is merely adequate at the bulk single-entity read (Q1). This is why real feature platforms are polyglot: the system of record is relational, the online serving layer is a denormalized document/key-value store rebuilt from it (the aggregate per entity for Q1), and lineage lives in a graph-shaped derived index for Q4.

Worked number — the fan-out cost of denormalizing

When denormalization stops paying

Say you embed the feature name into every per-user value document to make Q1 self-contained. You have 50,000,000 user documents, each embedding 200 feature names. A feature definition changes 20 times a year. Each change must rewrite the embedded copy in every document that holds it — on average 50,000,000 docs touched per rename: 20 renames × 50,000,000 = 1,000,000,000 document rewrites/year purely to keep copies in sync. Reference the feature by id instead (normalize) and each rename is 20 × 1 = 20 row updates — at the cost of one extra lookup to resolve the name on read. The denormalized read is faster per query; whether it is worth a billion background writes depends on your read:write ratio. If Q1 runs 100,000×/sec and renames are rare, embedding still likely wins; if names changed hourly, normalize. The model choice is this arithmetic, not a religion.

Schema-on-write vs schema-on-read

One more axis cuts across all three. Schema-on-write (the relational default) validates structure when data enters: the table declares its columns and types, and a malformed row is rejected at the door. The schema is explicit and enforced — you cannot insert an order without a valid customer_id. Schema-on-read (common in document stores and raw data lakes) accepts whatever you write and interprets structure only when the data is read: the documents in a collection need not share fields, and the consumer copes with the variety.

Neither is "schemaless" — schema-on-read just moves the schema from the database into the application code that reads it. The trade is the familiar one: schema-on-write catches errors early and makes the data self-describing, at the cost of a migration whenever the shape changes; schema-on-read lets heterogeneous and evolving data land without ceremony — ideal at ingestion boundaries where you do not yet know the final shape — at the cost of every reader needing to defend against surprises. ML pipelines often use both deliberately: raw events land schema-on-read in a document/lake form to preserve messy source shape, then get validated and flattened into schema-on-write relational/columnar tables for training and analysis.

Failure modes

  • Document store doing a relational job. Many-to-many relationships forced into aggregates means either fan-out updates on shared facts or hand-rolled joins in app code — you pay the relational cost without the relational machinery.
  • Relational store doing a graph job. Deep recursive traversals expressed as recursive self-joins of unknown depth: correct, clumsy, and frequently slow on the hot path.
  • Embedding mutable shared facts. Denormalizing a value that changes (a name, a price, a definition) turns every update into a fan-out and invites silent drift between copies.
  • Schema-on-read everywhere. No structural validation at ingest means malformed records propagate until some reader crashes far downstream, where the cause is hardest to trace.
  • Graph as system of record at scale. Traversals ignore shard boundaries; a giant write-heavy graph is painful to partition. If you only need a few traversals, a derived graph index beats making the graph authoritative.

Decision checklist

  • List the 3-5 hottest queries first. Which entity boundary do they touch, and in which direction?
  • Is the dominant relationship one-to-many (nests — favors document) or many-to-many (favors relational/graph)?
  • Do the hot reads touch a whole self-contained aggregate? If yes, locality argues for a document shape.
  • Are there variable-depth, recursive traversals on the hot path? If yes, a graph (possibly derived) earns its place.
  • For any fact you consider embedding: how often does it change, and how many copies would a change touch? Run the fan-out arithmetic.
  • Where should the schema be enforced — at write (early errors, migrations) or at read (flexible ingest, defensive readers)?
  • Could you keep one authoritative model and derive the others as indexes rather than maintaining several systems of record?

Checkpoint exercise

Try it

Take a RAG corpus: documents, their chunks, the embeddings of those chunks, the source URLs, and the citation links between documents. Sketch which parts are best as relational rows, which as document aggregates, which as a graph (the citation links), and which as a vector index. For the hot query — "retrieve the top-k chunks for a query, with their source document metadata" — name the access pattern, decide whether to denormalize the source metadata into each chunk, and justify the choice using the fan-out arithmetic from §5. Then state where you would enforce schema-on-write vs schema-on-read across the ingest-to-serve path.

Where this points next

You can now pick a shape for the data. But a shape is silent until you can ask it questions, and the three models come with sharply different querying philosophies: SQL describes what result you want and lets the optimizer choose the plan, while a graph traversal or a hand-written document scan often describes how to get it. That split — declarative versus imperative — is not cosmetic: it decides who owns the execution plan, the application or the engine, and therefore who can make a query faster without rewriting the app. Lesson 02 takes the models you just built and turns to the languages over them, where access-pattern design becomes part of the contract.

Where is truth?

For a data model, truth lives in the system of record: the normalized relational tables where each fact is stored exactly once and integrity is enforced (one features row, one customers row). The copies / derived views are the denormalized shapes built for access patterns — the per-entity document/key-value aggregate for whole-entity serving reads, and the graph-shaped index for lineage traversals. Freshness budget: how stale a denormalized copy may run after a normalized fact changes (often sub-second for serving, looser for lineage). Owner: the team that owns the authoritative schema and the pipelines that rebuild the derived shapes. Deletion path: delete the fact once in the system of record, then propagate to every embedded copy — the fan-out arithmetic of §5 is exactly this cost. Reconciliation / repair: rebuild each derived view from the normalized source rather than patching copies in place. Evidence it is correct: foreign-key and uniqueness constraints on the system of record, plus row-count and checksum reconciliation between source and each derived copy.

Takeaway

A data model is the shape that makes some questions a single fetch and others a network-bound join, and the central tension is normalization (store each fact once, pay on join-heavy reads) versus denormalization (copy facts where they are read, pay on fan-out writes). The relational model splits facts by key and rejoins on demand — it owns many-to-many relationships and cheap shared-fact updates, at the cost of join work that turns into data movement across partitions. The document model stores a nested aggregate for locality — it owns whole-entity reads and flexible schema, but cracks on many-to-many and on mutable shared facts that must fan out. The graph model makes edges first-class — it owns deep recursive traversals and evolving relationships, at the cost of being hard to partition. No model wins every query in the worked feature-platform comparison, which is why production systems keep one authoritative model and derive the others, and why the right answer is fan-out arithmetic over your hottest queries, not a favorite paradigm. Schema-on-write catches errors early; schema-on-read defers structure to flexible ingest — the schema never disappears, it just moves into the reader.

Interview prompts