all_lessons / data_intensive_systems / 33 · case: model registry lesson 34 / 35 · ~18 min

Part 12 · Applied & interviews

Interview Case: Model Registry

Lesson 32 solved the dual-write trap: never write your source database and a derived search index in two separate steps — make one of them the log and let the other follow. This case applies that same discipline to a different artifact. A model registry is the system of record for trained ML models: the large immutable weights sitting in blob storage, the small critical metadata (version, metrics, lineage, stage) describing them, and — the part that bites in production — a single pointer that says which version is live right now. The crux is not storing files. It is making "promote version N to production" an atomic, linearizable flip that every serving replica agrees on, while keeping every artifact immutable so a rollback is just pointing back at a version that still exists, byte-for-byte.

DDIA source
Arc drawn from Martin Kleppmann, Designing Data-Intensive Applications — append-only logs and immutability (Ch.3, our L03), linearizability (Ch.9, L16), consensus and coordination services like ZooKeeper/etcd (Ch.9, L17), and derived data / lineage (Ch.12, L20). The model-registry framing is ours; the mechanisms are general.
Linear position
Prerequisite: append/log thinking and immutability (L03); linearizability — a register whose reads always see the latest committed write (L16); consensus and a coordination service that holds an authoritative value with watches (L17); derived data and lineage (L20); the no-dual-write discipline (L32).
New capability: design a registry that separates a strongly-consistent metadata store from a durable blob store, makes artifacts content-addressed and immutable, exposes one linearizable production pointer per model so the whole serving fleet flips atomically, records enough lineage to answer "what produced this model", and rolls back safely in seconds.
The plan
Six moves, the shared case template. (1) The prompt, the functional/non-functional requirements, and the clarifying questions a strong candidate opens with. (2) Back-of-envelope sizing — model count, artifact size, promotion frequency, read QPS from the serving fleet. (3) The design walk, building the registry from the track's mechanisms with an ASCII architecture diagram. (4) One concrete failure timeline — a split-brain promotion — and its mitigation. (5) A four-bucket answer rubric: good, better, red flags, likely follow-ups. (6) Takeaway plus interview prompts with answers.

1 · The prompt and what to ask first

Prompt. "Design a model registry. It is the system of record for our trained ML models. Teams push model versions; the registry stores the weights and the metadata about each version; and there is a notion of which version is currently serving production traffic. Make promotion and rollback safe, and make sure the whole serving fleet agrees on the live version."

The weak candidate starts drawing an S3 bucket and a Postgres table. The strong candidate first separates the two halves of the problem — a small but critical metadata store and a large but simple artifact store — and then asks the questions that determine the consistency model.

Functional
Register a new immutable version (weights + metadata). List/query versions and their metrics. Promote a version to a stage (staging, production). Roll back. Read "which version is production for model M" — the call the serving fleet makes. Read the artifact bytes for a given version. Record lineage: training data snapshot, code/commit, hyperparameters, parent version.
Non-functional
Artifacts immutable — a version's bytes never change once registered. Promotion atomic and linearizable — no replica ever serves a half-applied flip; once promotion returns, every reader sees the new version. Durable artifacts (you can never lose a production model's weights). Auditable — who promoted what, when, from which lineage. Fast, high-QPS reads of the production pointer; promotion itself is rare.
Clarifying questions
How big are artifacts — hundreds of MB or tens of GB? How many models and versions total? How often does a promotion happen — hourly or weekly? How many serving replicas read the pointer, and how do they react to a flip (poll, or get pushed)? Is a stale read of the production pointer for a few seconds acceptable, or must it be linearizable? Do we need point-in-time "what was production on date D" for audit?
The reframe
This is two stores with opposite needs glued by one pointer. The blob store wants throughput and durability and does not need transactions. The metadata store wants strong consistency for a tiny amount of data. The pointer flip is the only operation that needs linearizability and consensus. Get those boundaries right and the rest is plumbing.

The single most important clarification is the one about the production pointer's consistency. If the answer is "a few seconds of staleness is fine" you can serve the pointer from an eventually-consistent cache. If the answer is "every replica must agree the instant promotion returns" — the usual answer for production model serving — you need a linearizable register backed by consensus, and that decision shapes the whole design.

2 · Back-of-envelope sizing

Numbers turn vague requirements into a concrete design. Take a mid-size ML org.

Worked sizing
Inventory. Say 500 models, each with ~40 retained versions = 20,000 artifacts. Average artifact 2 GB (a few are 30 GB LLMs, most are small trees and rankers). Artifact bytes ≈ 20,000 × 2 GB = 40 TB raw; at 3× replication for durability that is 120 TB in the blob store — large, but trivially within object-storage scale.

Metadata. Each version's metadata row (version, metrics, lineage refs, stage, checksums) is a few KB. 20,000 × ~4 KB ≈ 80 MB. The entire metadata store fits in RAM many times over. This is the key asymmetry: 40 TB of artifacts, 80 MB of metadata. They do not belong in the same engine.

Production pointers. One small record per model — "model M → version V" — so 500 pointers, a few hundred bytes each. Tiny. This is the linearizable state, and it is essentially nothing in size.

Promotion frequency (write rate). Even a busy org promotes maybe 50 times a day across all models — under 0.001 writes/sec to the pointer set. Promotion is rare. Consensus is famously slow (a round-trip among a quorum, see L17), but at this write rate its cost is irrelevant.

Read QPS (the serving fleet). Here is the real load. Suppose 2,000 serving replicas across all models. If each replica polled "what is production for my model" every 5 s, that is 2,000 / 5 = 400 reads/sec on the pointer — modest, but it grows linearly with the fleet and adds up-to-5-second staleness. If instead each replica watches the pointer (a coordination service push on change), steady-state read load on the source of truth drops to near zero and a flip propagates in well under a second. The asymmetry — rare writes, fleet-scale reads, sub-second propagation wanted — points straight at a watched coordination register, not a polled database.

Artifact read bandwidth. A replica fetches weights only on cold start or a flip, not per request: 2,000 replicas each pulling a 2 GB artifact during a fleet-wide promotion is 4 TB of egress in a burst. That is a real number — it argues for caching artifacts on the serving hosts and staggering the roll, not a thundering-herd pull from the blob store the instant the pointer flips.

The sizing settles three design choices before any diagram: (a) artifacts and metadata get separate stores; (b) the production pointer is a tiny linearizable register with rare writes and is read via watches, not polling; (c) a flip triggers a bounded artifact-fetch burst that must be staggered.

3 · Design walk

Build it from the track's mechanisms. Four components, each with a job it is good at and a consistency need it must meet.

3.1 · Immutable, content-addressed artifacts (L03, L20)

Weights go in a blob/object store (S3, GCS) that gives durability and throughput but no transactions — and it does not need them. The discipline that makes the registry safe is immutability: a registered artifact's bytes never change. The cleanest way to enforce that is content addressing — name each artifact by the hash of its bytes (e.g. sha256:9f86d0...), exactly the append-only / immutable-log thinking from L03. The hash is the address, so the same bytes always land at the same key, a different model can never overwrite an existing one, and a download can be verified against its name. The version's metadata row stores that hash; "the weights for version V" is a content hash that resolves to bytes that cannot have drifted. This also gives free dedup (two versions with identical weights share storage) and makes lineage tamper-evident.

3.2 · Strongly-consistent metadata store (L13, L16)

Metadata — version number, metrics, the artifact hash, lineage refs, stage, checksums, who/when — lives in a small relational store (Postgres) that gives you transactions (L13) and is easy to query ("best F1 among production-eligible versions of model M"). Registering a version is one transaction: write the metadata row referencing the already-uploaded, content-addressed artifact. Crucially, register the artifact first (upload bytes, get the hash), then commit the metadata row that points at it. That ordering — durable artifact before the row that references it — means the metadata never points at bytes that do not exist, the registry's version of "no dual write" (L32): one store is the follower of the other, never two independent writes that can half-fail.

3.3 · The linearizable production pointer (L16, L17)

This is the heart of the case. "Which version is live for model M" is a single mutable value that the whole fleet must agree on. The non-negotiable property is linearizability (L16): the pointer behaves as one register, and the moment a promotion's write returns, every subsequent read — from any replica — sees the new version; there is never a window where two replicas serve different "production" versions because one read a stale copy. A normal asynchronously-replicated database (L08) does not give you this: a follower can lag and hand a serving host the old version after the flip "succeeded".

The right home for the pointer is a coordination service — etcd or ZooKeeper — which is a consensus system (L17) purpose-built for exactly this: a tiny amount of strongly-consistent state, linearizable writes via a quorum, atomic compare-and-set, and watches that push a notification to every interested client the instant the value changes. (Linearizable reads need a small care: etcd serves them through the Raft leader by default, while ZooKeeper reads can be served stale from a follower unless you issue a sync first — but the watch-driven flow below means replicas are pushed the new value rather than relying on a fresh read.) Promotion is a single linearizable compare-and-set: "set /models/M/production from version 41 to version 42, only if it is still 41." It succeeds atomically or fails — never half-applies. Watches mean the 2,000 replicas learn about the flip in sub-second time without polling (the sizing in §2). Because promotion is rare (§2), the cost of consensus is irrelevant; what you buy is that the fleet can never disagree about the live version.

MODEL REGISTRY push version 42 promote 42 -> prod (train job) (release tool) | | v v +----------------+ 1. upload bytes +--------------------+ | ARTIFACT / |<------- (content hash) -------| metadata write | | BLOB STORE | sha256:9f86d0... | (register V42 row,| | (S3/GCS) | | refs the hash) | | immutable, | +---------+----------+ | 3x durable, | | | 40 TB | v +-------+--------+ +--------------------+ ^ | METADATA STORE | | 4. fetch weights | (Postgres) | | by hash (on flip / | versions, metrics,| | cold start, staggered) | lineage, stage | | | ~80 MB, txns | | +---------+----------+ | | | 2. compare-and-set | | /models/M/prod v | 41 -> 42 +-------------------+ | (atomic) | COORDINATION SVC | | | etcd / ZooKeeper | | | LINEARIZABLE | | | prod pointer, | | | 500 tiny records | | +---------+---------+ | | | 3. WATCH push: "M is now V42" (sub-second) | | +--------------------------------------------------+ | | | | | v v v v v +--------+ +--------+ +--------+ +--------+ ... SERVING FLEET |replica | |replica | |replica | |replica | (2,000 replicas) | M:V42 | | M:V42 | | M:V42 | | M:V42 | all agree on V42 +--------+ +--------+ +--------+ +--------+

Read the flow as: artifact bytes are uploaded and addressed by hash (1); the metadata row is committed referencing that hash (2 on the write side); promotion is a single linearizable compare-and-set on the coordination service (2 on the pointer side); every replica is watching and is pushed the new version (3); each replica then fetches the new weights by hash from the blob store, staggered to avoid a stampede (4). The only linearizable operation in the whole system is the pointer flip; everything else is immutable bytes or rarely-changing metadata.

3.4 · Audit, lineage, and rollback (L20)

Lineage answers "what produced this model" — which training-data snapshot, which code commit, which hyperparameters, which parent version. This is derived-data thinking (L20): a model is derived from data + code, and recording the inputs makes the derivation reproducible and the system auditable. Because artifacts are content-addressed (§3.1) and the data snapshot and code commit are themselves immutable references, the lineage is a tamper-evident chain — you can recompute the model from the recorded inputs and confirm you get the same hash.

Audit of promotions falls out almost for free: log every pointer change as an append-only record — (model, from_version, to_version, who, when). That log is the answer to "what was production on date D" and "who promoted the bad model", and it is itself an immutable log (L03). Rollback is then trivial and safe: because every prior version's artifact is immutable and still present (never overwritten, never mutated), rolling back is just another compare-and-set flipping the pointer back to a version that still exists byte-for-byte. There is no "restore from backup" and no risk that the old weights drifted — immutability is what makes rollback a sub-second pointer flip rather than a recovery operation.

ComponentJobConsistency need
Artifact / blob storeHold large immutable weights, durably, at throughputDurability + immutability (content-addressed); no transactions needed
Metadata storeVersions, metrics, lineage, stage; rich queriesStrong consistency / transactions; tiny data, registered after its artifact
Production pointerThe one live version per model the fleet readsLinearizable, consensus-backed, atomic compare-and-set + watches
Audit / lineage logWho promoted what; what produced each modelAppend-only / immutable; derived-data provenance

4 · Failure timeline — split-brain promotion

The failure that separates a good design from a fragile one: two promotions racing, or a stale-read replica, producing a fleet that disagrees about the live version.

SCENARIO: two release tools promote model M at nearly the same time, and the pointer lives in an async-replicated DB (the WRONG choice). t0 pointer (leader) = V41. Followers = V41. t1 Release tool A: "promote V42" -> writes leader, leader = V42 t2 Release tool B: "promote V43" -> also writes leader (last-writer-wins), leader = V43. A's promotion silently lost. t3 Replica X reads a LAGGING FOLLOWER -> still sees V41 (stale, L08 lag) Replica Y reads the leader -> sees V43 Replica Z cached A's transient V42 -> serves V42 t4 SPLIT BRAIN: three different "production" versions serving at once. No audit of who won. Rollback unclear: roll back to which version? ------------------------------------------------------------------------- FIX with the linearizable pointer (L16/L17): t1 Tool A: compare-and-set /models/M/prod 41 -> 42 ==> SUCCEEDS, now 42 t2 Tool B: compare-and-set /models/M/prod 41 -> 43 ==> FAILS (it is 42, not 41). B is told it lost; it retries from 42 or aborts. No lost update. t3 WATCH pushes "M = V42" to ALL replicas; reads are linearizable, so no replica can read a stale V41 after the set returned. Fleet converges. t4 Audit log has exactly one record: (M, 41 -> 42, toolA, t1). One truth.

Root cause of the broken version: the pointer was treated as ordinary replicated data, so concurrent writes did last-writer-wins (a lost update) and lagging followers served stale reads. Mitigation: the pointer is a linearizable register with atomic compare-and-set on a coordination service — concurrent promotions are serialized (one wins, the other is told it lost and must retry against the new value), reads cannot go stale after a successful write, and watches push the single agreed value to the fleet. The conditional compare-and-set is the same fencing idea from L15: an action that must not stomp a newer state checks the version it expects before it commits.

5 · Answer rubric

Good answer (baseline)

  • Separates the large artifact store (blob, durable, throughput) from the small metadata store (strong consistency, queryable) — does not jam both into one engine.
  • Makes artifacts immutable and versioned; never overwrites a registered model's bytes.
  • Has an explicit production pointer per model and a promote operation distinct from registering a version.
  • Supports rollback by re-pointing at a prior version, and records who promoted what.

Better answer (senior signal)

  • Names linearizability explicitly and puts the pointer in a consensus-backed coordination service (etcd/ZooKeeper), with atomic compare-and-set for promotion so concurrent flips can't lose updates.
  • Uses watches to push flips to the fleet sub-second and justifies it against polling with the read-QPS sizing.
  • Makes artifacts content-addressed (hash = name) for immutability, verification, and dedup; registers the artifact before the metadata row (no dual write).
  • Treats lineage as derived-data provenance (data snapshot + code commit + parent) and the promotion log as an append-only audit; rollback is a sub-second flip because prior bytes are immutable.
  • Quantifies: 40 TB artifacts vs 80 MB metadata; rare writes vs fleet-scale reads; staggers the post-flip artifact-fetch burst.

Red flags

  • Storing weights as a BLOB column in the metadata DB, or putting tiny critical metadata in the blob store — wrong store for each.
  • Mutable artifacts ("just re-upload to the same key") — destroys reproducibility and makes rollback unsafe.
  • Putting the production pointer in an async-replicated database and assuming reads are fresh — invites the split-brain timeline.
  • A non-atomic promote (delete-then-write, or two separate writes) that can leave the pointer in a half-applied state.
  • Polling the pointer at high frequency from thousands of replicas with no watch mechanism, or fetching artifacts in a thundering herd on flip.
  • No lineage / no audit log — cannot answer "what produced this" or "who promoted the bad model".

Checkpoint exercise

Try it
You run a registry with 300 models and 1,500 serving replicas. (a) Sketch the four components and state, for each, its consistency need and roughly how much data it holds. (b) Write the exact promotion operation as a compare-and-set on the production pointer and explain why a plain "UPDATE pointer SET version=42" on an async-replicated database would let two replicas disagree. (c) A replica reads the pointer every 3 s today; compute the steady-state pointer read QPS, then explain how switching to watches changes both the QPS and the flip-propagation latency. (d) A bad model V42 is live; describe the rollback to V41 and state the single property of the artifact store that makes that rollback safe in one pointer flip. (e) The audit team asks "what was production for model M on 2026-03-01" — which structure answers it, and why is it an append-only log?

Where this points next

The registry is a system of record with a tiny linearizable core and a large immutable periphery — the consistency burden concentrated in one pointer. The next case inverts that profile. A metrics / observability dashboard has almost no strong-consistency requirement but enormous write fan-in: millions of high-cardinality time-series samples per second pouring in from the same serving fleet, stored columnar and rolled up, queried fresh-vs-historical with approximate aggregates. Lesson 34 builds that ingest-and-query path — where the hard problems are write throughput, downsampling, and hot-recent-vs-cold-historical storage rather than a single atomic flip.

Where is truth?
System of record: three authoritative pieces — the content-addressed artifact bytes in the blob store, the metadata rows in Postgres, and the linearizable production pointer in the coordination service (etcd/ZooKeeper). The registry is the source of truth here; nothing downstream is more authoritative. Copies / derived views: each serving replica's locally cached weights and its last-watched copy of the pointer; the model itself is derived from its lineage (data snapshot + code + hyperparameters). Freshness budget: the pointer flip propagates to the fleet sub-second via watches (no stale window after the compare-and-set returns, because the pointer is linearizable); artifact bytes never change, so they have no freshness concern. Owner: the ML-platform team owns the registry; a release tool owns promotions. Deletion path: artifacts are immutable and retained so rollback targets always exist; "removing" a version means de-referencing it (it can never be production-pointed) and garbage-collecting only versions no pointer or retention policy still needs — never an in-place overwrite. Reconciliation/repair: a model is recomputable from its recorded lineage and verified by re-deriving the same content hash; a replica that missed a watch re-reads the linearizable pointer and re-fetches by hash. Evidence of correctness: content hashes prove the bytes never drifted (download verification, tamper-evident lineage); the append-only promotion log proves exactly who promoted what and when; the compare-and-set proves no concurrent flip was lost.
Takeaway
A model registry is two stores glued by one pointer. The artifact store holds large immutable weights — make them content-addressed (hash = name) so they can never drift, dedup for free, and verify on download (L03). The metadata store holds tiny strongly-consistent rows (versions, metrics, lineage, stage), registered after their artifact so the registry never dual-writes. The crux is the production pointer: "which version is live" must be linearizable (L16) so the whole fleet agrees the instant a promotion returns, which means a consensus-backed coordination service (etcd/ZooKeeper) with an atomic compare-and-set promote and watches that push flips sub-second (L17) — rare writes, fleet-scale reads. Lineage (data snapshot + code commit + parent) is derived-data provenance and the promotion log is an append-only audit (L20). Because every artifact is immutable, rollback is just a compare-and-set back to a version that still exists byte-for-byte — a sub-second flip, not a recovery.

Interview prompts