Design Google Docs-style collaboration
Collaborative editing is a consistency problem with humans in the loop. The system must converge — every replica ends at the same document — while preserving each user's intention closely enough that the editor still feels like it's doing what they typed.
The hinge
The prompt looks like "build a realtime text box." It becomes a forcing architecture the moment you ask the one question that separates the two real designs: do users edit offline? If editing is online-only, a central session owner can impose a single order on operations and transform each one against what came before — Operational Transform (OT). If users can disconnect, edit for an hour, and reconnect, you need operations that commute regardless of order and merge deterministically with no central referee — a CRDT. Everything downstream — payload size, metadata cost, recovery story — follows from that one fork. This is DDIA Ch. 5's "Detecting Concurrent Writes": Kleppmann walks exactly this ground (siblings, version vectors, mergeable structures, CRDTs) because the database-replication problem and the collaborative-editing problem are the same problem at different latencies.
Clarify the contract
Treat the prompt as a product contract before drawing boxes. The system must:
- Let many users edit one document concurrently and see each other's edits within ~100 ms.
- Converge — every connected client renders identical content once the network quiesces.
- Persist history and recover — reopen a doc, reconstruct it, survive a server crash without losing accepted edits.
- Show presence — cursors, selections, "Alice is here" — as soft realtime state.
- Tolerate offline edits (depending on the answer to the hinge) and reconnect cleanly.
Equally important is what it need not do. It is not a transactional store: there is no "reject this edit because of a constraint violation" — every well-formed keystroke is accepted, the only question is how it merges. Presence is not document state: a lost cursor update corrupts nothing. And we do not need cross-document transactions — each document is its own consistency island, which is what lets us shard cleanly by document ID and put one session owner per doc.
Put numbers on it
The numbers that steer this design are not QPS totals — they're the two ratios that decide where state lives.
Ratio 1 — why you cannot replay forever. Take a mature 50-page document. Over its lifetime it has accumulated, say, 1,000,000 operations (every keystroke, paste, and delete is an op). To open it by replaying the whole log, at a generous 10 µs per op:
1{,}000{,}000 ops × 10 µs = 10{,}000{,}000 µs = 10 s to open the doc
Ten seconds to open a document is unshippable. The fix is a snapshot: periodically materialise the document and discard the need to replay ops before it. Open-latency becomes (replay only the ops since the last snapshot). If we snapshot every N = 1,000 ops, worst-case replay is 1{,}000 × 10 µs = 10 ms — three orders of magnitude better. N is the knob, and it trades three things against each other:
| Snapshot interval N (ops) | Worst-case open latency (replay since snapshot) | Snapshot storage / write amplification |
|---|---|---|
| 100 | 100 × 10 µs = 1 ms — instant | High — a full ~doc-sized snapshot every 100 ops; heavy write amp, lots of storage churn. |
| 1,000 | 1{,}000 × 10 µs = 10 ms — imperceptible | Moderate — the usual sweet spot for an interactive editor. |
| 10,000 | 10{,}000 × 10 µs = 100 ms — noticeable on open | Low — snapshot rarely, store mostly the cheap op log. |
| ∞ (never) | 10 s — unacceptable | Minimal snapshot storage, but unusable open latency. |
This is exactly DDIA Ch. 3 / Ch. 11's log-plus-snapshot pattern (and LSM-tree compaction): the append-only log is the source of truth and is cheap to write; the snapshot is a periodically materialised view that bounds the work to reconstruct state. Snapshot interval is the same tuning knob as LSM compaction frequency — compact too often and you burn write bandwidth, too rarely and reads (here, opens) get slow.
Ratio 2 — why presence stays off durable storage. While ten people read and two people type, everyone's cursor moves. Cursor and selection updates routinely run 10× the edit volume — a single drag-select fires a stream of position updates carrying zero document content. If those hit the op log, you have just multiplied your durable write rate 10× to persist data nobody will ever recover. So presence rides a separate, lossy, in-memory pub/sub channel that never touches the durable path. Dropping a cursor update costs a momentarily stale caret; dropping an edit costs a corrupted document. They must not share a pipe.
Data model & API
The data model is the architecture here. Five things, two of them durable:
| Entity | Durable? | What it is |
|---|---|---|
| Op log | Yes — source of truth | Append-only, per-document sequence of operations, each stamped with a monotonic version number. Both the replication stream and the recovery record. |
| Snapshot | Yes | Materialised document content at version V, written every N ops. Lets open replay only ops > V. |
| Version | Yes (in log) | The monotonic counter that gives every accepted op a single linear position — the basis for ordering and for clients to say "I'm based on version V." |
| Presence session | No — ephemeral | Who is connected, where their cursor is. In-memory, lossy pub/sub, TTL'd on disconnect. |
| Comment | Yes (separate) | Anchored to a text range; stored apart from the op log so comment churn doesn't bloat the edit stream. |
The API is deliberately tiny — three calls — and crucially submit_operation carries the client's base version, which is what lets the server detect concurrency and transform/rebase:
| API / operation | Why it exists |
|---|---|
open_document(id) | Returns latest snapshot + the op tail since that snapshot + current version. The 10 ms open path. |
submit_operation(doc, base_version, op) | The whole game. base_version tells the owner what the client thought the doc was; the owner transforms op against everything accepted since. |
subscribe(doc) | Opens the live channel: receive broadcast ops (durable) and presence (ephemeral) over the same socket, different streams. |
Linearized design
Walk one edit through the system in the order it actually moves. The bottleneck — the single session owner per document — appears naturally at step 2.
- Open. Client calls
open_document. Server loads the latest snapshot and replays only the op tail since it (the 10 ms path from above). Client now holds content + current version V. - Route to the session owner. All ops for this document go to one coordinating node — the session owner. One owner per doc gives a single point to impose order. This is the bottleneck and the SPOF; we handle its death in failure modes.
- Transform / merge. The owner takes the incoming op tagged with the client's
base_version, and (OT) transforms it against every op accepted since that base version, or (CRDT) merges it commutatively. Result: an op that applies cleanly to the current state. - Append + version. The transformed op is appended to the durable op log and assigned the next monotonic version. The log is now both the history and the thing followers/recovery replay.
- Broadcast. The owner pushes the accepted op, with its version, to every subscriber. Clients apply it; because versions are monotonic and the owner is the single sequencer, everyone converges.
- Snapshot. Every N ops, the owner (or a background job) materialises a snapshot to durable store, bounding future open/recovery cost.
- Presence, separately. Cursor/selection updates ride the lossy in-memory pub/sub — broadcast to subscribers, never logged, never snapshotted.
Deep dives
1. OT vs CRDT — the offline question decides it
These are two answers to "how do I make concurrent edits converge while preserving intention," and they sit at opposite ends of the coordination spectrum.
Operational Transform (OT) assumes a central transform authority — the session owner. Each op is small (e.g. insert("x", pos=12)), and when two ops were generated against the same base version, the server transforms one against the other: if Alice inserted at position 5 and Bob inserted at position 3 concurrently, Alice's op is shifted to position 6 before applying, so both intentions survive. The payloads are tiny and the editor model is mature (Google Docs is OT). The cost is that transform correctness is genuinely hard — you must define a transform function for every pair of op types and prove it satisfies the convergence property (TP1, and TP2 for the fully decentralised case). It also leans on the central sequencer: OT is happiest online, where the owner sees ops promptly.
CRDTs (e.g. RGA, Logoot, Yjs/Automerge) make operations commutative by construction: each character gets a globally unique, densely-orderable identity, so any two replicas that have seen the same set of ops compute the same document regardless of arrival order, with no central referee. This is what makes them offline- and P2P-friendly — edit on a plane for an hour, sync later, merge deterministically. The cost is heavier metadata: every character carries identity/tombstone bookkeeping, so payloads and memory are larger (mitigated in modern libraries, but never zero). This is precisely DDIA Ch. 5's CRDT discussion — "mergeable persistent data structures" that converge without coordination.
2. The versioned op log is both the replication stream and the recovery record
The single best structural idea in this design is that there is one append-only, monotonically-versioned op log per document, and it plays two roles at once:
- Replication stream. Broadcasting "op @ version 8, op @ version 9 …" to subscribers is log shipping — the same total-order broadcast that DDIA Ch. 9 describes. Because the owner is the single sequencer assigning monotonic versions, every replica applies the same ops in the same order and converges. A reconnecting client just says "I'm at v7, send me everything after" — log replay from an offset, identical to a follower catching up (DDIA Ch. 5).
- Recovery record. When the owner dies, the new owner reconstructs state by loading the last snapshot and replaying the log tail — the same operation as a client open. Snapshots bound recovery time; the ops between snapshots preserve full history (audit, undo, "see version from last Tuesday"). You keep log retention beyond the latest snapshot precisely so a corrupt snapshot can be rebuilt from an earlier one plus ops.
The elegance is that you don't build a separate replication system and a separate backup system — the durable log is both. This is event sourcing (DDIA Ch. 11): the log of changes is primary, the current document is a derived, rebuildable view.
3. Presence is not document state — and that's a load-bearing decision
It is tempting to treat the cursor as "just more state on the document." Resist it. From the numbers above, presence is ~10× the edit volume and carries no recoverable value. If presence shared the durable op-log path, three bad things happen: durable write rate multiplies ~10×, the op log fills with noise that slows replay and bloats snapshots, and — worst — cursor chatter now competes with edits for the edit-latency budget, so the thing that matters (your keystroke showing up) gets queued behind the thing that doesn't (someone else's caret wiggling). Keeping presence on a separate, lossy, in-memory pub/sub channel with throttling and a disconnect TTL is what protects the edit path. The general principle: data without a recovery requirement does not belong on the durable, ordered path — give it a cheaper channel sized to its real importance.
Trade-offs
| Choice | Buys | Costs | Choose when |
|---|---|---|---|
| Central session owner vs peer merge | Single sequencer → simple total order, smaller payloads | Session owner is a per-doc bottleneck and SPOF; needs failover | Online collaborative docs (the common case) |
| OT vs CRDT | OT: tiny ops, mature editor model | OT: hard transform correctness, leans on central authority | Mostly-online editing → OT; offline-first/P2P → CRDT |
| Full history vs compact snapshots | Audit, undo, time-travel | Storage + replay cost; snapshot interval is the knob | Regulated / audited docs keep more; consumer docs snapshot aggressively |
| Durable comments inline vs separate | Inline: consistent single-export document | Inline: schema coupling, comment churn bloats the edit stream | Separate when comments are range-anchored and high-volume |
The sharpest one is OT vs CRDT, and the reason it's sharp is that it's secretly the session-owner question wearing a different hat. Choosing OT commits you to a central transform authority, which buys small payloads and a battle-tested model but makes the session owner load-bearing and makes offline editing awkward (you must rebase a long divergent history against the server on reconnect). Choosing CRDT buys you genuine offline/P2P merge — no authority required — at the price of per-character metadata you carry forever. There is no free lunch: you are trading coordination (OT's server) against metadata (CRDT's identities). Pick by where your users are, not by which is "more elegant."
Failure modes
| Failure | Mitigation |
|---|---|
| Session owner dies | Elect/assign a new owner (DDIA Ch. 8 — leader failure, fencing tokens to stop a zombie old owner from accepting ops); rebuild state from last snapshot + op-log tail. Clients reconnect with their last-seen version and resync. See lesson 13. |
| Client submits a stale op (base_version behind current) | Owner transforms it against the ops accepted since that base (OT), or merges it (CRDT). If transform is impossible/ambiguous, ask the client to rebase onto current and resubmit. |
| Snapshot corruption | Retain op-log history beyond the latest snapshot + checksum snapshots; on corruption, rebuild from an earlier snapshot + replayed ops. The dual-role log pays off here. |
| Presence storm (big meeting, many cursors) | Throttle/coalesce cursor updates, use lossy pub/sub, TTL stale sessions. Never let presence back-pressure the edit path. |
| Hot document (one owner, huge meeting) | Owner is single-threaded per doc by design; scale subscribers via a broadcast fan-out tier (lesson 11) rather than overloading the sequencer. Document sharding is by ID, so other docs are unaffected. |
Interview Q&A — what a senior says
- Two users edit the same word offline, then reconnect — what does each algorithm do? (senior answer) Both must converge to identical bytes; the difference is the mechanism and the intention outcome. CRDT: each character has a unique, densely-ordered identity, so the two divergent edits merge deterministically with no referee — concurrent inserts interleave by ID, concurrent deletes tombstone, and both replicas compute the same result regardless of who syncs first. Intention is preserved structurally; the merged word may interleave both edits. OT: on reconnect the client's offline op chain is transformed against everything the server accepted meanwhile (and vice-versa) and replayed in the owner's order — workable but you're rebasing a long divergent history against a central authority, which is exactly why OT is awkward offline. Both converge (the hard guarantee); CRDT preserves intention without coordination, OT preserves it via the server's transform. Last-write-wins would converge too — by silently dropping one person's word, which is why nobody ships it.
- Why can't you just replay the op log to open a document? (senior answer) A mature doc has ~1M ops; at 10 µs/op that's 10 s to open — unshippable. You snapshot every N ops and replay only the tail: N=1,000 → 10 ms. N trades open-latency against snapshot storage/write-amp — the LSM-tree compaction trade-off (DDIA Ch. 3/11) applied to a document.
- Why keep presence off durable storage? (senior answer) Cursor/selection updates run ~10× edit volume and carry zero recoverable value. On the durable path they'd multiply write rate 10×, bloat the log/snapshots, and — worst — contend with edits for the latency budget. Lossy in-memory pub/sub with throttling and a disconnect TTL is the right channel: dropping a cursor update costs a stale caret, dropping an edit costs the document.
- OT or CRDT — how do you decide? (senior answer) Off the contract, not taste. Online-only, server-coordinated, want minimal payloads → OT (it's what Google Docs uses). Offline-first / P2P / local-first → CRDT, because operations commute by construction and need no central authority, at the cost of per-character metadata. The decision is really "do I require a central session owner," because OT presumes one.
- The session owner crashes mid-edit. What's lost? (senior answer) Only ops it accepted but hadn't durably appended — so append-before-ack. A new owner is elected (DDIA Ch. 8) and rebuilds from the last snapshot + op-log tail, the same path as a client open. Fence the old owner so a zombie can't keep assigning versions. Clients reconnect with their last-seen version and resync from the log offset — follower catch-up (DDIA Ch. 5).
- How do versions give you convergence? (senior answer) The single session owner assigns a monotonic version to each accepted op — total-order broadcast (DDIA Ch. 9). Because every replica applies the same ops in the same order, they converge.
base_versionon submit lets the owner detect concurrency (client based on an older version) and transform/merge accordingly. Causality and happens-before — siblings and version vectors (DDIA Ch. 5), total ordering (Ch. 9) — are the formal backbone here. - One document, a 500-person all-hands viewing it — does the owner fall over? (senior answer) The owner only sequences writes, which are few; the load is broadcast fan-out to 500 subscribers. Separate the fan-out tier (lesson 11) from the sequencer so the owner never touches 500 sockets directly. Presence is throttled and lossy. Sharding is per-document, so this hot doc can't hurt others.
- Where do comments live — in the op log or separate? (senior answer) Separate, range-anchored. Comments are high-churn and shouldn't bloat the edit stream or slow replay; they reference a text range (which itself must be transformed as the underlying text shifts). Inline-in-the-doc buys a single consistent export but couples schemas — choose it only when export consistency dominates.
Related lessons
This case is a tour of the consistency lessons. The convergence-vs-intention split and the "no single correct merge" framing are lesson 09's consistency models made concrete — CRDTs are the eventual-consistency answer that still converges without coordination. The op broadcast to subscribers is the queue/fan-out machinery of lesson 11, and the same lesson is where you put the broadcast tier that absorbs a 500-viewer meeting. The snapshot + op-log durable model is lesson 04's log-plus-snapshot / event-sourcing storage pattern. And the single-session-owner bottleneck — its election, fencing, and replay-on-failover — is lesson 13's leader failover applied to a per-document coordinator.