Block, file, and object storage primer

Three storage interfaces, one machine underneath. The interface you pick is a promise about latency, sharing, and mutation — and that promise quietly dictates the whole product above it.

Primer under C16 / C17 DDIA Ch. 3 Interface = contract

First principle

Every storage system is the same physical media — spinning rust or flash, addressed in sectors — wrapped in an interface that hides some of it. Block hides nothing above the sector: you get a raw array of fixed-size blocks and bring your own structure. File adds a namespace (paths, directories, permissions) and POSIX mutation semantics. Object throws away the namespace and the in-place mutation entirely, keeping only PUT key / GET key over HTTP — and in exchange gets effectively infinite, cheap, durable scale. The interview question "block, file, or object?" is therefore never about storage trivia. It is: which promise does my workload actually need, and what am I willing to give up to get scale?

The hinge: the interface is the contract

This case sits underneath two others. C16 (design an object storage service) and C17 (resumable large-file upload) both assume you already know why object storage behaves the way it does — why uploads are multipart, why there's no append, why you keep a separate metadata database. This page is the prerequisite that makes those designs obvious instead of arbitrary. If C16/C17 are the building, C18 is the foundation: master the three interfaces here and those two cases stop being memorization.

The forcing function is that the three interfaces are not interchangeable layers of abstraction you can swap freely — each one throws away a capability to buy something. Block gives you bytes-level random access but no sharing and no naming. File gives you a shared hierarchy but pays for it with a metadata server that becomes the bottleneck. Object gives you planet-scale cheap durability but takes away in-place mutation, low latency, and rename. Picking the wrong one doesn't just cost performance; it forces you to rebuild the discarded capability in application code, badly.

Clarify the contract

What this primer must let you do in an interview:

Explain each interface from its access unit: block = fixed-size sectors at offsets; file = byte streams under named paths; object = whole immutable blobs under flat keys.
Map a workload to an interface by its access pattern — random small IO vs. shared hierarchy vs. write-once read-many blobs.
Reason quantitatively about latency, IOPS, throughput, and — the one everyone forgets — metadata operation rate.
Name what each interface makes hard, so you can pre-empt the follow-up.

And explicitly what this case need not do: it is not the design of a full object storage service (that's C16), not the upload protocol (C17), and not a benchmark shootout between vendors. It's the mental model that makes those decisions fall out cleanly.

Put numbers on it: the latency / IOPS / throughput ladder

The single most load-bearing fact about these three interfaces is their latency ladder, and it spans three orders of magnitude. Everything else follows from it.

Block — random IO latency

0.1–1 ms

File (NFS) — metadata op RTT

~1–5 ms / op

Object — GET first byte

10–100+ ms

Object — access unit

whole object

Property	Block	File	Object
Access unit	fixed-size block (512 B–4 KB) at an LBA offset	byte range within a named file	the whole object, by key
Latency (typical)	0.1–1 ms (flash), microseconds on NVMe	RTT to NFS server per metadata op (~1–5 ms); data follows	10–100+ ms to first byte (HTTP + lookup + fetch)
Mutation	overwrite any block in place	seek + write any byte range; append cheaply	replace whole object; no in-place edit, no real append
Naming	none — caller owns the addressing	hierarchical paths + directories	flat key space (prefixes look like folders)
Sharing	one mounter at a time (usually)	many clients, POSIX locking	unlimited concurrent readers; many writers via separate keys
Scale ceiling	one volume = one disk-sized array	metadata server / inode pressure	effectively unbounded (exabytes)
Cost / GB	highest (provisioned, attached)	high (NAS / managed file)	lowest (cheap, tiered)
Examples	EBS, local SSD, iSCSI LUN	NFS, SMB, EFS, NAS	S3, GCS, Azure Blob

Small files vs. one big file — the metadata ceiling

Take two workloads of identical total bytes: 1 million 1 KB files vs. one 1 GB file. They look the same on a capacity graph and behave nothing alike. The big file is a bandwidth problem — one open, one metadata touch, then stream a gigabyte; the disk's sequential throughput (say 500 MB/s) drains it in ~2 s, and the metadata server is bored. The million small files are a metadata-QPS problem — each needs a path lookup, a permission check, an open, a stat. At ~3 ms per metadata round trip a single client doing them serially needs 10⁶ × 3 ms ≈ 3000 s ≈ 50 min, with the disk almost idle the whole time. The bytes never were the bottleneck — the operation count was. This is why "we have lots of tiny files" is the phrase that should make you reach for object storage (or pack the files), and why file systems quietly fall over long before they run out of disk.

The three-layer stack: what each interface hides

The clearest way to hold all three in your head is to draw them as the same media seen through three different windows. The application talks to an interface; the interface decides how much of the machine below it the app is allowed to see.

APPLICATION │ ┌───────────────────┼───────────────────┐ ▼ ▼ ▼ ┌─────────┐ ┌──────────┐ ┌───────────┐ │ BLOCK │ │ FILE │ │ OBJECT │ │ read/ │ │ open(path)│ │ PUT key │ │ write │ │ seek/read │ │ GET key │ │ block@ │ │ write/ │ │ (HTTP) │ │ offset │ │ append │ │ │ └────┬────┘ └────┬─────┘ └─────┬─────┘ │ │ hides: │ hides: hides: nothing paths,inodes, keys→shards, above the sector dir tree,locks replicas,erasure │ │ exposes │ coding,GC. │ POSIX byte-level exposes ONLY │ mutation. whole-object put/get. ▼ ▼ ▼ ┌─────────────────────────────────────────────────────┐ │ PHYSICAL MEDIA (flash / disk sectors) │ └─────────────────────────────────────────────────────┘ "raw array" "+ namespace & mutation" "+ infinite scale, − mutation & latency"

Read the diagram left to right as a ladder of abstraction and a ladder of surrender. Block hides nothing above the sector, so you must bring a filesystem or a database to give the bytes meaning — that's exactly what a database does. File hides the inode/directory machinery and hands you POSIX, so you can seek and overwrite byte 5,000,000 of a 1 GB file directly. Object hides everything — sharding, replication, erasure coding, garbage collection — behind two verbs, and the price of that magic is that the only mutation it offers is "replace the whole object."

Data model & API

The APIs are tiny on purpose; the constraints live in what they omit.

Interface	Core operations	Conspicuously absent
Block	`read(lba, n)`, `write(lba, buf)`, `flush()`	any notion of a file, name, or sharing
File	`open(path)`, `read`, `write`, `seek`, `append`, `rename`, `fsync`	nothing — that's the point; it's the rich one
Object	`PUT key→bytes`, `GET key`, `DELETE key`, `LIST prefix`	no append, no in-place write, no atomic rename, no fsync

When object storage backs your product, the durable bytes live in the object store but the mutable truth lives elsewhere: a metadata database keyed by object key, holding the version, owner, ACL, content hash, and the pointer that says "object v3 is the current one." That two-system split (cheap immutable bytes + small mutable index) is the canonical object-storage pattern and the seed of C16's whole design.

Linearized design: choosing the interface

Walk the decision the way it actually unfolds — start from the access pattern, not from a brand name.

Name the access pattern. Is it random small reads/writes at arbitrary offsets, a shared hierarchy many machines mutate together, or write-once-read-many whole blobs? This one question usually decides it.
Random IO + durability of writes ⇒ block. Databases and VM disks sit on block because they need to overwrite a 4 KB page in place and fsync it. (See the deep dive — this is non-negotiable.)
Many clients, shared mutable hierarchy ⇒ file. A team drive, a build farm reading a shared toolchain, legacy apps that expect a POSIX path. Watch the metadata ceiling.
Blobs: media, backups, logs, static assets, ML datasets ⇒ object. Cheap, durable, infinitely scalable, served straight to clients via signed URLs / CDN.
Keep mutable metadata in a database when object storage holds the bytes — version pointers, ACLs, the key↔record link.
Design lifecycle from the interface. Object → lifecycle rules and tiering (hot→cold→archive); block → snapshots; file → quotas and backup windows.

Deep dives

1 — "Object storage is not a filesystem" (the load-bearing insight)

This is the single idea that, internalized, makes you sound senior on C16 and C17. Object storage looks like a filesystem because the console shows folders and the SDK has list("photos/2026/"). It is a costume. The key space is flat: photos/2026/cat.jpg is one opaque string, and the slashes are just characters. The "folder" is a query (LIST with a prefix), not a directory object.

That illusion shatters on two operations every filesystem makes cheap:

Rename / move is O(copy). There is no "change the path" — you PUT the bytes under the new key and DELETE the old one. Renaming a 5 GB object means copying 5 GB. "Renaming a folder" of a million objects means a million copy-and-deletes. (Cloud stores offer a server-side copy that avoids the round trip back to your app, but it is still a full byte copy under the hood, billed and rate-limited as such.)
Append doesn't exist. To "add a line to a log object" you must GET the whole object, append in memory, and PUT the whole thing back — read-modify-write of the entire blob, with a lost-update race if two writers do it at once. This is precisely why append-heavy small writes are the canonical anti-pattern: buffer them in memory or a log system and flush as large, immutable objects.

The senior move

Because the natural mutations are missing, you don't fight the object store — you build product semantics above it deliberately. Need versions? Write a new key (doc/v1, doc/v2) and move a pointer in your metadata DB. Need rename? Rename the pointer, leave the bytes. Need append? Write a sequence of immutable chunks and stitch them at read time, or use multipart upload (C17) to assemble one object from parts. The bytes are immutable and cheap; the mutable story is a small database transaction you own. That deliberate split is the whole art — and it is DDIA Ch. 3's distinction between a log of immutable writes and the mutable index over them, applied at the storage tier.

2 — The filesystem metadata bottleneck

File storage's gift is the namespace; its scaling wall is the same namespace. Every path operation — open, stat, readdir, create, unlink — touches a metadata server (or the inode/dentry structures on a local FS). Unlike data reads, which parallelize across disks, metadata ops serialize on shared structures.

Two pathologies dominate at scale. First, the small-file storm from the box above: a million 1 KB files is a million metadata ops, and the metadata path — not the data path — saturates first. Second, the hot directory: a single directory with millions of entries, or one every job writes into at once. readdir on a huge directory is expensive, and concurrent creates into one directory contend on the same directory inode's lock. A build farm where 5,000 jobs each write output into /shared/results/ doesn't run out of disk — it runs out of metadata lock, and every job blocks on the same hot inode. This is the file-system cousin of lesson 07's hot shard: a single key (here, a directory) absorbing a disproportionate share of operations. The mitigations rhyme too — fan out across many directories (a sharded path layout like results/ab/cd/<id>), or step off the file interface entirely onto object storage where there is no shared directory inode to contend on.

3 — Why a database needs block, not object

This is the deep dive that ties the latency ladder to a hard architectural rule. A transactional database (DDIA Ch. 3 — B-trees and LSM-trees) has three requirements the object interface structurally cannot satisfy:

Low-latency random IO. A B-tree lookup walks a few pages; an LSM compaction rewrites SSTables. These are sub-millisecond random reads/writes at specific offsets. Block delivers ~0.1–1 ms; object delivers 10–100+ ms and only whole-object access. Putting a B-tree on object storage means every page read is a multi-hundred-millisecond whole-object fetch — three orders of magnitude too slow, hundreds of times per query.
In-place / fine-grained writes. Updating one row touches a 4–8 KB page. Object storage has no in-place write — you'd rewrite the whole file (or whole SSTable) per change. Read-modify-write of a multi-GB data file per UPDATE is absurd.
fsync durability. Durability and crash recovery depend on the write-ahead log being flushed to stable media and acknowledged in order — DDIA Ch. 3's WAL. Block storage (and a real filesystem on it) honors fsync: when it returns, the bytes are durable, in order. Object storage has no fsync and no ordered partial writes; its consistency is per-object and eventual-ish, with no fine-grained durability barrier. Without an ordered, acknowledged flush, a database cannot guarantee crash recovery.

Sharpest senior question: "Why can't you point Postgres's data_directory at S3?"

Three reasons, in priority order. (1) fsync semantics: Postgres's WAL relies on fsync returning only when bytes are durably on stable storage, in order — object stores expose no such barrier, so a crash can lose or reorder writes Postgres believed were committed, breaking durability and recovery. (2) Random IO: the access pattern is small random page reads/writes at offsets; S3 only serves whole objects at 10–100+ ms, so every page touch is hundreds of times too slow. (3) In-place mutation: updating one row means rewriting a page, which on an object store means rewriting the entire file. The right architecture is the inverse: run Postgres on a block volume (low-latency random IO + fsync), and use object storage for what it's good at — base backups, WAL archive, and cold/lake tables (Parquet on S3) read in bulk. Newer "data lakehouse" engines and separated-storage databases (Aurora, Neon, Snowflake) do use object storage, but only by interposing a caching/page-server layer that re-creates block-like random access and durability above the object tier — proving the rule rather than breaking it.

Trade-offs

Choice	Buys	Costs	Choose when
Block vs object	random IO, low latency, fsync, in-place writes	highest $/GB; one mounter; you manage the filesystem/DB above it	databases, VM root disks, anything needing fine-grained durable writes
File vs object	shared POSIX hierarchy, locking, append	metadata server is the scaling ceiling; hot-directory contention	team drives, build farms, legacy apps that expect a path
Object vs DB blob	cheap, exabyte-scale, durable bytes; serve via CDN/signed URL	two-system consistency (bytes vs. metadata); no transactions over bytes	large media, backups, logs, static assets, datasets
Mutable file vs immutable object versions	familiar in-place edits	harder point-in-time recovery; no natural audit trail	small collaborative files where edit-in-place is the product

The sharpest trade-off is object vs. database blob, because it's the one people get wrong by reflex. Stuffing a 50 MB video into a Postgres bytea column "keeps everything in one place" and gives you transactions over the bytes — but it bloats the database, wrecks backup/restore times, and pushes blob bytes through your most expensive, hardest-to-scale tier. The standard answer is to store the bytes in object storage and a row of metadata + the object key in the database, accepting the one real cost: the two systems can drift (a committed DB row pointing at an object that the upload never finished, or an orphaned object whose DB row was rolled back). You buy that back with a transactional metadata commit (write the bytes first, commit the pointer last) plus a garbage collector that sweeps objects with no live pointer — exactly the reconciliation pattern C16 builds out in full.

Failure modes

Failure	Mitigation
Append-heavy small writes to an object store	Buffer/batch into large immutable objects, or use a real log/queue system; never read-modify-write a whole object per line.
Hot prefix / hot partition in object store	Modern S3 auto-partitions, but for extreme rates spread keys (high-entropy prefix) and measure before assuming it's solved.
Rename / move storm	Model rename as a pointer update in the metadata DB, not an O(copy) of the bytes.
Hot directory (file storage)	Shard the path tree (`aa/bb/<id>`); cap entries per directory; or move to object keys with no shared inode.
Orphaned object / dangling pointer	Write bytes first, commit pointer last (transactional metadata commit); background GC sweeps unreferenced objects.
Database on object storage	Don't. Put the DB on block; use object storage for backups, WAL archive, and lake tables only.

Interview Q&A

Questions a senior interviewer will actually ask

Block, file, or object — how do you decide? (senior answer) Start from the access pattern, not the brand. Random fine-grained durable writes (a DB or VM disk) → block. A shared mutable hierarchy many clients touch → file. Write-once-read-many blobs at scale → object. Then sanity-check against the latency ladder: block ~0.1–1 ms, file pays a metadata RTT per op, object 10–100+ ms whole-object — and the metadata-QPS ceiling if it's lots of small files.
Why isn't object storage just a filesystem with HTTP? (senior answer) Because it has no real rename or append and no in-place mutation. The key space is flat; "folders" are prefix queries. Rename is O(copy), append is whole-object read-modify-write. You build mutable semantics — versions, pointers, append — in a metadata DB above it deliberately, rather than expecting the store to provide them.
Why can't you point Postgres's data directory at S3? (senior answer) fsync semantics (no ordered durability barrier → no crash recovery), random IO (whole-object 10–100+ ms is orders of magnitude too slow for page reads), and no in-place writes (a row update would rewrite a whole file). Run the DB on block; use object storage for backups, WAL archive, and lake tables. Lakehouse engines that "use S3" interpose a page/cache layer that re-creates block-like access.
A million 1 KB files vs. one 1 GB file — same bytes, what's different? (senior answer) The big file is a bandwidth problem (one metadata touch, then stream); the million small files are a metadata-QPS problem — ~3 ms per op means ~50 minutes serially with the disk idle. Small files stress metadata; large files stress bandwidth. "Lots of tiny files" should make you pack them or reach for object storage.
Where does the file-storage metadata bottleneck bite, and how do you fix it? (senior answer) Path ops serialize on metadata structures; the small-file storm and the hot directory (everyone createing into one dir, contending on its inode lock) saturate metadata before data. It's lesson 07's hot shard at the filesystem layer — fix it by sharding the path tree across many directories, or stepping off the file interface onto object keys with no shared inode.
You're storing user video — object store or a blob column in the DB? (senior answer) Bytes in object storage, metadata + key in the database. Blob columns bloat the DB and wreck backups, and push bytes through the priciest tier. The cost is two-system drift; pay it back with write-bytes-first/commit-pointer-last and a GC sweep for orphans.
What does object storage hide that block exposes, and why care? (senior answer) Object hides sharding, replication, erasure coding, and GC behind PUT/GET — giving exabyte durability with zero ops, at the cost of latency and mutation. Block hides nothing above the sector, so you control structure and get fsync/random IO but must bring your own filesystem or DB and can't share it. The "hiding" is exactly the trade.
How would you implement rename and append on top of an object store? (senior answer) Don't touch the bytes. Rename = update the path→key pointer in the metadata DB. Append = write a sequence of immutable chunks and stitch at read time, or use multipart upload (C17) to assemble parts into one object. The bytes stay immutable and cheap; mutation is a small transaction you own.

Related lessons

This primer feeds the storage cases directly. C16 (object storage service) is this page made concrete — the metadata-DB-plus-immutable-bytes split, the GC, the hot-prefix handling all start from "object storage is not a filesystem" here. C17 (large-file upload) is the upload protocol that exists because there's no append and PUT is whole-object: multipart upload is how you assemble one big immutable object from parts. On the foundation side, lesson 04 (data modeling & storage) and DDIA Ch. 3 are the spine for why a DB wants block (B-trees, LSM-trees, WAL, fsync, random vs. sequential IO), and lesson 07 (partitioning) is the hot-shard pattern that the hot-directory and hot-prefix failures are special cases of.

C16 · Object storage C17 · Large-file upload 04 · Data modeling & storage 07 · Partitioning 08 · Replication

Takeaway

All three interfaces sit on the same media; the interface is a promise about latency, sharing, and mutation. Block hides nothing above the sector — random IO, fsync, in-place writes — which is exactly why databases and VM disks live there and why you can't point Postgres at S3. File adds a shared POSIX namespace and pays for it with a metadata server that becomes the bottleneck (the small-file storm, the hot directory). Object trades away mutation and low latency for planet-scale cheap durability, and its flat, immutable, no-append nature is not a bug to fight but a contract to build above — the load-bearing senior insight of the whole storage track. Pick by access pattern, check it against the latency/IOPS/metadata ladder, and remember that the capability the interface throws away is the one you'll otherwise rebuild in application code.