Concurrency II: atomics, the memory model, and lock-free thinking

Lesson 17 left you with one weapon against the data race: a std::mutex, taken with a RAII lock, that serializes access so only one thread touches shared state at a time. That is the right default, and you should reach for it nearly always. But it raised two questions it could not answer. First, why is an unguarded ++counter a race at all — what exactly is indivisible and what is not? Second, can you ever share data without a lock — and if so, what are you trading away? Answering both forces us to confront the one piece of C++ we have so far taken on faith: that memory operations happen in the order you wrote them. They do not. The compiler (Lesson 01) and the CPU both reorder your reads and writes, and the only reason your single-threaded code still works is that they preserve its observable behavior. The moment a second thread observes that memory, the reordering becomes visible — and that is the subject of this lesson. The tool that tames it is std::atomic, and the contract it lives under is the C++ memory model.

The thesis, here

The memory model is the zero-overhead principle applied to ordering. The standard does not promise that writes from one thread appear to another in program order, because guaranteeing that on every access would tax every program — including the single-threaded majority that never shares memory — with fences the hardware would otherwise skip. Instead it gives you atomics with selectable orderings: you pay for exactly the synchronization you ask for, and not a cycle more. The bill is itemized down to the individual memory operation.

Linear position

Prerequisite: Lesson 17 — you must already know what a data race is (two threads, one memory location, at least one write, no synchronization), why shared mutable state is the enemy, and how a std::mutex with a RAII lock_guard prevents the race. This lesson explains the layer underneath the mutex.
New capability: read and write shared state correctly without a lock for the cases that warrant it — explain why an ordinary int can tear and why ++ isn't atomic, choose a memory ordering (seq_cst / acquire-release / relaxed) by reasoning about what it permits, and recognize when lock-free is the wrong tool, which is most of the time.

The plan

Six moves. (1) Decompose ++counter into its three machine steps and pin the race from Lesson 17 to exactly one of them. (2) Show that ordinary reads and writes aren't even guaranteed to be atomic — they can tear. (3) Introduce std::atomic<T>: the operation made indivisible. (4) The hard truth — the compiler and CPU reorder memory, and why that is legal and usually invisible. (5) The three orderings — seq_cst, acquire/release, relaxed — explained as how much reordering you forbid, with a worked publish/consume example. (6) The honesty section: lock-free is hard, error-prone, and rarely worth it; prefer the mutex.

1 · Why `++counter` is three steps, not one

In Lesson 17 the canonical race was two threads each running ++counter on a shared int, and the final value coming out too small. To see why, you have to stop reading ++counter as a single action. It is not. On essentially every CPU, incrementing a variable that lives in memory is a read-modify-write sequence — three distinct machine operations:

// counter is a shared int in memory.   ++counter  becomes, roughly:
int tmp = counter;   // 1. READ   — load the current value into a register
tmp = tmp + 1;       // 2. MODIFY — add one, in the register
counter = tmp;       // 3. WRITE  — store the result back to memory

A register is one of the CPU's handful of tiny, ultra-fast storage slots where arithmetic actually happens; memory cannot be added to directly, so the value must be loaded, modified, and stored back. Between any two of those three steps, the operating system can suspend this thread and run the other one. Trace it with both threads starting from counter == 0:

Worked example — the lost update, step by step

thread A: READ counter -> tmp_A = 0 thread B: READ counter -> tmp_B = 0 (A hasn't written yet!) thread A: MODIFY -> tmp_A = 1 thread B: MODIFY -> tmp_B = 1 thread A: WRITE -> counter = 1 thread B: WRITE -> counter = 1 (overwrites A's 1 with its own 1)

Two increments happened; counter ended at 1, not 2. One update was lost. The bug lives entirely in the gap between READ and WRITE: B read a stale value because A's read-modify-write was not indivisible. Run a million such increments across many threads and you lose tens of thousands. This is the exact race from Lesson 17, now located at the instruction level — and the fix is to make those three steps one step that nothing can interleave.

2 · Ordinary reads and writes can tear

You might hope the problem is only the multi-step ++, and that a plain assignment like x = 42 or a plain read like int y = x is at least safe on its own. It is not guaranteed to be. The standard says that if one thread writes a non-atomic variable while another reads or writes the same variable with no synchronization, the program has a data race, which is undefined behavior (Lesson 19) — full stop, regardless of how simple the operation looks.

There is even a concrete mechanism behind that rule, called tearing. A store of a value that is wider than the CPU's atomic write granularity can be split into two parts. If a 64-bit value is written as two 32-bit stores, another thread can read after the first half and before the second, observing a value that is neither the old one nor the new one but a mangled splice of both — a value that was never assigned at all. Whether a given type tears depends on size and alignment (Lesson 08) and the specific hardware, which is precisely why the standard refuses to promise atomicity for ordinary types: it would force the compiler to emit guarantees the program may not need. You cannot rely on "it's just an int, the write is one instruction." Sometimes it is; the standard does not say so; and code that depends on accidents of the target is not correct C++.

3 · `std::atomic<T>` — the operation made indivisible

An atomic operation is one the system guarantees cannot be interrupted partway through: it either has not happened or has fully happened, and no other thread can observe an in-between state. std::atomic<T> (from <atomic>) wraps a type T and turns its loads, stores, and read-modify-write operations into single indivisible steps. The three-step increment from §1 collapses into one:

#include <atomic>
#include <thread>

std::atomic<int> counter{0};        // an atomic integer, started at 0

void work() {
    for (int i = 0; i < 1'000'000; ++i)
        ++counter;                  // ONE indivisible read-modify-write
}                                   // no torn reads, no lost updates

int main() {
    std::thread t1{work}, t2{work};
    t1.join();
    t2.join();
    // counter is GUARANTEED to be exactly 2'000'000.
    // counter.load() reads the current value; .store(v) writes one.
}

The hardware provides this with a dedicated atomic read-modify-write (on x86 a lock-prefixed add, which reserves the cache line — Lesson 08 — for the duration of the operation; other CPUs such as ARM use a load-linked/store-conditional retry loop), so there is no mutex, no thread ever blocks, and the cost is a single, slightly more expensive instruction rather than a kernel call. atomic<T> also offers compare_exchange_weak/strong (atomic "if the value is still what I last saw, swap in this new value, otherwise tell me the current value"), which is the primitive most lock-free algorithms are built from. Crucially, atomicity is not the whole story: the increment is now indivisible, but we have said nothing yet about the order in which other memory operations around it become visible to other threads. That is the next, harder, half.

4 · The hard truth: memory operations get reordered

Here is the assumption every program in this series has quietly relied on and which is now false: that the statements in your function execute, and become visible to memory, in the order you wrote them. Two layers are allowed to reorder them.

the compiler reorders

The optimizer (Lesson 01, Lesson 09) freely rearranges, merges, and eliminates reads and writes — hoisting a load out of a loop, caching a value in a register, reordering two independent stores — whenever doing so produces faster code with the same single-threaded result.

the CPU reorders

Even after the compiler emits instructions in some order, the processor may execute them out of order and let writes sit in a per-core store buffer, so the order in which one core's writes become visible to another core is not the program order either.

why this is legal

Both are bound by one rule: preserve the behavior of the program as if it ran on one thread. Reordering two independent stores changes nothing observable to a single thread, so it is allowed — and it is a huge source of speed, which is exactly why the standard permits it by default.

The catch is the phrase "as if single-threaded." A second thread watching the same memory is an observer outside that as-if guarantee. It can see your two stores land in the opposite order, because nothing in your thread's behavior depended on the order, so the system felt free to swap them. This is why the naive flag pattern below is broken:

int data = 0;
bool ready = false;            // plain bool — no synchronization

// producer thread                 // consumer thread
data = 42;                         while (!ready) { /* spin */ }
ready = true;                      use(data);   // expects data == 42

It looks airtight: the producer writes data first, then sets the flag; the consumer waits for the flag, then reads data. But the producer's two stores are independent in single-threaded terms, so the compiler or CPU may make ready = true visible before data = 42. The consumer then sees ready, breaks out of the loop, and reads data while it is still 0. (And because ready is read and written concurrently without synchronization, this is a data race — UB, Lesson 19 — so the compiler is additionally free to assume it never happens and, say, hoist the ready check out of the loop entirely and spin forever.) To fix it you must forbid the reordering across the flag, and that is what memory orderings are for.

5 · The three orderings: how much reordering you forbid

Every atomic operation takes an optional memory order argument that controls how memory operations around it may be reordered relative to it, and therefore what other threads are guaranteed to see. Read them as a dial from "most ordering, easiest to reason about" to "least ordering, fastest, most dangerous."

Ordering	What it guarantees	Use it for
`memory_order_seq_cst`	Sequential consistency — the default. All `seq_cst` operations across all threads appear to happen in one single global order that every thread agrees on. Nothing is reordered past them. This is the model your intuition already has.	Everything, until profiling proves you need less. The safe default.
`release` / `acquire`	Pairwise synchronization. A release store "publishes": every write the thread did before it is guaranteed visible to any thread that does an acquire load which reads that value. The acquire "sees" everything before the matching release. Ordering is only between this pair, not globally.	Producer/consumer hand-off, publishing a built object behind a flag, lock implementations.
`relaxed`	Atomicity only, no ordering. The operation is indivisible (no tearing, no lost updates) but imposes no constraints on how surrounding memory operations are ordered or when they become visible.	Counters and statistics where you need a correct total but never use the count to gate access to other data.

The mental model for acquire/release is a publish / subscribe handshake. A release store is a sealed package: everything you wrote before sealing it is inside. An acquire load that receives that exact package is guaranteed to find everything that was inside — nothing the producer did before the release can "leak out" to after it, and nothing the consumer does after the acquire can be pulled before it. That one-directional barrier is all you need to fix §4's flag:

Worked example — a correct lock-free publish

std::atomic<bool> ready{false};
int data = 0;                       // ordinary int is fine now — the
                                    // atomic flag carries the ordering

// producer                              // consumer
data = 42;                               while (!ready.load(
ready.store(true,                            std::memory_order_acquire))
    std::memory_order_release);              { /* spin */ }
                                         use(data);   // GUARANTEED 42

Why it works. The release store on ready forbids the prior data = 42 write from being reordered after it — when the flag becomes visible, the data write already is too. The acquire load on the consumer side forbids the later use(data) read from being hoisted before the flag check. The moment the consumer's acquire reads the true that the producer's release wrote, the pair synchronizes and the producer's earlier writes are visible. No mutex, no blocking, no torn data — and notice we paid for ordering on exactly one variable (ready) and nowhere else. That is the zero-overhead principle (Lesson 00) operating at the granularity of a single memory access.

And the counter from §3? Its increments don't gate access to any other memory — you only ever want a correct final total — so it is the textbook case for relaxed: counter.fetch_add(1, std::memory_order_relaxed). Each add is still atomic (no lost updates), but you save the CPU the cost of ordering you do not need. The default ++counter from §3 is seq_cst, which is correct but stricter — and slower — than this counter requires.

6 · Honesty: lock-free is hard — prefer the mutex

Everything in §5 is real, but it comes with a warning that is part of the engineering, not a footnote to it. Reasoning about which reorderings each ordering permits is genuinely difficult; the bugs are non-deterministic (they appear once in ten million runs, on one CPU model, under load), they often cannot be reproduced in a debugger because attaching one changes the timing, and acquire/release code that looks obviously correct is routinely wrong in ways that take experts days to find. Lock-free data structures (queues, stacks) additionally have to solve hazards like the "ABA problem" that the mutex version never faces.

Failure modes this lesson is honest about

Reaching for atomics to "go faster" without measuring. A mutex protecting a short critical section is often faster than hand-rolled atomics and is vastly easier to get right. Profile first (Lesson 17's rule); a lock is not slow until you have proof it is your bottleneck.
Using relaxed for a flag that gates other data. Relaxed gives atomicity but no ordering, so the §4 reordering bug comes straight back. Relaxed is for counters that gate nothing.
Assuming std::atomic makes a sequence of operations atomic. Each individual operation is atomic; if (a.load()) a.store(...) is two atomic operations with a race-able gap between them. Use compare_exchange or a lock for compound updates.
Mixing atomic and non-atomic access to the same variable. Still a data race, still UB (Lesson 19).

The rule that should survive this lesson: use a mutex. Reach for atomics only for the genuinely simple cases (a shared counter, a single publish flag) or when profiling has proven a lock is the bottleneck and you have the expertise to verify the alternative. Lock-free is a specialist tool, not a default.

Checkpoint exercise

Try it

Take the §1 race: two threads each doing ++counter a million times on a shared int. (1) Run it and print the result a few times — you'll see values below 2'000'000 that vary run to run. (2) Confirm the diagnosis by building with -fsanitize=thread (ThreadSanitizer, the TSan you'll meet again in Lesson 19); it will report the data race with both stack traces. (3) Change int to std::atomic<int> and rerun — the result is now exactly 2'000'000 and TSan is silent. (4) Now switch the increment to counter.fetch_add(1, std::memory_order_relaxed): still exactly 2'000'000, because the count gates no other memory. (5) Finally, try the §4 broken flag with a plain bool under TSan, then fix it with the §5 acquire/release pair and watch the race report disappear. Predict each outcome before you run it; the surprises are where your mental model still trusts program order.

Where this points next

Notice what made the broken flag in §4 "broken": not a crash, but a program whose behavior the standard declares undefined the instant two threads touch one location without synchronization — which is exactly why the compiler felt free to reorder, cache, and even delete the operations that produced the bug. A data race is undefined behavior, and undefined behavior is what licenses the optimizer to assume your races never happen. Lesson 19, the capstone, makes that idea the whole subject: undefined behavior as the deliberate price of the zero-overhead principle stated back in Lesson 00. You will see why "all bets are off" is not a threat but a contract — the same contract that let every abstraction in this series compile down to hand-written speed — and how it can bite by deleting your null checks and traveling backward in time. The reordering you just met is one face of it.

Takeaway

A plain ++counter is a three-step read-modify-write (read, add, store), and a thread switch in the gap loses updates — the Lesson 17 race, pinned to the instruction level; even a single ordinary read or write can tear and is a data race (UB) under contention. std::atomic<T> makes each load, store, and read-modify-write indivisible, fixing atomicity. But atomicity is not ordering: the compiler and CPU freely reorder memory operations because they only owe you single-threaded behavior, and a second thread observing that memory sees the reordering — which silently breaks naive flag patterns. The C++ memory model lets you forbid exactly as much reordering as you need: seq_cst (one global order, the safe default), acquire/release (a pairwise publish/subscribe hand-off — a release publishes everything before it, a matching acquire sees it all), and relaxed (atomicity only, for counters that gate nothing). This is zero-overhead applied to ordering: pay per memory operation. And the honest conclusion is to prefer the mutex — lock-free is hard, its bugs are non-deterministic and nearly un-debuggable, and a lock is not your bottleneck until profiling proves it.

Interview prompts

Why is ++counter on a shared int a race even though it's one line of source? (§1 — it compiles to a read-modify-write of three machine steps; a thread switch between the read and the write lets a second thread read a stale value, so one update is overwritten and lost.)
Aren't plain reads and writes of a single int at least atomic? (§2 — not guaranteed; a wide store can be split into parts and another thread can observe a torn, never-assigned value, and concurrent unsynchronized access is a data race / UB regardless of how simple the operation looks.)
What does std::atomic<T> guarantee, and what does it not? (§3, §5 — it makes each individual load/store/RMW indivisible (no tearing, no lost updates); it does not by itself make a sequence of operations atomic, and ordering of surrounding memory depends on the chosen memory_order.)
Why can the compiler and CPU reorder your memory operations, and when does it bite? (§4 — both need only preserve single-threaded observable behavior, so independent reads/writes may be reordered for speed; a second thread is outside that as-if guarantee and can observe the reordered order, breaking naive flag/data hand-offs.)
Explain acquire/release in terms of publish/subscribe. (§5 — a release store publishes a package containing every write before it; an acquire load that reads that value receives the whole package, so prior writes are visible — a one-directional barrier between exactly that pair.)
When is relaxed the right ordering, and when is it a bug? (§5, §6 — right for a counter/statistic whose value gates no other memory; a bug for any flag used to publish other data, because relaxed gives atomicity with no ordering and the reordering hazard returns.)
You think atomics will speed up your hot path — what do you do first? (§6 — measure; a mutex over a short critical section is often faster and far easier to get right, lock-free bugs are non-deterministic and nearly un-debuggable, so you only go lock-free when profiling proves the lock is the bottleneck and you can verify the alternative.)