all_lessons / system_design / cases / C36 C36 / C44

Payment security, SWIFT, and foreign exchange

Three adversaries push on this design at once: attackers who want the card numbers, regulators who want a screened-and-auditable trail, and a foreign-exchange market that re-prices the deal every second while your money is in flight. The architecture's job is to make the unsafe states unrepresentable.

Payments Security & compliance External rails Trade-off first
First principle
In most systems a bug costs you a retry. In payments, three different forces turn a bug into a permanent loss. (1) Secrets: a leaked PAN (primary account number) is fraud you can't recall. (2) Regulation: wiring money to a sanctioned party is a fine and a criminal-liability event, not a 500. (3) Markets: an FX rate is only true for a few seconds — settle on a stale rate and someone eats an unhedged loss. So the design is governed by a single discipline: shrink the blast radius of each. Keep raw secrets in the smallest possible zone, make the screening decision before money moves, and make the rate-expiry check atomic at the instant of commit. Everything else is plumbing.

The shape of the problem

Strip away the jargon and a cross-border payment is four sequential gambles, each with its own failure currency:

This case is the security, FX, and compliance layer. We deliberately defer two things it sits on top of: the double-entry ledger that records the money movement lives in C33 (debits/credits, available vs posted balance), and the idempotency machinery that makes a retried "confirm" safe lives in C35 (insert-if-absent key, request hash, TTL). Here we cover only our delta: where raw secrets are allowed to exist, the atomicity of quote expiry, and synchronous-vs-asynchronous screening.

The trust-zone diagram: where raw PAN never crosses

The single most important picture in a payments design is the data-classification boundary. The raw PAN is radioactive: it must exist in exactly one narrow zone (the tokenization vault) and be replaced by an opaque token everywhere else. The compliance scope (PCI-DSS) is exactly "every system the PAN can reach," so the design goal is to make that set as small as possible.

TRUST ZONES — the PAN lives in ONE box; a token crosses every other line. Zone 0: CLIENT Zone 1: VAULT (PCI scope) Zone 2: CORE LEDGER Zone 3: EXTERNAL RAILS ┌──────────────┐ ┌───────────────────────────┐ ┌────────────────────┐ ┌──────────────────────┐ │ browser / │ PAN │ tokenization vault │ │ payment service │ │ SWIFT / correspondent│ │ mobile app │═══════▶│ HSM-backed, encrypted │ │ + double-entry │ │ bank / card network │ │ │ (TLS, │ store. PAN ⇄ token map. │ │ ledger (C33) │ │ │ │ enters card │ one │ │ │ │ │ async, T+0..T+2 │ └──────────────┘ hop) │ returns ► token "tok_9f4" │ │ sees only token │ │ sees only token / │ └───────────────┬───────────┘ └─────────┬──────────┘ │ masked PAN/IBAN │ │ token only │ token only └──────────────────────┘ └───────────────────────▶│──────────────────────▶ ╔══════════════════════════════════════════════════════════════════════════════════════════╗ ║ PAN crosses ONLY the double line (client → vault). Token crosses every single line. ║ ║ Breach of Zone 2 or 3 leaks tokens, not card numbers → blast radius contained to Zone 1. ║ ╚══════════════════════════════════════════════════════════════════════════════════════════╝

Read that boundary as a containment argument, not a checklist. The vault is the only system in PCI scope; it is small, audited, HSM-backed, and rarely changed. The big, fast-moving application — the part with a thousand commits a month and the largest attack surface — only ever holds tok_9f4…, which is worthless off-platform. The whole point of tokenization is that the systems most likely to be breached hold the data least worth stealing.

Put numbers on it

Two numeric forces drive the architecture. First, the FX quote lifecycle: a quote is valid for a short window, but settlement on the external rail completes at T+2 (two business days later for a wire). The provider who locked your rate is exposed to market drift for that entire holding period.

Worked example — rate-drift risk over the holding period
A quote on EUR/USD locks a rate for a notional of $10{,}000{,}000. Annualised volatility is σ = 8%. The provider is exposed from quote-lock until the hedge settles — call the holding period 2 business days. Scaling vol to that horizon (√-of-time):

σ2d = σ · √(t / 252) = 0.08 · √(2 / 252) ≈ 0.08 · 0.0891 ≈ 0.71%

A one-standard-deviation move on $10M is 0.0071 · 10{,}000{,}000 ≈ $71{,}000 of drift risk per locked deal, per holding period. That number is why the quote window is seconds, not hours, and why an expired-quote-used-anyway bug is a direct, uncapped loss — every second of staleness is unhedged exposure the provider never priced in.

Now the much shorter window the user lives in: the quote is valid 30 s. That is the budget between "show the user a rate" and "user clicks confirm." It is deliberately short — short enough that σ over 30 s is negligible (0.08·√(30/(252·6.5·3600)) ≈ 0.018%, basically zero), so the provider can honour it for free. Stretch it to 30 minutes and the same math says the provider must price in a spread to cover the drift, or re-quote.

The second force is sanctions screening. Every transfer is screened against watchlists before release. Screening adds latency and produces false positives that humans must clear:

Screening latency added
~200 ms / transfer
False-positive rate
~2%
Transfers / day
500,000
Manual reviews / day
≈ 10,000

The review load is the load-bearing number: 500{,}000 · 0.02 = 10{,}000 false positives a day land on a human review queue. At, say, 3 minutes per case that is 10{,}000 · 3 / 60 = 500 analyst-hours/day ≈ 63 full-time reviewers at 8 h/day. That cost is what makes "synchronous vs asynchronous screening" a real architectural decision and not a checkbox — block every transfer on a human and you've throttled the whole product to the review queue's throughput; let everything through and async-screen and you risk releasing a sanctioned payment you can't recall.

Data model & API

Keep the surface small and make the dangerous fields impossible to misuse. Note that instrument stores a token, never a PAN, and quote carries its own expiry so staleness is a first-class field, not an afterthought.

EntityKey fieldsWhy it's shaped this way
instrumenttoken, masked_pan (last 4), typeThe app never sees the PAN. Last-4 is for display/disputes only.
quotequote_id, pair, rate, notional, expires_at, stateExpiry is explicit. State machine: OFFERED → LOCKED → CONSUMED | EXPIRED.
transfertransfer_id, quote_id, idempotency_key (→ C35), screen_status, rail_statusBinds to exactly one quote so a confirmed transfer can't drift to a different rate.
screening_resultlist_version, verdict, evidence, decided_atPins the list version — you must be able to prove what you knew when.
audit_eventseq, actor, action, before/after, tsAppend-only, immutable. The regulator's source of truth.

API: POST /quotes (offer a rate), POST /transfers (confirm against a quote_id, carrying an idempotency key), GET /transfers/{id} (poll rail status). The confirm endpoint is the one with teeth — see the deep dive.

Linearized design: follow a transfer through the zones

  1. Capture. The client posts the card directly to the vault (zone 1), which returns a token. The application server never touches the PAN — it starts the flow already holding only tok_9f4….
  2. Quote. The pricing service offers a rate with expires_at = now + 30 s, state OFFERED. The user sees "100 USD → 91.80 EUR, expires in 30 s."
  3. Confirm. The user clicks. POST /transfers arrives with the quote_id and an idempotency key (C35). This is the atomicity-critical step (deep dive 2): the quote-expiry check and the state transition LOCKED → CONSUMED happen in one atomic step. A stale quote is rejected here, never silently used.
  4. Screen. Both parties are checked against the current watchlist version. Clean → proceed. Hit → the transfer goes to HELD and onto the review queue (deep dive 3). A confirmed false positive is released with its evidence recorded.
  5. Post & release. The double-entry ledger (C33) records the movement; the transfer message is handed to the external rail with the idempotency key as the dedup handle.
  6. Settle. The rail is async (T+0…T+2). We hold rail_status = PENDING until an ack (→ SETTLED) or a rejection (→ REVERSED, triggering a compensating ledger entry). Every state change appends an immutable audit event.

The first bottleneck is naturally exposed at step 4: if screening is synchronous and blocking, the 200 ms (and the worse-case review wait) is on the critical path of every transfer. That tension is deep dive 3.

Deep dives

1. Tokenize vs store credentials — a blast-radius argument

The naive design stores the PAN encrypted in the application database. It feels fine — it's encrypted! — but it gets the threat model backwards. The question isn't "is it encrypted at rest" (it always is); it's "how many systems can produce the cleartext, and how big is each of those systems' attack surface?" If the application that decrypts cards for "convenience" is the same sprawling service with a thousand monthly commits, three contractor teams, and a public API, then your most-attacked system can mint cleartext PANs. Encryption-at-rest doesn't help when the breach is application-level (SQL injection, leaked credentials, a malicious insider with DB access).

Tokenization inverts this. The PAN exists in cleartext only inside the vault — a small, HSM-backed, rarely-changed, heavily-audited service whose only job is the PAN ⇄ token mapping. The application holds tok_9f4…, which is useless anywhere else: it can't be charged off-platform, it carries no value, and a full dump of the application database leaks zero card numbers. The blast radius of an application breach drops from "every card we've ever seen" to "a list of opaque tokens." You've also collapsed PCI-DSS scope to one box, which is the difference between auditing a 200-service estate and auditing one vault. This is the same minimize-the-trusted-set instinct as a fencing token in foundation lesson 10, applied to secrets instead of leadership.

2. The FX quote-expiry atomicity (the sharp one)

Here is the race. A quote is valid 30 s. The user clicks confirm at 29.9 s. The request travels, hits a queue, waits behind a GC pause — and by the time the confirm handler runs the quote is at 30.2 s: expired. What now? The wrong answers are instructive:

The correct design makes the expiry check and the state transition a single atomic conditional, executed where the data lives:

CONFIRM handler — one atomic compare-and-set on the quote row: UPDATE quote SET state = 'CONSUMED' WHERE quote_id = :qid AND state = 'LOCKED' AND expires_at > now() ◀── expiry is part of the SAME guarded write -- rows affected: 1 → quote was valid AND unconsumed → proceed to screen + post 0 → either EXPIRED or already CONSUMED → do NOT post if rows == 0: re-fetch quote.state ├── EXPIRED → return 409 + a FRESH quote; user re-confirms explicitly └── CONSUMED → idempotent replay (C35): return the original transfer, do NOT double-post

The guard AND expires_at > now() AND state = 'LOCKED' rides inside the single row update, so the database's row-level isolation makes check-and-consume atomic — no window exists between deciding "valid" and acting on it. This is the linearizable decision DDIA Ch. 9 calls for: every observer must agree on a single answer to "was this quote consumed, and was it valid at consume time?" with no in-between. An eventually-consistent read of the quote state would reintroduce exactly the stale-rate bug. The product rule that falls out: on expiry, re-quote and require an explicit re-confirm — never auto-renew, never approximate. The user always commits to a rate they actually saw.

Note how cleanly this composes with C35: the state = 'CONSUMED' branch is the idempotent-replay case — a retried confirm of an already-consumed quote returns the original transfer rather than posting twice. We don't re-derive the idempotency-key mechanics here; we just bind the quote state machine to it.

3. Realtime vs async sanctions screening

The 10,000 daily false positives force the choice. Two failure currencies pull in opposite directions:

The senior answer is not "pick one" — it's risk-tier the decision. Screening that can produce a true positive must be synchronous and blocking before the funds touch an irreversible rail; that is a hard regulatory constraint, not a tunable. What you optimise is the false-positive cost: tighten matching, pre-screen and cache verdicts for known-clean repeat counterparties (so a returning payee isn't re-queued), and stage the pipeline — a fast deterministic exact-match pass clears the 98% in 200 ms, and only fuzzy/near-matches escalate to the human queue. The async pattern is reserved for screening that cannot release bad money — e.g. post-hoc anomaly detection on already-settled, low-risk domestic transfers, which is monitoring, not gatekeeping. The list version is pinned on every decision (DDIA Ch. 11-style event sourcing): when a watchlist updates, you can prove what you knew at decision time and re-screen the backlog against the new version.

Trade-offs

ChoiceBuysCostsChoose when
Tokenize (vault) vs store encrypted PAN in appTiny PCI scope; app breach leaks only worthless tokensAn extra hop + a vault to operateAlmost always — the default for any system touching cards
Realtime blocking screening vs async post-hocNo sanctioned payment ever leaves200 ms on critical path; legit transfers can stall in reviewAny transfer to an irreversible rail (i.e. nearly all)
Locked quote vs market rate at settlementUser certainty on the displayed priceProvider carries the drift risk (~$71k/holding period @ $10M)Consumer checkout, where price certainty is the product
Detailed immutable audit vs minimal logsProvable compliance; can reconstruct any decisionStorage + privacy/retention burdenRegulated payments — non-negotiable

The sharpest trade-off is the locked-quote one, because it's where the FX risk math becomes a business decision. Offering a locked rate is a free option you're writing to the user: they will only confirm if the market hasn't moved against them, so you systematically eat the adverse moves. The 30 s window is the lever — short enough that the option is nearly worthless (drift over 30 s is ~0.018%), so you can give it away. The instant the window stretches, the option has value and you must either hedge it or price a spread into the rate. The atomicity of deep dive 2 is what stops that option from being exercised for free past expiry.

Failure modes

FailureMitigation
Expired quote used (stale rate)Atomic expires_at > now() AND state='LOCKED' guard inside the confirm update; on 0 rows, re-quote + explicit re-confirm. Never auto-renew.
Double-confirm / retry double-postsIdempotency key (C35) + the CONSUMED branch returns the original transfer. The atomic CAS makes only one confirm win.
Credential leakPAN exists only in the vault; app holds tokens. Rotate tokens; HSM-backed keys; vault is the only PCI-scoped box.
Sanctions false negativeBlock-before-release for irreversible rails; pin list_version on every verdict and re-screen the backlog when lists update.
External rail rejection / no-ackModel rail status as an explicit state machine (PENDING → SETTLED | REVERSED); rejection triggers a compensating ledger entry (C33). The rail is an unreliable async system (DDIA Ch. 8) — design for timeouts and reconciliation, not request/response.

Interview prompts you should be ready for

  1. A locked FX quote expired between confirm and capture — what does the system do? (senior answer) The expiry check is not a separate step; it's a guard inside the same atomic write that consumes the quote (UPDATE … WHERE state='LOCKED' AND expires_at > now()). If zero rows change, the quote is expired or already consumed — we reject with 409 and a fresh quote that the user must explicitly re-confirm. We never silently use the stale rate (that's unhedged loss) and we never auto-renew (the user must commit to a rate they actually saw). This is a linearizable decision (DDIA Ch. 9): one agreed answer to "valid and unconsumed?", no TOCTOU window.
  2. Why tokenize instead of just encrypting the PAN in your DB? (senior answer) Encryption-at-rest doesn't stop an application-level breach — if the app can decrypt for "convenience," your most-attacked system can mint cleartext. Tokenization confines cleartext to a small HSM-backed vault and gives the sprawling app only worthless tokens, so an app breach leaks nothing chargeable and PCI scope collapses to one box. It's a blast-radius argument, not an encryption one.
  3. Synchronous or asynchronous sanctions screening? (senior answer) Screening that can produce a true positive must block before money touches an irreversible rail — async release means you may have already wired to a sanctioned party with no claw-back. That's regulatory, not tunable. What you optimise is the ~2% false-positive cost: stage a fast exact-match pass that clears 98% in ~200 ms, cache verdicts for known-clean repeat payees, and escalate only fuzzy matches to the human queue (~10k/day at our volume).
  4. The external rail (SWIFT) never acks a transfer — what's the state? (senior answer) It stays PENDING; you do not assume success or failure. SWIFT is an unreliable async system (DDIA Ch. 8) — design for timeouts and reconciliation, never request/response. A rejection transitions to REVERSED and fires a compensating ledger entry (C33). The idempotency key (C35) is the rail's dedup handle so a resend doesn't double-send.
  5. Where does the raw PAN ever appear in your diagram? (senior answer) On exactly one edge: client → vault, over TLS, one hop. After that it's a token everywhere — ledger, screening, rails all see only the token (or a masked last-4 for display). If the PAN appears anywhere else, the design has a bug.
  6. How do you prove to a regulator what you knew at decision time? (senior answer) Append-only immutable audit log (DDIA Ch. 11 event sourcing) with the screening list_version pinned on every verdict. When a watchlist updates you can re-screen the backlog and still reconstruct exactly which list version cleared a given transfer and when.
  7. Why is the quote window 30 s and not 30 minutes? (senior answer) A locked quote is a free option you write to the user; they only confirm if the market hasn't moved against you, so you eat the adverse moves. Over 30 s, vol-scaled drift is ~0.018% — the option is nearly worthless, so you give it away. Stretch the window and the option has real value (~0.71% over T+2 on the holding period); then you must hedge or price a spread.
  8. Isn't the ledger the hard part here? (senior answer) The double-entry ledger and idempotency are real but they're solved elsewhere — C33 owns the debit/credit + available-vs-posted model, C35 owns the idempotency key. I'd link to those and spend my time on this case's delta: the trust-zone boundary, the atomic quote-expiry CAS, and the screening risk-tiering.

Related lessons

This case is a satellite of two anchors. The money-movement record — debits, credits, available vs posted balance — is taught in full in C33 (ledger); here we only bind the quote state machine to a ledger post. The make-a-retried-confirm-safe machinery (insert-if-absent key, request hash, TTL) is taught in C35 (avoid double charge); our CONSUMED branch is just that mechanism applied to a quote. For the underlying primitives: the atomic confirm is foundation lesson 12 / DDIA Ch. 7's transaction isolation, the "one agreed answer to consumed?" property is lesson 09's linearizability, and the credential-abuse / replay defence at the edge leans on lesson 15's admission control to stop carding attacks before they reach the vault.

C33 · Ledger (anchor) C35 · Idempotency (anchor) 09 · Consistency / linearizability 12 · Distributed transactions 15 · Rate limiting & edge
Takeaway
A cross-border payment fights three adversaries with three failure currencies — stolen secrets, regulatory fines, and FX loss — so the whole design is a discipline of shrinking blast radius. Confine the raw PAN to one HSM-backed vault and let everything else hold worthless tokens. Make the FX quote-expiry check atomic with the consume so a stale rate can never be silently used — on expiry, re-quote and re-confirm, never auto-renew. Screen synchronously before money touches an irreversible rail, and optimise only the false-positive cost, not the safety. Lean on C33 for the ledger and C35 for idempotency; spend your own words on the trust boundary, the atomic CAS, and the screening tier.