User privacy protection

A recommender is, structurally, a machine for learning intimate facts about people: every click, dwell, and skip is logged, joined, and turned into a prediction. The same pipeline that powers personalization is a standing privacy hazard. The spine of this lesson is one curve — privacy ↔ utility — and the one number that quantifies it: ε.

The contract of this lesson

Privacy is not a feature you bolt on — it is a constraint you design against, exactly like latency or memory. The right mental model is a budget: every released statistic, every query, every model update spends some quantifiable privacy. The job is to maximize model quality subject to not overspending that budget. Everything below is a way to account for, or reduce, the spend.

Why a recommender is a privacy hazard

Behavior logs are not anonymous by nature — they are re-identifying by nature. A handful of watched videos, a few purchased items, or a sequence of timestamped locations is often unique to one person. The textbook result: in any large behavioral dataset, the combination of just a few "quasi-identifiers" (zip + birthdate + gender, or three timestamped check-ins) singles out the overwhelming majority of individuals.

So the threat is not "someone steals the user_id column." The threat is linkage: an attacker holding an auxiliary dataset — public reviews, a leaked table, their own knowledge of you — joins it against your "anonymized" release and recovers identities. The information that makes data useful for recommendation (the unique behavioral fingerprint) is exactly the information that re-identifies. This is the foundational fact the rest of the lesson is built to handle.

The Netflix-Prize lesson

Netflix released 100M ratings with user-ids replaced by random numbers — "anonymized." Researchers cross-referenced it with public IMDb reviews and re-identified specific subscribers, exposing their full (including politically and medically sensitive) rating histories. Stripping identifiers is not anonymization. Two people who each rated a few of the same obscure films are almost certainly the same person. The fingerprint survives.

Anonymization and why it fails: k-anonymity

The first-generation answer was k-anonymity: generalize and suppress the quasi-identifiers until every record is indistinguishable from at least k−1 others. Age 25 → [20,30); full GPS → city; a specific video → its category. A release is k-anonymous if every combination of quasi-identifier values appears in ≥ k rows, so any single lookup returns an "equivalence class" of size ≥ k, not one person.

It is intuitive and cheap, and it is not enough. Two classic attacks break it:

Homogeneity attack. If all k rows in an equivalence class share the same sensitive value — everyone in the "[20,30), female, this neighborhood" bucket watched the same medical-condition video — then identifying the group reveals the secret without ever singling out the individual. k-anonymity protects the identifier, not the attribute. The patches ℓ-diversity and t-closeness demand varied sensitive values within a class, and are themselves brittle.
Background-knowledge / linkage attack. k-anonymity only generalizes the columns you labeled as quasi-identifiers. An attacker with side information can treat any field as an identifier. The guarantee is relative to an assumed attacker, and real attackers do not respect your assumptions.

The deep reason both attacks land: k-anonymity is a property of the output table, not of the process that produced it. It makes no promise about what an adversary can infer when armed with outside data. The fix is to flip the question — instead of "is the released data hard to re-identify?", ask "does any single person's data measurably change what I release?" That is differential privacy.

Differential privacy — the rigorous answer

Differential privacy (DP) does not try to hide individuals inside a crowd. It makes a stronger, process-level promise: the output of an analysis is (almost) the same whether or not your data was included at all. If your presence in the dataset barely moves the result, then nothing about the result can be blamed on you — so observing the result tells an attacker almost nothing about you, regardless of what side knowledge they hold. This is exactly what makes DP immune to the linkage attacks that kill k-anonymity.

Neighboring datasets

Two datasets D and D′ are neighbors if they differ by exactly one person's records (one row added or removed). DP bounds how differently a randomized algorithm M can behave on neighbors. "Your data was in" vs. "your data was out" are neighbors — so a DP guarantee is precisely a per-person guarantee.

The ε-definition

A randomized mechanism M satisfies ε-differential privacy if for every pair of neighboring datasets D, D′ and every possible set of outputs S:

Pr[ M(D) ∈ S ] ≤ e^ε · Pr[ M(D′) ∈ S ]

Read it plainly: the probability of any outcome changes by at most a multiplicative factor e^ε when one person joins or leaves. ε is the privacy budget (also called the privacy loss):

ε → 0: e^ε → 1, the output distribution is identical with or without you — perfect privacy, and (with enough noise) useless data.
ε small (≈ 0.1 – 1): strong privacy. Each person is nearly invisible.
ε large (≥ 10): weak — the output may swing wildly on one person, so the "guarantee" is vacuous.

The relaxed form (ε, δ)-DP lets the bound fail with tiny probability δ (think δ < 1/N, smaller than one over the dataset size): Pr[M(D)∈S] ≤ e^ε · Pr[M(D′)∈S] + δ. The slack δ is what lets us use Gaussian noise (below), which is the convenient choice in ML.

The mechanisms: calibrate noise to sensitivity

You achieve DP by adding random noise to the result, scaled to how much one person could possibly change it. That worst-case influence is the sensitivity of the query f:

Δf = max over neighbors D, D′ ‖ f(D) − f(D′) ‖

For a count ("how many users watched cooking videos?"), one person changes it by at most 1, so Δf = 1. The two workhorse mechanisms:

Laplace mechanism (pure ε-DP) : M(D) = f(D) + Lap(0, Δf / ε) Gaussian mechanism ((ε,δ)-DP) : M(D) = f(D) + 𝒩(0, σ²), σ = Δf · √(2 ln(1.25/δ)) / ε

The single most important relationship is in the denominator: noise scale ∝ Δf / ε. Smaller ε (more privacy) ⇒ bigger noise ⇒ noisier answer. Larger sensitivity ⇒ bigger noise. This is the privacy–utility tradeoff written as one fraction.

Two properties that make ε an actual budget

Composition. Run an ε₁-DP analysis and then an ε₂-DP analysis on the same people, and the total leakage is at most (ε₁ + ε₂)-DP. Privacy loss accumulates across every query and every training step — which is why a real system tracks a running ε ledger and stops releasing once the budget is spent. (Advanced composition and Rényi-DP accounting give tighter-than-linear bounds, which is what real DP-SGD libraries use.)
Post-processing immunity. Anything you compute from a DP output — aggregations, a trained model, a dashboard — is still DP with the same ε. You cannot "un-protect" data by analyzing it further. This is why a DP-trained model can be shipped freely.

DP for training: DP-SGD in one line

To train a model with DP, make each gradient step DP: clip every per-example gradient to a fixed norm C (this caps the sensitivity at Δf = C), then add Gaussian noise 𝒩(0, σ²C²) to the batch-averaged gradient. Composition over all steps — tracked by a "moments accountant" — gives the final (ε, δ). The noisy, clipped gradients are exactly what stop the model from memorizing any single user, at a measurable cost in accuracy. Random subsampling of batches amplifies privacy, buying back some of that cost.

The privacy–utility curve — made concrete

Everything above collapses to one tradeoff: more noise ⇒ more privacy ⇒ a worse estimate / weaker model. The widget makes the fraction Δf/ε tangible. A platform might count how many of its users watched cooking videos; DP lets it publish that count without the count itself becoming evidence about any one viewer — but only by blurring it. Slide ε down for stronger privacy and watch the relative error degrade; then drag the true count toward 100 to see why long-tail signals are the hard case.

Worked numbers — DP is nearly free on aggregates and brutal on the tail

Laplace noise has scale b = Δf/ε; with Δf = 1 and ε = 1 the typical error is ~1, regardless of the true count. So the same fixed ±1 noise lands completely differently depending on what you're measuring: a count of 1,000,000 → relative error ≈ 1/10⁶ = 0.0001% (effectively free); a count of 1,000 → ≈ 0.1% (fine); a count of 10 (a niche subgroup) → ≈ 10% (unusable). The lesson for a recommender: publishing DP population statistics (overall CTR, top categories) costs almost nothing, but DP estimates for small segments, rare items, or new creators are swamped by noise — which is exactly where you'd most want the signal, and exactly why per-user personalization under tight ε is the genuinely hard regime.

The bill, in production numbers

The relative-error story above is the statistical cost of DP. A shipped system pays three more bills — in training time, serving latency, and a recurring ε spend — and an interviewer wants the order of magnitude of each, not just the direction. These are the numbers worth memorizing for a recsys privacy design:

What you pay	Mechanism	Order of magnitude	Why
Model quality	DP-SGD on a ranker (e.g. DeepFM)	AUC down 3–5% at useful ε; a tight ε = 0.1 ≈ 2% AUC drop, clawed back to ≈ 0.5% with feature compensation (extra context/content features that don't need noise)	clipped + noised gradients can't memorize the sharp per-user signal
Training time	DP-SGD	+20–40% wall-clock to converge	per-example gradient clipping kills the vectorized batch grad, and the noise slows convergence so you need more steps
Serving latency	crypto path (federated + HE/MPC inline)	50 ms → 200 ms (≈ 4×); keep any GPU HE inference < 50 ms or it breaks the budget	ciphertext ops + interactive rounds on the request path
Memory / bandwidth	homomorphic encryption	~1000× ciphertext blow-up, ~100× compute	a single scalar becomes a large polynomial; FHE is still infeasible for training, additive-HE (Paillier) only viable for the sum in secure aggregation

The ε ledger — composition turns one number into a running balance

Composition (above) means ε is not spent once — it accrues over every repeated query on the same users, day after day. So a real system runs a privacy ledger, exactly like a cost budget. A concrete policy: grant each user a daily budget ε_daily = 0.1, and cap the monthly total at ε_total < 1.0 via advanced (Rényi) composition.

Naïve linear composition would let 0.1 × 30 = 3.0 through in a month — already a weak guarantee. Advanced and Rényi composition bound the cumulative loss sub-linearly: the spend grows asymptotically like √k·ε rather than k·ε for k queries. Be careful with the constant, though — the advanced-composition bound carries a √(2 ln(1/δ)) factor (≈ 4.8 at δ = 10⁻⁵), so at small k (a few dozen queries) it barely beats the linear sum; the dramatic savings appear at the thousands-of-steps scale of DP-SGD training, where linear composition would be hopeless. This is why production DP libraries track the budget with a Rényi-DP (moments) accountant rather than the textbook sum — and why the running ledger, not a back-of-envelope √k, is what actually enforces the cap. When the ledger hits its cap, you stop releasing — the same discipline as a spend limit. For the regulatory framing of this ledger (consent, audit, the lawful-basis layer it sits under), see 39 · Privacy, compliance & computing.

Interactive · ε ↔ noise ↔ utility

A DP count query under the Laplace mechanism. One user can change the true count by Δf, so the noise has scale Δf/ε. The published value is one random draw; the typical error is the noise scale itself. The diagnosis calls out which regime you are in.

A differentially-private count

Defaults are a strong, shippable setting: ε = 1, a large population count, Δf = 1. Slide ε down for more privacy, or the count down toward the long tail, and read what breaks. The numbers are exact for this query; the verdict is the interesting part.

ε (privacy budget) 1.00 true count 50000 sensitivity Δf 1

noise scale b = Δf/ε

—

e^ε (max odds shift / person)

—

published ≈

—

relative error

—

Reading

—

Two lessons fall out of playing with it. (1) For a large aggregate (a population-level count), even strong privacy (small ε) costs almost nothing in relative terms — DP shines on big aggregates. (2) The same noise on a small or per-user quantity is devastating — which is exactly why protecting long-tail / niche interests is hard, and why over-aggressive DP flattens personalization for minority tastes. The standard mitigation is to spend a smaller ε (more noise) on the dense head and reserve structural protections for the sparse tail, rather than noising everything uniformly into uselessness.

Architectural privacy — shrink the data before you protect it

DP bounds leakage from data you hold. The cheaper move is to not hold it. Three architectural levers, in rough order of how much they change the system:

Data minimization. The legal "minimum necessary" principle, as engineering: collect only fields the objective needs (clicks, not the photo library), at the coarsest granularity that works (city, not GPS), retained for the shortest useful window (a 7-day sequence for the real-time model; 180 days for long-term interest), used only for the stated purpose. Every byte you never collect is a byte you never have to protect, leak, or delete.
On-device inference. Run the ranking model (or its last layers) on the phone so raw behavior never leaves it; only the chosen items, or coarse aggregates, come back. The user's fingerprint stays local.
Federated learning. "Move the model, not the data." Devices train on local data and upload only gradient updates, which a secure-aggregation protocol (SecAgg) sums so the server sees only the aggregate — never an individual update. Crucially, gradients still leak (membership-inference, gradient-inversion), so production federated systems also clip and add DP noise to the updates — DP and federation are complementary, not alternatives. Full treatment in 23 · Federated learning.

Technique	What it protects	Guarantee strength	Main cost	Beats linkage?
Masking / hashing IDs	direct identifiers	weak (reversible via linkage)	~none	No
Generalization + k-anonymity	quasi-identifiers in a release	weak (homogeneity, background knowledge)	coarser features	No
Differential privacy (ε)	any single person's contribution	provable, attacker-agnostic, composable	noise ⇒ accuracy loss (worst on small / sparse)	Yes
Federated learning	raw data location	medium (gradients leak; needs DP + SecAgg)	comms, Non-IID, device compute	Partly
On-device inference	raw data location at serving	medium (the model itself can leak)	device compute, model-size limits	Partly
Homomorphic enc. / MPC / TEE	data in use (compute on encrypted)	strong (cryptographic)	10–100× latency / orchestration	Yes (confidentiality)

Encryption and DP solve orthogonal problems: encryption stops an eavesdropper from seeing the computation; DP stops the legitimate output from revealing an individual. You can need both — a federated update can be encrypted in transit and DP-noised so the aggregate itself is safe. Note also local vs. central DP: in local DP each device noises its own data before upload (no trusted server, but more total noise); in central DP a trusted aggregator noises the aggregate (less noise, stronger trust assumption). Federated systems usually want the local flavor.

Privacy-computing primitives — HE vs SMPC vs TEE

The table's "compute on encrypted data" row hides three genuinely different primitives, and interviewers like to hear them separated. They all answer "protect the data in use" — orthogonal to DP, which protects the legitimate output — but they make very different trust and cost bargains.

Homomorphic encryption (HE) lets you compute directly on ciphertext and decrypt the result: no party ever sees the plaintext, so there is no trusted party at all. Partially-HE (additive only, e.g. Paillier) supports the one operation secure aggregation actually needs — summing encrypted gradients — and is the practical flavor. Fully-HE evaluates arbitrary circuits but remains orders of magnitude too slow for training. Secure multi-party computation (SMPC) has several parties jointly compute a function over secret-shared inputs (via secret sharing or garbled circuits) so no single party reconstructs the raw data; it is the basis of secure aggregation in federated learning (lesson 23) and of vertical FL across organizations that each hold different columns for the same users. Its cost is communication: many interactive rounds between parties.

Trusted execution environments (TEE) take the opposite route — a hardware enclave (Intel SGX, ARM TrustZone) isolates the computation so even the host OS cannot read it, at near-native speed. The catch is the trust model: you trust the hardware vendor's root of trust and accept exposure to side-channel attacks, and you pay an attestation overhead to prove the enclave is genuine.

Technique	What it protects	Trust model	Cost	Typical recsys use
Homomorphic encryption (HE)	data in use — compute on ciphertext	no trusted party (cryptographic)	very high compute (FHE infeasible for training; additive-HE viable)	secure aggregation of gradients (Paillier)
Secure MPC (SMPC)	secret-shared inputs across parties	no single party sees raw data (cryptographic)	high communication (many rounds)	secure aggregation; cross-org vertical FL
Trusted execution env. (TEE)	computation inside a hardware enclave	hardware root of trust (vendor); side-channel exposure	low compute (near-native) + attestation	latency-sensitive joint compute on shared data

The verdict: HE and SMPC are cryptographic — the strongest trust model, paid for with heavy compute or communication; TEE is hardware — cheap and fast, but its guarantee is only as good as your trust in the silicon. In practice, secure aggregation (additive-HE or SMPC) combined with the differential privacy above is the deployed combo for federated recsys — encryption hides individual updates in transit, DP bounds what the aggregate itself leaks — while TEE is reserved for latency-sensitive joint computation where cryptographic overhead is unaffordable. None of these replaces DP: they protect the data going in, DP protects the answer coming out.

Regulation — what the law forces you to build

Privacy is also a compliance surface, and the regimes shape architecture directly. The dominant frameworks:

GDPR (EU) — a lawful basis is required to process personal data; explicit, purpose-bound consent; data-subject rights (access, rectification, portability, and erasure); data minimization and storage limitation as legal duties; and accountability — you must be able to demonstrate compliance via data lineage and consent records.
CCPA / CPRA (California) — a notice-and-opt-out regime: users can know what is collected, demand deletion, and opt out of "sale/sharing." Lighter on up-front consent than GDPR, but the deletion and disclosure duties are similar. China's PIPL adds explicit consent for sensitive data plus data-residency rules.

Cross-border serving multiplies this: data-residency rules (keep certain data in-region) plus GDPR's transfer requirements push toward regional data planes — raw EU behavior stays on EU nodes, and only anonymized / aggregated model parameters cross borders. The architectural levers above are how you satisfy the law cheaply.

The right to be forgotten → machine unlearning

The hardest requirement to engineer is erasure. When a user withdraws consent, you must delete their data — but their data is also baked into the weights of every model that trained on it. Deleting the row in the database does not remove its influence on the model. Removing that influence is machine unlearning, and it is genuinely hard:

Retrain from scratch without the user's data — perfect unlearning, and usually unaffordable to run per deletion request at scale.
SISA (sharded training) — train many models on disjoint data shards and ensemble them; a deletion forces retraining of only the one shard (and its checkpoints) that held the user. Bounds the cost, at some accuracy hit.
Influence-based / gradient unlearning — estimate the user's contribution to the weights and subtract a correction. Fast and approximate; proving the influence is actually gone is the open problem.
DP as prophylaxis — a model trained with small ε barely memorized any one user in the first place, so the residual influence to erase is provably tiny. The cleanest unlearning is having never learned the individual.

Why "just delete the row" is a trap

Models leak membership. A membership-inference attack asks "was this user in the training set?" by probing the model's confidence on their data — and frequently succeeds. So a deletion that removes the database row but leaves the trained weights untouched does not satisfy the spirit (or, increasingly, the letter) of erasure: the model can still betray that the person was once a user. Unlearning has to reach the weights, and proving it did is a frontier problem — design for it up front (sharding, DP, retrain cadence) rather than bolting it on.

The recommender leaks through its own output

Everything so far protects the data you collect and the model you train. But a recommender has a third leak that is specific to recsys and easy to miss: the recommendation list itself is an output computed from the user's history, so it leaks that history. The model is trained to be a near-deterministic function of what you watched; an attacker who only sees what it serves you can run that function backwards.

Membership inference vs. sequence inference

These are two related attacks against the trained model, and interviewers expect you to separate them:

Membership inference (the general ML attack, met in the unlearning box above) asks a yes/no question: "was this person in the training set?" It exploits the fact that a model is more confident on examples it trained on — probe its confidence on a candidate's data and threshold it.
Sequence inference (the recsys-specific attack) is stronger and reconstructive: from a user's recommendation list — possibly observed over several refreshes — the attacker reverse-infers the interaction history that produced it. The recs are a low-entropy fingerprint of the watch sequence, so recovering "you've been watching X, Y, Z" needs no access to the logs at all, only to the surface the user already sees. This is why a sequence model (lesson 13 · User-behavior sequences) that is too tightly fit to the last few interactions is a privacy liability, not just an accuracy win.

Worked: how a too-confident rec list hands the attacker the answer

The leak is driven by the entropy of the served distribution. Suppose a niche user's true next-item interest, given their history, is essentially one item — the model's softmax over the catalog puts p ≈ 0.95 on item A and spreads 0.05 over everything else. The top-5 list is then near-deterministic: observe it once and you have recovered the history that forces it, because almost no other history produces that exact ranking. Attack-success-rate (ASR) — the fraction of users whose history the attacker reconstructs above some accuracy — runs high, say ≈ 80% in this confident regime.

Now blunt the output. Flatten the served distribution toward p ≈ 0.4 on A with real mass on B–E (an output-perturbation / max-entropy defense, below). The same top-5 is now consistent with many plausible histories, so each observation pins down far less and ASR falls to, say, ≈ 35%. The cost is exactly the cost of every other privacy mechanism in this lesson: a flatter list is a less sharply personalized list. Confidence is the currency the attacker spends — which is the same reason a model trained with small ε (it never memorized the sharp signal) leaks less here too.

The defense: perturb the output, regularize for entropy, and forget

Because the leak is through the output distribution, the defenses act on that distribution rather than on the data:

Max-entropy regularizer. Add a term to the training loss that rewards higher entropy in the predicted distribution (penalizes over-confident, near-one-hot recommendations). The model is pushed to keep a deliberate spread of plausible items, so no single list uniquely betrays the history. This is the in-training analogue of post-hoc output perturbation.
Forgetting mechanism. Decay or drop the oldest interactions from the user's state so the served list reflects a bounded, recent window rather than the user's full reconstructible past — which also dovetails with the data-minimization and storage-limitation duties above. Less history encoded ⇒ less history recoverable.
Output-side DP / exponential mechanism. Add calibrated noise to the ranking scores (the exponential mechanism on a top-K selection is the standard tool; the collection-side and recall-side flavors are detailed in 39 · Privacy, compliance & computing). Same lever, with a provable ε attached.

Measure the defense — ASR is the metric, not a vibe

You do not get to assert that a list is "private enough." You run the attack: train (or simulate) a sequence-inference / membership-inference adversary against your deployed model and report its attack-success-rate as a privacy KPI, sitting alongside AUC and watch-time. A defense is shippable when it pushes ASR down to an acceptable floor (e.g. the 80% → 35% move above) at a tolerable accuracy cost — the privacy–utility curve, now drawn with ASR on the privacy axis instead of ε.

The engineering reality

Step back and the whole field is one optimization: maximize utility subject to a privacy budget. Every mechanism is a point on that curve. The practitioner's checklist:

Measure, don't assert. Quantify privacy with ε (and δ); quantify the cost with the metric that pays the bills — AUC, watch-time, CTR — via A/B tests of the protected vs. unprotected pipeline. "ε ≤ 2 with < 3% AUC loss" is a shippable target; "we anonymized it" is not.
Layer defenses. Minimize first (cheapest), keep raw data on-device / federated, add DP to whatever still leaves the device, and reserve crypto (HE / MPC / TEE) for the few places that truly need confidential compute on shared data.
Spend the budget where it hurts least. DP is nearly free on large aggregates and brutal on sparse, long-tail, per-user signals — so protect the head aggressively and find structural protections (or coarser features) for the tail.
Design for deletion now. Sharded training plus a DP budget is far cheaper than discovering, after launch, that every erasure request triggers a full retrain.
Treat the output as a leak too. An over-confident recommendation list reconstructs the history that produced it. Regularize for entropy / perturb the served scores, bound the history window (forgetting), and report attack-success-rate as a first-class privacy metric — not just ε.

Interview prompts you should be ready for

"Why isn't stripping the user_id column enough to anonymize a behavior log?" (Linkage. The behavioral fingerprint — a few rated films, a few check-ins — is itself near-unique; an attacker joins an auxiliary dataset against it and re-identifies. Netflix Prize is the canonical case. The information that's useful for recommendation is the information that re-identifies.)
"State the ε-DP definition and explain what ε means." (Pr[M(D)∈S] ≤ e^ε·Pr[M(D′)∈S] over neighboring D, D′ differing in one person. ε bounds the multiplicative change in any outcome's probability when one person joins/leaves; small ε = strong privacy = the result barely depends on you. (ε,δ) lets it fail with prob δ, enabling Gaussian noise.)
"How do you actually make a query DP, and how is the noise scaled?" (Add noise calibrated to sensitivity Δf — the worst-case change from one person. Laplace with scale Δf/ε for pure ε-DP; Gaussian with σ = Δf√(2ln(1.25/δ))/ε for (ε,δ)-DP. For a count, Δf = 1. Noise ∝ Δf/ε is the whole tradeoff.)
"Walk me through DP-SGD." (Per-example gradient clipping to norm C caps sensitivity at C; add Gaussian noise 𝒩(0,σ²C²) to the averaged gradient; track cumulative (ε,δ) with a moments / Rényi accountant across steps; subsampling amplifies privacy. The model becomes unable to memorize any single user.)
"Why is DP cheap for a global CTR but expensive for long-tail personalization?" (Relative error ≈ b/value = (Δf/ε)/value. On a count of 10⁶ the noise is negligible; on a per-user or niche-segment quantity the same noise dominates. DP shines on big aggregates and crushes sparse signals — protect the head, find structural protections for the tail.)
"Federated learning keeps raw data on-device — is that enough for privacy?" (No. Uploaded gradients leak via membership-inference and gradient-inversion. You still need clipping + DP noise on the updates and SecAgg so the server sees only the aggregate. Federation reduces the attack surface; DP bounds the residual leakage. They compose.)
"A user invokes the right to be forgotten. Deleting their row isn't enough — why, and what do you do?" (Their data is baked into the weights; membership-inference can still betray them. Options: full retrain (perfect, costly), SISA sharded retraining (bounded cost), influence-based subtraction (fast, hard to prove), or training with small ε so there was little to forget. Design for it up front.)
"Your model never exposes the training logs, yet an attacker reconstructs a user's watch history. How?" (Sequence-inference attack: the recommendation list is a low-entropy function of the history, so observing the served top-K reverse-infers what produced it — no log access needed. A near-one-hot, over-confident list is uniquely explained by one history (high ASR); the defense flattens the served distribution via a max-entropy regularizer / output-perturbation (exponential mechanism on top-K) and a forgetting window, then validates by measuring attack-success-rate. Distinguish from membership inference, which only answers the yes/no "was this user in training?".)
"Put real numbers on the cost of DP and crypto in a recsys serving stack." (DP-SGD: AUC down ~3–5% at useful ε — ε=0.1 ≈ 2%, recoverable to ~0.5% with feature compensation — and +20–40% training wall-clock from per-example clipping + noise. Crypto path: 50ms → 200ms latency, keep GPU HE inference < 50ms; HE itself is ~1000× ciphertext blow-up / ~100× compute, so only additive-HE for the secure-aggregation sum is viable, not FHE training. And ε is a running ledger: ε_daily≈0.1 capped to ε_total<1.0/month via a Rényi-DP accountant (sub-linear, not linear k — the √k win is asymptotic, real at DP-SGD's thousands of steps) — exceed it and you stop releasing.)

Takeaway

k-anonymity protects a table; DP protects a person — only the latter survives an attacker with side information. The one number is ε: small ε = strong privacy = big noise = worse model, and it composes (adds up) across every query and training step, so it is a finite budget you spend down. Architecture (minimize, on-device, federate) shrinks what you must protect; regulation (consent, erasure → unlearning) forces you to be able to take it back. The deliverable is always a chosen point on the privacy–utility curve — ε on one axis, your business metric on the other.