Privacy, compliance, and privacy computing - useful, lawful, and measurable
The book's privacy material is not just a list of techniques. This lesson linearizes how recommender systems can keep personalization useful while respecting data minimization, consent, localization, deletion, anonymization limits, federated learning, differential privacy, secure computation, and the privacy-effectiveness trade-off.
1 · Start with the legal and product question
Privacy engineering does not start with encryption. It starts with purpose: what data is needed, why it is needed, whether the user knows, how long it is retained, where it is processed, and whether the same recommendation value can be achieved with less sensitive data.
| Question | Recommender-system interpretation |
|---|---|
| Purpose limitation | Use watch history for personalization only if that purpose is declared and justified. |
| Data minimization | Prefer coarse region over precise location if coarse region is enough. |
| Consent and transparency | Users should understand personalization and meaningful data uses. |
| Retention | Raw logs should expire or aggregate; old behavior should not live forever by default. |
| Deletion | Deletion must affect raw logs, feature stores, embeddings, training exclusions, and backfills where required. |
| Cross-border transfer | Data localization, contracts, transfer assessments, and audit trails may shape architecture. |
The book repeatedly mentions GDPR, China's Data Security Law, and Personal Information Protection Law. The interview-level point is not to recite statutes; it is to show that compliance changes system boundaries.
2 · Map data sensitivity before choosing technology
| Data | Recommendation value | Privacy risk | Safer alternative |
|---|---|---|---|
| Watch history | Sequence modeling and interest profile | Sensitive interests and re-identification | Short retention, local summary, coarse topics, federated training |
| Location | Language, regional content, legal region | Precise tracking | Coarse region, user-declared locale |
| Device fingerprint | Fraud detection and cold start | Persistent tracking | Rotating pseudonymous ID and strict anti-fraud purpose |
| Social graph | Creator/friend recommendations | Third-party privacy | Explicit consent, limited graph features |
| Embeddings | Compact user/item representation | May encode sensitive behavior | Access control, DP training, deletion/retraining policy |
A user embedding is not automatically anonymous. If it is derived from identifiable behavior and linked over time, it can remain personal data in practice.
3 · Desensitization is not anonymity
The book separates desensitization, anonymization, k-anonymity, differential privacy, and privacy computing. Treat them as a ladder, not synonyms.
| Technique | What it does | Limitation |
|---|---|---|
| Pseudonymization | Replaces direct identifiers with stable tokens | Still linkable across events |
| Hashing | Hides raw ID string | Dictionary attacks and linkability remain |
| Generalization | Coarsens age, time, or location | Loses utility and may still be unique in combination |
| k-anonymity / l-diversity | Prevents small identifiable groups | Weak against auxiliary data in high-dimensional logs |
| Differential privacy | Bounds single-user contribution through noise | Privacy budget and accuracy trade-off |
4 · Federated learning keeps raw data local
The book's federated-learning chapters repeat one core idea: data does not move; model updates move. In recommendation, clients or data holders train locally, then a coordinator aggregates encrypted or protected updates.
| Benefit | Recommendation example | Cost |
|---|---|---|
| Privacy and compliance | Raw watch logs stay on device or in region | Harder debugging and monitoring |
| Data-silo collaboration | Short-video and e-commerce platforms learn shared signals | ID alignment and feature heterogeneity |
| Cold-start transfer | New app/region borrows global representation | Domain mismatch |
| On-device personalization | Local ranker adapts to recent behavior | Device compute, battery, network constraints |
Federated learning is not a complete privacy solution by itself. Updates can leak information through gradients, so the book pairs it with secure aggregation, differential privacy, encryption, and attack monitoring.
| Federated type | Data split | Short-video fit |
|---|---|---|
| Horizontal FL | Participants share feature schema but hold different users | Common for cross-device or cross-region user behavior training |
| Vertical FL | Participants share users/items but hold different feature columns | Useful when one party has content features and another has behavior or commerce features |
| Federated transfer | Domains differ but share transferable representations | Useful for cold-start regions/apps when raw data cannot be pooled |
5 · Handle Non-IID data and multi-objective ranking
Federated recommendation is harder than centralized training because users, regions, devices, and partners have different data distributions. The book names Non-IID as a central challenge.
| Problem | Strategy | Reason |
|---|---|---|
| Regions behave differently | Personalized federated learning, local heads, FedProx-style constraints | Global average may erase local preference |
| Devices differ in compute | Client sampling, asynchronous updates, model compression | Mobile participation is unreliable |
| Objectives conflict | Federated multi-task heads for CTR, completion, interaction | Watch-time and interaction labels differ by segment |
| Gradient leakage risk | Clipping, noise, secure aggregation, attack testing | Updates may reveal behavior |
| Poisoned client updates | Robust aggregation, anomaly filters, trust scoring, known-good validation | Malicious clients can degrade or bias the global model |
| Communication cost | Quantization, sparse updates, low-frequency upload | Recommendation models can be large |
Robustness is part of privacy computing. A system that protects raw data but accepts poisoned gradients can still harm users. In interviews, name defenses such as update clipping, anomaly detection on client updates, robust aggregation, secure aggregation with auditability, and canary evaluation before model rollout.
6 · Differential privacy gives a measurable privacy knob
The book's DP sections emphasize the epsilon budget and the trade-off with model effect. DP adds noise so the presence or absence of one user's data has bounded impact on the output.
| Where to use DP | Example | Trade-off |
|---|---|---|
| Analytics | Noised category CTR or aggregate watch-time reporting | Less accurate small segments |
| Feature engineering | DP-protected group profiles | Long-tail signal weakens |
| Model training | DP-SGD with gradient clipping and noise | AUC/watch-time prediction may fall |
| Federated learning | Client-side or server-side DP on updates | Privacy budget management and convergence cost |
Do not say "add DP" without saying what epsilon, what data, what threat, and what utility loss are acceptable. The book's framing is a privacy-effect Pareto problem, not a binary switch.
7 · Match privacy computing to the threat model
| Technology | Protects against | Good fit | Cost |
|---|---|---|---|
| Federated learning | Raw data leaving device/region/partner | Cross-device or cross-region training | Communication, Non-IID, debugging |
| Differential privacy | Single-user contribution inference | Analytics, gradients, aggregates | Noise and utility loss |
| Secure aggregation / MPC | Server or parties seeing individual updates | Federated updates, cross-party statistics | Protocol complexity and latency |
| Homomorphic encryption | Computation over encrypted values | Small high-value encrypted joins or scores | Large compute overhead |
| TEE | Host/operator access to plaintext during compute | Sensitive joins or controlled inference | Hardware trust and side-channel concerns |
| On-device inference | Sending raw behavior to server | Fast local personalization | Model size and update management |
The interview move is to pick the simplest technology that matches the actual risk. Encryption does not fix excessive collection. Federated learning does not fix unauthorized use. DP does not fix a missing deletion path.
8 · Evaluate privacy and utility together
The book repeatedly asks how to balance privacy protection and recommendation effect. Treat privacy changes as product and model changes with their own experiment plan.
- Define protected data. Watch history, location, social graph, device ID, embeddings, or gradients?
- Define threat model. Curious server, cross-party leakage, membership inference, re-identification, model inversion?
- Choose mechanism. Minimization, local summary, DP, FL, MPC, HE, TEE, or on-device inference.
- Measure utility. AUC/GAUC, NDCG, calibration, per-user watch time, retention, cold-start lift.
- Measure privacy. Epsilon, attack success rate, re-identification risk, access logs, deletion coverage.
- Ship with guardrails. Decide acceptable utility loss for risk reduction; monitor segments and long-tail impact.
Mini-scenario: replacing raw watch-history training with federated learning plus DP may reduce offline AUC by 1%, improve privacy-risk scores, increase communication cost, and affect new users more than mature users. The decision is not "privacy won" or "model lost"; it is a Pareto read across utility, privacy, cost, and segment fairness.
9 · Compliance changes data lifecycle
| Requirement | System implication |
|---|---|
| Minimum necessary principle | Feature review asks whether a less granular feature can achieve similar lift. |
| User deletion | Delete or exclude raw logs, feature rows, embeddings, training datasets, and future backfills where required. |
| Consent changes | Feature generation and model serving must be consent-aware. |
| Data localization | Regional training, federated aggregation, or local serving may be required. |
| Auditability | Maintain lineage from raw event to feature to model to decision. |
| Model memorization | Test membership inference, rare-pattern leakage, and sensitive-feature reconstruction. |
10 · Local DP - protect the data before it leaves the device
Sections 6 and 7 use differential privacy on gradients and aggregates - that is central DP: a trusted server collects raw data and noises the answer. But the engineering question for a recommender is collection: the raw click belongs to the user and the server is exactly the party we may not want to trust. Local DP (LDP) moves the trust boundary to the client - the device perturbs each value before upload, so the raw signal never exists off-device. The mechanics of ε (sensitivity Δf, Laplace/Gaussian scale, composition) are derived in lesson 22; here we cover the collection-side mechanisms that 22 does not.
| Stage | Central DP (server-side) | Local DP (client-side) |
|---|---|---|
| Who sees raw data | Trusted server | Nobody - perturbed on device |
| Where noise is added | Aggregate / gradient | Each raw value before upload |
| Noise needed for same ε | Lower (one query) | Higher (per record), so debiasing matters |
| Recsys use | Category CTR reporting, DP-SGD | Behavior collection, interest-tag upload |
Randomized response - worked debiasing
For a categorical or boolean signal (did the user watch a health video? did they like?) the classic LDP primitive is randomized response: the device reports the truth with probability p and flips with probability 1 − p. A single report is plausibly deniable, but the aggregate is still estimable - you just have to undo the flip. The flip probability sets ε via p / (1 − p) = eε (binary case); see lesson 22 for the ε derivation.
Let the observed "yes" rate over n reports be ŷ and the true rate be π. Each report is "yes" either because it was truly yes and kept (πp) or truly no and flipped ((1 − π)(1 − p)):
ŷ = πp + (1 − π)(1 − p) ⇒ π̂ = (ŷ + p − 1) / (2p − 1)Worked example. Pick p = 0.75 (so ε = ln(0.75/0.25) ≈ 1.10). Suppose 30% of users truly watched a sensitive category, π = 0.30. The raw observed rate is biased toward 0.5:
ŷ = 0.30(0.75) + 0.70(0.25) = 0.225 + 0.175 = 0.400Naively reading 40% would over-report by a third. Debiasing recovers the truth:
π̂ = (0.400 + 0.75 − 1) / (2(0.75) − 1) = 0.150 / 0.500 = 0.300 ✓Per-feature ε budgets
One global ε is the wrong knob: a static demographic and a 30-event click sequence are not equally sensitive, and they do not carry equal utility. Spend the budget per feature, by sensitivity:
| Feature | ε | Reasoning |
|---|---|---|
| Gender / age bucket | 0.1 (strong) | Highly identifying, low marginal lift once coarse |
| Click / watch sequence | 1.5 (weaker) | Core ranking signal; over-noising kills personalization |
| Coarse region | 0.5 | Sensitive but needed for locale routing |
A useful refinement the book repeats: scale noise inversely to user activity - heavy users already emit many reports, so the same ε per report buys them a tighter aggregate; sparse users get more protection per event. The total spend is then tracked by a budget ledger (Section 12).
Exponential mechanism on Top-K recall
Randomized response and Laplace noise assume a numeric or boolean output. The recall layer instead emits a discrete selection - which K items to surface. The right primitive is the Exponential mechanism: sample item v with probability ∝ exp(ε · s(v) / 2Δs), where s(v) is a utility score. A practical scoring blend is
s(v) = η · CTR(v) + (1 − η) · noise, η = 0.8so high-CTR items still dominate the Top-K but the exact ranking is randomized enough that the list does not deterministically reveal the user's strongest interests. This also blunts the sequence-inference attack (reconstructing history by reverse-inferring from the recommendation list) - the same attack the book pairs with a max-entropy regularizer; see lesson 22 for the membership-inference family. Note the tension with transparency: a user-facing reason (lesson 21) that names the exact triggering video re-exposes the very history this mechanism is hiding, so explanation templates should cite coarse interest tags, not raw items.
On-device pre-aggregation
The cheapest privacy is to never upload the raw event at all. Before any DP noise, the device can summarize:
| Technique | What leaves the device | Why it helps |
|---|---|---|
| Top-K interest tags | "pets, cooking, fitness" not the 400 video IDs | Collapses a unique sequence into a coarse, shared vocabulary |
| Thousand-user bucketing | A group ID; features updated per bucket, not per user | Any upload is shared by ≈1000 users - k-anonymity by construction |
| Bloom-filter encoding | A fixed-width bit array of the history set | Membership is queryable; exact items are not recoverable |
Pre-aggregation and LDP compose: bucket on device, then add randomized-response noise to the bucket-level counts. The trust boundary stays at the device, and the server only ever sees coarse, noised, debiasable statistics. This is also the collection layer that feeds federated training - the device noises and summarizes before any update is computed, so client-side DP and on-device pre-aggregation are the first line of defense even when raw gradients never leave.
11 · Desensitization that preserves utility - and how to prove it
Section 3 listed the desensitization ladder (pseudonymization → hashing → generalization → k-anonymity → DP) and warned that removing names does not remove identity. The missing engineering is the recsys-specific question: how do you mask IDs without destroying the structure the model relies on, and how do you prove the masked data is still usable?
Similarity-preserving masking and rare-ID handling
| Technique | Mechanism | Why naive masking fails without it |
|---|---|---|
| LSH for IDs | Locality-sensitive hashing maps similar users to nearby buckets, so the hash preserves the neighbor structure | SHA-256 of a user ID is uniform - it destroys all similarity, so collaborative signal (similar users → similar recs) is lost |
| Rare-ID clustering | Merge singleton / long-tail item IDs into a parent class (e.g. plays < 100 → "other") | A video watched by one user is a unique fingerprint; left alone it re-identifies that user even after the user ID is hashed |
| GAN synthesis | Train a generator on the real distribution, then share synthetic rows / face-removed thumbnails for external research | Any real row is linkable; synthetic rows keep the distribution but cut the link to a real person |
The engineering line: hashing for anonymity and hashing for utility pull in opposite directions. A cryptographic hash maximizes unlinkability and minimizes usefulness; LSH deliberately keeps collisions among similar users so clustering survives - which means LSH alone is not anonymization (similar users still cluster, by design) and must be paired with DP or access control on the sensitive features.
Validate utility with a divergence check - worked
Desensitization is only safe to ship if the masked distribution still matches the original. The book's rule of thumb: compare pre/post distributions with KL or JS divergence and require statistical deviation < 5%. Worked example on a 3-category interest distribution before and after generalization:
| Category | P (original) | Q (desensitized) |
|---|---|---|
| pets | 0.50 | 0.46 |
| cooking | 0.30 | 0.32 |
| fitness | 0.20 | 0.22 |
Jensen-Shannon is the symmetric, bounded choice. With M = (P + Q)/2 and JS = ½KL(P‖M) + ½KL(Q‖M), using log base 2 so JS ∈ [0, 1]:
M = (0.48, 0.31, 0.21) KL(P‖M) = 0.50·log₂(0.50/0.48) + 0.30·log₂(0.30/0.31) + 0.20·log₂(0.20/0.21) ≈ 0.00118 KL(Q‖M) ≈ 0.00118 ⇒ JS ≈ 0.00118, √JS ≈ 0.0343 = 3.4%The JS distance (3.6%) is under the 5% gate, so this generalization ships. Had we collapsed pets and cooking into one bucket, P and Q would diverge past the line and the check would block it - a concrete, automatable guardrail rather than "it looks fine." Pair the distribution check with an A/B read on AUC/GAUC, because a small global divergence can still hide a large long-tail hit.
12 · Put concrete numbers on the privacy-utility bill
"Add privacy" is not a plan until the cost is quantified. The recurring numbers from the book, assembled so a design review can price the trade-off (the ε mechanics behind these live in lesson 22):
| Mechanism | Utility / cost hit | Engineering consequence |
|---|---|---|
| DP-SGD training | Convergence 20-40% slower; AUC down ~1-3% at ε ≤ 2 (DeepFM 3-5% in worse cases) | Budget extra GPU-hours per training run; long-tail users lose the most |
| ε = 0.1 on a feature | ~2% AUC drop, recoverable to ~0.5% with feature compensation (context features) | Strong per-feature privacy is shippable if you add back signal elsewhere |
| Crypto serving path (FL + HE/MPC) | Latency 50ms → 200ms (4×) | Reserve the encrypted path for high-value joins; keep GPU HE inference < 50ms target |
| Homomorphic encryption | ~1000× ciphertext blow-up, ~100× compute | Offline feature engineering or small encrypted scores only - not the full ranker |
The ε ledger - privacy is a spent budget, not a setting
ε is consumed every time a user's data touches a noised computation, and composition theorems add it up. So privacy is accounted like a budget that depletes: set a per-day cap and a per-month ceiling, and stop (or fall back to non-personalized serving) when the user's ledger is exhausted.
Advanced (or Renyi) composition is what makes this viable: naive composition would sum ε linearly and blow the monthly ceiling in days, whereas advanced (Rényi) composition grows the cumulative bound sub-linearly — asymptotically like √k for k queries, though the bound carries a √(2 ln 1/δ) constant so the real savings show up at the thousands-of-steps scale, not after a handful of reads — letting a real system run many noised reads under a fixed budget. The ledger is the operational artifact that turns "we use DP" into an auditable, enforceable claim.
Interview prompts you should be ready for
- "How do you personalize while collecting less data?" Minimize raw data, use coarse/local summaries, shorten retention, move training/inference on-device or federated when justified, and measure utility loss.
- "Is hashing user IDs enough?" No. Stable behavior logs remain linkable and re-identifiable; you still need purpose, access control, retention, aggregation, and stronger mechanisms where needed.
- "How would you use federated learning for recommendation?" Local training, clipped/noised/encrypted updates, secure aggregation, global/personalized model, Non-IID handling, communication compression, and A/B evaluation.
- "How do you choose between FL, DP, MPC, HE, and TEE?" Name the threat model first, then select the lowest-cost mechanism that protects against that threat while preserving utility.
- "How do you handle deletion requests?" Trace raw logs, online/offline features, embeddings, training snapshots, derived datasets, and future backfills; maintain auditability.
- "Local vs central DP - which would you use to collect click data, and what's the catch?" Local DP if the server is the untrusted party: perturb on device (randomized response) so raw data never leaves. The catch is variance - debiasing is unbiased but the estimator's variance scales like 1/(n(2p−1)²), so LDP only works on high-volume signals; sparse/long-tail events stay central or get pre-aggregated (Top-K tags, thousand-user buckets, Bloom filters) first.
- "You hashed every user ID with salted SHA-256 - is the data now both anonymous and still useful for collaborative filtering?" No, those goals conflict. A crypto hash destroys similarity, so neighbor-based signal is gone; LSH preserves it but is therefore not anonymization. And rare item IDs re-identify users even after the user ID is hashed, so you also cluster long-tail IDs. Prove utility survived with a KL/JS divergence < 5% check plus an A/B on AUC, not by eyeballing.