User segmentation & profiling
A single global model already personalizes per user — every user gets their own embedding, their own predicted score. So why bother carving the population into segments at all? Because segments are where you bootstrap the users you barely know, where you read and debug a system whose aggregate metrics lie to you, and where you apply policy the model itself can't express. Profiling is the representation; segmentation is the partition over it. This is the last lesson, and it is the one that makes you sound senior: always slice your metrics.
The user profile / portrait — the representation you segment over
Before you can partition users you need a representation of them. In the Chinese literature this is the 用户画像 ("user portrait"): the structured, multi-dimensional description of who a user is and what they want. It is not one object — it is a stack of layers, each built from a different data source and serving a different consumer.
Read that diagram top-to-bottom and a duality jumps out. The top layers are human-readable tags ("lives in SF", "interested in cooking", "high-value", "new this week"); the bottom layers are a dense learned embedding the model consumes but no human can read. You keep both, deliberately. The tag profile is what lets ops write a business rule, lets legal exclude a protected attribute, lets you debug ("why did this user see that?"), and lets you build segments at all. The embedding is what actually maximizes the objective. Throwing away the tags because "the embedding already encodes it" is a classic mistake — you lose every consumer except the model.
| Layer | Example fields | Built from | Primary consumer |
|---|---|---|---|
| Static / demographic | age band, gender, signup country, device tier | registration, device fingerprint | coarse priors, legal exclusion, ops rules |
| Declared / stated | onboarding topic picks, followed creators, stated language | explicit user input | cold-start prior, hard filters |
| Behavioral-aggregate | RFM scores, top-5 categories, avg session length, daypart | roll-ups over the event log | RFM segments, dashboards, business rules |
| Sequence-derived | last-N item IDs, real-time session intent | the click stream (see lesson 13) | the ranker, fast session profile |
| Learned embedding | dense user vector u ∈ ℝ^d | trained jointly with the model | the ranker only (opaque) |
This lesson is the discipline of profiling and segmenting; the sequence-derived layer — how a click stream becomes a user vector — is its own deep dive in lesson 13, and I won't re-derive it here. What matters at this altitude is that the sequence layer is the hot, per-request part of the portrait, and the rest is comparatively slow.
Freshness tiers — why the profile is a tiered cache, not one table
Read that table by a different axis — how often each field changes — and the layers collapse into three freshness tiers, and that re-grouping is the whole reason a profile is never one table. A naive design stores every field in one row and recomputes it on one schedule. But the fields have wildly different clocks: gender effectively never changes, "top categories this month" changes nightly, and "current GPS / this session's intent" changes every few seconds. Recompute everything on the fast clock and you burn compute re-deriving a gender that didn't move; recompute everything on the slow clock and your session intent is stale by the time it lands. So you tier the profile exactly like a CPU cache — hot/warm/cold — and pay for freshness only where freshness is worth paying for.
The trade-off is freshness against cost, and it is monotone: the fresher the tier, the more it costs per update and the smaller the value-density of each refresh. Putting a static attribute (gender) in the real-time tier is pure waste — you pay hot-path stream-compute to re-derive a constant. Putting session intent in the static tier is the opposite failure — you serve a "what they want now" that is a day old, which for a one-off gift session is simply wrong (the two-clocks failure made concrete; the same slow/fast split returns below as the modeling view). The freshness tier is the engineering decision about which clock a field runs on, and getting it wrong shows up as either a bloated feature-compute bill or a profile that describes a stale person.
| Tier | Example field | Update cadence | Store |
|---|---|---|---|
| Static | gender, signup country, device tier | ≈ never (write once) | user DB row (cold) |
| Dynamic | 30-day top categories, RFM, daypart histogram | T+1 batch / hourly | feature store (warm) |
| Real-time | current GPS, this-session click intent | seconds (per event) | Redis / in-memory (hot) |
The six data sources — and the dominant weakness of each
The fields filling those tiers come from six data sources, and the senior move is to name not just what each source gives you but its dominant weakness — because the weakness is what decides how much to trust the source and where it must not be used. No source is good at everything; you fuse them precisely to cover for each other's failure mode.
| Source | What it gives | Strength | Dominant weakness |
|---|---|---|---|
| Declared / basic attributes | age, gender, region, onboarding picks | broad coverage, available at signup → good cold-start prior | low precision; people lie / leave defaults; coarse |
| Behavioral events | clicks, plays, scroll, dwell | real-time, high-volume, reveals revealed preference | noisy — mis-taps & passive exposure; needs sessionization (lesson 13) |
| Content consumption | tags/categories of items watched | directly reflects interest | exposure bias — you only see what the system already showed |
| Social interaction | follows, comments, shares | reveals interest circles the behavior log misses | sparse — most users follow few accounts |
| Device / environment | network type, GPS, OS | cheap, always present → context features | privacy-sensitive; legally constrained as a policy lever |
| Transactional | purchases, gifts, ad conversions | highest value-density signal there is | covers few users — most never transact |
Tag systems — rule/statistical tags vs model-inferred tags
The human-readable layers are populated by tags, and tags come in two flavors that you must not conflate:
- Rule / statistical tags are deterministic roll-ups: "watched > 10 cooking videos in 30 days → tag
cooking_enthusiast." Cheap, exact, auditable, and trivially explainable — but coarse and brittle at the threshold. - Model-inferred tags come from a classifier or from thresholding a learned representation: "P(interested in finance) = 0.82 → tag
finance." Richer and able to generalize from sparse signal, but probabilistic and harder to defend.
Every tag should carry two things beyond its value: a confidence (a rule tag is 1.0; a model tag is its probability) and a decay. Interest is perishable — a "World Cup" tag earned in July is noise by September. Model the tag weight as exponential decay on time-since-last-supporting-event, w(t) = w_0 · e^(−λ Δt), so stale interests fade instead of accumulating forever. Without decay your profile becomes a landfill of everything the user ever touched, and your "interests" describe a person who no longer exists.
RFM — the classic interpretable segmentation
The oldest segmentation that still earns its keep is RFM, born in direct-mail marketing: score each user on three axes — Recency, Frequency, Monetary — and segment on the combination. For an engagement platform you adapt the axes to behavior rather than purchases:
- Recency — how long since the last visit. (A user last seen an hour ago is worth more than one last seen in March.)
- Frequency — sessions per window. (How habitual is the relationship?)
- Monetary → value — total watch-time, ad revenue, or whatever your north-star value is. ("Monetary" generalizes to "how much value does this user create or capture?")
The scoring is deliberately dumb and that is its strength. You don't model anything — you just bucket each axis into quantiles (quintiles are traditional) and concatenate. With per-axis quantile scores R, F, M ∈ {1..5} the composite is simply
score = 100·R + 10·F + M (or, for ranking, the sum w_R·R + w_F·F + w_M·M)
A user scoring 555 is a recent, frequent, high-value "champion"; a 155 is a high-value user who used to be frequent and just lapsed — the single most valuable group to win back. The whole appeal is that the segments are named, actionable, and explainable to a non-ML stakeholder: "whales", "at-risk", "hibernating", "new". A growth PM can act on an RFM segment in an afternoon.
Clustering — finding segments you didn't hand-design
RFM partitions on axes you chose. Clustering lets the data choose. Two distinct flavors, with very different trade-offs.
K-means on engineered features
Take a feature vector per user — RFM scores, category affinities, daypart histogram, device — and partition into k clusters minimizing within-cluster squared distance to the centroid:
min{S₁..S_k} Σj=1..k Σx ∈ S_j ‖ x − μ_j ‖² μ_j = mean of cluster j
Two non-negotiables. First, you must scale the features first — K-means optimizes a Euclidean distance, so an unscaled "total watch-seconds" axis (range 0–100,000) will completely dominate a "sessions per week" axis (range 0–20) and the clustering degenerates into a 1-D split on the big number. Standardize or quantile-transform every axis before you cluster; this is exactly the discipline from lesson 28 · feature engineering, and forgetting it is the most common K-means bug. Second, you must choose k — there is no true answer, but the elbow method (plot within-cluster inertia vs k, take the kink) and the silhouette score (how well-separated clusters are) give you a defensible pick.
Embedding clustering — cluster the learned vectors
The deeper move: cluster the learned user embeddings instead of engineered features. The embedding already encodes interaction patterns no hand-built feature captures — that a user who watches woodworking and espresso content behaves like a coherent "maker" type even though no single feature says so. Clustering in embedding space surfaces segments RFM and feature-K-means structurally cannot, because those interactions live in the dense vector, not in any column you engineered.
The cost is interpretability. A feature-space cluster is describable ("high-frequency, low-value, mobile, evenings"); an embedding cluster is a blob in ℝ^d you have to reverse-engineer a label for, usually by inspecting the items its members engage with. You trade the readability of RFM for the expressiveness of the embedding — which is the same tag-vs-vector duality from the top of this lesson, now at the segment level.
| RFM | K-means on features | Embedding clustering | |
|---|---|---|---|
| Input | 3 hand-chosen axes | engineered feature vector | learned user embedding |
| Captures interactions? | No | Only what you engineered | Yes — that's the point |
| Interpretable? | Very — named segments | Yes — describe by centroid | Poor — reverse-engineer a label |
| Needs feature scaling? | N/A (quantiles) | Yes — or it degenerates | Already in a metric space |
| Cost to build | Trivial (SQL) | Low | Needs a trained model first |
| Best for | Ops vocabulary, guardrails, cold-start prior | Discovering behavioral groups | Finding latent taste communities |
Which clustering algorithm — the data shape decides, not the fashion
"Cluster the users" is under-specified: K-means is one choice among several, and each makes a different shape assumption and a different cost/scale promise. Pick the one whose assumption your data actually satisfies — the failure mode of every clustering algorithm is running it on a data shape it cannot represent and trusting the (always non-empty) output.
| Algorithm | Complexity | Pick k? | Assumes / yields | Use when · fails when |
|---|---|---|---|---|
| K-means (++ / Mini-Batch) | O(n·k·i) — scales | you set k | convex, roughly equal-size spherical blobs; hard assignment | large n (Mini-Batch past ~100k users). Fails on elongated/nested shapes & outliers |
| Hierarchical (agglomerative) | O(n³) — won't scale | no — cut the tree | a dendrogram → multi-granularity at once | small n, you want a lifecycle hierarchy (sub-segments nest). Fails past ~10k rows (memory) |
| GMM (EM) | O(n·k·d·i) | you set k | overlapping Gaussian components; soft P(cluster) | a user is genuinely 60% "foodie" / 40% "traveler". Fails when components aren't Gaussian |
| DBSCAN | O(n log n) (indexed) | no — set ε, minPts | density-connected arbitrary shapes; labels outliers as noise | non-convex blobs + you want outliers flagged, not forced in. Fails on varying-density data |
| StreamKM++ | streaming, ~O(1)/point | you set k | online centroids over a coreset | behavior drifts and you must re-cluster continuously without a full rebatch |
The decision tree is short. Big and roughly spherical → K-means (Mini-Batch if n is huge). Need nested sub-segments for a lifecycle view, small data → hierarchical. Users belong to several segments at once → GMM's soft membership beats forcing a hard label. Weird shapes and you care about outliers → DBSCAN. A live stream that drifts → StreamKM++. Notice K-means' assumption is the strongest (equal-size convex blobs) precisely because it's the cheapest — the recurring engineering tension, cost bought with an assumption you must then verify holds.
Did the clustering work? — quality metrics, and where each lies
Clustering always returns something; you need a number for whether that something is real structure or just a partition of noise. Four metrics, and the senior point is knowing what each one can't see:
| Metric | Type | Measures | Misleads when |
|---|---|---|---|
| Silhouette ∈ [−1, 1] | internal | per-point: cohesion vs separation, (b−a)/max(a,b) | assumes convex clusters — penalizes a correct DBSCAN crescent as "bad" |
| Calinski–Harabasz (↑) | internal | between- / within-cluster dispersion ratio | biased toward many small equal-size clusters; rises as k rises |
| Davies–Bouldin (↓) | internal | avg worst-case cluster similarity (lower = better) | also assumes convex/spherical; sensitive to scale |
| NMI ∈ [0, 1] | external | info overlap vs a known label (e.g. paid-tier) | needs ground-truth labels you usually don't have; label ≠ "good clustering" |
The trap they share: every internal metric (silhouette, CH, DB) bakes in a convexity prior, so all three will rate a tidy-but-meaningless K-means split above a true-but-non-convex DBSCAN result. They measure geometry, not business meaning — a clustering can score a beautiful silhouette and still cut the population along an axis no PM can act on. Use them to compare candidate k's and to reject obviously-bad partitions, never as the final word over interpretability and a live A/B on the resulting policy.
Silhouette scores one point at a time. For point i: let a(i) = its mean distance to the other points in its own cluster (cohesion), and b(i) = its mean distance to the points of the nearest other cluster (separation). Then s(i) = (b(i) − a(i)) / max(a(i), b(i)), and the clustering's score is the mean over all points. Near +1 = well inside its cluster; near 0 = on a boundary; negative = probably mis-assigned.
Take six users on one engineered axis — say a standardized "sessions/week" score — split into two clusters:
A = {1, 2, 3} B = {8, 9, 10}Work point x = 2 (the centre of A). Cohesion: distances within A are |2−1| = 1 and |2−3| = 1, so a = (1+1)/2 = 1. Separation against B: |2−8|, |2−9|, |2−10| = 6, 7, 8, so b = (6+7+8)/3 = 7. Then s = (7 − 1) / max(1, 7) = 6/7 ≈ 0.857 — strongly in-cluster, as expected for a centre point.
Now an edge point, x = 3. Cohesion: |3−1|, |3−2| = 2, 1 → a = 1.5. Separation: |3−8|, |3−9|, |3−10| = 5, 6, 7 → b = 6. So s = (6 − 1.5)/6 = 0.75 — still healthy, but lower than the centre, exactly because it sits nearer B. By symmetry the whole clustering averages high (≈ 0.80), confirming two genuinely separated groups.
The instructive case is a mis-split. Re-draw the line as A' = {1, 2}, B' = {3, 8, 9, 10} and re-score the lone point x = 3: now its own cluster B' gives a = (|3−8|+|3−9|+|3−10|)/3 = (5+6+7)/3 = 6, while the nearest other cluster A' gives b = (|3−1|+|3−2|)/2 = (2+1)/2 = 1.5. Then s = (1.5 − 6)/max(6, 1.5) = −4.5/6 = −0.75 — sharply negative, the metric screaming that point 3 was put in the wrong cluster. That sign flip is the whole value of silhouette: it localizes a bad assignment to the individual point, which inertia (the K-means objective) never tells you because inertia is only ever a single global sum.
A third, orthogonal flavor worth naming: behavioral cohorts — group users by a shared event time, classically signup week, and track each cohort's curve over calendar time. Cohorting is how you read retention honestly: a flat aggregate DAU can hide that every recent signup cohort is churning faster than the old ones, masked by a shrinking-but-loyal legacy base. Same idea as segmentation, partitioned on time instead of attributes.
Two clocks — fast session profile and slow interest profile
A user is two things at once. There is the slow "who you are" — your durable interests built over months (you like cooking, indie music, soccer). And there is the fast "what you want right now" — this session you are shopping for a birthday gift, which has nothing to do with your durable taste and will be gone in twenty minutes. Collapse them into one profile and you get mush: the long-term profile drowns the urgent session intent, or a single anomalous session corrupts the durable profile.
This is the two-clocks idea, and it ties directly to lesson 13: the fast profile is essentially the session sequence summarized in real time, while the slow profile is the aggregate. The fusion can be a fixed blend u = α·u_slow + (1−α)·u_fast, or — better — a learned attention gate that decides per-request how much to trust the session over the durable profile (high session signal → lean fast; thin session → fall back on slow).
Freshness is a first-class concern across both clocks. The slow profile needs decay (the e^(−λΔt) from earlier) so it tracks a changing person. And you need drift detection: when a user's recent behavior diverges sharply from their durable profile — a new parent, a new job, a new city — you want to reset or rapidly re-weight the profile rather than keep predicting against a person who no longer exists. A profile that can't change its mind is worse than no profile.
Segment-conditioned evaluation — the senior payoff
Here is the reason a senior engineer reflexively segments, and the reason this is the closing lesson. Aggregate metrics lie. A single top-line number — overall CTR, overall watch-time — averages over wildly different populations, and the average can move in the opposite direction from every population that composes it. This is Simpson's paradox, and it is not a curiosity — it is a routine, ship-blocking event in any A/B test where the traffic mix differs between arms.
The mechanism: aggregate CTR is a mix-weighted average of per-segment CTRs. If a treatment shifts the traffic mix toward a structurally lower-CTR segment (say it retains more new users, who click less than returning users), the aggregate CTR can fall even while CTR rises in the new segment and the returning segment. Every slice improves; the blend regresses. Or the reverse. The widget below makes this concrete and reversible.
Segments-as-stereotypes is the modeling failure: once you bucket a user as "sports fan", serve only sports, observe only sports clicks, and confirm the stereotype — a filter bubble manufactured by your own segmentation. A segment is a prior to start from and update away from with exploration, never a cage. This is the diversity / fairness concern from lesson 20, and it is why segment priors must always sit alongside an exploration budget.
Interactive · Simpson's paradox slicer
Two segments — new and returning users. Set each segment's CTR under control vs treatment (treatment is pre-set to win in both segments). Then shift the traffic mix: what fraction of each arm's impressions comes from new users. Watch the aggregate reverse the per-segment verdict purely from mix — every slice green, the blend red.
Differentiated policy — segments feeding back into the model
The third job. Once you can name a segment, you can apply policy the global objective can't express on its own:
- Per-segment exploration budget. New and low-history users have the most to gain from exploration and the least to lose; whales have a finely-tuned profile you should mostly exploit. Spend exploration where uncertainty is highest — that's a segment-conditioned bandit budget, the direct tie to lesson 15 · cold start (a segment prior beats a random one), lesson 38 · cold-start advanced for the meta-learning and uncertainty-driven version, and the explore/exploit machinery.
- Per-segment objective weights. A multi-task ranker blends heads (CTR, watch-time, retention) with weights. Those weights need not be global: for new users you might up-weight retention and diversity over immediate CTR (so you don't clickbait someone into churning on day one); for whales you might trust engagement more. Segment-conditioned objective weights are how business strategy enters the model. Which segment a treatment actually moves is an uplift question, not a correlation — target the policy at the segment with the highest incremental response, the heterogeneous-treatment-effect machinery from lesson 27 · causal & uplift (a segment that would have converted anyway is wasted treatment).
- Per-segment guardrails. Every launch decision carries a guardrail set — new vs returning, whales vs casual, by region — and a change ships only if no protected segment regresses, regardless of the aggregate. This is the operational form of "always slice", and it's the same fairness instinct as lesson 20.
Interview prompts you should be ready for
- "A single global model already personalizes per user. Why segment at all?" (Three jobs the global model is bad at: a prior for low-history users (cold start), a lens to read per-population behavior the average hides, and a policy handle — exploration budget, objective weights, guardrails — applied coarser than the individual. Segmentation complements personalization; it doesn't replace it.)
- "Walk me through a user profile / 用户画像. What's in it?" (A stack of layers: static/demographic, declared, behavioral-aggregate, sequence-derived, learned embedding. The duality is the senior point — human-readable tags for rules/ops/explainability vs a dense embedding for the model — and you keep both because the embedding can't be queried by a PM or audited by legal.)
- "What is RFM, and why does it survive in the deep-learning era?" (Recency / Frequency / Monetary→value, quantile-scored and concatenated into named segments. It survives not as a predictor but as the business's vocabulary, the guardrail dimensions you slice on, and a cheap cold-start prior. Coarse and backward-looking — pair it with the model, don't replace it.)
- "When would you cluster learned embeddings instead of engineered features — and what's the catch with K-means?" (Embedding clustering captures interaction structure RFM/feature-K-means can't — latent taste communities — at the cost of interpretability. K-means catch: scale features first or the largest-range axis dominates the Euclidean distance; choose k via elbow/silhouette.)
- "Your user profile is one table recomputed nightly. What's wrong, and how do you fix it?" (One schedule for fields on three different clocks: you waste compute re-deriving static fields (gender) every night and your session intent is up to 24h stale. Tier it like a cache — static (write-once) / dynamic (T+1 batch in the feature store) / real-time (per-event in Redis) — and pay for freshness only where the field actually changes fast. The tier is the decision about which clock a field runs on; wrong tier = bloated bill or stale person.)
- "You ran K-means, got a silhouette of 0.7, and shipped. What could still be wrong?" (Silhouette and the other internal metrics (Calinski-Harabasz, Davies-Bouldin) bake in a convexity prior, so a high score only says the partition is geometrically tidy — not that it's meaningful or that K-means' equal-size-convex-blob assumption fits the data. If the true structure is non-convex or density-based, DBSCAN's correct answer would score worse on silhouette. Pick the algorithm by data shape (GMM for soft/overlapping membership, DBSCAN for arbitrary shapes + outliers, StreamKM++ for drift), and validate the segments by interpretability and a live A/B on the resulting policy, not by the internal metric alone.)
- "Aggregate CTR rose in the A/B test but you're nervous. What do you check?" (Slice by segment for Simpson's paradox / mix shift: the aggregate is a mix-weighted average of per-segment CTRs, so a change to the traffic mix can move the blend opposite to every slice. Confirm per-segment guardrails — new vs returning, by region — before trusting the top line.)
- "How do you model a user who is shopping for a one-off gift today but normally browses cooking?" (Two clocks: a slow durable-interest profile and a fast session-intent profile, fused by a tuned or learned-attention blend. The session profile carries the gift intent without corrupting the durable profile, and decays/resets after the session.)
- "How can segmentation hurt the user, and how do you prevent it?" (Segments-as-stereotypes → filter bubble: bucket as 'sports fan', serve only sports, confirm the stereotype. A segment is a prior to update away from, not a cage — pair every segment prior with an exploration budget and diversity guardrails (lesson 20). Also never use a protected attribute as a hard policy lever.)