User-behavior sequence modeling

A recommender never sees a user's mind — only their click stream. Every label it trains on, every interest vector it builds, is reconstructed from behavior the system itself shaped. This lesson follows one click from screen to gradient: how a tap becomes a label, why that label is biased before it is recorded, and how an ordered list of behaviors becomes the user vector that the ranker actually scores.

Two kinds of feedback, and why neither is the truth

Behavior splits into explicit feedback — actions whose purpose is to express preference (a like, a rating, a "not interested" tap, a follow) — and implicit feedback — the exhaust of normal use that we interpret as preference (a click, watch-time, dwell, a scroll-past, a replay). The trade is the eternal one between volume and clarity.

Explicit feedback is rare but high-precision: a 5-star rating means something. Implicit feedback is abundant — typically 100–1000× more of it — but every signal is contaminated. A 6-minute watch might mean fascination or a phone left face-up on a couch. A skip might mean dislike, "already seen it," or a passing distraction. Behavior modeling assigns each signal a strength and a noise level, then lets abundance compensate for ambiguity.

How a click becomes a label — and why it is biased at birth

The flywheel is seductive: show items, log clicks, train on clicks, show better items. The hidden danger is in the very first arrow. The model trains on (item, clicked?) pairs — but the items in that log are not a random sample of the corpus. They are exactly the items a previous model chose to show, placed in positions the UI chose. The label is corrupted by the system that produced it, in three compounding ways.

• Selection / exposure bias. You only observe feedback on items you displayed. Everything the model never surfaced has no label — and "no label" gets silently treated as "negative" when you sample un-clicked items as negatives. The model learns to like what it already showed; the un-shown long tail stays un-shown. Survivorship bias in a recommender's clothes.
• Position bias. An item at rank 1 is clicked far more often than the same item at rank 10, purely because eyes and thumbs reach it first. Train CTR naively and the model attributes the click to the item when much of it belongs to the slot — it learns "things at the top are good," a tautology that, once deployed, cements whatever was already on top.
• Presentation bias. Thumbnail, title, autoplay-on, surrounding items — packaging shifts the click independent of content. Clickbait is position bias's cousin: it games the label, not the preference.

The clean way to think about position bias is a factorization. A click happens only if the user examined the slot and found the item relevant — the examination hypothesis:

P(click | item, rank) = P(examine | rank) · P(relevant | item)

The first factor is the position propensity p_r — pure UI artifact. The second is what you actually want. As a rough calibration, a typical feed sees p_1 ≈ 1.0, p_3 ≈ 0.5, p_10 ≈ 0.2 — so an item at slot 10 needs to be ~5× more relevant to earn the same observed click rate as slot 1. If you can estimate p_r (by randomizing positions in a small traffic slice, or by "swap" experiments), you recover relevance two ways:

• Position as a feature. Feed rank into the model at train time and fix/zero it at serve time — the model attributes the slot-driven part of the click to the position embedding instead of the item. (Lesson 4's "position at training only" toggle.)
• Inverse propensity weighting (IPW). Divide each example's loss by its propensity, L_IPW = Σ (1 / p_{r_i}) · ℓ(ŷ_i, y_i), so rarely-examined positions count more.

This is the same debiasing machinery as lesson 07. The mental shift is the whole game: a click is an observation of a biased process, not a measurement of preference. Model the process.

From a sequence of behaviors to an interest vector

A single click is a weak signal; an ordered sequence of clicks is a trajectory through interest space. Sequence modeling turns a user's recent interactions — [v₁, v₂, …, v_n], each item an embedding plus side-features (category, dwell, timestamp) — into one user vector, then matches it against a candidate. Four generations of mechanism, each fixing the one before:

DIN — target attention over the behavior sequence

A user who watches cooking, gaming, and travel videos has no single "interest." Pooling produces a muddy average that matches nothing well. DIN's (Deep Interest Network, Zhou et al. 2018) insight: the relevant slice of history depends on what you're scoring. When ranking a recipe, attend to the cooking clips and ignore the rest; when ranking a game trailer, do the reverse. The user vector becomes a candidate-conditional weighted sum of the history:

u(cand) = Σᵢ αᵢ · vᵢ ,   αᵢ = softmax( a(vᵢ, v_cand) )

where a(·,·) is a small MLP — the "activation unit" — fed [vᵢ, v_cand, vᵢ − v_cand, vᵢ ⊙ v_cand]. The explicit difference and product terms hand the unit the cross features instead of forcing it to learn subtraction. Note that u changes per candidate: the same history yields a different user vector for every item scored. (DIN's original paper softmaxes without normalization to preserve interest intensity — a user with 20 cooking clips should activate harder than one with 2 — a detail worth naming in an interview.)

DIEN — modeling how interest evolves

DIEN (Deep Interest Evolution Network, Zhou et al. 2019) adds time. DIN attends over raw clicked-item embeddings; DIEN first threads a GRU through the history to extract an interest state at each step (the "interest extractor," supervised by an auxiliary loss that predicts the next behavior), then runs a second AUGRU — a GRU whose update gate is scaled by the target-attention weight — to model how interest grows and fades toward the candidate. A fading interest contributes less than a surging one. The cost is the GRU's sequential dependency: it does not parallelize the way self-attention does, which is why high-QPS systems often prefer DIN or a Transformer block (BST) with a hard length cap.

Both DIN and DIEN live inside the ranker from lesson 04 — they are the sub-network that consumes the "sparse multi-valued" user-history feature and emits a pooled vector that concatenates with the dense and sparse-ID embeddings before the top MLP.

Lifelong sequences — why a flat attention over 10k events is dead on arrival

DIN, DIEN, and a BST-style Transformer all share an unspoken assumption: the history is short — last 50, maybe a few hundred events. That cap is not a stylistic choice; it is forced by the serving budget. The deeper the user, the more this hurts: the behavior that explains this click — a phone bought eleven months ago, a recipe saved last spring — is often thousands of events back, exactly where the length cap throws it away. So the question that dominates senior interviews: how do you attend over a lifelong sequence of 10,000+ behaviors?

Start with why you cannot just raise the cap. Walk the cost of a single ranking request, batch of B = 1000 candidates, history length n, embedding dim d = 64:

The self-attention row is the killer: n² means going from 100 → 10,000 events is not 100× the cost, it is 100² = 10,000×. Even the linear DIN row, multiplied across 1000 candidates, blows a 50 ms budget. And it is wasteful: of those 10,000 events, maybe 200 have anything to do with the candidate. The fix in every production answer is the same shape — retrieve first, attend second. Three named architectures, each a different bet on where you pay the cost.

SIM — two-stage GSU → ESU (retrieve, then attend)

SIM (Search-based Interest Model, Pi et al. 2020) splits the problem into a cheap General Search Unit (GSU) and an expensive Exact Search Unit (ESU). The GSU pre-filters the lifelong sequence down to the few-hundred behaviors relevant to the candidate; the ESU then runs full DIN-style target attention over only that subset. Two flavors of GSU:

• Hard search. Index the user's lifelong behaviors by category (or a coarse cluster ID). At request time, look up the candidate's category and pull only the history items in that category. No model, just a key lookup against a per-user inverted index — O(1) amortized. A user with 10,000 lifelong events but 300 in the candidate's category hands the ESU a length-300 sequence. This is what ships at scale: the index can live in an offline table, refreshed daily.
• Soft search. Pre-compute an embedding per behavior; retrieve the top-K by inner-product with the candidate embedding (the same ANN machinery as retrieval, lesson 03). More precise than category match — it catches cross-category affinity — but needs an embedding index kept consistent with the model, which is the operational tax.

Worked saving. Lifelong n = 10,000, candidate category holds 250 of them, K = 200 retained. The ESU now runs DIN over 200 items, not 10,000: 200 · 64 ≈ 1.3×10⁴ MAC per candidate vs 6.4×10⁵ for flat DIN — a ~50× cut — and the GSU cost is a hash lookup, not a matrix multiply. The bet SIM makes: the candidate's category is a good enough filter that the dropped 9,750 events held no signal for this item. Usually true; the failure mode is a cross-category interest (cooking history predicting a kitchen-gadget purchase) that hard search misses and only soft search recovers.

MIMN — fold the sequence into a fixed memory, so serving cost stops growing

SIM still stores the full lifelong sequence and searches it per request. MIMN (Multi-channel user Interest Memory Network, Pi et al. 2019) makes a different bet: never store the raw sequence at serve time at all. It keeps a small fixed-size memory matrix M — say m = 4 slots × d = 64, one "channel" per coarse interest — and a Neural-Turing-Machine controller that incrementally writes each new behavior into the slots as it streams in:

Mₜ = (1 − wₜ ⊗ eₜ) ⊙ Mₜ₋₁ + wₜ ⊗ aₜ

where wₜ ∈ ℝ^m is a soft write-address over the m slots, eₜ and aₜ ∈ ℝ^d are the erase and add vectors over the d feature dimensions, and ⊗ is the outer product — so each of wₜ ⊗ eₜ and wₜ ⊗ aₜ is an m×d matrix shaped exactly like M (the address selects the slot, the vector fills its row), making the update well-typed. The point is the dimensions: M is 4×64 = 256 numbers no matter whether the user has 100 lifelong events or 100,000. Serving reads a fixed 256-float state — cost decoupled from sequence length entirely. The write happens once per behavior in the streaming pipeline (below), never at request time. The trade is lossy compression: four slots cannot hold the fidelity of 10,000 raw events, so MIMN is paired with a separate short-term DIN tower for recent precision — memory for the lifelong gist, attention for the fresh detail.

DSIN — the sessionization the lesson described, made concrete

Later this lesson sessionizes the stream conceptually. DSIN (Deep Session Interest Network, Feng et al. 2019) is the architecture: cut the sequence into sessions at the ~30-min gap, then process within and across sessions separately.

• Intra-session: a Bias Encoding (a learned position-in-session + session-index offset added to each item) plus multi-head self-attention — within a 20-minute camping burst, every behavior is coherent, so self-attention earns its O(k²) on the small k of one session.
• Inter-session: a Bi-LSTM over the per-session interest vectors models how intent evolves across sessions (tents this week → sleeping bags next). Bi-directional because a later session can re-interpret an earlier one.

The cost discipline is the same retrieve-then-attend idea wearing different clothes: self-attention is confined to within a session (small k), and the expensive cross-session modeling runs over a handful of session vectors, not thousands of raw events.

Short- vs long-term interest — two clocks

Behavior carries two timescales that fight each other. Long-term interest is stable taste: someone is, over years, "into" sci-fi and indie music. Short-term interest is the session's intent: tonight they're shopping for a tent, and every recommendation should bend toward camping for twenty minutes — then snap back. Collapse the two and you get a recommender that is either too sluggish to react or too jittery to be coherent.

The standard architecture keeps two encoders: a long-term tower over months of history (heavier, recomputed offline / daily) and a short-term tower over the last N events (recomputed in near-real-time, seconds-fresh), fused by a learned gate:

u = σ(g) ⊙ u_long + (1 − σ(g)) ⊙ u_short

The gate σ(g) depends on context — when the session signal is strong and decisive (a burst of camping clicks), the gate shifts weight to u_short. This is the offline / nearline / online split: the long tower can be a cached two-tower user embedding refreshed nightly; the short tower must run on the freshest event stream.

Time decay: making "recent" precise

Inside both the gate and the attention, recency is encoded by a decay weight on each past behavior. The workhorse is exponential decay on the gap Δt since the action:

wᵢ = w_base(actionᵢ) · e^(−λ Δtᵢ)

Two knobs do the work. w_base encodes signal strength — a share weighs more than a click (e.g. share 5, like 3, watch 1). λ sets the half-life: t½ = ln 2 / λ. A short-video feed uses a fast decay (interest turns over in hours); a movie service uses a slow one (taste persists for months). Choosing λ is choosing how long the past gets to vote — one of the most impactful, least-glamorous hyperparameters in the system.

Work the numbers. Pick a 48-hour half-life, so λ = ln 2 / 48 ≈ 0.01444 per hour. A behavior from Δt = 24 h ago keeps e^(−0.01444·24) = e^(−0.347) ≈ 0.71 of its weight; from 72 h ago, e^(−0.01444·72) = e^(−1.04) ≈ 0.35; from one week (168 h) ago, e^(−2.42) ≈ 0.089 — under a tenth, effectively forgotten. Halve the half-life to 24 h (λ ≈ 0.0289) and that same 72-hour behavior drops to e^(−2.08) ≈ 0.125 instead of 0.35 — the feed reacts roughly 3× faster to fresh intent but throws away more of yesterday. That single knob is the dial between "jittery" and "sluggish."

Scalar decay vs learned time embeddings (Time2Vec)

Exponential decay imposes one assumption: forgetting is monotonic — older always matters less. That is wrong for any periodic behavior. A user who opens a cooking feed every Sunday morning has a strong signal at Δt = 7 days that a decay curve actively suppresses, because by then e^(−λ·168h) has crushed it. The fix is to stop hand-coding the time response and let the model learn it.

• Time-bucketing. Before encoding, slot each behavior into coarse age buckets — e.g. <5 min / <1 h / <24 h / older — and give each bucket its own learnable embedding. Cheap, and it lets "5 minutes ago" and "yesterday" carry qualitatively different meaning rather than just a scaled-down version of the same thing.
• Time2Vec. Embed the raw timestamp (or the inter-event gap Δt) as a vector with one linear term and several periodic terms: t2v(τ)[i] = ω₀·τ + φ₀   (i = 0);   sin(ωᵢ·τ + φᵢ)   (i ≥ 1) The linear term captures trend (recency); the sin terms, with learned frequencies ωᵢ, capture periodicity — a frequency whose period lands near 24 h or 7 days lets the model represent "same hour daily" or "same day weekly." This vector is concatenated onto each behavior embedding and fed into the attention, so the model learns the time response instead of you imposing e^(−λΔt).

The trade is the usual one: scalar decay is one hyperparameter, robust, and free; Time2Vec adds parameters and needs enough data per user to learn a stable periodicity, so it earns its keep on heavy users and over-fits on sparse ones. Many systems use both — decay as a feature and a small learned time embedding — and let the attention weigh them.

Behavior denoising — what reaches the sequence in the first place

Everything above assumes the sequence is clean. It is not. A real click stream is laced with mis-taps, inertial fast-scrolls, and autoplay impressions — behaviors that carry no intent but, if fed to the attention, get a weight and corrupt the user vector. Denoising happens at two stages, and the trade-off between them is concrete.

• Hard pre-filters (before the model). Drop a play with watch-time < 500 ms, or a slide whose dwell < 300 ms, before the behavior ever enters the sequence. A thumb that flicks past four items in a second generated four "impressions"; none is intent. Measured trade-off from production: direct filtering raises sequence purity ~15–20% (the fraction of behaviors that reflect genuine interest), but mis-kills ~5% of real fast browsing — a power-user who genuinely skims at 400 ms gets some true behaviors dropped. Net positive, and it is the cheap win because every filtered event is one fewer item the attention must score downstream.
• Attention denoising (inside the model). Keep the noisy behaviors but let the model learn to down-weight them — the activation unit assigns a near-zero αᵢ to an anomalous event. This retains latent signal a hard filter would have killed (the 5% false-kill recovered), but costs ~30% more attention compute, because you are now scoring the full, un-filtered sequence.

So the recipe is layered: hard-filter the obvious noise to shrink the sequence (purity + cheaper attention), then let attention soft-denoise the rest. Two refinements from practice: a noise classifier trained on a user's own positives (their liked/completed videos) scores whether a new behavior is anomalous for that user; and contrastive learning builds augmented views of a behavior sequence (drop/mask/reorder some events) and trains the encoder to map them close, which makes the user vector robust to exactly the noise you could not filter.

Sessionization — cutting the stream into intents

The raw event log is one unbroken stream per user. But a session — a bounded burst of activity with a coherent intent — is the natural unit of short-term interest. You sessionize by cutting the stream wherever the gap between consecutive events exceeds a threshold (commonly ~30 minutes of inactivity, tuned per product): new session if (tᵢ − tᵢ₋₁) > τ_gap.

Sessionization earns its keep three ways. It scopes short-term interest (the camping intent lives in this session, not last week's). It de-noises sequences: behaviors from different sessions shouldn't blur together in the GRU, since a 30-minute gap usually means the intent reset. And it powers session-based metrics — sessions/user, depth-per-session, completion within session — that often track satisfaction better than raw event counts.

The real-time behavior pipeline — putting a number on "seconds-fresh"

The short-term tower is only as good as the freshness of the sequence it reads. "Seconds-fresh" is not a vibe; it is a latency budget with a specific architecture behind it. The path from a thumb-tap to an updated user vector, with the numbers a short-video stack actually targets:

Read the budget as a chain of constraints. Edge pre-aggregation merges a burst of identical events (four fast swipes → one summarized event) so the wire and Kafka see ~30% less traffic and the obvious mis-taps die before they cost anything. Partition by user-hash is the load-bearing detail: a sequence model is meaningless if events arrive out of order, so every event for a user must route to the same Kafka partition, which preserves per-user ordering for free. Flink's 30 s window / 10 s slide sets the recompute cadence — a fresh short-term vector every 10 seconds. Dual-write Redis + HBase splits roles: Redis serves the online read on the request path under ~200 ms; HBase is the durable store for backfill and the lifelong sequence SIM searches. End to end, tap → usable feature lands under 3 s, and a Douyin-class intent-prediction loop targets < 200 ms for the real-time sequence update itself. (This is the streaming machinery of lesson 17 (real-time streaming) and lesson 36 (streaming pipeline ops) — see them for the Kafka/Flink internals; here we only care that it delivers a fresh, ordered sequence on budget.)

Sudden interest shifts — reacting in a session, not a quarter

Time-decay and sessionization handle gradual drift. They are too slow for a mutation — a user who pivots from entertainment to study content tonight, a signal the slow tower won't register for days. Treating this as ordinary drift means the feed keeps serving the old self through the whole pivot. The named recipe:

• Detect. Track sliding-window statistics — e.g. the variance of CTR or category entropy over the last ~10 behaviors. A sharp jump flags a candidate mutation, via a lightweight CUSUM change-point test or an LSTM anomaly detector.
• Confirm, don't trust. Most spikes are noise — one study video then straight back to comedy is not a pivot. Filter false mutations by behavior-chain analysis: a genuine shift sustains across several behaviors and sessions; a blip reverts immediately.
• Expand & explore. Extract the new interest (cluster the mutation behaviors), then use a knowledge graph to fan out to related verticals ("study" → programming, language-learning), and launch a bandit that hands the new interest a temporary exposure boost — on the order of ~30% — to gather signal fast. Monitor D7 retention to confirm the pivot was real and the response helped, then let the boost decay back.

This is the sharp, named version of the earlier interest-drift callout: detect the change-point, gate out the false alarms, and buy fast evidence with a bounded exploration budget.

Interactive · target attention in action

One toy history of 8 past behaviors. Pick the candidate you're scoring — the widget computes a DIN-style attention weight αᵢ for each past item (softmax over category match × recency), and shows which behaviors "light up." Watch the same history produce a completely different user vector depending on what you score.

Behavior becomes features — the handoff to the rest of the stack

Everything above lands as features feeding the ranking model. Three families, increasingly powerful and increasingly perishable:

• Aggregate statistics — counts, rates, decayed sums: user CTR over 7 days, completion rate per category, sessions this week. Cheap, robust, the backbone.
• Sequence features — the raw ordered list of last-N item IDs + side info, handed to a DIN / DIEN / Transformer block inside the model. Where the attention machinery lives.
• Learned interest vectors — the gated short/long user embedding, precomputed and cached for retrieval (the two-tower user vector) and recomputed online for ranking.

Two failure modes, both already named: negative signals are features too — skips, "not interested," and fast-scrolls are not the absence of data, they are weak-negative labels (treating un-clicked-but-shown items as hard negatives is the cheap version); and optimizing only immediate behavior is a trap — clicks and watch-time are the easiest labels to get and the easiest to over-fit, which is precisely how a feed wins every short-term metric while quietly burning long-term retention. That next-click-vs-long-term-value tension is the heart of multi-task / multi-objective ranking.

Turning raw signals into a shippable strength score

The opening table assigned each signal a qualitative "strength." Here is the concrete recipe that turns that into a number the model can consume — every line a guard against a specific pathology:

• Action weights, set by business logic. A composite implicit score with hand-set (or SHAP-derived) weights, e.g. play = 1, like = 3, share = 5 — a share is worth ~5 plays because it is rarer and far more intentful. These are the same w_base weights from the time-decay formula.
• Log-transform dwell time. Dwell is brutally long-tailed — a few sessions run for an hour, most for seconds — and raw dwell lets those few dominate. Use log(dwell): a 600 s watch becomes log(600) ≈ 6.4, a 6 s watch log(6) ≈ 1.8, so the long watch counts ~3.6× the short one, not 100×. The composite becomes score = α·log(dwell) + β·click + γ·engage.
• Truncate the extremes. Cap dwell at the 95th percentile before the log so a phone-left-on-the-couch outlier (the idle-time confounder from the opening callout) cannot masquerade as deep engagement.
• Completion as a hard floor. An item in the low-completion bucket gets its exposure capped even if its CTR is high — this is the "clickbait, high-CTR-low-completion" trap from the opening callout, made operational. Completion gates, CTR ranks.
• Second-chance / regret rule. A behavior that was skipped but had long dwell is ambiguous, not negative — re-recommend it once rather than burying it. The skip might have meant "not now," and the dwell says the interest was real.

Interview prompts you should be ready for