Cold start advanced - priors, probes, exploration, and graduation

The book returns to cold start many times because it is where modeling, product design, exploration, content understanding, privacy, and system latency meet. This lesson linearizes the advanced version: define the cold entity, choose priors, collect early signals, allocate exploration, protect quality, transfer carefully, and graduate into the main ranker.

1 · Cold start is three different problems

The book's first cold-start distinction is essential. Do not answer "use popularity" before naming which data is missing.

Short video makes all three harder: content lifecycle is short, feedback arrives quickly, new videos arrive constantly, and the first few recommendations shape whether a user returns.

2 · Build a relay, not one cold-start model

A mature design changes policy as evidence accumulates. The book hints at this with rules, content-based recommendation, real-time feedback, bandits, and main models. Put them into a relay:

The relay needs two explicit thresholds: when to enter cold-start treatment and when to leave it. Without a graduation rule, users remain trapped in early assumptions and new items stay over- or under-explored.

3 · New-user cold start: ask, infer, probe, adapt

The book names several new-user signals: registration information, lightweight interest selection, social authorization, device/context metadata, first-screen interaction, dwell, completion, swipe speed, and cross-domain behavior. Use them in order from least invasive to most behavior-specific.

A good onboarding feed should be both safe and informative. If all probes are globally popular comedy videos, you learn little. If probes are too random, you waste the user's first session. The book's "hot content + random exploration + content matching" pattern is a practical starting point.

Concrete interview knob: for the first few exposures, a common starter policy is majority safe/popular content, some content-matched recommendations, and a controlled exploration slice. The book gives examples such as hot-list fallback plus random/long-tail exploration and 10%-20% probe traffic. Do not treat those numbers as universal; treat them as a way to explain that exploration is budgeted, measured, and reduced as evidence arrives.

4 · New-item cold start: content first, then qualified exposure

For a new video, collaborative filtering has no interactions to work with. The book therefore emphasizes content-side information: tags, category hierarchy, cover image, title, ASR text, background music, visual embeddings, quality scores, and similarity to existing items.

5 · Exploration-exploitation is a budget

The book explicitly names epsilon-greedy, UCB, Thompson sampling, Bandit, LinUCB, and multi-objective exploration. The practical question is not "which algorithm is coolest?" It is "how much user experience are we willing to spend to learn?"

In a short-video feed, exploration should be capped by safety and quality. Do not give unlimited trial exposure to low-trust content simply because uncertainty is high.

5a · Tiered cold-start flow pools (engineering the exploration budget)

"Exploration is a budget" is the principle; the flow pool is how production actually spends it. A new item is not dropped into the main ranker. It is routed through a ladder of escalating exposure pools and must clear a quality bar to climb each rung. The why is cost control with fairness: a bad item's blast radius is capped at the pool it is stuck in, while every item still gets a fair, bounded test before judgment. This is the structured alternative to "give 10% random traffic and hope."

Two pieces make this work in practice, both straight from the playbook:

• Tiered allocation by exploration strength. The book splits trial traffic into Tier-1 ≈ 5% strong exploration (allocated by a contextual bandit such as LinUCB, see lesson 16, so probes go to plausibly-matched users) and Tier-2 ≈ 15% weak exploration (uniform-by-category, so no niche is starved). Treat these as a ratio to explain, not a universal constant.
• Sliding-decay exposure weighting. A new item gets a guaranteed base exposure in its first window (e.g. its first hour), after which exposure is dynamically scaled by real-time CTR/CVR. An early-signal amplification step up-weights likes and completions to fight the sparse-feedback problem (5 likes out of 40 plays is a strong signal you must not average away), while a fast negative-feedback reclaim down-weights low performers quickly so spend follows evidence.

Worked promotion check

Suppose the Pool 0 → Pool 1 bar is "completion rate ≥ 0.35 AND report rate ≤ 0.5% over ≥ 400 qualified impressions." A probe item logs 500 impressions, 190 completions, 1 report:

All three clear, so the item promotes to Pool 1 (the ~5k rung). Had report rate hit 1.2%, it would be held or reclaimed regardless of CTR — the safety bar dominates the engagement bar, so a clickbait item that wins clicks but draws reports cannot buy its way up.

5b · Uncertainty-aware ranking: variance as an exploration bonus

Bandits (section 5) explore in a separate re-rank layer. A complementary mechanism puts exploration inside the ranker as a feature. Estimate the model's predictive uncertainty — the output variance of a Bayesian neural network (BNN) or, cheaply, the variance across several MC-dropout forward passes — and add it as a UCB-style bonus to the score:

where μ is the mean predicted engagement, σ its standard deviation across stochastic passes, and β the exploration strength. The book frames a related form as a convex blend, final = α·score + (1−α)·uncertainty — a different parameterization of the same instinct: "boost what we are unsure about, because that is where a probe buys the most information." (The two aren't identical — the additive bonus and the convex blend trade off mean vs. uncertainty differently — but both push exploration toward high-variance items.) An item is flagged cold by a hard rule: exposures < 100.

The bonus must α-decay (equivalently, decay β) as exposures accumulate: more data shrinks σ naturally, and you also actively pull the multiplier down so a graduated item stops getting exploration help it no longer needs.

Worked UCB score

A cold item (60 exposures, so < 100) gets 8 MC-dropout passes giving mean pCTR μ = 0.040 with σ = 0.018 (high variance — the model has barely seen it). A mature item has μ = 0.052, σ = 0.002. With β = 0.5:

On raw mean the cold item loses (0.040 < 0.052), but the bonus closes most of the gap (0.0490 vs 0.0530), earning it exposure to resolve the uncertainty. After ~200 more impressions its σ might fall to 0.004 and we decay β → 0.2:

Now the bonus is worth 0.0008 — negligible. The item rises or falls on its real mean, exactly as graduation intends.

This is "exploration as a model feature," and it composes with the flow pools above: the bonus decides ranking within a pool's traffic; the pool decides how much traffic exists.

6 · Transfer learning helps only when domains align

The book mentions cross-domain migration: social, e-commerce, search, regional data, graph embeddings, and federated transfer. Transfer can shorten cold start, but it can also import wrong assumptions.

Use transfer as a prior, not as truth. Let early local behavior override it quickly when evidence contradicts the prior.

Also monitor drift by source. A social graph prior may be useful for creator affinity but weak for entertainment topic preference; a device or region prior may help bootstrapping but become unfair if it dominates after real behavior appears.

7 · Meta-learning is fast adaptation, not zero-signal magic

The book mentions MAML/meta-learning for small-sample cold start. The right explanation is:

Meta-learning helps after a few informative signals arrive. It does not replace content features, privacy-safe context, exploration, or quality gates. In interviews, pair it with the relay rather than presenting it as a magic model.

7a · Beyond MAML: prototypical init and synthetic interactions

MAML is the optimization-based meta-learning family. The book names two others, and a synthesis trick worth knowing because they trade serving cost differently.

The engineering point: prototypical-net init is often the pragmatic default for brand-new entities because the embedding is assigned by a similarity lookup against precomputed prototypes — no per-user fine-tuning in the serving path, unlike MAML. You pay the fine-tune cost only when a few real signals justify it.

Synthetic interactions attack the same sparsity from the data side: generate plausible early behavior to warm the embedding before any real impressions exist.

• GAN-generated pseudo interactions — synthesize user–item edges and inject them into the training graph to densify a sparse cold node, so collaborative signal exists to learn from.
• LLM-generated virtual behavior sequences — prompt an LLM to produce a plausible early watch/like sequence for the new entity from its content description (cross-link the multimodal features in lesson 14).

7b · GNN / LightGCN: borrow neighbors' embeddings through the graph

A complementary warm-up uses the structure the platform already has. Build a user–item bipartite graph (edges weighted by interaction count or dwell), initialize new nodes from attributes (category tags, demographics), then run LightGCN-style multi-hop neighbor aggregation so a cold node inherits high-order features from its few neighbors and their neighbors. A cold user's embedding becomes a weighted blend of the items they touched plus the users those items connect to. Add a cross-domain contrastive term to the BPR objective that pulls a cold user toward similar warm users in embedding space (PinSage is the production-scale example for densifying sparse behavior). See lesson 24 for the message-passing mechanics.

So the relay now has three warm-start sources that compose rather than compete: content (always available), prototype/MAML (after a few signals), and graph propagation (when the node has neighbors). Pick by what the new entity actually has.

8 · Graduate cold entities explicitly

Graduation can be one-way or staged. For users with interest drift, the system may re-enter a "session cold-start" mode for a topic even when the user is not globally new.

9 · Evaluate cold start separately

The book emphasizes cold-start-specific evaluation: first-screen CTR, first-session value, D7 retention, new-item exposure, exploration efficiency, and long-term value. Global averages hide cold-start failures because mature users dominate traffic.

9a · Cold-start A/B specifics

General A/B design lives in lessons 33 and 08. Cold start adds four problems that change the experiment: the metric you care about (D7 retention) is far downstream, cold traffic is a thin slice, the first days are dominated by a novelty effect, and a bad strategy can hurt real users fast. The book's recipe addresses each.

The pattern to articulate: a cold-start A/B is a variance-reduction and safety exercise. CUPED and the holdout fight the small-sample noise; dropping day 1–3 and watching first-30-min plays fight the timing problem; the ≥ 15% neg-feedback halt and p < 0.01 bar bound the downside. Read the new-user cohort separately — a global watch-time win can hide a new-user retention loss the holdout would have caught.

Interview prompts you should be ready for