Multi-task & multi-objective ranking

No feed is graded on clicks alone. This lesson is how one model predicts click, watch-time, like, share and conversion at once — why naïve sharing makes every head worse, and how MMoE, PLE and ESMM each remove one specific failure before collapsing the heads into a single ranking score.

Why one objective is never enough

Open a short-video feed. The product wants the user to click a video, watch most of it, then like / comment / share / follow, and — the metric that actually pays salaries — keep coming back next week. These are different events with wildly different statistics. The base rates alone tell you that you cannot train them the same way.

That last row is the one juniors forget. Production rankers carry negative heads — swipe-away rate (the user flicks past in <1s) and report rate — predicted exactly like the positive heads but subtracted in fusion. Without them the model happily learns that any attention-grabbing thumbnail is good, even one users immediately bounce from or flag. A swipe-away head is a cheap, dense dissatisfaction signal (every impression has a dwell time) that catches the failure long before the sparse report head fires; the report head is the safety floor. We return to where these enter the score in the fusion section.

You could train five separate models, but that throws away the central asset of a recommender: the same user and item features predict all five. The user embedding that helps you guess a click also helps you guess a like. And the sparse targets — share, follow, conversion — have so few positives that a model trained on them alone overfits hard. With a 0.1% follow rate, 100M impressions buys you only ~100k positives across the entire vocabulary of items; a from-scratch follow model never sees enough signal per item to learn. They need to borrow representational strength from the dense tasks. That borrowing is the whole point of multi-task learning (MTL).

Crucially, the objectives conflict. Clickbait raises CTR while wrecking completion rate. Optimizing likes pushes toward agreeable, low-diversity content. On a real short-video corpus, comedy clips have high CTR and short watch-time; knowledge clips are the reverse — CTR and completion are outright negatively correlated within some segments. So MTL is not a free lunch. Sharing the wrong thing actively hurts, and that tension drives the entire architecture progression below. This is the same label-definition problem from 05 · Losses, now multiplied across heads.

Shared-bottom — MTL as transfer learning

The first idea is the obvious one. Stack a tower of shared layers — the "bottom" — that reads raw features into a common representation, then hang a small task-specific head on top for each objective. Every task backpropagates into the same bottom.

h = Bottom(x)  (shared)  ·  ŷ_k = Tower_k(h)  ·  L = Σ_k w_k · L_k(ŷ_k, y_k)

This is transfer learning happening in both directions at once: the dense CTR task teaches the bottom a good user/item representation, and the sparse like/share tasks ride on it instead of starving. When tasks are positively correlated, shared-bottom is excellent — one forward pass through the expensive bottom, cheap heads, statistically strong. It is also the cheapest thing to serve, which matters under the few-hundred-µs budget from 04 · Ranking models.

The fix cannot be "share everything" (negative transfer) nor "share nothing" (lose transfer, starve the sparse tasks). It must be conditional, learned sharing — let the network decide, per task, how much of the shared computation to use. That is exactly MMoE.

MMoE — experts plus per-task gates

MMoE (Multi-gate Mixture-of-Experts, Ma et al. 2018) replaces the single shared bottom with a bank of n parallel sub-networks called experts, and gives each task its own gating network that mixes the experts with task-specific weights. The experts are shared; the recipe for combining them is not. For task k the gate is a softmax over experts conditioned on the input:

Read it intuitively. There are n experts, each free to specialize — one learns "price sensitivity," another "novelty," another "session intent." Each task's gate is a tiny softmax that emits a probability distribution over those experts, per example. CTR can lean on experts 1 and 3; watch-time on 2 and 4; when their interests overlap they share an expert, when they conflict the gates route around each other. Conflicting tasks are no longer forced through one bottleneck — they take different weighted paths through one expert pool. That is the soft, learned parameter sharing shared-bottom lacked.

Two design notes. First, the gate is conditioned on x, so routing is per-user, per-item: the same model leans on the "explorer" expert for a new user and the "deep-consumer" expert for a power user. Second, MMoE is still cheap to serve — all experts run once (say 8 experts × a 256-wide MLP), and the per-task gates and towers are rounding error. With 4 tasks, you pay 8 expert passes versus 4 full bottoms for 4 separate models. It is a near-drop-in upgrade from shared-bottom.

The seesaw effect, and PLE's fix

MMoE softens conflict but does not eliminate it, because every expert is still shared by every task. When two objectives genuinely pull apart — short-video watch-time (needs deep interest signal) versus interaction rate (rewarded by shallow, immediate appeal) — you observe the seesaw effect: tuning the model to lift one objective reliably drags the other down. The shared experts cannot simultaneously hold a representation good for both. In practice teams see it as a Pareto frontier you slide along, never a configuration that lifts both at once.

PLE (Progressive Layered Extraction, Tencent 2020) attacks this by making the separation explicit in the architecture rather than hoping the gates discover it. Its core block is CGC (Customized Gate Control): split the expert pool into task-specific experts (private to one task) and shared experts (common to all). A task's gate may select from its own specific experts plus the shared ones — but never from another task's private experts.

The "progressive layered" part stacks several CGC blocks: each layer refines the separation, the shared experts feed the next layer's gates, and information is gradually decoupled into shared-vs-private channels rather than in one shot. The task-private experts give each conflicting objective a protected place to learn what only it needs — exactly what the seesaw was punishing. The cost is parameters and a harder serving footprint: private-expert pools complicate sharding and dynamic loading, whereas MMoE's uniform expert bank is the friendliest to serve.

The reported payoff is large enough to justify that cost when the seesaw is real (numbers are vendor-claimed, cite as such). On Kuaishou's short-video stack, PLE versus MMoE cut inter-task gradient conflict by 47%, lifted a sparse long-tail head (e.g. favorite) AUC +1.2%, and added +2.3% per-user watch-time — attributed to PLE more faithfully capturing the progressive watch → like → follow funnel. Be honest about the cost direction: PLE adds parameters and FLOPs over MMoE (more experts, one gate per task), so at face value its inference is more expensive, not less. The serving cost is clawed back at inference time by gate-threshold expert-skipping — PLE's gates are peaky, so most experts can be skipped per example (see the serving-knobs section below) — not by the architecture being intrinsically cheaper.

Deltas are vendor-reported (Google for MMoE; Kuaishou short-video for PLE) — read them as the shape of the win, not a guarantee for your corpus.

CVR's hidden trap — sample-selection bias

So far tasks differed only in correlation. Conversion rate (CVR) differs in something deeper: where you can observe its label at all. In an ad or e-commerce funnel the events are sequential — impression → click → conversion — and a purchase is only possible after a click. So the natural CVR model is "given a click, will it convert?" and you train it on the clicked samples only.

That is quietly broken, for two compounding reasons:

• Sample-selection bias (SSB). The model trains on the click space (the slice of impressions that got clicked) but at serving time must score the entire impression space — every candidate, most of which would never be clicked. Train and serve distributions differ, so the estimates are miscalibrated on exactly the items you rank. This is a cousin of the exposure bias in 07 · Bias.
• Data sparsity. Clicks are already 1–10% of impressions; conversions are a small fraction of those. With a 5% CTR and 3% CVR-on-click, conversions are ~0.15% of impressions — and the clicked-only training set is ~20× smaller than the full log, so the CVR tower overfits.

ESMM — model on the entire exposure space

ESMM (Entire-Space Multi-task Model, Alibaba 2018) fixes both problems at once with a single identity from the funnel's structure. The thing you actually want for ranking ads is pCTCVR — the probability that an impression is both clicked and converted. By the chain rule of the funnel:

ESMM builds three outputs from two towers. A CTR tower and a CVR tower each consume a shared embedding layer — the sparse-feature embeddings are shared, which is where the sparse CVR task borrows strength from the dense CTR task. Then:

• pCTR = CTR tower output — supervised by the click label, over all impressions.
• pCVR = CVR tower output — never directly supervised; it has no standalone loss.
• pCTCVR = pCTR × pCVR — supervised by the conversion label, also over all impressions.

The trick: both losses are computed over the entire impression space, not the click space. CVR is learned only implicitly, squeezed out as the factor that makes pCTR × pCVR match the conversion label. Because pCVR is never trained on the clicked-only subset, the sample-selection bias is gone by construction. And because its embeddings are shared with the high-volume CTR task, the sparsity problem is mitigated too. Two diseases, one cure. Note the failure mode: if pCTR is poorly calibrated (lesson 05), the error propagates multiplicatively into pCVR, so ESMM raises the stakes on CTR calibration.

Weighting and reconciling conflicting losses

Whatever the architecture, you still combine task losses into one scalar to backprop. The baseline weighted sum L = Σ_k w_k L_k has two real problems: tasks live on different scales (a 0–1 cross-entropy vs an MSE on watch-seconds that ranges 0–600), so the large-magnitude loss dominates the gradient; and hand-picking w_k is brittle — tiny weight changes swing business metrics. Three families of fix:

• Normalize the targets first. Z-score or quantile-normalize each objective so no task wins by sheer numeric magnitude. Quantile normalization is attractive because it preserves rank order, which is what ranking cares about; update the statistics on a sliding window for non-stationary feeds.
• Uncertainty weighting (Kendall et al. 2018). Make each task's weight learnable through its homoscedastic noise σ_k:
  L = Σ_k (1 / 2σ_k²) · L_k + log σ_k
  The model down-weights noisy/uncertain tasks automatically; the log σ_k term is the regularizer that stops it from driving all weights to zero. No more hand-tuning ratios — and it gives a new task a sane initial weight.
  Worked numbers — what the learned σ actually does

  Two heads: a clean CTR loss and a noisy watch-time MSE. Say the network settles at σ_ctr = 0.7 (confident) and σ_watch = 1.4 (twice as noisy). The effective loss weights are 1/2σ²:

  w_ctr = 1/(2·0.7²) = 1/0.98 ≈ 1.02  ·  w_watch = 1/(2·1.4²) = 1/3.92 ≈ 0.26

  The noisy head is auto-weighted to ~¼ of the clean head — a 4× ratio the model found, not one you guessed. Why doesn't it cheat by sending every σ → ∞ to zero out the data terms? Because the log σ penalty grows without bound: log(0.7) ≈ −0.36 vs log(1.4) ≈ +0.34, so inflating σ to dodge a loss costs you in the regularizer. The two forces balance at the noise level the data implies. Initialize the raw parameter as s = log σ² and predict w = ½·e^−s for numerical stability — optimizing σ directly invites divide-by-zero.

• Gradient-level reconciliation. Attack the conflict where it lives. GradNorm rescales per-task loss weights so all tasks learn at a similar rate — it reads each task's gradient norm at the last shared layer and minimizes an auxiliary balancing loss over the weights, which costs an extra backward pass per step and can be finicky to tune; budget for it. PCGrad detects when two task gradients oppose (cos < 0) and projects one onto the other's normal plane, removing the conflicting component before the update — directly targeting the negative-transfer mechanism above; reported to lift both like and completion ~+3% simultaneously in short-video ranking. CAGrad (Conflict-Averse Gradient) is PCGrad's successor for persistently, strongly opposed tasks: instead of one-shot projection it solves for an update direction that maximizes the worst-case per-task improvement while staying near the average gradient — so it still makes progress on the task PCGrad would have sacrificed, at the cost of a small per-step optimization. Rule of thumb: PCGrad when conflict is intermittent, CAGrad when cos(∇L_a, ∇L_b) sits stubbornly negative epoch after epoch.

A complementary lens is Pareto optimality: among all weightings, the interesting ones are the non-dominated solutions where you cannot improve one objective without hurting another. MGDA finds a common descent direction for all tasks; Pareto-front analysis lets the business see the trade-off curve and pick a point on it rather than guessing weights. The cost is solving an optimization sub-problem during training.

Score fusion — from many scores to one ordering

The model now emits several calibrated probabilities. The feed shows one list. Somewhere you must collapse the heads into a single ranking score — the multi-objective decision, separate from training.

Weighted sum is the workhorse — normalize first (the heads are already 0–1 probabilities here, which helps), then weight:

Weighted product is common in industry because it is unit-free and rewards items that are decent across the board rather than spiky on one head:

The exponents act as elasticities; a head with exponent 0 is dropped, and the multiplicative form means an item must be non-terrible on every weighted objective to rank high — a useful guard against single-metric gaming like clickbait, where one head spikes and the rest crater.

A concrete win that interviewers like: one platform found comedy clips had high CTR but short watch-time, knowledge clips the reverse. Re-tuning fusion lifted per-user watch-time +15% while CTR fell only −2% — a clearly good trade. The honest objective, though, is discounted long-term user value, which immediate clicks only proxy; any blend tuned purely to immediate signals can quietly erode 7-day / 30-day retention. That is the bridge to reinforcement learning, where the feed is recast as an MDP and the reward is retention.

The online weight-control loop — the mechanism behind "tune it online"

"Tune the weights with A/B" is the right instinct but a slow loop — days per variant. Mature systems run a continuous control loop that adjusts fusion weights in near-real-time as the feed shifts (a hot event lands, a user's interest drifts, a daypart turns over). Three mechanisms, in increasing autonomy, all sitting downstream of a frozen model — they re-blend the heads, they do not retrain:

• Per-user-state schedule (rule-based). The simplest: weights are a deterministic function of how mature the user is. New users get CTR:watch = 7:3 (lean on click to explore, because you have no watch history to trust); as the user crosses interaction milestones the schedule slides toward 5:5. Cheap, interpretable, and it encodes a real prior — covered as exploration in 02 · Candidate generation.
• PID feedback (reactive). Pick a guardrail to defend — say rolling completion-rate with target 32%. A PID controller watches the error e = target − measured and nudges the watch-time weight: Δw = K_p·e + K_i·Σe + K_d·Δe. Worked step: completion dips to 30%, so e = +0.02; with K_p = 5 the proportional term alone bumps the watch weight by +0.10, the integral term keeps pushing if the dip persists, and the derivative term damps overshoot so you don't oscillate. It reacts to drift without a human in the loop.
• Thompson-sampled arms (exploratory). Treat a handful of weight blends as bandit arms; keep a Beta posterior on each arm's success rate. After each impression, update the chosen arm's Beta(α, β) with the realized outcome (e.g. a satisfying watch → α += 1, a swipe-away → β += 1); next request, sample a weight combo proportional to posterior belief. This explores the weight space instead of greedily locking onto a local optimum — the same Thompson machinery as bandit retrieval, now over fusion weights rather than items.

None of these ship without guardrails, because a runaway controller can wreck the feed faster than any offline bug:

A concrete trace tying it together: a controller notices a user segment's golden-hour completion rate fell 5%, so it raises the watch-time weight 0.40 → 0.55 (EMA-smoothed over several ticks, never one jump) and trims the like weight by 0.10 to keep the blend normalized — all inside the click-≤-0.60 bound, with the KL fuse armed. No retrain, no deploy: the model is frozen, only the fusion blend moved.

Interactive · seesaw between click-AUC and watch-time

Slide the per-task loss weight for watch-time. The widget shows how the shared bottom reallocates capacity — click-AUC and watch-time move on a seesaw — and what the fused ranking score does at three sample weights. The numbers are a toy model of the trade-off; the diagnosis is the point.

Engineering reality — delay, heads, and consistency

• Label delay (the conversion problem). A click is logged in seconds; a conversion may land hours or days later. Train on a fixed window and you mislabel recent converters as negatives; wait for all labels and your model is stale. Real systems use delayed-feedback corrections — model the delay distribution, or re-ingest late conversions and reweight. This is the same tension that makes real-time training hard.
• Serving multiple heads. One forward pass through the shared body, several cheap heads — the serving win of MTL over N separate models, and it matters under the ranking budget from 04 · Ranking models. PLE's private-expert pools cost the most parameters and complicate sharding; MMoE's uniform bank is friendliest. At billion-sample scale, shard embeddings via parameter server and sync the dense expert/tower params via all-reduce; consider 1-bit gradient quantization to cut sync traffic.
• Conditional-compute serving — the expert bank is the latency budget. A mixture-of-experts model's cost is dominated by running every expert on every request, but the gates already tell you which experts a given example ignores. Three knobs exploit that, and they're a strong senior signal:
  • Gate-threshold expert skip. If a gate weight g^k_i(x) < 0.1, that expert contributes almost nothing to task k, so skip its forward pass for this example. An expert needed by no task is dropped entirely. In practice this cuts ~30% of FLOPs at negligible quality loss, because gate distributions are usually peaky — most experts sit near zero for most inputs. The trade-off: batches now have ragged compute, so you pay it back in batching complexity.
  • Mixed precision split. Keep embeddings in FP16 (the table is huge, bandwidth-bound) and run the dense expert/tower MLPs in INT8 (compute-bound, robust to quantization). You shrink memory bandwidth where the model is fat and speed up arithmetic where it's hot.
  • P99 degradation fallback. Define an SLA (e.g. 50ms P99). When live P99 latency crosses ~80ms — a traffic spike, a slow shard — auto-fall back to a distilled shared-bottom model that drops the experts and gates for one cheap forward pass. You serve a slightly worse ranking instead of timing out, and recover when load drops. A Triton concurrent model pool can also pin the highest-priority head (watch-time) to its own GPU stream so it never queues behind the others.
• Adding a task without a full retrain. Introducing a new objective (say share-rate) risks regressing the mature ones, so evaluation must include an order-preservation check: did the existing tasks' offline AUC / NDCG hold while the new one improved? Uncertainty weighting gives the newcomer a sane initial weight; parameter isolation (a private expert/tower) limits its blast radius. And separate correlation from causation — videos that get likes are naturally longer-watched, so a like head can look like it lifts watch-time without causing it. The usual culprit is a shared confounder: a hot author inflates both CTR and watch-time, so the two heads look mutually reinforcing when neither drives the other. To strip it you need a causal estimate, not a correlational one — Double Machine Learning (DML) residualizes out the confounder (regress both the objective and the "treatment" on the hot-author features, correlate the residuals) before you trust that lifting one head will move the business metric. This is the cross-objective version of the debiasing in 07 · Bias; the full machinery lives in 27 · Causal & uplift.
• Train–serve consistency. Normalization statistics and gating behavior must match between offline training and online scoring, or you get a silent train–serve skew bug — the fused score drifts even though every head looks calibrated in isolation.
• How to actually evaluate an MTL model. Per-head AUC is necessary but blind to the interactions that make MTL hard. A senior eval battery adds three layers: (1) task-relationship — the inter-task gradient-cosine matrix (not just one scalar pair; you want the whole NxN, since adding a task can flip a previously-friendly pair negative), and HSIC (Hilbert–Schmidt Independence Criterion) plus probing tasks on the shared bottom to confirm the shared representation actually carries task-relevant signal rather than mush. (2) Order-preservation / ablation — joint-vs-single-task: did the new head improve while every mature head's offline AUC/NDCG held? A regression here is the abort signal. (3) Guardrails & truth — the negative heads (swipe-away / report rate) get hard thresholds that alarm, and the lagging-truth metric is 7-day retention, because every offline number above is a proxy for it. Full eval philosophy in 08 · Evaluation.

Interview prompts you should be ready for