Industry case studies — reading production recommenders

This capstone is not about memorizing YouTube, Airbnb, Alibaba, or Meta by name. It is about learning how to read any production recommender with the same disciplined chain: product surface → objective → retrieval → ranking → feedback loop → serving system → evaluation → failure modes. Once you can do that, every case study becomes a reusable design pattern instead of a trivia card.

The worksheet — one shape for every case

Before looking at any famous system, write the blank worksheet. This prevents the common interview failure: name-dropping architectures before naming the product constraints. A recommender is a decision system under constraints; the model is only one box inside it.

Case 1 · YouTube-style video recommendation

Surface. The home feed and watch-next surfaces rank videos for a user in a session. The inventory is huge, the user intent changes quickly, and the system is judged less by a single click than by whether the session is satisfying enough for the user to keep watching and return later.

Linear read. Start with objective: raw CTR is a trap because thumbnails can win clicks while producing short, unhappy sessions. That pushes you toward watch-time, completion, explicit feedback, return probability, and safety/quality guardrails. Then read retrieval: no single source covers the catalog, so candidate generation is multi-source — subscriptions, similar-video co-watch, embedding retrieval, fresh/trending content, and exploration buckets. Then read ranking: sequence features matter because the last few watches often dominate the current intent; a user's durable profile and session profile should be fused rather than collapsed.

Interview move. Say the words "home feed is not a one-label classification problem." Then draw retrieve → rank → rerank, name the multi-objective score, and explicitly call out the feedback loop: a model that only exploits past watch-time will narrow the catalog and starve new creators. That single sentence connects multi-task learning, cold start, exploration, and diversity.

Case 2 · Airbnb-style marketplace search and recommendation

Surface. Marketplace ranking is not a pure taste feed. The item must be available, geographically plausible, priced within range, and acceptable to both sides of the market. Search intent is explicit: location, dates, guests, filters. The recommender cannot rank a beautiful but unavailable home.

Linear read. Start with constraints. Availability, location, capacity, policy filters, and price range are hard gates; retrieval happens inside the feasible set. Next read objective: booking is delayed and sparse, so you need proxy labels such as clicks, saves, contact/checkout starts, and calibrated conversion probability. Then read two-sided marketplace effects: the platform is optimizing guest satisfaction and host marketplace health, so ranking by expected booking alone can over-concentrate demand on already-popular listings.

Interview move. Do not lead with "two-tower embeddings." Lead with "this is constrained search plus ranking." Then explain where embeddings help: similar listings, personalized retrieval within a city, cold-start listing representation from text/images/amenities, and candidate expansion when exact filters are too narrow. That is the educational order: constraints first, model second.

Case 3 · Alibaba-style e-commerce ranking

Surface. E-commerce ranking blends recommendation, search, and ads-like conversion economics. The user may browse, search, add to cart, buy later, return, or churn. Labels are hierarchical and delayed: impression → click → detail page → cart → purchase → repeat purchase.

Linear read. Start with label structure. CVR is only observed after click, so training a conversion model on clicked examples introduces sample-selection bias. This is where ESMM-style "entire space" thinking matters: model click and conversion jointly over the impression space rather than pretending clicked traffic is the whole world. Next read behavior sequences: recent category views, search terms, cart events, and target-item attention are strong signals; DIN/DIEN-like target attention is not a decorative architecture choice, it answers "which prior behaviors are relevant to this candidate?" Finally read feature and system scale: huge sparse IDs, price/discount features, seller features, and real-time events demand careful feature engineering and train-serve parity.

Interview move. If asked to design an e-commerce ranker, say "the funnel labels are nested and biased." That phrase earns its keep: it explains why naive CVR training fails, why multi-task learning is natural, why calibration matters, and why offline metrics can lie when the old ranker chose which products got seen.

Case 4 · Generative recommendation and semantic IDs

Surface. The LLM/AIGC era changes two pieces of the stack, not all of it. Models can generate richer content representations, semantic item IDs, explanations, synthetic user sessions, and candidate sets. But the hot path is still bound by latency, cost, freshness, and evaluation. The educational mistake is to say "use an LLM for recommendation" without naming the layer.

Linear read. Place the generative model at a layer, then run the same constraints. If it creates item embeddings, ask how often they refresh. If it proposes candidates, ask how recall is measured. If it reranks, ask how many candidates it can afford. If it judges quality, ask how it is calibrated against humans and online metrics. If it generates content, ask how the platform avoids flooding itself with optimized sameness.

Case 5 · SparrowRecSys-style teaching and prototyping stack

Surface. A small open or teaching recommender is valuable for a different reason: it makes the full loop visible. You can implement recall, ranking, feature generation, model training, and serving without hiding behind a large-company platform. The goal is not to match YouTube scale; it is to make the abstractions concrete.

Linear read. Treat a toy stack as a microscope. Implement matrix factorization or two-tower retrieval, ANN lookup, ranking features, a small MTL head, offline metrics, and a simple serving endpoint. Then deliberately add one production hazard at a time: cold-start items, stale features, biased logs, delayed labels, train-serve skew, bad calibration, and a fallback path. This is the fastest way to convert syllabus knowledge into engineering taste.

Interactive · the conversion-funnel label explorer

Every case in this lesson hinges on one shared fact: the label you actually care about (purchase, booking, retention) sits at the bottom of a funnel, where it is sparse and selection-biased. Drag the per-stage rates and watch how fast the deep label vanishes — and read the verdict on whether a naive deep-funnel model is even trainable, or whether you need entire-space / multi-task modeling. The numbers are illustrative; the collapse is real.

Putting the cases side by side

How to answer a case-study interview

1. Name the surface and the decision. "We rank videos for a home feed" is different from "we rank available homes for a dated search query."
2. Choose the objective and critique it immediately. Say why CTR, watch-time, booking, or revenue is biased, delayed, gameable, or incomplete.
3. Draw the cascade. Retrieval, ranking, reranking, business rules, serving and fallback. Put latency budgets next to each stage.
4. Explain the feature clocks. Static, batch, nearline, online/session features. Name how train-serve skew can enter.
5. Handle cold start and exploration explicitly. New users and new items are not edge cases; they are daily traffic.
6. Close with evaluation and operations. Offline metrics, counterfactual caveats, A/B tests, segment slices, monitoring, rollback, degradation.

Interview prompts you should be ready for