Design a Recommendation System — System Design Interview Walkthrough

"Design a recommendation system" is the most common machine-learning-flavoured system design question, and it is often confused with a feed question. They are different problems. A news feed shows you content from accounts you already follow — the candidate set is given, and the hard parts are fan-out and ranking within a known graph; that is covered in Design a News Feed. A recommender has no such graph to lean on. Its job is to surface items the user has never seen and has no explicit relationship with, chosen from a catalogue of millions. Selecting the candidate set is the problem.

The good news is that the interviewer is not asking you to derive a model. They are asking whether you can build a system around one: what gets computed offline, what gets computed at request time, where the latency goes, and how you know a change helped. Use this alongside the system design interview cheat sheet, and see the machine learning interview guide if the loop leans ML-heavy. The structure below mirrors what works in a live system design interview.

1. Clarify the requirements

Functional requirements

• Given a user and a context (surface, device, time), return a ranked list of items the user is likely to engage with.
• Exclude items already seen, already consumed, or ineligible for that user.
• Incorporate recent in-session behaviour, not just yesterday's batch job.
• Support multiple surfaces (homepage rail, related-item module, notifications) from shared infrastructure.

Non-functional requirements

• Latency: the whole recommendation call typically has tens of milliseconds inside a page's budget.
• Scale: assume millions of items and hundreds of millions of user-item interactions per day.
• Freshness: new items should be recommendable quickly; user interest shifts within a session.
• Diversity and safety: avoid degenerate, repetitive, or policy-violating results.

Pin down the objective early, because everything else follows from it. "Maximise clicks" and "maximise long-term satisfaction" produce very different systems, and naming that tension is a strong senior signal.

2. The classic approaches

Collaborative filtering

Collaborative filtering uses the interaction matrix — who engaged with what — and nothing about the items themselves. Three flavours are worth naming:

• User-user: find users with similar interaction histories, recommend what they engaged with. Conceptually clean, but user vectors shift constantly and the neighbourhood is expensive to maintain.
• Item-item: for each item, precompute the items most often engaged with by the same users. Item similarity is far more stable than user similarity, so it can be computed offline and served as a lookup. This is the workhorse of "customers who liked X also liked Y".
• Matrix factorisation: approximate the sparse interaction matrix as the product of two low-rank matrices, giving every user and every item a dense vector in a shared latent space. Relevance becomes a dot product, which is exactly the property that makes fast retrieval possible.

Collaborative signals are powerful because they capture taste patterns nobody wrote down. Their weakness is sparsity: they need interactions, and they have nothing to say about an item nobody has touched yet.

Content-based filtering

Content-based filtering represents items by their attributes — category, tags, text, audio or visual features — and builds a user profile from the attributes of things they engaged with. It works from the moment an item exists, which is exactly where collaborative filtering fails, but it tends toward sameness: it will happily recommend a fifth item near-identical to the last four.

Hybrid

Real systems are hybrids. The practical pattern is not to blend one score but to run several retrieval sources in parallel — collaborative, content-based, popularity, recent-session — and let a single ranking model arbitrate between their outputs. That composition is the architecture below.

3. The two-stage architecture

You cannot score ten million items with a heavy model in 30 milliseconds. So the system splits into two stages with opposite priorities.

Candidate generation must be fast and forgiving. It typically unions several cheap sources: embedding-based retrieval, item-item lookups from the user's recent history, popularity within the user's segment, and anything editorially pinned. Recall is what matters — an item missed here can never be recommended, no matter how good the ranker is.

Ranking can afford to be expensive because it only sees a few hundred items. It is usually a supervised model predicting engagement probability from hundreds of features spanning the user, the item, the pair, and the context. Gradient-boosted trees and neural rankers are both common; the interview point is the shape, not the choice.

Re-ranking applies what the model cannot express: deduplicate near-identical items, cap how many results come from one creator or category, blend in fresh content, and enforce policy filters. Skipping this stage produces technically-well-ranked lists that feel monotonous.

  Request (user_id, context)
        |
        v
  +-- Candidate generation (~10ms) --------------------+
  |   ANN over item embeddings  ->  ~200              |
  |   item-item from recent history ->  ~100          |
  |   popularity / trending in segment ->  ~50        |
  |   (union, dedupe, filter seen & ineligible)       |
  +---------------------+------------------------------+
                        |  ~300 candidates
                        v
  +-- Ranking model (~20ms) ---------------------------+
  |   features: user + item + pair + context          |
  |   from online feature store (low-latency KV)      |
  +---------------------+------------------------------+
                        |  scored & ordered
                        v
  +-- Re-rank: diversity, freshness, policy, explore --+
                        |
                        v
                  Final list -> client
                        |
                        v
             impression + engagement logs
                        |
        +---------------+----------------+
        v                                v
  Offline training (batch)        Streaming features
  embeddings, item-item tables    session, counters
        |                                |
        +---------> feature store <------+

4. Embeddings and ANN retrieval

Embeddings are how candidate generation gets fast. Users and items are mapped into a shared vector space where closeness means relevance, learned from interaction data (matrix factorisation, or a two-tower neural model with a user tower and an item tower trained so that engaged pairs land near each other). Item embeddings are computed offline and indexed; the user embedding is either looked up or computed at request time from recent behaviour.

Comparing one user vector against every item vector exactly is too slow at catalogue scale, so retrieval uses approximate nearest neighbour search. Graph-based indexes such as HNSW and partitioning or quantisation approaches such as IVF and product quantisation all trade a small amount of recall for a very large latency reduction. State that trade explicitly — "approximate" is a deliberate choice, not a compromise you're apologising for — and mention that the index is rebuilt or incrementally updated on a schedule, which is what bounds how quickly a brand-new item becomes retrievable.

5. Features: offline, online, and the feature store

Features arrive on three different clocks, and conflating them is a classic mistake:

• Batch (offline): long-window aggregates, embeddings, item-item similarity tables. Computed on a schedule over the full log.
• Streaming (near-real-time): counters and trends over minutes — what is spiking right now, how the user has behaved in the last hour.
• Request-time: device, surface, time of day, and the current session's actions.

A feature store exists to keep these consistent. It serves the online path from a low-latency key-value store and the training path from an offline store, using the same definitions. The failure it prevents is training/serving skew: computing a feature one way in the training pipeline and another way in the serving path, which silently degrades a model that looked fine offline. Related to this is leakage — training on a feature value that would not have been available at prediction time — which point-in-time correct joins are designed to avoid.

6. The cold-start problem

Cold start has two halves and they need different answers.

New user. There is no history to embed. Fall back to popularity and trending, conditioned on whatever context exists: locale, device, referral source, or answers to an onboarding prompt. Then adapt fast — a session-based model that consumes the last few interactions can personalise within minutes of the first click, long before any batch job runs.

New item. Collaborative signals do not exist yet, so retrieval must lean on content features and metadata so the item can surface before anyone has engaged with it. Beyond that, the item needs exposure to generate the data that would judge it; without deliberate intervention the system never shows it, so it never gets interactions, so it stays invisible. Give new items a controlled slice of impressions and evaluate them on that evidence.

7. Exploration vs exploitation

A recommender trained only on its own logs learns from a biased sample: it only observes outcomes for items it chose to show. Pure exploitation therefore narrows the catalogue over time and starves anything new. Exploration is the correction — reserving some traffic for items the model is uncertain about, whether through a simple epsilon-greedy slice, bandit approaches that weigh uncertainty, or the new-item exposure budget above. Frame it as a cost you pay for information: a small, bounded amount of short-term engagement traded for a healthier catalogue and a model that keeps learning.

8. Evaluation: offline metrics and online tests

Offline, you replay logged data and compute precision and recall at K, and ranking measures like NDCG or MAP, plus catalogue coverage and diversity. These are cheap and fast, and they are the right filter for deciding which candidate models deserve live traffic.

They are not the verdict. Logged data reflects what the previous system showed, so it cannot tell you how users would respond to something meaningfully different, and it says nothing about long-term effects on retention. The decision is made by online A/B testing: split traffic, run long enough for the metric to stabilise, and watch guardrails (latency, diversity, complaint rates) alongside the headline metric. Be explicit that offline and online results can disagree, and that when they do, online wins.

9. Serving, latency budget, and trade-offs

Talk through where the milliseconds go: feature fetch, ANN retrieval, ranking inference, re-ranking. Practical levers include caching candidate sets per user for a short window, precomputing recommendations offline for low-traffic surfaces, batching inference across candidates, and degrading gracefully — if the ranker times out, serve the retrieval order rather than an empty rail.

• Recall vs latency. More candidates and a bigger ANN search improve quality and cost time. Tune per surface, not globally.
• Freshness vs cost. Rebuilding embeddings and indexes more often means newer items surface sooner and the pipeline costs more.
• Precomputed vs real-time. Precomputing is cheap and stale; real-time is responsive and expensive. Many systems do both, keyed by surface value.
• Engagement vs long-term value. Optimising the easiest-to-measure signal can degrade the experience you actually care about; multi-objective ranking and guardrail metrics are the usual response.

FAQ

What is the difference between collaborative filtering and content-based filtering?

Collaborative filtering uses interaction patterns across users: it recommends items that similar users liked, without needing to understand the items themselves. Content-based filtering uses attributes of the items and a profile of what the user has engaged with, recommending items whose features resemble the user's history. Collaborative filtering usually finds more surprising and useful recommendations at scale but fails when interaction data is sparse, while content-based filtering works from day one for a new item but tends to keep recommending more of the same.

Why do recommendation systems use a two-stage architecture?

Scoring every item in a catalogue of millions with an expensive model for every request is not feasible inside a latency budget of tens of milliseconds. So the system splits the work: a cheap candidate generation stage narrows millions of items down to a few hundred plausible ones, usually with embedding retrieval and simple heuristics, then an expensive ranking model scores only that shortlist using rich user, item, and context features. Cheap and high-recall first, expensive and high-precision second.

What is approximate nearest neighbour search and why is it used?

Candidate generation often represents users and items as vectors in a shared embedding space, where relevance becomes geometric closeness. Comparing a user vector against every item vector exactly is too slow at catalogue scale, so an approximate nearest neighbour index such as an HNSW graph or an inverted-file or product-quantisation structure is used instead. It returns almost all of the true nearest neighbours in a fraction of the time, trading a small amount of recall for a large latency win.

How do you handle the cold-start problem?

Cold start comes in two forms. For a new user with no interaction history, fall back to popularity, trending items, and any context you do have such as locale, device, or signup answers, then switch to personalised retrieval as signals accumulate within the session. For a new item with no interactions, lean on content-based features and metadata so it can be retrieved before anyone has engaged with it, and deliberately give it some exposure so the system can learn whether it performs.

Why are offline metrics not enough to evaluate recommendations?

Offline metrics such as precision, recall, and ranking measures like NDCG are computed on logged historical data, and that data reflects what the previous system chose to show. A model can look better offline while changing behaviour in ways the log cannot reveal, and offline data cannot measure long-term effects on retention or diversity. Online A/B testing on live traffic is the decisive check, with offline metrics used as a fast filter to decide which models are worth testing.