System Design Building Blocks: Rate Limiter, Cache, Key-Value Store, URL Shortener and More

Nine building-block system design questions, the ones interviewers use to test whether you understand the parts large systems are made of: a rate limiter, a distributed cache, a key-value store, a URL shortener, a web crawler, search autocomplete, a leaderboard, an ad click aggregator and a recommendation system. Each walkthrough covers what to clarify, the core design, the deep dive and the trade-offs, in the order you would present them. For the wider plan, see our system design interview guide.

How to run any of these in 45 minutes

1. Clarify for five minutes. Pin down the two or three features that matter, the scale (daily users, reads versus writes, data size) and the one non-functional requirement that dominates, usually latency, availability or correctness.
2. Estimate in two. Rough numbers decide the design: requests per second at peak, storage per year, bandwidth for media. Round hard and say your assumptions out loud.
3. Sketch the API and the data model. Three to five endpoints and the main tables or keys. This is where you show you know what has to be stored and how it will be read.
4. Draw the high-level design. Clients, load balancer, stateless services, the stores, caches, queues and the CDN. Walk one request end to end.
5. Spend fifteen to twenty minutes on the deep dive. Each walkthrough below names the part interviewers push on. Lead there before you are asked.
6. Close with trade-offs. What breaks first at ten times the load, what you would monitor, and what you chose not to build.

These questions are smaller than a full product design, so interviewers expect more depth on the one hard part. New to the format? Start with what a system design interview is, then practice with a system design mock interview.

At a glance: the hard part of each question

Traffic and data infrastructure

Design a rate limiter

Clarify: where it runs (in an API gateway or inside each service), what it limits by (user, API key or IP address), the rules (for example 100 requests a minute), whether limits are hard or soft, and what the client sees: an HTTP 429 response with a retry-after header.

Algorithms: a token bucket refills at a fixed rate and allows short bursts up to the bucket size; a leaky bucket smooths output to a steady rate; a fixed window counter is simple but lets double the limit through at window edges; a sliding window log is exact but stores every timestamp; a sliding window counter weights the previous window and is the usual balance of accuracy and memory.

Deep dive: doing it across many servers. Keep counters in a shared in-memory store and update them atomically, with an increment and expiry or a small server-side script for the token bucket, so two servers cannot both let the last request through. Discuss the extra round trip and the option of local counters that sync periodically when approximate limits are acceptable.

Trade-offs: accuracy against latency, and the failure mode when the limiter's store is down: failing open keeps the product working, failing closed protects the backend. Multi-region systems usually enforce limits per region.

Design a distributed cache

Clarify: the read-to-write ratio, the latency target (sub-millisecond for in-memory reads), the data size, the eviction policy and how stale data can be.

Core design: a cluster of cache nodes with keys spread by consistent hashing with virtual nodes, so adding or losing a node moves only a small share of keys; a client library or proxy routes requests; replicas provide availability. Explain the access patterns: cache-aside (the application reads the cache, falls back to the database and fills the cache), read-through, write-through and write-behind, plus TTLs and LRU or LFU eviction.

Deep dive: hot keys (replicate them, add a small in-process cache, or split the key), the cache stampede when a popular key expires (coalesce requests so only one goes to the database, or refresh early with a little randomness), and invalidation on writes (delete the key after the database write and watch for the race where a slow reader puts the old value back; version numbers help).

Trade-offs: consistency against latency, memory cost against hit rate, and a cold cache after a failure, which is why warm-up matters.

Design a key-value store

Clarify: get and put on small values, the scale, durability, and the big choice: when the network partitions, do you stay available or stay consistent? Most interviewers expect a Dynamo-style, highly available design and then ask what changes for strong consistency.

Core design: partition keys with consistent hashing; replicate each key to the next N nodes on the ring (often three); use quorums, where a write waits for W replicas and a read asks R, and W plus R greater than N gives read-your-writes behavior. Membership and failure detection spread by gossip; hinted handoff covers a briefly failed node; Merkle trees find and repair divergent replicas in the background.

Deep dive: conflicts and storage. Concurrent writes produce conflicting versions, resolved with vector clocks or last-write-wins depending on what the data can tolerate. Each node stores data in an LSM tree: writes go to a commit log and an in-memory table, which flushes to sorted files that are compacted over time, with Bloom filters to skip files on reads.

Trade-offs: tunable consistency per request, write amplification from compaction against read amplification, and rebalancing cost when nodes join.

Links, crawling and search

Design a URL shortener

Clarify: create a short link, redirect, custom aliases, expiry and click analytics. Reads outnumber writes heavily. Do the capacity math: seven characters of base 62 give about 3.5 trillion codes, which is plenty.

Core design: a write service generates a unique code and stores code to long URL in a key-value store; a redirect service looks the code up, behind a cache that holds the popular links. Two ways to get codes: a counter (ranges handed to each server by a coordinator, then base-62 encoded) or a hash of the URL truncated to seven characters with collision checks. The counter is simpler and collision-free; add randomness if sequential codes would let people enumerate links.

Deep dive: 301 or 302. A 301 is permanent, so browsers cache it and later clicks never reach you, which is cheaper but loses analytics; a 302 sends every click through the service. Analytics are logged asynchronously so redirects stay fast. Abuse matters too: screen destination URLs for malware and phishing.

Trade-offs: custom aliases need a uniqueness check, expiry needs a cleanup job, and the cache hit rate decides how small the database tier can be.

Design a web crawler

Clarify: the purpose (a search index), the scale (billions of pages a month), politeness, how fresh pages must be, content types and duplicate handling.

Core design: seed URLs feed a URL frontier with priority queues and per-host queues; fetchers pull from it using a DNS cache and a robots.txt cache; parsers extract links and content; a content fingerprint catches duplicate pages; a seen-URL check, often a Bloom filter, drops links already crawled; new links go back to the frontier and content goes to storage for indexing.

Deep dive: politeness (one connection per host at a time, a delay between requests, obey robots.txt), prioritization (important and frequently changing pages first), and traps (infinite calendars and session IDs in URLs, handled with URL normalization and depth limits). Distribute work by hashing the host name so one host's queue lives on one machine.

Trade-offs: freshness against coverage, and Bloom filter false positives, which skip a few new URLs in exchange for a lot of memory saved.

Design search autocomplete

Clarify: how many suggestions (five to ten), ranked by what (popularity, sometimes personalized), how fast (under about 100 milliseconds as the user types), and how fresh (daily is usually fine, with trending as a stretch goal).

Core design: an offline job aggregates query logs into counts; a builder turns them into a trie and stores the top suggestions at every node, so a lookup is one walk down the prefix; serving nodes hold the trie in memory, sharded by prefix; browsers and the CDN cache responses for common prefixes.

Deep dive: caching the top suggestions at each node trades memory for speed, which is the point of the design; updates are built offline and swapped in, with a small streaming layer for trending queries merged at read time; shards are split by measured load rather than by letter, because some prefixes are far busier; and offensive or unsafe suggestions are filtered before serving.

Trade-offs: freshness against rebuild cost, and a custom trie against prefix queries on an existing search engine.

Counting, ranking and recommending

Design a leaderboard

Clarify: score updates, the top N, a player's own rank, the players just above and below, real-time updates, millions of players, and periods such as daily, weekly and all-time.

Core design: a sorted set in an in-memory store does most of the work: add or increment a score, read the top range, and ask for a player's rank, each in logarithmic time. The database keeps the durable scores and game history, and the sorted set is the ranking index, rebuilt from the database if lost. Each period gets its own key.

Deep dive: scale beyond one node. Exact global ranks across shards are hard, so one answer is to shard by score range and add up the counts above a player's shard; another is approximate ranks from a score histogram for very large games, with exact ranks for the top of the board. Ties need a rule, such as who reached the score first.

Trade-offs: exact against approximate ranks, and update frequency against cost. Fraud and abuse checks on submitted scores are a common follow-up.

Design an ad click aggregator

Clarify: count clicks per ad per minute, serve queries on the aggregates, keep counts accurate enough to bill advertisers, tolerate a few minutes of delay, and filter duplicate and invalid clicks. Volumes run to billions of clicks a day.

Core design: click events go to a durable log; a stream processor counts them in one-minute windows keyed by ad; results land in an analytics store built for time-series queries; raw events are kept in cheaper storage so totals can be recomputed and reconciled in batch.

Deep dive: correctness. Use checkpointing and idempotent or transactional writes so a restart does not double-count; handle late events with watermarks and a lateness allowance; spread hot ads by salting the key and combining partial counts; deduplicate by click ID; and run a daily batch recount to reconcile with the stream.

Trade-offs: latency against accuracy, and one streaming path against a stream plus batch design, which costs more but catches errors.

Design a recommendation system

Clarify: what is recommended, on which surface (a home feed or related items), the objective (clicks, watch time, purchases), the latency budget (often under 200 milliseconds) and how to handle new users and new items.

Core design: a multi-stage pipeline. Candidate generation narrows millions of items to hundreds using embeddings with approximate nearest-neighbor search, co-occurrence and popularity; ranking scores those candidates with a model using user, item and context features; re-ranking adds diversity, freshness and business rules. Offline pipelines train the models; a feature store keeps training and serving features consistent; every impression and click is logged for the next round of training.

Deep dive: embeddings and nearest-neighbor indexes, cold start (content features, popularity and onboarding choices), evaluation (recall and ranking metrics offline, then A/B tests online), and feedback loops that keep recommending what was already shown.

Trade-offs: precomputed recommendations are cheap but stale; real-time ranking is fresh but costs latency; bigger models must fit the budget.

To see these blocks inside full products, read the feed, messaging and video walkthroughs and the marketplace, maps and payments walkthroughs; for the full prep plan, see our system design interview guide.

Where CoPilot Interview fits

System design is a conversation, so the fastest way to improve is to say a design out loud and hear where it wobbles. CoPilot Interview is an AI interview assistant whose free AI mock interview can run a rate limiter, cache or key-value question against your résumé and the job description, read it aloud, score your spoken answer out of ten and show a model answer. The free plan gives 15 minutes a day of live help in sessions of up to 15 minutes, with AI mock interviews included. For the live round itself, follow your employer's rules: many companies now state their policy on AI tools in the invitation, and design rounds are where interviewers most want to hear your own reasoning.

Disclosure: CoPilot Interview is our product, so weigh what we say about it accordingly.

Frequently asked questions

Why do interviewers ask building-block questions instead of full products?

They are small enough to go deep in 45 minutes, and they show whether you understand the parts every large system is made of. A candidate who can explain consistent hashing, quorums and cache invalidation can usually assemble a full product design too.

Which algorithm should I pick for a rate limiter?

A token bucket is the common default because it allows short bursts while holding the average rate, and a sliding window counter is a good choice when you want smoother limits with little memory. Say why, and say what happens when the shared counter store is unavailable.

How deep should I go on consistency models?

Deep enough to explain quorums, what eventual consistency means for the user, and one way to resolve conflicting writes. You do not need to cite papers, but you should be able to say what your design does during a network partition.

Is the URL shortener question too easy for senior roles?

Interviewers use it at every level and raise the bar with follow-ups: multi-region writes, analytics at scale, abuse prevention and expiry. Treat the basic design as the first ten minutes, not the whole answer.

Where are the old single-question pages?

Each of the nine walkthroughs that used to be a separate page now lives in a section of this page, and the old addresses redirect to the matching section.