Real-world case studies

How to read these — and how not to use them

Everything below is drawn from publicly discussed architecture: conference talks, engineering blog posts, papers. Each is a snapshot of a system at a point in time, usually years ago, and in several cases the company has since rebuilt the thing described. None of it is current internal truth, and you should say so if you cite it. Each section below names its source year and carries a "what has changed since" line, so you can see how stale the snapshot is. Where a claim could not be traced to a dated primary source, the text says "reported" or "secondary write-up" instead of stating it as fact. The value is not the architecture; it is the reasoning, and specifically the part that most retellings skip: what the clever decision cost.

Used badly, case studies are name-dropping — "I'd do it like Twitter" signals that you have read a blog post, not that you can design. Used well, they are a compressed argument: "there is a known pattern for the celebrity problem; it is a hybrid, and the threshold is a tunable I'd choose from the follower histogram and a cost model." Each section below ends with the one transferable move.

Twitter timelines — the hybrid fan-out

The problem, as Twitter engineers described it in QCon talks in 2012 and 2013 (the numbers here come from a 2013 summary of the "Timelines at Scale" talk): home timeline reads ran at about 300,000 requests per second against tweet writes of roughly 6,000 per second. A read-time query — "fetch every tweet from everyone this user follows, merge, sort" — is a scatter across thousands of partitions on the hottest path in the product. The fix was to invert it: on write, push the tweet id into a precomputed per-follower timeline list held in memory (Redis), capped at 800 entries. The read then becomes a single list fetch. Reads went from a distributed join to an O(1) lookup.

The cost is write amplification, and it is brutal at the tail of the follower distribution. The same summary cites an account with about 31 million followers, where one tweet means about 31 million list insertions and, in the worst case, up to five minutes to reach everyone. Across the network, total fan-out ran to about 30 billion timeline deliveries a day. The summary also reports where Twitter was heading: exploring a hybrid, fanning out on write for ordinary accounts and, for the small set of very-high-follower accounts, skipping fan-out and merging their recent tweets in at read time. Note the tense: in 2013 that hybrid was being explored, not finished, so "Twitter uses a hybrid" is an inference from a snapshot, not a settled fact.

What to steal in an interview: whenever a workload has a power-law distribution, propose treating the head and the tail differently, and name the threshold as a tunable. "Fan-out on write below some follower count, fan-out on read above it, and I'd make that a config value because the right number depends on the follower histogram and a cost model" is a complete answer to a whole family of feed, notification and subscription questions. There is no industry-standard number: reasonable thresholds sit anywhere from roughly 100,000 to 1,000,000 followers.

What has changed since (source years 2012-2013): the design above is a 2013 snapshot. X's open-sourced "For You" repository (updated in 2026) describes a different pipeline: in-network candidates come from an in-memory store of recent posts by accounts you follow, out-of-network candidates come from machine-learned retrieval, and a transformer model ranks them. The precomputed Redis list of 800 ids is no longer how that feed is described.

Netflix — precomputed placement, and failure as a habit

Netflix's Open Connect programme, publicly described from around 2012, inverts the usual CDN model. Instead of caching reactively on a miss, Netflix offers physical appliances that sit inside ISP networks, and fills them during off-peak windows with content chosen by per-region popularity (Netflix's own overview says it can predict with high accuracy what members will watch and when). Video is unusually well suited to this: the catalogue is finite, changes slowly, and next-Tuesday's demand is genuinely predictable from this-Tuesday's. So the cache miss — the expensive event that dominates ordinary CDN design — is mostly engineered out of existence rather than optimised.

The cost is that you are now running a hardware logistics operation: manufacturing, shipping (Netflix's current ISP page quotes about a week), ISP agreements, and operating boxes you cannot physically reach. That is a real organisational commitment, and it only pencils out because streaming video is a large enough fraction of internet traffic to justify it. Do not propose it for a service serving 200 KB images.

The second Netflix idea is cultural rather than architectural: Chaos Monkey, introduced in Netflix's 2011 Simian Army write-up, which randomly terminates production instances on purpose. The reasoning is that redundancy you have never exercised is a claim, not a property — failover paths rot, runbooks go stale, and the first real test of your multi-AZ story should not be at 3am. The cost is that you must build the resilience first; switching on failure injection in a system that isn't ready is just an outage you scheduled.

What has changed since (source years 2011-2012): Netflix's current Open Connect pages still describe appliances embedded in ISP networks with nightly fills, so the idea has held up. The wider Simian Army project, by contrast, is retired and unmaintained; Chaos Monkey itself lives on as a standalone service that is configured through Spinnaker.

What to steal: two sentences. "If demand is predictable, precompute placement instead of caching reactively." And, at the end of any availability discussion, "I'd verify the failover path with scheduled failure injection, because an untested failover is a broken failover." The second one costs you fifteen seconds and reliably lands.

Uber — a spatial index is a sharding decision

Matching riders to drivers is a continuous geospatial query: given a point, find nearby available supply, fast, while every driver's position updates every few seconds. Uber's publicly described answer (the H3 library, announced on Uber's engineering blog in June 2018) indexes the world as a hierarchy of hexagonal cells, each addressed by a 64-bit id, at sixteen resolutions from continent-sized down to cells of under a square metre. A proximity query becomes "look up this cell and its ring of neighbours" — a key lookup, not a geometric search.

Hexagons rather than squares for a specific reason: a hexagon has six neighbours and all six centroids are equidistant. A square grid has four edge-neighbours at distance d and four corner-neighbours at d√2, so "expand the search by one ring" means different things in different directions, and any smoothing or gradient over the grid is distorted. The cost is that hexagons do not tile hierarchically: you cannot subdivide a hexagon into smaller hexagons exactly, so containment across resolutions is approximate (H3's documentation says geographic containment is approximate while the logical parent-child relation in the index is exact), and wrapping a sphere in hexagons mathematically requires exactly twelve pentagons (H3 orients its icosahedron so all twelve fall in the ocean). That is a genuine correctness footnote you inherit.

The deeper point is how a spatial index relates to sharding. Uber's H3 post describes using the grid for pricing, dispatch and analysis; it does not say H3 cells are database shard keys. The sharding claim comes from a separate, older source: secondary summaries of Uber's 2015 QCon talk on its dispatch platform report that it used S2 cell ids (a different spatial index) as the sharding key. Treat that as a 2015 secondary-source description of a system that has since been replaced. Sharding by geography is the natural choice, and it imports the natural failure: geography is not a uniform load distribution. A stadium at kickoff is a hot cell, and no amount of consistent hashing helps because the load genuinely is in one place. The mitigations are the usual ones from the sharding chapter — split hot cells to a finer resolution dynamically, and keep the dispatch tier able to rebalance ownership between nodes.

What to steal: for any "find things near me" prompt, name a concrete index (H3, S2, geohash, or a PostGIS R-tree) and immediately name the resolution tradeoff — coarse cells mean fewer lookups but more candidates to filter; fine cells mean more lookups but tighter results. Then volunteer the hot-cell problem before the interviewer does.

What has changed since (source years 2015 and 2018): H3 is still maintained as open source with public documentation. The dispatch architecture from the 2015 talk is more than a decade old; use it as a pattern for geographic sharding, not as a description of Uber today.

WhatsApp — constraining the product is a scaling lever

The figures with a traceable source: in February 2014, when Facebook bought it, Sequoia's note counted more than 450 million active users and 32 engineers (a ratio of about 14 million users per engineer). WhatsApp's own 2012 blog post described holding more than two million concurrent TCP connections on a single machine (2.28 million sockets on a 24-logical-CPU server with about 96 GB of RAM), running Erlang on FreeBSD. Erlang's runtime is the reason it is even plausible — millions of cheap preemptively-scheduled processes, per-process heaps so garbage collection is per-connection and measured in microseconds rather than a stop-the-world pause, and supervision trees that restart a crashed connection process without touching its neighbours. Add heavy kernel tuning of socket buffers and file descriptor limits. The post does not state a per-connection memory figure; dividing the 35 GB of active memory in its output by 2.28 million sockets gives roughly 15 KB each, which is our rough estimate from active memory alone, not a figure from the post. (The Erlang runtime properties above are general features of that platform, not claims from the post.)

But the runtime is only half of it. The other half is that WhatsApp refused features. The server did not keep message history: WhatsApp's privacy policy still says messages are stored on your device and deleted from its servers once delivered, and that an undelivered message is held in encrypted form for up to 30 days. That single product decision is worth revisiting with the arithmetic from the capacity chapter: retaining history costs petabytes a year; retaining only undelivered messages costs tens of gigabytes. Minimal server-side state and a narrow feature surface are a plausible reason the team could stay small, though the sources listed at the end of this chapter give the headcount, not the cause.

The cost is severe specialisation. Hiring Erlang engineers is hard, the ecosystem is small, and the architecture is unusually inflexible in the direction of adding features — which is precisely what a company adds after an acquisition.

What has changed since (source years 2012 and 2014): the product surface has grown. In June 2025 Meta announced channel subscriptions, promoted channels and ads in the Updates tab (which hosts Status and Channels), while saying personal chats stay end-to-end encrypted and are not used for ads. A WhatsApp with no advertising surface and no feed-like tab is therefore a pre-2025 picture, and the 2012-2014 numbers describe a much smaller product than Meta runs now.

What to steal: when the requirements phase is happening, ask whether a feature is actually required, and put a number on what it costs if it is. "Do we need server-side history? If yes I need a petabyte-scale message store; if no, a 40 GB queue covers it" is the highest-leverage question in the whole interview, and almost nobody asks it. Also: name your per-connection memory budget on any real-time design.

Instagram — boring database, aggressively applied

Instagram is the standing counterexample to "web scale needs a new database." Through its hypergrowth years its core data lived in PostgreSQL, sharded by user id, with two decisions that made it work. (It was not Postgres alone: see the "what has changed since" note below.)

First, blobs left the database immediately — photos went to object storage behind a CDN, and Postgres held only metadata rows. Re-run the capacity arithmetic and it is obvious why: the blob path is measured in petabytes and the metadata path in terabytes. Keeping them in the same system means sizing your relational store for the wrong workload.

Second, logical shards. Rather than mapping users onto physical machines, they mapped users onto a few thousand logical shards (Postgres schemas), then mapped many logical shards onto each physical machine. Growing the cluster means moving logical shards between machines — no re-hashing, no application change. Their id generation scheme, described in a 2012 post, packs the shard into the primary key itself (the post's own example uses 2,000 logical shards and a shard computed as user id modulo the shard count; 13 bits allows up to 8,192):

64-bit id =  41 bits  millisecond timestamp   // 2^41 ms is ~69.7 years from a custom epoch (the 2012 post says "41 years")
           + 13 bits  logical shard id        // 8,192 logical shards
           + 10 bits  per-shard sequence      // per-table auto-increment mod 1,024: unique only while a shard mints at most 1,024 ids per ms

  ids are roughly time-sortable  -> good index locality on an append-heavy table
  the shard is IN the id         -> route a request without a lookup table
  generated in the database      -> no separate id service to keep available

The cost is everything you give up by sharding a relational database: no cross-shard joins, no cross-shard transactions, no global secondary indexes, and application code that must always know the shard key. Connection management becomes its own project — thousands of app processes against a Postgres backend that forks a process per connection means a connection pooler such as PgBouncer is usually needed.

What has changed since (source year 2012): Instagram's 2018 engineering post says it began using Cassandra in 2012 to replace Redis for fraud detection, Feed and the Direct inbox, first on AWS and later on Facebook's infrastructure when the rest of Instagram moved. So the full picture is sharded Postgres for core relational data plus Cassandra for append-heavy, large-scale feed-style data, and the infrastructure it runs on is no longer the one the 2012 post describes.

What to steal: "I'd start with Postgres" is a legitimate and often correct answer, and the way to make it credible is to pair it with the scaling path — blobs out to object storage, logical shards from day one so rebalancing is a data move rather than a rewrite, and a shard-encoded id. Say the sentence "sharding costs me joins and cross-shard transactions" before the interviewer asks.

Discord — coalescing in front of a hot partition

Discord's two posts on message storage — billions of messages on a 12-node Cassandra cluster (January 2017), and trillions of messages after a move from 177 Cassandra nodes to 72 ScyllaDB nodes (March 2023, describing a cutover completed in May 2022) — are unusually honest about what went wrong in between, which makes them the most instructive pair on this list.

The original data model is worth memorising as a template: partition key of (channel_id, bucket) where the bucket is a fixed time window, clustered by a snowflake message id in descending order. The bucket exists because a partition keyed on channel alone grows without bound; in the 2017 post a bucket covers about ten days and keeps a partition under roughly 100 MB, which makes "load the most recent messages" a single partition read with no sorting.

What broke was not throughput but tail latency, and the causes are a tour of distributed-storage failure modes. Hot partitions: a handful of enormous channels received a wildly disproportionate share of reads. Tombstones: deletes in an LSM store are writes, and a range scan must read past every tombstone in the range, so a heavily-moderated channel became slow to read. And JVM garbage collection: stop-the-world pauses on Cassandra nodes produced multi-second stalls that showed up purely as p99 spikes — invisible in an average, invisible in the error rate, exactly the pattern the observability chapter warns about.

The 2022 migration (written up in 2023) had two parts. ScyllaDB — a C++ reimplementation of Cassandra with a shard-per-core architecture and no garbage collector — removed the GC pauses. More interesting is what they put in front: intermediate "data services" written in Rust that coalesce concurrent identical requests. When ten thousand clients ask for the same channel page in the same instant, the first request goes to the database and the rest wait on its result; consistent-hash routing sends each channel to the same service instance so the crowd actually meets in one place.

What to steal: request coalescing (also called single-flight or request collapsing) is a genuinely reusable move, and mentioning it marks you as someone who has fought a thundering herd. It is also the correct answer to cache-stampede questions from the caching chapter: on a miss, one request refills while the others wait, instead of ten thousand simultaneous refills.

What has changed since (source years 2017 and 2023): the 2017 Cassandra design was replaced in 2022 (177 Cassandra nodes became 72 ScyllaDB nodes, each with about 9 TB of disk). I found no later Discord write-up that supersedes the March 2023 post, so treat it as the latest public snapshot rather than a statement about today.

The six systems, seven decisions, on one line each

Using a case study without sounding like you read a blog post

• Lead with the problem, not the company. "This has the celebrity fan-out shape" is analysis; "Twitter uses Redis" is trivia. Name the company second, as a citation.
• Date it and hedge it. "As of their 2017 write-up" costs you four words and protects you completely — the interviewer may work there and know it changed.
• Always state the cost. A candidate who says "and that cost them cross-shard joins" has understood the decision. A candidate who lists only the benefit has memorised it.
• Never argue from authority. "Netflix does it" is not a reason. "Our demand is predictable in the same way theirs is, which is what makes precomputed placement work here" is a reason.
• Scale-check the analogy out loud. Most of these solve problems that appear at a much larger scale than a small product has. If the prompt is a 100k-user product, the correct use of the case study is to explain why you are not doing it yet — which sets up the tradeoff-thinking chapter's central point.
• The pitfall: importing a whole architecture instead of one idea. These companies each had a specific constraint that justified a specific cost. Steal the reasoning; leave the org chart.

Dry run: using the Twitter case study inside a live interview answer

Prompt: "Design a social feed where a handful of accounts have 10M+ followers." Applying the six moves above to that one case study, in order.

Every row is doing the same test: does this sentence teach the interviewer something about this design, or does it just prove you read a blog post? A case study only earns its place in the answer when the answer would be weaker without it.