Design an Instagram-style ranked feed

I would ask four things before drawing anything. Is the home feed only accounts the viewer follows, or does it also mix in recommended posts from accounts they do not follow? How fresh must a new post be: seconds or minutes? What does the follower distribution look like, in particular how big is the largest account? And is the ranking model in scope, or do I treat the ranker as a service that takes candidates and returns scores?

Assumed scope: 500 million daily users, 15 feed opens each, photo and short-video posts, a follow graph, a ranked home feed with cursor paging, like and comment counts, impression logging. Out of scope: the ad auction, model training, search, direct messages, and the transcoding pipeline beyond "store media and serve it through a CDN".

These are planning figures for the exercise, not facts about any company.

• Feed opens: 500 M × 15 ÷ 86,400 s is about 87,000 a second on average, and about 260,000 at a 3x peak.
• Posts: 10% of users post 1.5 times a day, so 75 M a day ÷ 86,400 is about 870 a second. That is a read-to-write ratio near 100:1.
• Fan-out on write at 200 followers each: 870 × 200 is about 174,000 feed appends a second. Far past 5,000 writes a second, so the feed store is partitioned and fed through a queue.
• Ranking: say 500 candidates per request, cut by a cheap score to 100 for the heavy model. At peak that is 260,000 × 100 = 26 M scores a second. If one ranker box does 10,000 scores a second (an assumption), that is about 2,600 boxes. This is the biggest cost line, so it gets a deadline and a fallback.
• Media: 260,000 feeds × 6 images × 150 KB is about 234 GB a second. A CDN carries that.
• Feed lists: 1 B accounts active in 30 days × 500 entries × 40 B = 20 TB. Keeping the top 100 entries in memory is 1 B × 100 × 40 B = 4 TB.

What the numbers force: precompute feeds (reads are over 20,000 a second), keep media off the feed path, partition and queue the writes, and treat the ranker as the scarce resource.

Components and the happy path:

1. Post service. Stores the post (post_id as a time-ordered ID, author_id, media keys, caption, visibility, created_at) partitioned by post_id. Media goes straight to object storage through a presigned URL; the post becomes visible only once renditions exist. After commit it publishes a post-created event to a durable log using an outbox row written in the same transaction, so a crash between commit and publish loses nothing.
2. Graph service. Follows are stored twice, as (follower_id, followee_id) and (followee_id, follower_id), because fan-out needs "who follows X" and the read path needs "whom does X follow".
3. Fan-out workers. Consume the log partitioned by author_id. For an author under the follower threshold (assume 100,000, a tunable between roughly 100k and 1M), append the post_id to the feed list of each follower active in the last 30 days. The append is a set-add keyed by post_id, so replays are harmless. Authors above the threshold are not fanned out at all.
4. Feed store. Key is user_id, value is a capped list of post IDs (about 500) in an in-memory sorted structure with a disk-backed tail. IDs only, never bodies, so an edit or delete touches one place.

Read path, with a deadline on every step:

1. Read the stored list for the viewer, and fetch the cached recent-posts list for each large account they follow (a handful; cap it at the top 20 by affinity).
2. Merge, then filter: posts the viewer has already seen (a per-user recent-impression set or Bloom filter), blocked or deleted authors, and visibility checks done at read time so privacy changes apply instantly.
3. A cheap scorer (recency, affinity) trims about 500 candidates to about 100. The heavy ranker scores those using features from a feature store: viewer-author affinity, post age, early engagement. Then apply diversity rules, for example no more than two in a row from one author.
4. Hydrate the top 20 through a post cache multi-get and a separate counter cache, return media as CDN URLs plus an opaque cursor.

Hard problem one is the very large account: pushing one post to 50 million lists floods the queue and delays everyone behind it, so those authors are pulled at read time and merged. This is the hybrid from the feed walkthrough.

Hard problem two is stable paging under ranking. If page two is re-ranked from scratch, items move and the user sees repeats or gaps. The first request freezes a ranked candidate list for the session (cached a few minutes) and the cursor carries the snapshot time, the model version and the last position. New posts appear at the top on pull-to-refresh, not by reshuffling mid-scroll.

Impressions and actions are logged asynchronously to the same log, feeding the seen filter and training, never written synchronously on the read path. If the ranker misses its deadline (say 80 ms) the service returns the stored order, which is newest-first. New users with no follows and inactive users with no stored list get a feed built on demand from popular and recommended candidates.

• Fan-out on write vs fan-out on read. Write-time push makes reads one list lookup but costs 174,000 appends a second and breaks on huge accounts. Read-time pull is cheap to write and expensive to read. Pick the hybrid because the follower distribution is skewed. Flip the threshold lower when queue lag shows up, higher when read-time merges get slow.
• Heavy ranking per request vs a cached ranked page. Fresh ranking is more relevant but the scoring bill is the largest line item. Pick fresh ranking of a short candidate list with a deadline and a newest-first fallback. Flip to reusing a ranked page for a minute or two if the cost or tail latency is unacceptable and users tolerate it.
• Store IDs vs store post copies. Copies make reads one fetch but multiply storage and make edits and deletes a chase. Pick IDs plus a hydration cache. Flip only for immutable content with a tiny fan-out.
• Strict freshness vs availability. A feed that is a few seconds stale is fine, an unavailable one is not. Pick asynchronous fan-out with eventual consistency, and give the author a read-your-writes shortcut by merging their own latest post locally. Flip when a feature (a live event) needs sub-second delivery.

• Describing a CRUD posts table and a join on the follow table at read time, without ever doing the read-rate arithmetic that kills it.
• Choosing pure fan-out on write and never mentioning the account with tens of millions of followers until the interviewer does.
• Copying full post bodies into every follower's list, so edits and deletes become impossible to apply.
• Using OFFSET paging on a list that changes at the head, which produces duplicates and gaps.
• Treating "ranking" as a box labelled ML with no candidate stage, no feature source, no deadline and no fallback.
• Serving media from the API tier instead of object storage and a CDN.
• Fanning out to every follower, including people who have not opened the app in a year.