Design a URL shortener

Before drawing anything I would ask: how many links are created a day, and what is the read-to-write ratio? Do links expire, and can the owner disable or edit one? Do we need click analytics, and how fresh must the counts be? Can users pick a custom alias? Must keys be unguessable? What is the redirect latency target, and is the audience global? How fast must a reported phishing link stop working?

Assumed scope: create a link (optional alias and expiry), redirect, disable a link, approximate click counts. Out of scope: link previews, user dashboards, billing, and a full abuse-detection pipeline (I would only show the hook where it plugs in).

These are planning figures for the exercise, not facts about any real service.

• Creates: 10,000,000 / 86,400 = about 116 a second; at a 3x peak, about 350 a second.
• Redirects: 100 x 10 million = 1 billion a day. 1,000,000,000 / 86,400 = about 11,600 a second; at 3x, about 35,000 a second.
• Rows: 10 million x 365 x 5 years = 18.25 billion. At about 500 bytes a row including indexes, that is 9.1 TB, or about 27 TB with three replicas. Storage is not the hard part.
• Keyspace: 62^6 is about 56.8 billion, so six base62 characters leave 18.25 / 56.8 = 32% used. Seven characters give 3.52 trillion and 0.52% used.
• Cache: ten days of creates is 100 million entries x 200 bytes = 20 GB.
• Click events: one per redirect, so 11,600 a second on average. That is more than twice the 5,000 committed writes a second one primary handles.

What the numbers force: the 35,000 a second peak is well past the roughly 20,000 reads a second of one primary, so a cache is mandatory. At a 95% hit rate the store sees 35,000 x 0.05 = 1,750 a second at peak. Creates at 350 a second fit on one primary easily, so the mapping table does not need sharding yet. The click events do not fit on it at all, so a click counter must never be a row update.

1. Create service (stateless). Validates the URL (http or https only, length cap, not pointing back at our own domain), takes an Idempotency-Key header so a client retry after a timeout does not mint two keys, calls an abuse-check hook, then asks the key allocator for a key.
2. Key allocation. A tiny counter service hands each create instance a block of 1,000 ids (one atomic UPDATE ... RETURNING on a single row, or a coordination service). At 350 creates a second that is one block grant every three seconds or so. The instance passes each id through a keyed bijection over the 56.8 billion space, then base62-encodes it to six characters. A counter gives zero collisions by construction, and the scramble stops people walking the keyspace. It is obfuscation, not secrecy: a truly private target needs authentication. A crashed instance wastes at most one block, which is nothing against 56.8 billion.
3. Custom aliases. A conditional insert on the primary key. A conflict returns 409 and the caller picks another. Keep alias keys in a different length class so they can never collide with generated ones.
4. Mapping store. One table, links(short_key PK, long_url, owner_id, created_at, expires_at, state). Reads are always by key, so the key is the only index that matters. Start with one primary and two replicas in different AZs. If you outgrow it, partition by hash of the key: every redirect knows its partition from the URL alone. The destination is immutable; only state changes.
5. Read path. Load balancer to redirect service, which checks an in-process cache (1 to 5 second TTL), then a shared cache (TTL with jitter), then the store. Cache unknown keys for 30 to 60 seconds so random-key scans do not reach the database. Answer with a 302 and Cache-Control: no-store by default, because analytics and takedown both need to see every click.
6. Click path. The redirect handler appends an event (key, time, coarse country, referrer class) to an in-process buffer that flushes to a durable log. An aggregator consumes the log and writes per-key, per-minute counts to a counts store. Losing a few seconds of events in a crash is acceptable for a click counter.
7. Takedown. Set state = disabled on the primary, delete the shared-cache entry, and broadcast an invalidation to the in-process caches. The worst case is the in-process TTL, a few seconds. If an edge cache is used, purge by tag as well.
8. Expiry. The redirect path checks expires_at on every read, so a late sweeper can never serve an expired link. The sweeper itself deletes in small batches by index, never one huge delete. Do not recycle expired keys quickly, because old links are still pasted in documents.

Hard problem one: unique, unguessable keys without a coordination call per write. Block allocation moves coordination from every create to every thousandth create. The alternative is seven random characters with a conditional insert and a retry: the chance a fresh random key is already taken reaches 0.52% by year five, so retries are rare but the write path needs a loop.

Hard problem two: read-your-write after create. The creator often opens the link within a second. Populate the shared cache after the commit (write-through), and on a miss that finds nothing on a replica, fall back once to the primary under a rate cap, so replica lag rarely turns into a false 404 and a scan of random keys cannot become a read storm on the primary.

• Counter blocks vs random keys. Counter blocks give zero collisions and a single unconditional insert, at the cost of a small allocator to run. Random seven-character keys need no allocator but add a retry loop. Pick blocks. Flip to random keys if you must create links in several regions with no shared allocator, or give each region a disjoint id range.
• 302 with no-store vs a cacheable 301. The 302 costs one request per click and gives exact analytics and a takedown measured in seconds. A 301 saves load but can sit in browsers for a long time. Pick the 302. Flip to a 301 with a short private max-age only if analytics is not required and traffic cost dominates.
• Relational primary vs partitioned key-value store. At 350 creates a second and 9 TB, one primary with replicas is simpler and has real transactions. Pick it. Flip to a partitioned store with native TTL when storage no longer fits one volume, writes pass about 5,000 a second, or you need active-active writes in several regions.
• Exact vs approximate counts. A log plus aggregator gives counts within seconds and keeps writes off the redirect path. Pick it. Flip to exact, transactional counting only when clicks are billed, and then count in the log with de-duplication by event id.

• Incrementing a clicks column on every redirect. That turns a read-heavy system into one with 11,600 writes a second on a table built for 350.
• Truncating a hash of the URL to seven characters and never handling the collision.
• Using plain sequential ids, so anyone can enumerate every link, and the total count leaks too.
• Choosing a 301 by default without noticing it kills analytics and makes takedown slow.
• Adding a cache with no invalidation story, then being unable to say how a bad link is removed.
• Reaching for a sharded wide-column store and a stream processor in the first five minutes for a mapping that fits one primary.
• Skipping the expiry check on read and trusting the sweeper to be on time.