Design a subscription billing platform on top of a payments system

I would ask: is billing in advance (pay for the coming period) or in arrears (pay for usage)? Are prices per seat or flat? Do all customers bill on the same calendar day, or on the anniversary of signup? Which currencies and which taxes? Must access stop the moment a payment fails, or after a grace period? Is the payments layer an existing internal service with idempotency keys and webhooks?

I would assume advance billing at flat prices, anniversary billing, an existing payments service I can call with idempotency keys, a grace period on failure, and 5 million subscriptions (about 80% monthly, 20% yearly). In scope: plans, subscription state, scheduler, invoices, proration, retries (dunning), entitlement events. Out of scope: the payments layer, tax rules and revenue accounting beyond posting entries.

Planning figures for the exercise, not facts about any company.

• Charges: 4,000,000 monthly subs ÷ 30 = about 133,000 a day; 1,000,000 yearly ÷ 365 = about 2,700 a day. Total about 136,000 a day, which is 136,000 ÷ 86,400 = 1.6 a second on average.
• The calendar trap: if every monthly customer bills on the 1st and the job runs over 4 hours, 4,000,000 ÷ 14,400 = about 278 charges a second. At about 2 seconds per provider call, that is roughly 556 calls in flight, and it lands on the provider's rate limits, not on your database. Anniversary billing flattens it by a factor of about 30.
• Failures: at an assumed 5% first-attempt failure, 133,333 × 0.05 = about 6,700 a day. Four retries bound the extra attempts at roughly 27,000 a day, tiny.
• Invoices: about 4.1 million a month at 1 KB of data each is about 4 GB a month, roughly 50 GB a year. Rendered PDFs at 50 KB are about 200 GB a month, which belongs in object storage, produced asynchronously.

The numbers say one relational primary is plenty. The real limits are the payments provider, correctness, and the day everyone bills at once.

Components. A plan catalogue (versioned prices), a subscription service (the state machine), a billing scheduler, an invoice service, the payments service (existing), a dunning service, an event stream for entitlements, and a ledger hook.

Data model.

• subscriptions(sub_id, customer_id, plan_id, price_version, status, current_period_start, current_period_end, next_billing_at, anchor_day, trial_end, cancel_at_period_end, version). Index on (status, next_billing_at).
• invoices(invoice_id, sub_id, period_start, period_end, status, total, currency) with a unique index on (sub_id, period_start).
• invoice_lines(invoice_id, kind, amount) where kind is plan, proration credit, proration charge, tax or discount.
• payment_attempts(invoice_id, attempt_no, idempotency_key, psp_ref, outcome).

Subscription status runs trialing, active, past_due, then canceled or unpaid, with paused as a side state. Transitions are compare-and-set on version, with each move logged.

Billing cycle.

1. Workers claim due subscriptions in small batches with SELECT ... FOR UPDATE SKIP LOCKED on next_billing_at, so many workers share the work without colliding.
2. In one transaction, create the invoice (the unique index means a rerun creates nothing), advance the period and next_billing_at, and write an outbox row.
3. A collection worker calls the payments service with key invoice_id:attempt_no. The key stays the same across retries of the same attempt, so a timeout cannot produce two charges. A timeout is unknown, not failed; reconcile before trying anything new.
4. The webhook or response marks the invoice paid, de-duplicated by event id, and emits "subscription renewed". A failure moves the subscription to past_due and starts dunning.

Proration. Prices are copied by version onto the subscription so a catalogue change never silently reprices existing customers. For an upgrade on day 10 of a 30-day cycle, from 10,000 to 20,000 minor units: credit for unused time is 10,000 × 20/30 = 6,667, charge for the new plan is 20,000 × 20/30 = 13,333 (both rounded), net 6,666 due now. Use integer minor units with a fixed rounding rule. Charge upgrades immediately; apply downgrades at the end of the period.

Trials. A trial sets next_billing_at to the trial end. A reminder goes out a few days earlier. Ending the trial is simply the first invoice.

Dunning. Classify declines. A hard decline (invalid or closed card) stops retries and asks for a new payment method; a soft one (insufficient funds, issuer unavailable) retries on a schedule such as days 1, 3, 5 and 7. Respect card network and provider guidance on retry limits (check the current rules). A new payment method triggers an immediate retry. After a grace period, entitlement ends and the subscription becomes unpaid or canceled.

Entitlements. Product services consume subscription events and cache the answer; they never call billing on a user request.

Dates. Store periods as UTC instants and keep the anchor day separately, so a subscription anchored on the 31st bills on the last day of short months and returns to the 31st afterwards.

Immutability. A finalised invoice is never edited. Corrections are credit notes, and the ledger gets entries for receivable and cash.

• Anniversary vs calendar billing. Anniversary spreads load evenly and avoids a thundering herd on the provider. Calendar billing makes finance reports and business invoices simpler. Pick anniversary for self-serve; flip to calendar for business customers who want one invoice a month, and then stagger by customer group.
• Polling the database vs a delay queue. A polling scheduler on an indexed column is simple, restartable and visible. A delay queue is a second store that can drift from the subscription table. Pick polling; flip to partitioned schedulers when one table scan no longer fits comfortably in the window.
• Immediate vs deferred proration. Immediate charges surprise customers but keep revenue accurate. Deferring to the next invoice is gentle but hides cost. Pick immediate for upgrades and deferred credits for downgrades; flip when customer support sees too many disputes.
• Cut access at once vs a grace period. Cutting off is strict, but many failures are temporary. Pick a grace period; flip to a short one for high-cost services where unpaid use is expensive.

• Retrying a timed-out charge with a new idempotency key, which can create a second charge. Only a definite decline earns a new attempt number.
• Billing everyone at the same time on the 1st with no jitter and no check of the payment provider's rate limits.
• No unique key on (subscription, period), so a restarted scheduler double-invoices.
• Editing invoices after finalisation instead of issuing credit notes.
• Using floating-point math for proration, or ignoring the end-of-month case.
• Calling the billing service on every user request to check access.
• Retrying declines forever, which annoys customers and can breach network rules.
• Treating entitlement as a column the app reads at request time, so a billing outage logs every paying customer out.