I10 · Part 2 · Intermediate
Microservices and service discovery
When splitting a system pays for itself, what it costs, and how one service finds another once the addresses keep changing.
07On this page
- A microservice is a team boundary before it is a technical one
- Where to cut: data ownership decides the boundary
- How services find each other: service discovery
- The gateway, the mesh and who does what
- Dry run: five services, one product page
- Migrating without a rewrite: the strangler fig
- Recognizing it in an unseen problem
A microservice is a team boundary before it is a technical one
A microservice is a small service that is deployed on its own, owns its own data and is built around one business capability. That is the shape Martin Fowler and James Lewis described in their 2014 essay, and the reason is organisational: when fifty engineers share one deploy, every release waits for the slowest change, and a bug in the reviews code can take down checkout. Splitting buys independent deploys and independent scaling. It does not buy simplicity. The complexity of the whole system stays; it moves from inside a process, where a function call cannot fail halfway, to the network, where it can.
Fowler's advice is blunt: almost every successful microservice system started as a monolith that grew too big to work in, and almost every system built as microservices from day one ran into trouble. His "microservice premium" makes the same point as a cost curve. Below some level of complexity the extra machinery costs more than it returns, and only above it does the productivity gain win. In an interview, "I would start with a well-structured monolith and split when a team or a scaling reason forces it" is a stronger answer than drawing eleven boxes for a product with ten engineers.
| What you gain | What you pay for it |
|---|---|
| Deploy one service without redeploying the rest | Every call between services can now time out, fail halfway or return a stale answer |
| Scale the hot service alone (search, not billing) | Latency adds up along the call chain, and availability multiplies down it |
| A failure stays inside one service if you build it to | No joins and no transaction across services; consistency becomes your problem (sagas, outbox) |
| Each team picks its own stack and release pace | Contracts between services need versioning, and a change to one can break another silently |
| Smaller codebases that a new hire can hold in their head | Debugging needs distributed tracing, and a local test run no longer covers the system |
⚠ The distributed monolith
If two services must be deployed together, share a database table or call each other in a tight loop for
every request, you have a monolith with network calls in the middle. You pay every cost in the right-hand
column and collect none of the gains in the left. It is the most common way a microservice migration goes wrong.
Where to cut: data ownership decides the boundary
The test for a good boundary is not "does this look like a noun" but who owns the data. Each service owns its tables and nobody else reads them directly; other services go through its API or consume events it publishes. This is the database per service pattern, and it is what makes independent deploys real: you can change a table without asking anyone. It is also what takes away joins and cross-service transactions, which is exactly the price.
Signs you cut in the wrong place: two services that always change in the same pull request, a request that needs six calls to assemble one screen, a table that two services both write, or a business operation (place an order) that needs a distributed transaction every time. The usual fix is to merge the services back, or to move the shared concept into one owner and let the others hold a read-only copy fed by events.
Because there is no join, a read that spans services is built one of three ways. API composition: a gateway or the calling service fetches from several and stitches the result, which is simple and costs latency. A local read model: a service keeps its own denormalised copy of the data it needs, updated from events, which is fast and means it can be stale (see the consistency chapter). A saga for writes that must happen across services, with a compensating action for each step that can be undone; the transactions and payments chapters work this through.
How services find each other: service discovery
In a fixed data centre you could write an address in a config file. With autoscaling, deploys and node
failures, instances appear and disappear every few minutes and keep getting new addresses. Service
discovery is the mechanism that maps a stable name such as inventory to the set of
instances that are alive right now. Three patterns cover almost everything.
| Pattern | Who picks the instance | Example | Trade-off |
|---|---|---|---|
| Client-side discovery | The caller asks a registry for the list, then balances across it itself | gRPC with a resolver, Envoy fed by an xDS control plane, Consul, and the older Netflix Eureka | No extra hop and smart balancing, but every client needs the logic and a language-specific library |
| Server-side discovery | The caller sends to a stable virtual address; a proxy or balancer picks | A Kubernetes Service, a cloud load balancer | Simple clients, one more hop, and the balancer is a component you must keep available |
| DNS-based | The caller resolves a name to one or more addresses | Cluster DNS, Consul DNS, a record per service | Works with anything, but answers are cached by clients and resolvers, so changes spread slowly |
Kubernetes is the version most interviews assume. A Service gives a set of pods one stable virtual IP and
a DNS name of the form inventory.shop.svc.cluster.local (service, namespace). The control plane
records the pods behind a Service, and whether each one is ready, in EndpointSlice objects, by default no more than 100 endpoints each, and
kube-proxy on every node turns that into routing rules. Only ready pods receive traffic, so a
failed readiness check removes a pod from rotation without anyone editing a config. A service with 450 pods
therefore needs at least 5 slices (slices are also split by IP family and port set), and splitting a large service into slices is what keeps one pod change from
re-sending a giant object to every node.
Outside Kubernetes the same ideas appear as a registry (Consul, ZooKeeper or etcd). An instance either registers itself and sends a heartbeat before a time-to-live expires, or a platform component registers it and runs health checks. Envoy shows the modern form of the client-side idea: a control plane pushes the endpoint list to each proxy over its discovery API, so the data path never asks a registry per request.
Discovery has to degrade gracefully
- Keep the last known good list. If the registry is unreachable, callers should keep using the addresses they already have, not drop to zero and take the whole system down with the registry.
- Stale entries are the normal failure. A crashed instance keeps getting traffic until its registration expires and every cache between you and the registry refreshes. Pair discovery with timeouts and retries to a different instance (see the fault tolerance chapter).
- Drain before you stop. A deploy should remove an instance from discovery first, wait for in-flight requests, then stop it (see the load balancing chapter on connection draining).
Dry run: how long does a dead instance keep getting traffic?
| Step | What happens | Time |
|---|---|---|
| 1 | An instance crashes. It sent its last heartbeat just before. | t = 0 s |
| 2 | The registry waits out a 30 s time-to-live before it drops the entry | entry gone at t = 30 s |
| 3 | Callers refresh their cached list every 15 s, so the worst case is one full refresh interval after the entry disappears | last caller updated at t = 45 s |
| 4 | Requests sent in that window fail or time out and are retried on another instance | up to 45 s of failed attempts |
| 5 | An active health check from the proxy (two failures, 5 s apart) would have ejected it much sooner | about 10 s |
The numbers are an example configuration, not a standard. The point is the shape: staleness is the sum of every timer on the path, and the cheapest fix is a fast local health check at the caller or proxy rather than a shorter registry time-to-live.
The gateway, the mesh and who does what
Two proxies sit on different paths and are often confused. The API gateway handles north-south traffic, from outside into the system: authentication, TLS, rate limits, routing and sometimes response assembly (the APIs chapter covers what belongs there). A backend for frontend is a gateway tuned to one client type, so the mobile app gets one lean response instead of five calls.
A service mesh handles east-west traffic, between services. Each service gets a proxy next to it (a sidecar, which in Istio is an Envoy proxy) that adds mutual TLS, retries, timeouts, load balancing and metrics without changing application code. Istio also offers an ambient mode that moves the common work into one shared proxy per node, with an optional waypoint proxy, per namespace or per service, only where application-layer features such as retries are needed, to cut the per-pod overhead. A mesh is worth its cost when many services written in different languages need the same security and traffic policy. For five services it is usually a library and a deploy script.
⚠ Retries stack across layers
A mesh that allows up to three attempts in front of an application that also allows up to three turns one failure
into nine attempts, and across three layers into twenty-seven (counting three retries as four attempts, the SRE book's
own example reaches sixty-four). Pick one layer to own retries and put a retry
budget on it (see the fault tolerance chapter).
Dry run: five services, one product page
A product page needs a catalogue record first (it supplies the product id), then pricing, stock, reviews and recommendations. Service times: catalogue 20 ms, pricing 15 ms, stock 25 ms, reviews 40 ms, recommendations 60 ms. Each service is 99.9% available.
| Design | Latency (sum of the critical path) | Availability | Downtime a year (365.25-day year) |
|---|---|---|---|
| All five called in sequence | 20 + 15 + 25 + 40 + 60 = 160 ms | 0.9995 = 99.50% | 43.7 hours |
| Catalogue first, then the other four in parallel | 20 + max(15, 25, 40, 60) = 80 ms | 99.50% (all still required) | 43.7 hours |
| Same, but reviews and recommendations are optional and the page degrades without them | 20 + 25 = 45 ms before the optional parts; 80 ms when they answer | 0.9993 = 99.70% (catalogue, pricing, stock) | 26.3 hours |
Parallelism fixes latency and does nothing for availability; deciding which dependencies are optional fixes availability and costs nothing in latency. That is the whole argument for graceful degradation, and it is also why a design with many hard dependencies in a chain should make an interviewer ask "which of these can fail without the page failing?".
Migrating without a rewrite: the strangler fig
The safe way out of a monolith is not a rewrite. Put a routing layer in front, build one capability as a new service, send a slice of its traffic there, and compare. When it behaves, send more; the old code for that capability is deleted last. Fowler calls this the strangler fig, after the vine that grows around a tree. Every step is reversible by changing the routing, which is the property a big-bang migration lacks. Start with an edge capability that has few inbound dependencies (notifications, search, reporting), not with the one the whole product leans on.
Say it like this → "I'd keep this as a modular monolith for now — one deploy, clear
module boundaries, one database with schemas per module. I'd split out search first because it has a different
scaling profile and an owning team, and I'd do it behind a routing layer so I can send ten percent of traffic
and roll back by changing a config. Each new service owns its data; where I need a view across services I'd
use a read model fed by events rather than a shared table. And I'd decide up front which calls are optional,
because that, not the parallelism, sets the availability of the page."
Recognizing it in an unseen problem
- Signals: "multiple teams", "independent deploys", "different scaling needs", or a prompt that lists many nouns (users, orders, payments, inventory). The question underneath is whether to split, not how many boxes to draw.
- The naive design is a service per entity with one shared database, or a request that walks six services in series. Name the shared-table coupling and the chain length as the problems before the interviewer does.
- Always give the cost with the benefit: network failure, latency along the chain, no cross-service transaction, contract versioning, and the tracing and on-call burden.
- Say how services find each other in one sentence (cluster DNS plus ready-pod routing, or a registry with health checks), and what happens when the registry is down.
- Separate gateway from mesh: gateway for traffic entering the system, mesh for service-to-service policy. Do not propose a mesh for a handful of services.
- The pitfall: starting with microservices because they are fashionable. A monolith with good module boundaries is a defensible, often better, first answer.
Before you move on
- Explain why the shared database makes a set of services a distributed monolith, and what you give up when each service owns its data.
- Compute the latency and the availability of a page that calls five services, once in series and once with two optional dependencies in parallel.
- Compare client-side, server-side and DNS-based discovery, and say what happens in each when an instance crashes.
- Say what a gateway does and what a mesh does, and name one situation where a mesh is not worth it.
- Describe how you would move one capability out of a monolith without a rewrite and how you would roll it back.
Sources and further reading
- Fowler and Lewis, Microservices — the definition, independent deploys, decentralised data, the costs
- Fowler, Monolith First — why most successful microservice systems began as a monolith
- Fowler, Microservice Premium — the cost curve and when the productivity gain pays for it
- Fowler, Strangler Fig Application — migrating by routing a slice at a time
- microservices.io: database per service — the pattern and its consequences for joins and transactions
- Kubernetes: Service — stable virtual IP, cluster DNS names, ready endpoints
- Kubernetes: EndpointSlices — 100 endpoints per slice by default, why slices scale
- Envoy: service discovery — endpoints pushed by a control plane, active and passive health checking
- Istio: ambient mode overview — the shared per-node proxy and the optional per-service proxy
- SRE book: addressing cascading failures — retry amplification across layers
Claims in this chapter were checked against these sources in October 2026. Figures marked "example" are illustrative configurations, not standards.
5 questions drawn from 7, scored out of 5. This is practice: it runs in your browser, saves nothing and does not change what you have marked read.
Finished reading?
0 of 36 chapters read in this series