Machine coding · Libraries and infrastructure

Load balancer with round robin

Reported once
Principal Software Engineer
Strategy
Facade
1 source

One candidate write-up. Treat it as a single data point.

The prompt

Build a load balancer that exposes HTTP endpoints, uses them to call an external service, and demonstrates round-robin distribution of the requests.

Restated in our own words, not a quote from Razorpay or any page. The answer below is written for this site.

Try it first.The answer is covered until you pick. Talk it through against a 60 minute clock, or just read it.

Clarify first

Pin the scope down before you name a single class.

Questions to ask: what do the endpoints do, register and remove backends, proxy a request, or both? Are the backends real HTTP services or simulated ones in the demo? Is round robin the only strategy, or must the strategy be replaceable? What counts as an unhealthy backend and how does one come back? Should a failed call be retried on another backend, and how many times? Are timeouts, sticky sessions or weights expected?

Assumed scope: three endpoints, one to add a backend, one to remove it and one to proxy a request. Strategies are round robin and least connections behind one interface, replaceable at runtime. A backend that fails several times in a row leaves the rotation and returns when a health probe passes. A failed call is retried on a different backend up to a limit. The HTTP call itself sits behind an injected function, and the balancing logic is kept apart from the HTTP layer so it can be run and checked without starting a server. The report names the endpoints, an external service and round robin; the strategies, health handling and retries are this answer's additions. Out of scope: weights, sticky sessions, TLS, rate limiting and running several balancer instances.

Entities and relationships

Who owns what, and who talks to whom.

LoadBalancer owns the backend list, the current strategy, the transport function, a failure threshold and an attempt limit. It exposes add, remove, setStrategy, acquire, release, route and checkHealth.
Backend holds an id, an address, its registration sequence number, a count of open requests, a count of consecutive failures and a healthy flag.
Strategy is the interface with one method, pick(candidates). RoundRobin and LeastConnections implement it.
Transport is an injected function that makes the real call and throws on failure. It is the external collaborator, and the only place a real HTTP client would appear.
Endpoints is a thin router from method and path to balancer calls and status codes: 201, 204, 400, 404, 409, 200 and 503.

Relationships: the balancer has many backends, uses one strategy and one transport, and the endpoints class uses the balancer.

The design

The class map, the main flow and the patterns that earn their place.

Class map of the load balancerThe balancer picks through a strategy and calls out through a transport. The endpoints only translate HTTP.

A client sends a request to one of the endpoints.
The endpoints class turns it into a call on the balancer.
The balancer asks the current strategy to pick from the healthy, untried backends.
It forwards the call through the transport and handles a failure by trying another backend.

Open on the whiteboardAll 2 diagrams on one tidy board, with a step-by-step walkthrough.

The balancer depends on two abstractions and owns everything else. Strategy is where the pattern earns its place: round robin and least connections differ only in how they choose from a list of candidates, so the balancer can switch between them at runtime and a third one is a new class. The Transport function is the seam to the outside world. In production it would be an HTTP client call with a timeout. In the demo and in tests it is a fake that fails on demand, which keeps the whole program deterministic and runnable without a server. Endpoints is a facade that translates POST /backends, DELETE /backends/id and POST /proxy into balancer calls and maps problems to status codes: 400 for a missing field, 409 for a duplicate id, 404 for an unknown id and 503 when no backend can serve the request.

RoundRobin remembers the registration sequence number of the last backend it picked, even if that call then failed, and picks the first candidate with a larger number, wrapping to the first one. Remembering a sequence number instead of an index keeps the rotation fair when a backend is removed or marked unhealthy, where a counter taken modulo the list length would skip or repeat backends. LeastConnections picks the candidate with the fewest open requests, and ties go to the earliest registered.

acquire builds the candidate list: backends that are healthy and not yet tried for this request.
The strategy picks one, its open request count goes up and the transport is called.
On success the failure count resets to zero and a lease holding the backend and the response is returned. The open count stays up until release, which models a response that is still streaming. Releasing the same lease twice is ignored, so the count cannot go negative. route is acquire followed by release for the ordinary case.
On failure the open count goes back down and the failure count goes up. At the threshold the backend is marked unhealthy. The loop then retries on a different backend until the attempt limit or until no candidates remain, and finally throws.
checkHealth takes a probe function and returns recovered backends to the rotation. A real system would run it from a timer.

This version is single threaded. A real one needs the round robin cursor and the open counters to be atomic, the backend list to be swapped as an immutable snapshot so a pick never sees a half-edited list, health probing on its own schedule, and retries only for idempotent requests. The trade-off is passive health detection from real traffic plus an explicit probe hook, instead of a full active health checker.

The code

Runnable JavaScript, with the checks it passes.

razorpay-load-balancer.js

class Backend {
  constructor(id, address, seq) {
    this.id = id;
    this.address = address;
    this.seq = seq;
    this.active = 0;
    this.failures = 0;
    this.healthy = true;
  }
}

class RoundRobin {
  constructor() {
    this.last = -1;
  }
  pick(candidates) {
    const next = candidates.find((backend) => backend.seq > this.last) || candidates[0];
    this.last = next.seq;
    return next;
  }
}

class LeastConnections {
  pick(candidates) {
    return candidates.reduce((best, backend) => (backend.active < best.active ? backend : best));
  }
}

class LoadBalancer {
  constructor(strategy, transport, options = {}) {
    this.strategy = strategy;
    this.transport = transport;
    this.maxFailures = options.maxFailures || 2;
    this.maxAttempts = options.maxAttempts || 3;
    this.backends = [];
    this.seq = 0;
  }
  add(id, address) {
    if (this.backends.some((backend) => backend.id === id)) throw new Error("duplicate backend " + id);
    this.backends.push(new Backend(id, address, this.seq++));
  }
  remove(id) {
    const before = this.backends.length;
    this.backends = this.backends.filter((backend) => backend.id !== id);
    return this.backends.length < before;
  }
  setStrategy(strategy) {
    this.strategy = strategy;
  }
  acquire(request) {
    const tried = new Set();
    for (let attempt = 0; attempt < this.maxAttempts; attempt++) {
      const candidates = this.backends.filter((backend) => backend.healthy && !tried.has(backend.id));
      if (candidates.length === 0) break;
      const backend = this.strategy.pick(candidates);
      tried.add(backend.id);
      backend.active++;
      try {
        const response = this.transport(backend, request);
        backend.failures = 0;
        return { backend, response };
      } catch (error) {
        backend.active--;
        backend.failures++;
        if (backend.failures >= this.maxFailures) backend.healthy = false;
      }
    }
    throw new Error("no backend could serve the request");
  }
  release(lease) {
    if (lease.released) return;
    lease.released = true;
    lease.backend.active--;
  }
  route(request) {
    const lease = this.acquire(request);
    this.release(lease);
    return { backend: lease.backend.id, response: lease.response };
  }
  checkHealth(probe) {
    for (const backend of this.backends) {
      if (!backend.healthy && probe(backend)) {
        backend.healthy = true;
        backend.failures = 0;
      }
    }
  }
}

class Endpoints {
  constructor(balancer) {
    this.balancer = balancer;
  }
  handle(method, path, body) {
    if (method === "POST" && path === "/backends") {
      if (!body || !body.id || !body.address) return { status: 400, body: "id and address are required" };
      try {
        this.balancer.add(body.id, body.address);
      } catch (error) {
        return { status: 409, body: error.message };
      }
      return { status: 201 };
    }
    if (method === "DELETE" && path.startsWith("/backends/")) {
      return { status: this.balancer.remove(path.slice("/backends/".length)) ? 204 : 404 };
    }
    if (method === "POST" && path === "/proxy") {
      try {
        return { status: 200, body: this.balancer.route(body) };
      } catch (error) {
        return { status: 503, body: error.message };
      }
    }
    return { status: 404 };
  }
}

Runs here in your browser, in a sandboxed worker. Nothing is sent anywhere.

Try your own input

Write JavaScript that calls the code above and prints with console.log. It runs after the solution.

Extend it

The change requests an interviewer throws next.

How would you add weighted round robin so that a backend with weight 3 receives three requests for each one sent to a weight 1 backend, without sending them in a burst?
How would you add sticky sessions so that requests with the same client id reach the same backend, and what happens to that mapping when the backend dies?
How would you make the health check active, probing each backend on a timer, and how would you stop every probe firing at the same moment?
How would you rate limit each client in front of the balancer without changing any strategy?
What changes if the balancer itself runs as several instances behind one address, given that the round robin cursor and the open counts are now local to each instance?

Where candidates slip

The mistakes this problem punishes.

Using a counter modulo the list length for round robin, which skips or repeats backends as soon as one is removed or marked down.
Building the HTTP server first and running out of time before the balancing logic works. Get the core running from code, then add the endpoints.
Calling an HTTP client directly inside the balancer, which makes it impossible to test failure paths.
Retrying the same failed backend, or retrying without a limit.
Not decrementing the open request count on the failure path, which makes least connections drift.
Letting the strategy pick from all backends instead of only the healthy untried ones, and not handling an empty pool.

Have a go before you read.

Interviews reward the sketch you make cold. Clarify out loud, draw it, then compare with the answer.

Always show the answer straight away

Now make it stick

Sketch the class map on the whiteboard Run a mock interview

Where this was reported

Public pages only. Companies do not confirm these, and they rotate their questions, so treat each link as one person's account or one list's tag.

Candidate write-up · 2025Razorpay Principal Software Engineer, India (Jointaro)

Now make it stick

Where this was reported

Read these chapters first

More on libraries and infrastructure