Low-level design · Schedulers and queues

Thread pool with a queue, a thread cap and idle release

Reported once
Software Engineer (L60)
Producer-consumer
Worker pool
State
1 source

One candidate write-up. Treat it as a single data point.

The prompt

Build a thread pool from scratch that runs submitted tasks in parallel, puts extra tasks in a queue without priorities when every thread is busy, never exceeds a maximum number of threads and releases threads that have no work instead of leaving them idle. The report mentions follow-up questions on synchronized and volatile, so this answer also explains what a real threaded version needs.

Restated in our own words, not a quote from Microsoft or any page. The answer below is written for this site.

Try it first.The answer is covered until you pick. Talk it through against a 60 minute clock, or just read it.

Clarify first

Pin the scope down before you name a single class.

Questions to settle first:

Do threads start up front or on demand? Assume on demand, up to the maximum, with no minimum kept alive.
Is the queue bounded? The report does not say. Assume an optional limit, and that a full queue rejects the task.
Do tasks have priorities? The report says no, so the queue is first in, first out.
How long must a thread be idle before it is released? Assume a configurable number of time units.
Do callers need results, and what does shutdown mean? Assume results are optional and shutdown is graceful.

Because real threads are not deterministic, the code models time as explicit steps and the design section describes the locking a real pool needs. Out of scope: task priorities, futures, work stealing, and a minimum core thread count.

Entities and relationships

Who owns what, and who talks to whom.

ThreadPool owns the workers, the queue, the completed list and the shutdown flag. It exposes submit, step, snapshot, shutdown, shutdownNow and isTerminated.
Worker stands for one thread. It is either idle, counting how long it has been idle, or busy with a task and the ticks that remain.
Task has a name, the number of ticks it needs and an optional function whose return value is stored when the task completes.
The task queue is the buffer between submitters (producers) and workers (consumers). Its optional limit is the pool's back-pressure.

The pool has many workers and one queue; a worker holds at most one task at a time; a task is either in the queue or on exactly one worker.

The design

The class map, the main flow and the patterns that earn their place.

Thread pool class mapSubmit prefers an idle worker, then a new worker, then the queue, then a rejection.

A caller submits a task.
If a worker is idle it takes the task; otherwise a new worker starts while below the maximum.
At the maximum, the task waits in the FIFO queue.
On each step an idle worker takes the oldest queued task.
A worker counts the task's ticks down and records the completion.

Open on the whiteboardAll 3 diagrams on one tidy board, with a step-by-step walkthrough.

The shape is producer-consumer: callers produce tasks, workers consume them, and the queue decouples the two. submit makes one of four choices, in this order: hand the task to an idle worker (reuse is cheapest); else start a new worker if fewer than maxThreads exist; else put the task in the queue if it has room; else reject by throwing. This grows to the maximum before queueing. Java's ThreadPoolExecutor with a core size prefers to queue once the core threads exist, so say which policy you chose and why.

Time advances through step, which does three things in a fixed order so runs are repeatable:

Every busy worker burns one tick. A worker that reaches zero completes its task and the name is appended to completed. The optional result is stored, or the error message goes to failed if the task threw, so one bad task never stops the step.
Every idle worker, in creation order, takes the next task from the front of the queue.
Every worker that was already idle when the step began, and is still idle, has its idle counter increased, and any worker whose counter reached the timeout is removed. The step in which a worker finishes does not count, so the timeout measures whole steps without work.

Worker is a small state machine: created busy, busy to idle when the task finishes, idle to busy when work arrives, idle to gone on timeout. A worker that just finished and picked up queued work in the same step never starts its idle clock.

What the real version needs. Each worker is a thread looping on queue.poll(idleTimeout) from a blocking queue, which is a lock plus a not-empty condition. A null result means it idled out, so it exits. The worker count is shared state with a check-then-act race: two submits can both see a count one below the maximum and both spawn. Guard the check and the increment with one lock, or use compare-and-set on an atomic counter. synchronized gives atomicity and visibility together. volatile gives visibility only, which is enough for a shutdown flag that one thread writes and others read, but not for count++. A worker must catch exceptions from a task so one bad task does not kill the thread. A worker that idles out must re-check the queue under the lock so a task submitted at that moment is not stranded.

Trade-off: the stepped model is fully deterministic and easy to assert on, but it hides contention. The array-backed queue here also makes dequeue linear in the worst case; a real queue is a linked list or ring buffer with O(1) enqueue and dequeue.

The code

Runnable JavaScript, with the checks it passes.

microsoft-thread-pool.js

class Task {
  constructor(name, ticks, run) {
    if (!name) throw new Error("a task needs a name");
    if (!Number.isInteger(ticks) || ticks < 1) throw new Error("ticks must be a positive integer");
    this.name = name;
    this.ticks = ticks;
    this.run = run ?? null;
  }
}

class Worker {
  constructor(id) {
    this.id = id;
    this.task = null;
    this.remaining = 0;
    this.idleFor = 0;
  }
  get busy() {
    return this.task !== null;
  }
  assign(task) {
    this.task = task;
    this.remaining = task.ticks;
    this.idleFor = 0;
  }
  advance() {
    this.remaining -= 1;
    if (this.remaining > 0) return null;
    const finished = this.task;
    this.task = null;
    this.idleFor = 0;
    return finished;
  }
}

class ThreadPool {
  constructor({ maxThreads, idleTimeout = Infinity, queueLimit = Infinity }) {
    if (!Number.isInteger(maxThreads) || maxThreads < 1) throw new Error("maxThreads must be a positive integer");
    if (!(idleTimeout >= 1)) throw new Error("idleTimeout must be at least 1");
    if (!(queueLimit >= 0)) throw new Error("queueLimit cannot be negative");
    this.maxThreads = maxThreads;
    this.idleTimeout = idleTimeout;
    this.queueLimit = queueLimit;
    this.workers = [];
    this.queue = [];
    this.completed = [];
    this.results = {};
    this.failed = {};
    this.nextId = 0;
    this.open = true;
  }
  submit(name, ticks, run) {
    if (!this.open) throw new Error("pool is shut down");
    const task = new Task(name, ticks, run);
    const idle = this.workers.find((worker) => !worker.busy);
    if (idle) {
      idle.assign(task);
    } else if (this.workers.length < this.maxThreads) {
      const worker = new Worker(this.nextId++);
      worker.assign(task);
      this.workers.push(worker);
    } else if (this.queue.length < this.queueLimit) {
      this.queue.push(task);
    } else {
      throw new Error("rejected: queue is full");
    }
  }
  finish(task) {
    this.completed.push(task.name);
    if (!task.run) return;
    try {
      this.results[task.name] = task.run();
    } catch (error) {
      this.failed[task.name] = error instanceof Error ? error.message : String(error);
    }
  }
  step() {
    const wasIdle = new Set(this.workers.filter((worker) => !worker.busy));
    for (const worker of this.workers) {
      if (!worker.busy) continue;
      const finished = worker.advance();
      if (finished) this.finish(finished);
    }
    for (const worker of this.workers) {
      if (!worker.busy && this.queue.length > 0) worker.assign(this.queue.shift());
    }
    for (const worker of this.workers) if (!worker.busy && wasIdle.has(worker)) worker.idleFor += 1;
    this.workers = this.workers.filter((worker) => worker.busy || worker.idleFor < this.idleTimeout);
  }
  snapshot() {
    const busy = this.workers.filter((worker) => worker.busy).length;
    return { workers: this.workers.length, busy, idle: this.workers.length - busy, queued: this.queue.length };
  }
  shutdown() {
    this.open = false;
  }
  shutdownNow() {
    this.open = false;
    return this.queue.splice(0).map((task) => task.name);
  }
  isTerminated() {
    return !this.open && this.queue.length === 0 && this.workers.every((worker) => !worker.busy);
  }
}

Runs here in your browser, in a sandboxed worker. Nothing is sent anywhere.

Try your own input

Write JavaScript that calls the code above and prints with console.log. It runs after the solution.

Extend it

The change requests an interviewer throws next.

Add task priorities so urgent tasks overtake queued ones. What replaces the FIFO queue, and what happens to ordering among equal priorities?
Keep a minimum number of core threads alive even when idle. Where does that rule live in the release logic?
Callers want the result of a task. How do you return a future-style handle without blocking the worker?
When the queue is full, what are the alternatives to throwing, such as running the task on the caller or dropping the oldest queued task?
How do you shut the pool down without losing queued tasks and without waiting forever on a stuck task?

Where candidates slip

The mistakes this problem punishes.

Checking the worker count and starting a thread as two unguarded steps, so the pool silently exceeds its maximum under load.
Using volatile for the counter and claiming that makes it safe. It makes the value visible but the read-modify-write is still not atomic.
Letting an exception from one task kill its worker thread, so capacity drops without any signal.
Releasing an idle worker without re-checking the queue, which strands a task submitted at the same moment.
Releasing idle threads down to zero by accident when the requirement says a minimum should stay, or the reverse.
Spinning on an empty queue instead of blocking on a condition, which wastes a core per worker.
Giving no answer for shutdown: new submits must be refused, queued work should drain, and callers need a way to learn the pool has stopped.

Have a go before you read.

Interviews reward the sketch you make cold. Clarify out loud, draw it, then compare with the answer.

Always show the answer straight away

Now make it stick

Sketch the class map on the whiteboard Run a mock interview

Where this was reported

Public pages only. Companies do not confirm these, and they rotate their questions, so treat each link as one person's account or one list's tag.

Candidate write-up · 2025Microsoft Software Engineer (L60) Interview Experience, Dec 2025 (GeeksforGeeks)

Now make it stick

Where this was reported

More on schedulers and queues