Your Node.js Server Is Blocked By A Thread Pool Of 4. Nobody Told You.

An 8-Core Box Doing 15 Percent CPU, And Every Request Timing Out

A while back on an internal service we watched something that should not have been possible. An 8-core Node.js box, load balanced with three siblings, and the p99 latency on a fairly ordinary endpoint quietly climbed from 80ms to 4 seconds over the course of a busy day. The requests were not doing anything exotic. Bcrypt a password, sign a JWT, do a DNS lookup, hit Redis, done.

Every graph looked wrong for what was happening. CPU on the box was sitting around 15 to 20 percent. Event loop lag was reported at under 5ms. Memory was flat. GC was fine. From every dashboard the process looked bored. From the client side, half the requests were timing out at 30 seconds.

We spent the better part of a day chasing ghosts. Was it the load balancer? Was DNS slow? Was Redis having a bad afternoon? None of it held up. The service metrics were all fine. It was Node.js itself that was queueing, and nothing in the standard observability stack was showing it.

What we eventually found is the single most consequential Node.js internals fact that most engineers who write Node for a living do not know. The one that makes the "JavaScript is single-threaded, but I/O is non-blocking" story a half-truth. Every book on Node draws the same picture: an event loop, a callback queue, some magic that makes I/O non-blocking. The magic has a name. It is called libuv. And by default it gives you exactly four threads to do that magic on.

The "Single-Threaded Event Loop" Story Skips A Whole Component

The story every engineer learns for Node.js goes something like this: Node has one event loop, running on one thread. JavaScript is single-threaded. When you call something like fs.readFile, Node hands the request off to the operating system, keeps running JavaScript, and calls your callback when the file is ready.

That story is not wrong, but it hides the part that matters. There is no "hands off to the operating system" that is free. Some system calls are genuinely asynchronous at the OS level - epoll on Linux, kqueue on macOS, IOCP on Windows. For those, Node really does hand the request off to the kernel and go do other things. But a lot of the operations you think of as async are not async at the OS level. They are synchronous system calls that Node hides behind an async facade by running them on a background thread.

That background thread is not one thread. It is a pool. And the pool has a specific size.

So the "single-threaded" answer is more accurately: the JavaScript execution is single-threaded, but the async operations Node depends on for its performance story run on a pool of C threads, and that pool is small enough to become a bottleneck on essentially any real production workload.

The C Library You Have Been Depending On Without Knowing It

libuv is a cross-platform asynchronous I/O library, originally built for Node.js and now used by a fair number of other projects. Written in C. It is what sits between V8 and the operating system. When your JavaScript calls fs.readFile, that call goes to Node's C++ bindings, which call into libuv, which decides how to actually perform the read.

libuv has two very different mechanisms for async work:

• Native async where the OS supports it. Network I/O uses epoll on Linux, kqueue on macOS, IOCP on Windows. libuv registers interest with the kernel and returns to the event loop. When the socket is ready, the kernel notifies libuv. No thread pool needed.
• Thread pool for everything else. When the underlying syscall is inherently blocking - filesystem reads on many systems, DNS resolution through the C library, CPU-bound crypto - libuv puts the work on a background thread from its pool, and signals the event loop when the thread finishes.

The second bucket is bigger than most people realise. On Linux specifically, filesystem operations are treated as blocking because the kernel's async filesystem support (io_uring is the modern answer) is still not universally usable, so libuv defaults to running fs work on threads. DNS lookups via the default dns.lookup function go through getaddrinfo, which is a blocking system call, and therefore also runs on a thread. All of Node's built-in crypto primitives that do heavy work (bcrypt, scrypt, pbkdf2, hash streams, zlib gzip/gunzip) are dispatched to the thread pool because they are CPU-bound and would otherwise pin the event loop.

Add all of that together, and a surprising fraction of the "async" surface of Node.js is not natively async at the OS level. It is thread-pool async. And the thread pool defaults to four.

How A Pool Of Four Becomes A Latency Nightmare

Back to the bug. Our endpoint was doing:

• One bcrypt.compare against the incoming password.
• One dns.lookup on a downstream hostname (because we used the http client's default resolver).
• One crypto.sign to mint the JWT.
• One zlib.gzip on the response body before sending.

Every single one of those operations goes to the libuv thread pool. Every single one. Four operations per request, four threads in the pool. One in-flight request is already saturating three quarters of the pool. Two in-flight requests are actively competing.

The math gets worse fast. Under peak load we were seeing 30 to 60 concurrent requests to this endpoint. That means, roughly, 120 to 240 thread-pool operations wanting to run at the same time, and exactly four slots to run them in. Requests were not slow because the work was slow. They were slow because they were sitting in a queue in libuv, invisible to any userland instrumentation, waiting for one of four threads to become free.

This is what makes the bug so hard to spot. The event loop lag metric that most Node teams watch is measured from the JavaScript side. It cannot see queueing inside the thread pool. CPU usage looks low because the JavaScript thread is bored and the four C threads are busy but capped. Memory looks fine because nothing is leaking. Every dashboard confirms the server is healthy. It is only when you correlate concurrent-in-flight-requests with p99 latency that the staircase shape shows up, and even then it looks like a downstream problem, not a runtime problem.

UV_THREADPOOL_SIZE, And Where To Put It

libuv exposes the pool size through a single environment variable. Set it before Node starts, and libuv sizes the pool accordingly.

# in your process manager, docker file, systemd unit, etc
UV_THREADPOOL_SIZE=16 node server.js

There are two critical details worth calling out, because both of them trip people up.

1. The variable must be set before Node starts

libuv creates the thread pool lazily, but it reads UV_THREADPOOL_SIZE exactly once, when the pool is first needed - typically during the first thread-pool operation. Setting the variable from within JavaScript, after Node has already started, does not always work reliably; even when it does, it is a race with the first pool use. The safe path is to set it in the environment before the process launches.

2. There is an upper bound

libuv caps UV_THREADPOOL_SIZE at 1024. Above that, it silently clamps. Below that, whatever you set is honoured. So you cannot ask libuv for 100,000 threads and have it accept the request - which is a good thing, because that would blow through your file descriptor and memory budgets in about a second.

In our bug, going from 4 to 32 collapsed the p99 latency by more than an order of magnitude and got the box back to behaving like a well-tuned server. CPU rose to a healthy 55-70 percent, which is what you want - a server running at 15 percent is a server that is being cheated of the work it could be doing.

The Actual Inventory Of Thread-Pool Work

If you are going to tune this number sensibly, you have to know what is actually using the pool. This trips people up because the docs are scattered and half the community advice is folklore. Here is the sharp version.

Uses the thread pool:

• Filesystem operations. fs.readFile, fs.writeFile, fs.stat, everything in fs/promises. On Linux especially, these run on the pool.
• DNS via dns.lookup. This is what happens when you use the default resolver for any http client - http.get, fetch in older Node versions, axios, most drivers. Under the hood it calls getaddrinfo, which is blocking and therefore pool-bound.
• CPU-bound crypto. bcrypt (via node-bcrypt), crypto.pbkdf2, crypto.scrypt, crypto.randomBytes in async form, and hashing / signing operations that are dispatched to the pool.
• zlib. Async compression and decompression - gzip, gunzip, deflate, brotli - all queue on the pool.
• Certain native addons. Any C++ addon that uses uv_queue_work to run work off the main thread ends up on the same pool. Which means library authors are silently sharing your budget.

Does not use the thread pool:

• Network I/O. TCP and UDP sockets, HTTP client and server sockets, TLS. These all use the OS-native async facilities (epoll / kqueue / IOCP) directly.
• dns.resolve and friends. Unlike dns.lookup, these use direct DNS network calls, which are network-async, not pool-bound. Which is why swapping dns.lookup for dns.resolve4 is often a real win in high-throughput HTTP clients.
• Child processes and worker threads. They are their own thing, not a libuv thread.
• Timers and immediates. Those live on the event loop itself.

The pattern that leaps out once you see this list is that anything CPU-bound is on the pool, anything network-bound is not. If your service is mostly a proxy in front of downstream APIs, the pool matters less. If your service is doing password hashing, compression, or heavy filesystem work, the pool is the single most important knob you have.

Why Raising The Pool Size Above Cores Still Helps

A reasonable objection: "if my box has 8 cores, having more than 8 threads is pointless because they cannot all run at once". This is true for pure CPU-bound work. It is not true for the mix a real Node server does.

Two things save you. First, most thread-pool work is a mix of CPU and waiting - a bcrypt is CPU-bound but a filesystem read on a network filesystem spends most of its time waiting on the disk / network, and a DNS lookup spends most of its time waiting on a UDP round trip. Threads that are waiting are not competing for CPU; they are competing for a slot in the pool. Having more slots means fewer requests sitting in the queue waiting for a slot they could have used to just wait.

Second, the OS scheduler is fine with more threads than cores. That is the OS's job. What it cannot recover from is queueing inside a userland pool that it cannot see. Moving the queueing from libuv (opaque, uncontrolled) to the OS (observable, priority-managed, preemptively scheduled) is a strict improvement for anything that is not compute-bound.

The one caveat is that if your workload is genuinely 100 percent CPU-bound - image resizing, ML inference, heavy encryption - piling up threads beyond the core count starts to hurt because of context switching. In that case, worker threads and a work-stealing queue are the right answer, not raising UV_THREADPOOL_SIZE. But for a typical mixed I/O and CPU workload, more pool threads is almost always a win up to the point where memory becomes the constraint.

The Signals That Tell You Your Pool Is The Problem

If the standard event-loop-lag metric cannot see thread-pool queueing, what can? Three concrete signals I have found useful.

1. Time from schedule to start for a known operation

Wrap a representative thread-pool operation (say, a small crypto.pbkdf2) and measure the wall-clock time from the moment you called it to the moment its callback fires. Then compare against a synchronous benchmark of the same operation. The difference is queue wait plus scheduling. If the queue wait grows with concurrent request count, you are pool-bound.

2. Perf-level thread state on the Node process

On Linux, perf or top -H shows you per-thread activity for the Node process. If you see the four libuv threads pinned at close to 100 percent while the main JavaScript thread is bored, and the box has spare CPU, the pool is the bottleneck. Even simpler: if you see exactly four busy worker threads and eight or more idle cores, the answer is right there.

3. Ratio of concurrent requests to pool size

If your steady-state concurrent-in-flight-request count is meaningfully higher than your UV_THREADPOOL_SIZE, and each request touches even one pool-bound operation, you are guaranteed to be queueing. This is not a subtle correlation. It is arithmetic.

Adding a small custom metric that measures pool wait time is one of the highest-value observability changes you can make to a Node service, and I have almost never seen it in place by default.

Two Common Mistakes When People Try To Fix This

Once teams learn about the pool, the pendulum sometimes swings too far the other way. Two anti-patterns I have seen in practice.

1. Setting UV_THREADPOOL_SIZE to a huge number and walking away

Setting it to 512 or 1024 is not free. Each thread has a stack allocation (default around 2MB on many platforms), plus file descriptors, plus the OS's own bookkeeping. A pool of 1024 threads is not a magical performance boost; it is a real memory and scheduling cost. The right number is "big enough to not be the bottleneck", not "the maximum the library allows".

2. Wrapping everything in a worker thread

Node's worker_threads module is a different animal. It is a proper JavaScript execution isolate for CPU-bound work. Moving all your thread-pool operations into workers does not fix the pool problem, because each worker has its own libuv thread pool of four. Now you have the same bottleneck, distributed across N workers, with cross-thread messaging overhead layered on top. Workers are the right tool for parallelising truly CPU-bound JavaScript. They are not the right tool for making libuv's pool wider.

The right tool for making the pool wider is UV_THREADPOOL_SIZE. It has one job.

Node's Best Kept Secret Is A Config Variable

The strange thing about libuv's thread pool is that it is one of the highest-leverage bits of Node.js knobs, and it is almost never taught. The books skip it. The bootcamps skip it. The docs mention it in a footnote. The default of 4 is a decade-old choice from when Node was primarily running on machines that looked nothing like modern servers, and it has stuck around because the failure mode of the default is not "your server crashes" - it is "your server quietly costs you three times as many boxes as it should".

Once you internalise this, a lot of vaguely-mysterious Node performance stories click into place. "Why does bcrypt make my API slow beyond what bcrypt itself costs?" Pool queueing. "Why does adding gzip compression tank throughput on a load-balanced fleet?" Pool queueing. "Why does DNS look fast in isolation and terrible at scale?" Pool queueing on getaddrinfo. The single-thread-plus-event-loop mental model does not have room for any of these. The event-loop-plus-libuv-thread-pool model has room for all of them.

Two concrete things you can do this week if you run a Node service in production.

• Set UV_THREADPOOL_SIZE explicitly in your process manager, docker file, or systemd unit. Not because 4 is wrong for every workload, but because "the default someone chose in 2011" should not be a load-bearing part of your production tuning. Pick a deliberate number, even if it happens to be 4.
• Instrument a tiny thread-pool queue-wait metric. A single-run crypto.pbkdf2 with a fixed cost, measured against wall clock at intervals, tells you exactly when the pool is a bottleneck. It is a five-line change and it is the difference between guessing and knowing.

Everything else follows from that. Once you can see it, you can tune it. And you will spend less time wondering why an 8-core box, running a healthy-looking Node process, is somehow only doing what a 1-core box should do.