2026-11-20 · 9 min read

Your Concurrency Limit Is a Budget You're Borrowing From Someone Else

Flat isometric illustration of many indigo request lines funneling into a server stack, most stalling in grey while ten glowing lanes pass through to a single green success node on a near-black background.

At 02:14 UTC, a scheduled billing job ran a single line: await Promise.all(invoices.map(syncInvoice)) over 1,200 customer records. Within 400 milliseconds the payment gateway started answering 429 Too Many Requests, and PostgreSQL began timing out while handing out connections. Then one promise rejected. Promise.all rejected with it, the job exited, and 1,199 writes that had already been set in motion kept going with nothing left to await them.

Every part of that line is idiomatic. Map an array, run the async function, await the whole thing. It's in the docs, it's in every tutorial, and in development with five mock records it is fast and correct. The syntax isn't the problem. The problem is that the limit on how much work you're doing at once was set by the length of an array — a number that came out of a database, not out of a capacity plan.

The mental shift worth making: concurrency is a budget you borrow. You don't own it. Your database owns some of it, your payment provider owns some, the TLS termination in front of you owns some. Firing 1,200 requests at once isn't being fast; it's spending someone else's capacity without asking, and the invoice arrives as a 429.

Promise.all synchronizes. It does not schedule.

This is where people get stuck, and it's a vocabulary problem more than a hard one.

JavaScript runs your code on a single thread with an event loop: a queue of callbacks executed one at a time. A tick is one pass through that queue. Array.prototype.map is a synchronous function — it calls your callback for every element, back to back, before it returns, all inside a single tick.

So when you write Promise.all(ids.map(fetchUser)), the order of events is: map invokes fetchUser 5,000 times; each invocation immediately starts a DNS lookup, a TLS handshake, a socket write, a database query; all 5,000 promises are in flight; and only then does Promise.all receive the array and begin waiting.

Promise.all is a join. It waits on work you already started. No queue, no throttle, no opinion about capacity. Waiting is the entire feature.

And the failure isn't slow. From a standing start, that call can saturate a connection pool in tens of milliseconds — which is exactly why it never shows up in testing. Five records never saturate anything.

Three ways it bites, and only one is a 429

  • The thundering herd. Your database pool is probably somewhere between 10 and 50 connections; a third-party API probably allows somewhere between 5 and 100 requests per second. Open 1,000 at once and you don't just fail your own request — you starve every other endpoint in the process, including the health check your load balancer is polling.
  • Orphaned side effects. A rejected promise is not a cancelled one. Nothing inside Promise.all reaches back into the other 999 promises and stops them. They keep writing. If the work isn't idempotent — if repeating it produces a duplicate rather than the same result — you now have partial records a retry will double, and "the job failed" and "the job half-succeeded" are very different states to reconcile at 3 a.m.
  • Memory and GC pressure. Thousands of response bodies and parsed JSON payloads buffered at the same moment means a heap spike, which means a stop-the-world garbage collection pause or an OOM kill — inside the same process that's holding the pool.

Notice the shape of that first one. The 429 is the polite failure: something downstream told you no, and you can read it. The pool timeout and the OOM kill are the impolite ones.

The number you need isn't in your code

Most advice stops at "use a concurrency limit." Fine — what limit? Picking the number is the entire job, and you can't pick it by feel.

Work backwards from the most constrained thing downstream, then divide by your replicas, because your limit is per process and the pool is shared.

A worked example. Your Postgres instance allows 100 connections. The pool library is configured for 50 per process. You run 5 replicas, and this billing path is not the only thing using the pool — say it gets a third of it. That's 50 divided by 5 replicas = 10 per process, times one third = 3. Not 10. Three is the number that doesn't degrade your other endpoints, and three is what belongs in config.

Do the same for the external API. If the contract says 50 requests per second and you run 5 replicas, the ceiling is 10 per second per replica — and aim below it, because retries share that budget too.

Take the minimum of everything you computed. That's your limit. Give it a name that says what it protects, and put it next to the code that reads it. A hardcoded 10 inside a helper function is a number nobody can audit and nobody will remember choosing.

And treat a 429 as a configuration bug, not weather. It means your limit is wrong — usually wrong by an order of magnitude — and in most systems it should page the way a 5xx does, because by the time the rate limiter answers you, the requests are already wasted.

Chunking looks like a fix and isn't one

The first repair everyone reaches for is slicing the array: take 10, await them, take the next 10. It does stop the 429s, and it's a three-line change. It also introduces head-of-line blocking: inside a batch, every fast task waits for the slowest one before the next batch can begin, and during that wait nine of your ten slots sit idle.

Concretely, a batch of 10 where 9 tasks take 20ms and 1 takes 900ms finishes in roughly 900ms, not 180ms. Your throughput isn't the sum of the work; it's the number of batches multiplied by the slowest item in each one. With 200 tasks that's 20 waves, and any wave containing an outlier costs you that outlier's full duration.

There's a second, subtler cost: chunked loops produce a sawtooth load pattern — burst, idle, burst, idle — which is harder on a downstream that does its own smoothing than a steady, saturated, predictable rate.

A worker pool, and the brake that makes fail-fast honest

A bounded worker pool fixes both problems. The shape: allocate N workers, where N is the limit you calculated. Each worker loops, pulls the next index from a shared counter, awaits its task, and pulls again. Workers refill their own slots, so one slow task occupies exactly one slot instead of stalling a whole batch. Throughput stays saturated until the queue drains.

In one published benchmark against a downstream that returns 429 above 10 concurrent connections, 200 tasks of 50ms each behaved like this: unbounded Promise.all crashed at about 52ms with zero tasks completed; a sequential for loop finished all 200 in about 10,180ms; the pool with a limit of 10 finished all 200 in about 1,045ms. Ten times faster than sequential, without dropping a request.

But the pool alone doesn't solve the orphan problem. You need a brake, and the brake is an AbortController — a standard object that carries a cancellation signal you can hand to fetch calls and pass into your worker function. On the first fatal error, abort the controller; each worker checks the signal before pulling its next item, and queued work stops instead of marching into a system that's already unhappy.

Here's the trade-off, stated plainly: fail-fast is only a virtue when you can also cancel. Without a signal, aborting a batch is the worst of both worlds — incomplete work plus background work you can no longer report on. If cancellation isn't available (some SDKs won't accept a signal), make the work idempotent and drain the batch instead of aborting it. Slow and complete beats fast and half-written.

The rule that covers most cases

  • A fixed tuple of heterogeneous calls — user, permissions, feature flags — Promise.all is correct. There are three of them, you need all three, they're all cheap. Leave it alone.
  • A dynamic array where every item must succeed — bounded pool, abort on first failure, and make the items idempotent so a retry is safe.
  • A dynamic array where partial success is acceptable — bounded pool, and catch errors inside the worker, returning a { status, value } object per item. You get the semantics of Promise.allSettled with a ceiling on in-flight work.
  • Anything with side effects and no idempotency key — fix that first. No concurrency strategy saves you from a retry that duplicates a write.

The audit is one search: find Promise.all followed by .map(, and look at what's being mapped. If that array's length comes from a query, a file, or a request body, then that length is your concurrency limit — and your concurrency limit is a decision somebody made by accident.

Two people can write identical Promise.all calls and get wildly different reliability, because the interesting variable was never in the code. It's how much capacity the other system had left when you showed up. Make that number deliberate, write it down, and put it next to the thing it protects.

Key Takeaways

  • Promise.all is a join, not a queue. It waits on promises you already started; it never throttles anything.
  • map is synchronous. Every callback fires in one event loop tick, so all the I/O starts before Promise.all even receives the array.
  • Size the limit from downstream, not from your array. Take the tightest constraint you can find, divide by replicas, and aim below it.
  • A rejected promise is not a cancelled one. Fail-fast without an AbortController produces orphaned writes, not a clean abort.
  • Chunking trades a 429 for head-of-line blocking. A worker pool keeps every slot busy; fixed batches wait for their slowest member.
  • Treat 429s as config bugs. They're a signal your limit is wrong, and they deserve alerting, not just retry logic.

[ Call to action ]

Ready to scale your engineering team?

Tell us the roles you need to fill and we'll get back within 24 hours.