Short answer: retry only failures that are safe and transient (429, 502, 503, 504, network errors), wait random(0, min(cap, base * 2^attempt)) between attempts ("full jitter"), obey Retry-After when the server sends one, and stop after a fixed number of attempts. Plain exponential backoff without the random part keeps every client in lockstep, so the retry wave hits the server all at once, again, on a schedule.
If you already read the rate limiting post, this is the other half: that one is about not tripping the limit, this one is about what your client does after something already failed.
The failure: a retry storm with good manners
Picture 1,000 clients that all got a 503 at the same moment because a deploy restarted the backend. Each one retries after 1s, 2s, 4s, 8s... Textbook exponential backoff. The problem is that they all failed at the same instant, so they all retry at the same instant. The server comes back up and gets hit with 1,000 requests at t=1s, then 1,000 more at t=3s, then 1,000 at t=7s. Each wave is as big as the original, and the server is probably still cold.
I simulated exactly this: 1,000 clients, all failing at t=0, base delay 1s, cap 30s, six attempts each, requests counted in 250ms buckets. The number to look at is the peak, the most requests landing in any single bucket:
| Strategy | Delay before attempt n | Peak requests per 250ms |
|---|---|---|
| No jitter | min(cap, base * 2^n) | 1,000 |
| Equal jitter | half fixed, half random | 500 |
| Full jitter | random(0, min(cap, base * 2^n)) | 402 |
Same total number of requests (6,000) in all three. Jitter doesn't reduce load, it spreads it. That is the whole trick, and it is why "we have backoff" is not an answer when someone asks whether your client is safe against a thundering herd.
The retry delay is a load-balancing decision, not a politeness setting. If every client computes the same delay, you have built a synchronized job scheduler pointed at your own backend.
Which jitter
The canonical comparison is Marc Brooker's Exponential Backoff and Jitter on the AWS Architecture Blog. It measured full, equal and decorrelated jitter against contended writes and found full jitter does the least total work, which is why it is the default I'd reach for.
- Full jitter:
random(0, ceiling). Best spread. The downside is that a client can occasionally retry almost immediately (a delay near 0), which is fine for an overloaded server and bad if the dependency needs real recovery time. Fix that with a small floor if you care. - Equal jitter:
ceiling/2 + random(0, ceiling/2). Guarantees some minimum wait, but in my run it only halved the peak, since the bottom half of the delay is still synchronized. - Decorrelated jitter:
min(cap, random(base, previous_delay * 3)). No dependence on the attempt number, only on the last delay. Spreads well but each client's delay grows with its own random walk, so it is harder to reason about worst-case total wait. Use it if you want stateless-ish retry logic; otherwise full jitter is easier to explain in a code review.
Retry-After beats your math
If the response carries a Retry-After header, the server is telling you when it expects to be ready. RFC 9110 section 10.2.3 allows two formats: a number of seconds (Retry-After: 120) or an HTTP date (Retry-After: Fri, 31 Dec 2027 23:59:59 GMT). Handle both. A parser that only does parseInt turns the date form into NaN, and setTimeout(fn, NaN) fires immediately, which is the opposite of what the header asked for.
Use the server's number when it exists, and keep your own jitter formula as the fallback. Add a little random spread on top of a hinted delay if you run many clients, since a fleet of workers all told "retry in 30" will also arrive together.
A wrapper you can paste
Tested against a stub that returns 503 twice and then 200:
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
function retryAfterMs(res) {
const h = res.headers.get("retry-after");
if (!h) return null;
if (/^\d+$/.test(h)) return Number(h) * 1000;
const t = Date.parse(h);
return Number.isNaN(t) ? null : Math.max(0, t - Date.now());
}
export async function fetchWithRetry(url, init = {}, { retries = 5, baseMs = 500, capMs = 30000 } = {}) {
for (let attempt = 0; ; attempt++) {
let res;
try {
res = await fetch(url, init);
} catch (e) {
if (attempt >= retries) throw e;
}
if (res && res.status !== 429 && res.status < 500) return res;
if (attempt >= retries) return res;
const ceiling = Math.min(capMs, baseMs * 2 ** attempt);
const hinted = res ? retryAfterMs(res) : null;
await sleep(hinted ?? Math.random() * ceiling);
}
}Two deliberate choices: it returns the last response instead of throwing once retries run out, so the caller still sees the real status and body; and it treats every 5xx as retryable, which is too broad for some APIs (see below). Narrow it to 502/503/504 if yours returns 500 for deterministic bugs.
What not to retry
Retrying is only safe when running the request twice is safe. The checklist I use:
- 4xx other than 429 and sometimes 408. A 400, 401, 403, 404 or 422 will fail identically on attempt six. Retrying them just burns quota. Not sure what a code means? The HTTP status code reference lists the semantics.
- Non-idempotent writes without an idempotency key. A POST that times out may have succeeded. Retry it blindly and you charge the card twice. Either send an
Idempotency-Key(more in the idempotency post) or don't retry. - Timeouts on slow work. If the server needs 40s and your timeout is 10s, retrying creates a queue of abandoned 40s jobs. Raise the timeout or make the endpoint async.
- Anything inside a request the user is waiting on, more than once or twice. Five retries with a 30s cap is a minute and a half of spinner. User-facing paths want 1 to 2 attempts and a fast failure; background jobs can afford more.
Retry at one layer only
Retries multiply across layers. If your browser retries 3 times, your API gateway retries 3 times, and your service client retries 3 times, one user click can become 27 requests against the database at the bottom, exactly when it is struggling. Pick one layer to own retries (usually the one closest to the failing call), and make the others fail fast. Google's SRE book makes the same point in its chapter on addressing cascading failures, where retry amplification is called out by name.
Common mistakes
No cap
base * 2^attempt reaches 17 minutes by attempt 10 at a 1s base. Always min(cap, ...).
No attempt limit
A retry loop with no maximum is a bug that waits for the one day the dependency is down for good.
Jitter on the wrong side
delay + random(0, 100) is not jitter at that scale. The random range has to be comparable to the delay itself, or the waves stay visible.
Retrying inside a retry budget you don't have
A per-request limit still allows a fleet-wide storm. If you can, add a client-wide budget: allow retries only while they stay under, say, 10% of recent requests. Without one, a partial outage turns your traffic into 1.5 to 6x its normal level.
FAQ
How many retries should I do?
Three to five for background work, one or two for anything a person is waiting on. If five attempts with a decent cap haven't worked, the problem isn't transient.
Is it okay to use the same base delay for every service?
As a default yes. Tune it per dependency once you know its recovery time: a cache that recovers in 200ms wants a smaller base than a database failover that takes 30s.
Should I add jitter if I only have one client?
It costs nothing, so yes, but the benefit is smaller. A single client can still synchronize with cron jobs or other services that start on the minute.