API Idempotency, Retries and Timeouts: Making Failure Predictable

Timeouts: three layers, not one

A single "total timeout" is usually not enough. Split it into three: a connect timeout (1–3s for TCP/TLS), a read timeout (based on the endpoint's P99), and an overall retry budget (a cap across all attempts). Without a budget, one blip can generate minutes of repeated requests and take the downstream down with it.

Retry only failures that can succeed

Exponential backoff with jitter

// Wait before attempt n (ms), with jitter to avoid a retry storm
const base = 200, cap = 10000;
const wait = Math.min(cap, base * 2 ** attempt);
const jitter = Math.random() * wait * 0.3;
await sleep(wait + jitter);

Backoff without jitter makes every client retry at the same instant, producing a retry storm. Jitter spreads attempts across a window and flattens the peak.

Idempotency: what makes retries safe

Idempotent means executing the same request many times has the same effect as once. Without that guarantee, retries cause duplicate charges, duplicate orders or duplicate messages. Common practice:

1. The client generates an idempotency key (e.g. a UUID) per business operation and sends it with the request;
2. The server records "key → first result" and replays that result instead of executing again;
3. Keys expire (e.g. after 24 hours) so storage does not grow forever;
4. Prefer writes that set a value over ones that increment (setBalance beats addBalance).

Four common mistakes

• Retries amplifying traffic: without caps and a budget, a blip becomes an avalanche;
• Retrying at several layers: gateway retries 3 and the app retries 3 — that is 9 requests;
• Retrying non-idempotent calls: duplicate money movement or messages;
• Over-long timeouts: connections and threads exhaust, and the failure spreads faster.

Observability and debugging

Attach a request ID (generate one with a UUID tool) to every call and log attempt count, duration and final status. When something breaks, look at the distribution of retry counts and latencies first — that usually separates a network blip, a slow downstream and a bad retry policy.

Try it: UUID generator — create request IDs and idempotency keys

Timeout and retry budgets

In distributed calls, timeouts and retries must be allocated as one budget rather than guessed per service.

1. Compute timeouts bottom-up: fix the outermost acceptable latency, then allocate inward, keeping each upstream timeout larger than the sum below it — or upstreams abandon and retry while downstreams are still working.
2. Bound and taper retries: attempts and backoff should decrease as you approach the edge, so one blip is not multiplied into historic traffic by many layers.
3. Retry only retryable errors: retrying a bad parameter or failed authentication is pointless and adds load; only connection failures, timeouts and some server errors deserve it.
4. Reads and writes differ: reads retry automatically; writes need an idempotency key or a retry duplicates the effect.
5. Measure retries: report retry rate, retry success rate and the extra latency they add. A persistently high retry rate means the dependency is unstable and needs fixing.

Idempotency key details

The client should generate the key with business meaning; the server stores the result for a retention window and returns the first outcome for duplicates. Mind storage cost and expiry, and serialise concurrent requests carrying the same key so both do not execute halfway.