Four algorithms compared
| Algorithm | Behaviour | Edge issue | Use for |
|---|---|---|---|
| Fixed window | Count per fixed interval | Double burst at window edge | Simple stats |
| Sliding window | Approx. smoother | Slightly more complex | Even throttling |
| Token bucket | Allows some burst | Needs bucket state | User-facing APIs |
| Leaky bucket | Constant outflow | Burst queued or dropped | Protect downstream,削峰 |
Distributed counting
With multiple instances use shared storage: Redis INCR + EXPIRE or a Lua script for atomicity. Otherwise each node counts alone and the limit is meaningless.
Three key practices
- Return 429 + Retry-After so clients know when to retry instead of hammering;
- Limit by identity: user, API key and tenant separately — not just IP, which hurts whole NAT groups;
- Backoff with jitter: client retries use exponential backoff plus random jitter to avoid retry storms.
Idempotency and retries
Rate limiting answers "don't hit too hard"; idempotency answers "will a retry double-apply". Together they make retries safe. See API idempotency, retries and timeouts.
Real-world cases: three problems caused by rate limiting
- Fixed windows burst at the boundary: twice the traffic arrives as the window flips. Sliding windows or token buckets smooth that spike.
- Limiting by IP only: legitimate users behind one NAT share the limit, while attackers rotate IPs to bypass it. Combine per-user, per-token and per-endpoint dimensions, with IP as a backstop.
- Returning 500 when limited: clients cannot tell "retry later" from "service broken", and retry storms worsen congestion. Return 429 with
Retry-After.
FAQ
Token bucket or leaky bucket? Token buckets allow configured bursts; leaky buckets emit more smoothly — user-facing APIs usually want token buckets. Where is the counter stored? Multiple instances need shared storage such as Redis, or each instance limits separately and the real allowance doubles. Should limits differ per endpoint? Yes — login, SMS and export endpoints are expensive and deserve stricter thresholds. How do I pick thresholds? Load-test for per-instance capacity, then set per-user and global limits with headroom.
Pairing with business policy
Rate limiting is not isolated middleware configuration — it only works alongside business rules.
- Tier by endpoint: strictest for login and SMS, looser for queries. One threshold for everything either hurts legitimate traffic or under-protects expensive endpoints.
- Separate bursts from scanning: allow the normal pattern of a user acting quickly several times, and watch for uniform high-frequency, enumerating access. Counting alone throttles users and crawlers together.
- Give callers feedback: beyond a status code and retry time, return remaining quota and reset time in headers so clients self-adjust instead of trial and error.
- Keep limits adjustable: sales, campaigns and incidents need looser or tighter limits on demand. Hot-reloadable configuration beats editing code and redeploying.
- Prevent bypass: aggregate multiple source addresses and devices for the same identity, or an attacker simply rotates addresses.
Post-launch verification
Verify three things after rollout: acceptable false-positive rate for real users, whether throttled requests really were anomalous, and whether limiting added latency. Unverified limits often turn out to be meaningless the day an attack arrives.
Connecting to billing and quotas
Beyond rate limiting you usually need quota management: the first prevents momentary overload, the second caps totals and supports billing. Both should share one metering basis, or you get "no limit triggered but quota exhausted" — impossible to explain. Metering accuracy and latency should match the usage users can see.