Rate limiting protects a service from being overwhelmed, whether by an abusive client, a buggy integration or a legitimate customer who just launched a campaign. Done well it is nearly invisible; done badly it breaks integrations in ways that are hard to diagnose from the client side.
The algorithms
Fixed window. Count requests per clock interval, reset at the boundary. Trivial to implement and has a well-known flaw: a client can send a full allowance at the end of one window and another at the start of the next, producing double the intended rate across the boundary.
Sliding window log. Store timestamps of recent requests and count those within the window. Exact, and memory grows with request volume.
Sliding window counter. Approximates the sliding window by weighting the previous window’s count. Nearly as accurate as the log, far cheaper. A good default.
Token bucket. Tokens accumulate at a fixed rate up to a maximum; each request consumes one. Allows controlled bursts while enforcing an average rate. The most common choice for public APIs because it matches how clients actually behave.
Leaky bucket. Requests queue and drain at a constant rate. Smooths traffic completely, at the cost of added latency. Useful in front of a downstream system that cannot tolerate bursts.
What to key the limit on
The choice matters more than the algorithm.
- API key or account — the usual default for authenticated APIs, and the fairest.
- IP address — necessary for unauthenticated endpoints, and blunt. Corporate NATs and mobile carriers put thousands of users behind one address.
- User within an account — prevents one user in a large tenant from consuming the whole allowance.
- Endpoint — expensive operations deserve tighter limits than cheap ones. A search endpoint and a health check should not share a budget.
Most mature APIs combine several: a global per-account limit, tighter limits on specific expensive endpoints, and an IP-based limit on authentication endpoints to slow credential stuffing.
Cost-based limiting
Counting requests treats a trivial lookup the same as a report that scans a million rows. For APIs with wide variation in operation cost, assigning each operation a weight and limiting on total weight per window is considerably fairer.
This is more work to implement and to document, and it is what makes limits feel reasonable to clients doing heavy work. It also maps naturally onto usage-based pricing if that is where the product is headed.
Headers: tell the client what is happening
A rate-limited response with no information forces the client to guess. At minimum, return the limit, the remaining allowance and when the window resets on every response — not only on rejections, so well-behaved clients can slow down before hitting the wall.
On a rejection, return HTTP 429 with a Retry-After header giving seconds to wait. This is the single most important header, because it is the one well-written clients act on automatically.
Use the standardised RateLimit header family if you are designing a new API; otherwise follow the convention your ecosystem already expects, and document whichever you choose precisely.
Status codes
429 Too Many Requests is correct for rate limiting. 503 Service Unavailable signals the service is down rather than the client being throttled, and causes clients to treat it as an outage. 403 is wrong and will send integrators looking at their credentials.
The response body should say which limit was exceeded and when it resets, in whatever error format the rest of the API uses. “Rate limit exceeded” with no detail generates support tickets.
Encouraging good client behaviour
The behaviour you want is exponential backoff with jitter: wait, double the wait on each failure, and add randomness so that clients that failed together do not retry together. Thundering herds after an outage are frequently worse than the original problem.
Three things make clients do this. Document the expected behaviour prominently. Publish official client libraries that implement it, since most integrators use the library rather than reading the docs. And return Retry-After, which well-written HTTP clients respect automatically.
Choosing the numbers
Limits that are too tight break legitimate use; too loose and they protect nothing.
A reasonable method: measure the actual distribution of request rates per account over a month, set the limit somewhere above the 99th percentile of legitimate use, and verify the resulting ceiling is still comfortably within what your infrastructure can serve. Then run in monitor-only mode for a fortnight, logging what would have been rejected, before enforcing.
That monitor-only period is the step most often skipped and the one that prevents an embarrassing rollback.
Operational details that matter
Distributed counters. With several API servers, counters must be shared, usually in a fast central store. Per-instance counters mean the effective limit is the configured limit times the number of instances.
Fail open or closed. Decide what happens when the counter store is unreachable. Failing open risks overload; failing closed turns a cache outage into a total outage. Most public APIs fail open with an alert.
A bypass list. You will need to raise a limit for a specific customer urgently. Build this before you need it at 2am.
Visibility for customers. Showing current usage against limits in the dashboard prevents most “why am I getting 429s” tickets.
Tiering limits
Different plans warrant different limits, and this is a legitimate and well-understood form of packaging. Keep the tiers simple enough to state on a pricing page, document the exact numbers rather than “fair use”, and make the upgrade path from one tier to the next obvious when a client starts hitting the ceiling — that moment is one of the better expansion triggers available.
Rate limiting is not the only throttle
Three related controls are often confused with rate limiting and solve different problems.
Concurrency limits cap how many requests a client can have in flight at once. For expensive, long-running operations this protects the service better than a per-minute rate, because ten simultaneous heavy queries hurt more than sixty spread over a minute.
Quotas are longer-period allowances — per day or per month — usually tied to a plan. They control cost rather than protecting capacity, and they need a different message when exceeded, because waiting sixty seconds will not help.
Load shedding rejects requests when the service itself is unhealthy, regardless of who sent them. This is the last line of defence and should prioritise: drop background and batch traffic before interactive requests.
A mature API has all four, each with its own error signal, so a client can tell the difference between “slow down”, “you have used your allowance” and “we are struggling”.
Rolling out a limit to an existing API
Introducing limits to an API that has never had them will break someone. A sequence that minimises the damage:
- Instrument first. Record per-client request rates for a month without enforcing anything.
- Set a provisional limit and log what would be rejected, still without enforcing.
- Contact the clients that would be affected individually. There are usually fewer than expected, and most will fix their integration if asked.
- Return the rate limit headers before enforcement begins, so well-behaved clients can adapt.
- Enforce, starting with a generous limit, and tighten over subsequent releases.
Announce each stage with dates. An unannounced limit that breaks a customer’s production integration costs more goodwill than the overload it prevented.
Frequently asked questions
What is a reasonable default rate limit?
There is no universal figure; it depends entirely on what each request costs you. Derive it from measured usage and capacity rather than copying another API.
Should rate limits apply to internal services?
Yes, at least as a circuit breaker. Internal clients cause overload as readily as external ones, usually through a retry loop in a batch job.
How should clients handle 429 responses?
Respect Retry-After when present, otherwise exponential backoff with jitter, with a maximum retry count. Never retry immediately in a tight loop.
Is rate limiting a security control?
Partly. It slows credential stuffing, scraping and enumeration, and it is not sufficient against a determined distributed attacker. Treat it as one layer alongside authentication controls and bot detection.
