Rate limiting is usually added after an incident, implemented with a counter and a TTL, and then discovered to be both too strict for legitimate users and too loose for the traffic it was meant to stop. The gap between those two outcomes is almost entirely about which algorithm you picked and which dimension you keyed it on.
The boundary problem
A fixed window — a counter per minute, incremented and expired — is the simplest implementation and has a structural flaw: a client can send its full allowance in the last second of one window and again in the first second of the next, delivering double the intended rate in a two-second burst. If your limit exists to protect a fragile downstream, that burst is exactly the thing you were defending against.
A token bucket fixes it and models the intent better. Tokens accrue at a steady rate up to a maximum; each request spends one. It permits a burst up to the bucket size, then enforces the sustained rate, which is usually what you actually want: tolerate a legitimate spike, refuse a sustained flood.
-- Token bucket in one atomic Redis call. Atomicity matters: the read-then-write
-- version has a race that leaks allowance under exactly the load you care about.
local key = KEYS[1]
local rate = tonumber(ARGV[1]) -- tokens per second
local capacity = tonumber(ARGV[2]) -- burst size
local now = tonumber(ARGV[3])
local cost = tonumber(ARGV[4])
local state = redis.call('HMGET', key, 'tokens', 'ts')
local tokens = tonumber(state[1]) or capacity
local last = tonumber(state[2]) or now
tokens = math.min(capacity, tokens + (now - last) * rate)
if tokens < cost then
redis.call('HMSET', key, 'tokens', tokens, 'ts', now)
redis.call('EXPIRE', key, math.ceil(capacity / rate) * 2)
return {0, math.ceil((cost - tokens) / rate)} -- denied, retry-after seconds
end
redis.call('HMSET', key, 'tokens', tokens - cost, 'ts', now)
redis.call('EXPIRE', key, math.ceil(capacity / rate) * 2)
return {1, 0}Limit the dimension that describes the abuse
Keying by IP address is the default and is wrong in most enterprise contexts: an entire customer behind one corporate gateway shares an address, so a limit strict enough to stop a scraper locks out a whole company. Key by API key or tenant for authenticated traffic, and layer rather than choose — a per-tenant limit for fairness, a per-endpoint limit to protect an expensive operation, and a global limit as the last line before the database.
Cost-weighted limits are the refinement worth adding once the basics work. Not all requests are equal: a bulk export may cost fifty tokens and a cheap lookup one. Spending tokens proportional to actual work aligns the limit with the resource you are protecting rather than with a request count that means very little.
- Always return Retry-After and the remaining allowance. A 429 with no guidance produces immediate retries, which is the opposite of the intent.
- Fail open on limiter infrastructure failure for most APIs, and fail closed only where the downstream genuinely cannot survive a flood. Decide deliberately.
- Put the counter close to the edge, and accept slight imprecision from local counters with periodic sync rather than a global round trip per request.
- Give internal and premium traffic separate buckets so a noisy integration cannot consume a customer's allowance.
- Log limit events with the key, endpoint and tenant. The first question after every incident is who was throttled and whether they should have been.
A rate limit is a statement about fairness, not a technical safeguard. Decide who you are protecting from whom before choosing an algorithm.
One last piece that gets skipped: the queue behind the limiter. Rejecting excess traffic only helps if accepted traffic is actually served, so pair the limiter with bounded concurrency. Otherwise you have built a sophisticated mechanism for admitting exactly as much work as it takes to fall over politely.