Why Rate Limiters Are Harder Than They Look
Most developers treat rate limiting as a solved problem. Slap a counter in Redis, increment it on every request, reject anything over the limit. Done, right? Not even close. I've built rate limiters that looked perfect in staging and fell apart spectacularly in production under real traffic. The failure modes are subtle, they often happen at the worst possible moments, and they can silently let through way more requests than you intended — or brutally block legitimate users.
The core challenge is that rate limiting is fundamentally a distributed systems problem disguised as a simple counter problem. Your app probably runs on multiple servers. Requests arrive at different instances simultaneously. Redis is fast but not instantaneous. Every operation has latency, and in the gap between reading a counter and writing it back, a dozen other requests might be doing the exact same thing. Understanding why each algorithm breaks is just as important as knowing how to implement it.
This guide walks through two of the most practical rate limiting strategies — the token bucket and the sliding window — with real Redis implementations, honest discussion of their tradeoffs, and the specific conditions that will cause each one to misbehave under load.
The Fixed Window Counter: The One You Should Probably Stop Using
Before getting to the good stuff, let's talk about the approach most tutorials teach first, because understanding its flaw explains why the better approaches exist. The fixed window counter works by dividing time into fixed buckets — say, one bucket per minute — and incrementing a counter per user per bucket. Simple, cheap, and tempting.
The problem is what I call the "boundary burst" — a phenomenon where a user can make double the allowed requests by straddling a window boundary. If your limit is 100 requests per minute and the window resets at :00, a user can fire 100 requests from :59 to :00, then immediately fire another 100 from :00 onward. In a two-second window, they've sent 200 requests, which completely breaks your intent. This isn't theoretical. Under any meaningful load, users (or bots) will hit this window edge regularly.
The fixed window isn't useless — it works fine for very loose rate limits where a 2x burst is acceptable, or for cases where you're just trying to prevent absolute abuse rather than enforce precise traffic shaping. But for anything requiring real precision, you need something better.
Token Bucket: The Classic Algorithm and Its Hidden Gotchas
The token bucket algorithm is elegant. Imagine a physical bucket that holds tokens. Tokens are added to the bucket at a fixed rate — say, 10 per second. Each request consumes one token. If the bucket is empty, the request is rejected. The bucket has a maximum capacity, which means tokens don't accumulate indefinitely. This naturally allows short bursts (consuming saved-up tokens) while enforcing a long-term average rate.
The implementation trick in Redis is that you don't actually add tokens on a timer. Instead, you store the last timestamp and calculate how many tokens should have been added since then when a request arrives. This is called a "lazy refill" — you compute the current token count on demand. It keeps your Redis writes focused only on actual request events, not on background timer ticks. Here's a working implementation in Python using Redis:
import time
import redis
r = redis.Redis(host='localhost', port=6379, decode_responses=True)
def token_bucket_allow(user_id: str, capacity: int, refill_rate: float) -> bool:
"""
capacity: max tokens the bucket can hold
refill_rate: tokens added per second
"""
key = f"rate_limit:token_bucket:{user_id}"
now = time.time()
pipe = r.pipeline()
pipe.hgetall(key)
results = pipe.execute()
data = results[0]
if data:
last_time = float(data['last_time'])
tokens = float(data['tokens'])
elapsed = now - last_time
tokens = min(capacity, tokens + elapsed * refill_rate)
else:
tokens = capacity
last_time = now
if tokens >= 1.0:
tokens -= 1.0
allowed = True
else:
allowed = False
pipe = r.pipeline()
pipe.hset(key, mapping={'tokens': tokens, 'last_time': now})
pipe.expire(key, int(capacity / refill_rate) + 10)
pipe.execute()
return allowedThis works, but there's a race condition lurking here that will bite you in production. Between the hgetall read and the hset write, another process can read the same stale token count and both processes will think they have permission to proceed. Under concurrent traffic from the same user — think a client retrying aggressively or a poorly implemented SDK — you'll let through significantly more requests than intended.
The fix is to use a Lua script, which Redis executes atomically (meaning no other command can interrupt it mid-execution). Lua in Redis is your best friend for rate limiting because it turns a multi-step read-modify-write operation into a single atomic unit:
-- token_bucket.lua
local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local refill_rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local data = redis.call('HMGET', key, 'tokens', 'last_time')
local tokens = tonumber(data[1])
local last_time = tonumber(data[2])
if tokens == nil then
tokens = capacity
last_time = now
end
local elapsed = now - last_time
tokens = math.min(capacity, tokens + elapsed * refill_rate)
local allowed = 0
if tokens >= 1.0 then
tokens = tokens - 1.0
allowed = 1
end
redis.call('HMSET', key, 'tokens', tokens, 'last_time', now)
redis.call('EXPIRE', key, math.ceil(capacity / refill_rate) + 10)
return allowedYou'd call this script using EVALSHA (which executes a cached Lua script by its hash, faster than sending the full script each time) with your key, capacity, refill rate, and current timestamp as arguments. The atomicity guarantee means concurrent requests for the same user will serialize properly through Redis.
Where Token Buckets Break Under Load
The token bucket shines for APIs where you want to allow brief bursts — a user can make 50 requests quickly if they've been idle, then get throttled. But there's a failure mode I've seen catch teams off guard: clock skew. If your application servers have slightly different system clocks (even a few hundred milliseconds difference), the elapsed time calculation becomes unreliable. Server A might think 1.5 seconds have passed since the last request while Server B thinks it's been 0.8 seconds. The result is inconsistent refill amounts and a rate limiter that behaves differently depending on which server happens to handle a request.
The mitigation here is to always use Redis's own time with redis.call('TIME') inside your Lua script instead of passing the timestamp from the application layer. Redis itself has a single clock, so every execution of your Lua script uses a consistent time source regardless of which app server triggered it.
Sliding Window Log: Precise but Memory-Hungry
The sliding window log is the most accurate approach. Instead of coarse time buckets, you store a timestamp for every request in a Redis sorted set (think of it as a list ordered by time). To check if a request is allowed, you count how many timestamps fall within the last N seconds. Remove anything older than that window, add the current timestamp, and check if the count exceeds your limit.
The precision is beautiful — there's no boundary burst problem, and the rate limit is always evaluated against an exact rolling window of time. But the memory cost is real. If you allow 10,000 requests per minute per user and you have 50,000 users, you're potentially storing millions of sorted set entries in Redis. For most production systems, this is impractical at scale. It's worth using for low-volume but high-precision use cases, like financial transaction limits or admin action throttling.
Sliding Window Counter: The Sweet Spot
The sliding window counter is a pragmatic hybrid that most production systems should be using. It doesn't store individual request timestamps. Instead, it keeps counters for two fixed windows (current and previous) and calculates an approximate sliding window value by weighting the previous window's count based on how far through the current window you are.
Here's the intuition: if your window is 60 seconds and you're 40 seconds into the current window, the previous window contributes (20/60) of its count to your current estimate. It's an approximation, but it's accurate enough for virtually every real-world use case and uses constant memory per user regardless of traffic volume.
const redis = require('redis');
const client = redis.createClient();
async function slidingWindowAllow(userId, limit, windowSeconds) {
const now = Math.floor(Date.now() / 1000);
const currentWindow = Math.floor(now / windowSeconds);
const previousWindow = currentWindow - 1;
const currentKey = `rate_limit:${userId}:${currentWindow}`;
const previousKey = `rate_limit:${userId}:${previousWindow}`;
const pipeline = client.multi();
pipeline.get(previousKey);
pipeline.get(currentKey);
const [prevCount, currCount] = await pipeline.exec();
const previous = parseInt(prevCount || '0', 10);
const current = parseInt(currCount || '0', 10);
// How far through the current window are we? (0.0 to 1.0)
const windowProgress = (now % windowSeconds) / windowSeconds;
// Weight the previous window's contribution
const estimatedCount = previous * (1 - windowProgress) + current;
if (estimatedCount >= limit) {
return false; // Rate limited
}
// Increment current window counter
const multi = client.multi();
multi.incr(currentKey);
multi.expire(currentKey, windowSeconds * 2); // Keep for 2 windows
await multi.exec();
return true;
}This has a subtle race condition too — between reading the counts and incrementing. For most applications, the approximation and slight over-allowance during races is completely acceptable. If you need strict enforcement, wrap the get-and-increment in a Lua script. In my experience, the strict version is overkill for most rate limiting scenarios, but the right call depends on your security requirements.
Where Sliding Windows Break Under Load
The sliding window counter's approximation can be gamed, though it takes a fairly determined attacker. Because the estimate uses weighted averages, sending requests at exactly the window boundary still allows slightly more than the strict limit — just not the dramatic 2x burst you'd see with fixed windows. Under extremely high concurrency (thousands of requests per second hitting the same user key), the non-atomic version of this will under-count and let through excess traffic.
The other failure mode is Redis connection exhaustion. Every rate limit check requires at least one round trip to Redis. Under heavy load, if your connection pool (the pre-established pool of Redis connections your app reuses to avoid the overhead of creating new ones for each request) is undersized, rate limit checks will queue up and add latency to every request — not just the rate-limited ones. I've watched applications where the rate limiter itself became the bottleneck, slower than the actual business logic it was protecting. Size your connection pool generously, use pipelining wherever possible, and consider caching the "definitely not rate limited" decision locally for a short window if your architecture allows it.
Handling Redis Failures Gracefully
This is the part most tutorials skip entirely. What happens when Redis is unavailable? Your rate limiter is now deciding the behavior of your entire application. There are two schools of thought: fail open (allow requests when Redis is down, because denying all traffic is worse than rate limit enforcement temporarily disappearing) and fail closed (deny all requests when you can't check limits, because your backend might not survive unthrottled traffic).
The right answer depends entirely on your context. A public API facing potential abuse should probably fail open with monitoring alerts — denying all traffic during a Redis blip would take down the service entirely. An internal payment processing endpoint might reasonably fail closed, because the cost of letting through unlimited requests is higher than a temporary outage.
What you should never do is let Redis exceptions propagate uncaught into your request handlers. Wrap every rate limit check in error handling that makes an explicit decision and logs the failure. Silence in these situations is how you end up debugging a cascade failure at 2am wondering why your rate limiter was silently returning false for six hours.
Choosing the Right Algorithm for Your Situation
The token bucket is your best choice when you're protecting an API that serves interactive users who need occasional burst capacity — think a search endpoint or a content delivery API where a user might legitimately need 20 requests in rapid succession after being idle. The natural burst-then-throttle behavior matches user experience expectations.
The sliding window counter wins when you need consistent, predictable enforcement without burst allowance — rate limiting login attempts, password resets, or SMS sends where any burst is a red flag. The constant memory usage and simple mental model are also significant operational advantages when you have millions of users.
Whatever you choose, build your rate limiter behind an abstraction layer from day one. The algorithm you start with is rarely the one you end up with, and nothing is more painful than untangling rate limit logic from your request handling code when you need to swap implementations. Treat your rate limiter like infrastructure: testable in isolation, replaceable without touching application logic, and observable with metrics on both allowed and rejected request rates. That last one especially — if you can't see how often your rate limiter is actually firing in production, you have no idea whether it's working.