Recently, I faced an API meltdown during a traffic surge. Our services buckled under unexpected load, exposing our rate limiting as a single-point failure. This experience drove me to design a distributed solution using Redis, Node.js, and Lua. Why these tools? They combine atomic operations, shared state management, and high throughput - essential for modern distributed systems.

Traditional approaches fail in distributed environments. Imagine ten servers each allowing 100 requests per minute. Without coordination, one client could send 1,000 requests across all servers. We need centralized state management. Redis solves this with its in-memory data store and atomic operations.

But race conditions lurk in naive implementations. Consider this flawed approach:

async function faultyRateLimit(key: string) {
  const current = await redis.get(key);
  if (parseInt(current) > 100) return false;
  // Race condition gap here
  await redis.incr(key);
  return true;
}

Between get() and incr(), other requests can slip through. How do we close this gap? Lua scripts execute atomically in Redis, eliminating race conditions.

Let’s implement the Token Bucket algorithm. It allows short bursts while enforcing long-term averages. We’ll store tokens and last refill time in a Redis hash:

-- Token Bucket Lua Script
local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local now = redis.call('TIME')
local current_time = tonumber(now[1]) * 1000 + math.floor(tonumber(now[2]) / 1000)
local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')

-- Calculate new tokens based on elapsed time
local elapsed = current_time - tonumber(bucket[2])
local new_tokens = elapsed * (tokens_per_ms)
local tokens = math.min(capacity, (tonumber(bucket[1]) or capacity) + new_tokens)

if tokens >= 1 then
  redis.call('HMSET', key, 'tokens', tokens - 1, 'last_refill', current_time)
  return {1, tokens - 1} -- Allowed
else
  return {0, tokens} -- Rejected
end

Notice we use Redis’ TIME command instead of server clocks. Why? Server time drift would cause inconsistencies.

For sliding windows, we combine sorted sets and transactions:

async function slidingWindow(key: string, windowMs: number, limit: number) {
  const now = Date.now();
  const windowStart = now - windowMs;
  
  const transaction = redis.multi();
  transaction.zremrangebyscore(key, 0, windowStart); // Clean old requests
  transaction.zcard(key); // Count current requests
  transaction.zadd(key, now, `${now}-${Math.random()}`); // Add new request
  transaction.expire(key, Math.ceil(windowMs / 1000) * 2); // Set TTL
  
  const [_, count] = await transaction.exec();
  return count < limit;
}

This maintains precise counts within moving time windows. But is it production-ready? Not yet - we need fault tolerance.

When Redis fails, we must fail open. Rejecting all requests during outages creates denial-of-service:

async function resilientRateLimit(userId: string) {
  try {
    return await strictRateLimit(userId);
  } catch (e) {
    metrics.increment('redis_failures');
    return true; // Fail open
  }
}

Monitor failures with metrics like:

• Redis latency
• Error rates
• Rejection percentages

For Express middleware, we inject rate checks before handlers:

app.use((req, res, next) => {
  const ip = req.headers['x-forwarded-for'] || req.socket.remoteAddress;
  const result = await rateLimiter.check(ip);
  
  if (!result.allowed) {
    res.header('Retry-After', result.retryAfter);
    return res.status(429).send('Too many requests');
  }
  
  next();
});

Include Retry-After headers - they’re crucial for good API citizenship.

Performance testing revealed optimizations:

• Pipeline batched operations
• Use EVALSHA instead of EVAL
• Set enableOfflineQueue: false in Redis config
• Local caches for frequent offenders

After implementing, our API handled 3x more traffic with zero outages. The system rejects abusive patterns while allowing legitimate bursts. What thresholds work for your use case? Experiment with different algorithms.

I’ve shared the core techniques that saved our infrastructure. If this helped you, share it with others facing similar scaling challenges. Have questions about implementation details? Let’s discuss in the comments!