Rate limits and idempotency

Key points

• A rate limit protects everyone else from one noisy client - a runaway script, a bad retry loop, or an attacker.
• Token bucket is the usual pick: steady refill, burst up to capacity. Fixed windows let a client double the rate across the boundary.
• Limit per API key or user, not per IP alone, and keep the counter in Redis so every instance shares it.
• Reject with 429 and a Retry-After header. A limit clients cannot see is a limit they will keep hammering.
• Retries cause duplicates. Accept a client Idempotency-Key, store it under a unique constraint, and replay the first response.
• Better still, design the operation to be naturally idempotent: set the balance to X can be repeated safely, add 5 cannot.

Example

// Token bucket: steady refill, bounded burst. One instance; use Redis to share it.
final class TokenBucket {

    private final double capacity;
    private final double refillPerSecond;
    private double tokens;
    private long lastNanos = System.nanoTime();

    TokenBucket(double capacity, double refillPerSecond) {
        this.capacity = capacity;
        this.refillPerSecond = refillPerSecond;
        this.tokens = capacity;
    }

    synchronized boolean tryConsume() {
        long now = System.nanoTime();
        double refill = (now - lastNanos) / 1_000_000_000.0 * refillPerSecond;
        tokens = Math.min(capacity, tokens + refill);
        lastNanos = now;
        if (tokens < 1.0) return false;             // caller answers 429 + Retry-After
        tokens -= 1.0;
        return true;
    }
}

// Idempotent writes: the database, not the application, guarantees "exactly once".
// CREATE TABLE payment_request (
//   idempotency_key TEXT PRIMARY KEY,
//   response_body   JSONB NOT NULL,
//   created_at      TIMESTAMPTZ NOT NULL DEFAULT now()
// );
// Insert first; on a duplicate key, return the stored response instead of charging again.

Rate limit at the edge, and make every retryable write idempotent by key.