Rate Limits, Retries and Exponential Backoff: A Practical Guide

Every serious API limits how fast you can call it. Hit the limit and you'll get 429 Too Many Requests; hit a bad moment and you'll see 503 or a timeout. How your client reacts decides whether users see a hiccup or an outage.

How rate limits are expressed

• Requests per second/minute, often per API key or per IP.
• Daily or monthly quotas tied to your plan.
• Concurrency limits: how many requests may be in flight at once.
• Token or cost-based limits, common for AI APIs (tokens per minute).

Look for response headers such as Retry-After, RateLimit-Remaining/RateLimit-Reset or provider-specific X-RateLimit-* headers, and use them instead of guessing.

What to retry

Only retry idempotent operations automatically (GET, PUT, DELETE), or POSTs that carry an idempotency key the API supports. Otherwise a retry might charge a card twice.

Exponential backoff with jitter

Wait longer after each failure (for example 0.5s, 1s, 2s, 4s…) up to a cap, and add randomness ("jitter") so thousands of clients don't retry in sync and knock the service over again.

JavaScript

async function withRetry(fn, { tries = 5, base = 500, cap = 8000 } = {}) {
  for (let i = 0; ; i++) {
    const res = await fn().catch((e) => ({ status: 0, error: e }));
    const retryable = res.status === 0 || res.status === 429 || res.status >= 500;
    if (!retryable || i === tries - 1) return res;
    const after = Number(res.headers?.get?.('retry-after'));
    const wait = after ? after * 1000 : Math.random() * Math.min(cap, base * 2 ** i);
    await new Promise((r) => setTimeout(r, wait));
  }
}

Python

import random, time, requests

def with_retry(call, tries=5, base=0.5, cap=8):
    for i in range(tries):
        try:
            r = call()
            if r.status_code != 429 and r.status_code < 500:
                return r
            wait = float(r.headers.get("Retry-After", 0)) or random.uniform(0, min(cap, base * 2 ** i))
        except requests.RequestException:
            if i == tries - 1:
                raise
            wait = random.uniform(0, min(cap, base * 2 ** i))
        time.sleep(wait)
    return r

Beyond retries

• Throttle proactively with a client-side token bucket so you rarely hit 429 at all.
• Cache responses you're allowed to store.
• Batch requests if the API offers batch endpoints.
• Use a circuit breaker: after repeated failures, stop calling for a short period and serve a fallback.
• Set timeouts on every request. No timeout means a hung request can tie up your workers indefinitely.

Reference: HTTP Status Codes.