Beyond setTimeout: Building Robust Retry Logic for LLM API Calls in Node.js

[chawkian]

If you've ever built a production app that calls LLM APIs, you know the pain: rate limits, transient 500s, and the dreaded 60-second timeout. A simple try/catch with a setTimeout retry isn't going to cut it.

The Problem with Naive Retries

// Don't do this in production
async function callLLM(prompt) {
  try {
    return await client.complete(prompt);
  } catch (e) {
    setTimeout(() => callLLM(prompt), 1000); // fires into the void
  }
}

This has zero backoff, no jitter, no max retries, and silently swallows the eventual failure.

What Good Retry Logic Looks Like

Here's a pattern we recommend for Node.js SDK consumers:

async function callWithRetry(fn, {
  maxRetries = 3,
  baseDelay = 500,
  maxDelay = 10000
} = {}) {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      return await fn();
    } catch (err) {
      const isLast = attempt === maxRetries;
      const retryable = err.status === 429 || err.status >= 500;
      if (isLast || !retryable) throw err;

      // Exponential backoff with jitter
      const delay = Math.min(
        baseDelay * 2 ** attempt + Math.random() * 250,
        maxDelay
      );
      await new Promise(r => setTimeout(r, delay));
    }
  }
}

Key Principles

1. Only retry idempotent or safely-repeatable calls — LLM completions are usually fine
2. Respect Retry-After headers — rate limit responses include this
3. Add jitter — synchronized retries from multiple clients make overload worse
4. Set a total timeout budget — don't let a single call eat 5 minutes

The AIWave Node SDK handles most of this automatically with built-in exponential backoff, but understanding the underlying logic helps when you need to customize for edge cases.

Open Question

Some teams are moving toward circuit breaker patterns — tripping after N consecutive failures to fail fast rather than queuing retries. Has anyone implemented circuit breakers for LLM API calls in Node? Curious whether it's overkill or genuinely useful for high-throughput scenarios.

---

Resources

• AIWave pricing
• AIWave developer blog
• AIWave Node.js SDK