Reliability and error handling

Two rules carry most of this page:

Branch on error.code

Not on the HTTP status. Three different codes share 429, and one of them can never succeed no matter how long you wait.

Quote x-request-id

On every response, success or failure. It is the only handle that reaches the whole path your request took.

The error shape

{
  "error": {
    "message": "Rate limit exceeded for this account. Retry after 30s.",
    "type": "rate_limit_error",
    "code": "TENANT_RATE_LIMITED",
    "param": null
  }
}

code is the stable part. Messages get reworded; codes are append-only and a retired one is never reused. /v1/messages returns Anthropic’s error shape instead, with the same code inside it.

What is worth retrying

of the codes are retryable — meaning the identical request may succeed later with nothing changed on your side:

Status	Code	What to do
400	`MEDIA_FETCH_FAILED`	Retry with backoff.
429	`TENANT_RATE_LIMITED`	Wait for `Retry-After`, then retry.
429	`UPSTREAM_RATE_LIMITED`	Wait for `Retry-After`, then retry.
500	`INTERNAL`	Retry with backoff.
503	`DEPLOYMENT_SCALING_UP`	Wait for `Retry-After`, then retry.
503	`STORAGE_UNAVAILABLE`	Retry with backoff.
503	`UPSTREAM_UNAVAILABLE`	Retry with backoff.
504	`REQUEST_DEADLINE_EXCEEDED`	Retry with backoff.
504	`UPSTREAM_TIMEOUT`	Retry with backoff.

Everything else needs something to change first: the request, the model, the account’s balance, or a permission. Retrying those unchanged burns your rate limit to arrive at the same answer.

TENANT_BUDGET_EXCEEDED is a 429 and is not retryable. It means the account hit its spending limit. A client that branches on the status alone will retry it forever and never succeed — which is exactly the loop that makes a quiet guardrail look like an outage.

Backing off

Where the message quotes a wait, Retry-After carries it as a header. Honour it — it is the platform telling you what it knows, which beats a fixed guess.

import time
import openai

RETRYABLE = {
    "TENANT_RATE_LIMITED", "UPSTREAM_RATE_LIMITED", "UPSTREAM_UNAVAILABLE",
    "UPSTREAM_TIMEOUT", "DEPLOYMENT_SCALING_UP", "REQUEST_DEADLINE_EXCEEDED",
    "INTERNAL", "STORAGE_UNAVAILABLE", "MEDIA_FETCH_FAILED",
}

for attempt in range(5):
    try:
        resp = client.chat.completions.create(...)
        break
    except openai.APIStatusError as e:
        code = (e.body or {}).get("error", {}).get("code")
        if code not in RETRYABLE:
            raise                                   # retrying cannot help
        wait = float(e.response.headers.get("Retry-After", 2 ** attempt))
        time.sleep(wait + random.uniform(0, 0.5))   # jitter, or you retry in lockstep

Without Retry-After, exponential backoff with jitter is the right fallback. Retrying immediately, in a loop, across every worker you have, turns a brief upstream wobble into a sustained one.

Timeouts

Set a client timeout longer than your max_tokens can plausibly take. A generation capped at 2000 tokens is not a two-second request, and a client that gives up at five seconds produces REQUEST_CANCELLED — one of the two codes that is billed, because the model was already working. Streaming changes this calculus: the first token arrives in a fraction of the total, so a streamed call can hold a much tighter first-byte timeout than a buffered one.

Idempotency, honestly

Idempotency-Key is accepted on every endpoint and echoed back. De-duplication of replays is not implemented yet, so today a retry with the same key is a second call and a second charge. It is in the contract now so that your client can start sending it and not change later. Until de-duplication ships, make retries safe on your side — which for inference usually means tolerating a duplicate answer rather than preventing one.

What failures cost

of the codes are free. The two that are not are both streams that came apart after the model had produced output — see Streaming. The full table, with the billing column, is on Errors.

Rate limits and quotas

The three 429s and the money-shaped stops.

Routing and providers

Why a retry may be served by something else entirely.

Get started

Guides

Platform

Reference

Reliability and error handling

Branch on error.code

Quote x-request-id

The error shape

What is worth retrying

Backing off

Timeouts

Idempotency, honestly

What failures cost

Next

Rate limits and quotas

Routing and providers

Branch on error.code

Quote x-request-id

​The error shape

​What is worth retrying

​Backing off

​Timeouts

​Idempotency, honestly

​What failures cost

​Next

Rate limits and quotas

Routing and providers

The error shape

What is worth retrying

Backing off

Timeouts

Idempotency, honestly

What failures cost

Next