explainer · 2026-09-26

How to document API errors so AI agents can recover

The error pages that passed give the wait, the backoff, the stop, and the code.

An AI agent calls your API, receives an HTTP 429 response, and turns to your documentation. The pages are its only guide. If they say what to do next, the agent follows them. If they stop at naming the error, the agent is on its own.

Pages that get an agent through a 429 name the limit that was hit, the wait, the backoff schedule, the stop condition, and code. Pages that stop at naming the error leave the agent to guess.

How the test scored recovery from a 429

We ran the AI Agent Readiness Test on 93 products from August 5 to September 22, 2026. One of the five jobs for each product was to recover from an error. For 48 of the 93 products, that job was recovering from an HTTP 429 response, the code a server returns when the client has sent too many requests. The job had 558 points available across the 93 products. The models lost 151 of those points, the second most of any job after finding the limits. Sixty-four of the 93 products lost at least one point on it. Across the 93 products, the votes came in at 52 PASS, 35 PARTIAL, and 6 FAIL.

A PASS means the models found one clear documented path and did no guessing. A PARTIAL means a model guessed, pieced the answer together from several pages, or chose between pages that disagree. A FAIL means the pages gave no confident answer.

Three models took each job: GPT 5.6 Sol, Claude Opus 5, and DeepSeek v4 Flash. Each worked alone with the same brief and no shared context. Each votes PASS, PARTIAL, or FAIL on the job, and every vote carries a verbatim quote from a page and that page’s URL. The same gap is worse for an agent than for a person. A person watches the dashboard, asks support, and tries a delay to see what happens. An agent has only the pages.

What the pages that passed contain

Notion names the field in the response that says which limit was hit. It defines Retry-After as a whole number of seconds. Retry-After is the response header that tells the client how long to wait before it retries. The page gives a four-step recovery procedure with backoff and jitter. Backoff waits longer between each retry. Jitter adds a small random change to each wait. The procedure caps the retries. Working code in JavaScript, Python, Go, and Java sits beside the steps.

Firecrawl gives the two 429 conditions separate rows on its Errors page. When the rate limit is exceeded, the client backs off and retries after the Retry-After value. When the concurrency limit is reached, the client waits for running jobs, lowers concurrency, or upgrades. A runnable backoff snippet in Python, Node, and cURL follows the rows.

WorkOS puts the recovery one link away. The 429 row in its error table links to the Rate limits page. That page gives the retry signal, what to do when the Retry-After header is absent, the backoff rule, and the fact that limits count per API key. The Node SDK page documents automatic retries with idempotency keys, which let the server recognize a repeated request.

n8n names the 429 symptom on one page and gives two fixes with every click spelled out. The first is Retry On Fail with Max Tries and Wait Between Tries. The second is a Loop Over Items cycle with a Wait node.

Together the passing pages answer the same questions: which limit was hit, how long to wait, how to back off, and when to stop. Then they give code. An agent that reads them does not invent any of it.

Where the other pages fall short

Each shortfall leaves at least one of the five parts out.

The first pattern names the error and stops. E2B documents a RateLimitError reference that names the cause, but no page gives the limit that triggers it, the retry timing, or backoff. CARTO has a rate limits page that says what triggers a 429, but gives no block duration, no retry timing, and no safe retry steps. The agent learns the error exists and nothing about recovering from it.

The second pattern gives a vague delay. DeepInfra says to retry after a short delay. The page gives no delay length, no backoff schedule, and no documented Retry-After header.

The third pattern omits the 429. PostHog lists the responses 200, 400, 401, and 503 in its API overview, and never 429. Massive recommends staying below 100 requests per second, and no page under /docs/ mentions HTTP 429, Retry-After, retry timing, or backoff. An agent that reads these pages finds no procedure to follow when the limit is hit.

The fourth pattern disagrees with itself. Orkes documents the cause of the condition and the queue-size default, but gives no backoff, no Retry-After, and no retry ceiling. Its architecture page says the same condition can return HTTP 400. Under our rules, two pages that disagree cap a job at PARTIAL.

Where this advice stops

The test reads the public documentation pages. It creates no accounts, makes no API calls, and runs no code. A page can therefore document a Retry-After header that the server never sends, and the test cannot catch it.

Three models are a small panel. Each vote in each report prints the quote and the page URL it came from, so you can check the model’s reading against the page yourself. The score appears as a percentage and a letter grade in our grades table.

What to put next to each error

The five parts belong next to the error entry, where the agent reads about it. Name which limit was hit and how the client can tell. Give the wait, using the Retry-After header when your API sends it. Give the backoff rule with jitter and a cap on retries, so the client knows when to stop. Give code in the languages your users call from. When one error code has two causes, give each cause its own row, the way Firecrawl does. When another page describes the same condition differently, fix that page before an agent has to choose between them.

Each product’s error recovery finding, with the quote and URL behind every vote, is printed in the reports.

Put another docs site through the battery.

Nominate a docs site