# Docs for Agents

A recurring reading test of developer documentation. Three AI models attempt five first-hour developer jobs using only a product’s public docs, then vote. Scorecards, verified receipts, and the full methodology.

How agent-ready are your docs? The Docs Agent-Readability Test, exactly as it runs.

Canonical URL: https://docsforagents.com/

# Methodology

The Docs Agent-Readability Test, exactly as it runs.

## A reading test, not an execution test

Each report asks one question of one product: can the public documentation get an AI agent through five first-hour developer jobs? The panel reads the docs and attempts each job using only what the fetched pages say. No accounts are created, no API calls are executed, no code runs.

A PASS therefore means the docs got an agent to a confident, unambiguous answer. It does not mean the integration was executed live. This constraint is also the point: every product gets identical treatment regardless of pricing tier, and every report is reproducible from public pages.

## The battery

The battery is fixed across reports so grades are comparable. Changing a task bumps the battery version, and the change is disclosed in the affected report. The current battery is v1.

- **Quickstart.** From zero, reach the product’s core action, such as sending the first email or accepting a test payment. Is there one clear path from credentials to a successful call?
- **Limits.** Find the exact rate limits and plan quotas. Are they stated, and stated consistently everywhere they appear?
- **Error recovery.** Starting from a common error such as a 429, can the docs get the agent from error to fix? Is there an error reference?
- **Webhooks end to end.** Create, receive, and verify a webhook. Is the full path documented, and do the pages agree with each other?
- **SDK coverage.** Find the primary-language SDK and a minimal working example. Does the example match the API shape shown elsewhere in the docs?

Products without a webhook surface or an SDK get a substitute task chosen before any panelist runs: authentication end to end, or pagination on the main list endpoint. Substitutions are printed on the scorecard.

## The panel

Three models run the identical brief independently: GPT 5.6 Sol, Claude Opus 5, and DeepSeek v4 Flash. No panelist is told about the others, and no panelist sees another’s output. DeepSeek is deliberately a budget-tier panelist: its verdicts indicate what docs look like to the model tier companies actually deploy at scale.

The panel is exactly three, every run, so grades stay comparable. A panelist that errors out gets one retry with the identical brief, and if it still fails the run waits rather than shipping a reduced panel. Any permanent change of membership is disclosed on the first affected scorecard.

Model self-reports are unreliable, so the run record, not the model’s own claim, names the panelists. Column headers on the scorecard print the versions as invoked.

## Verdicts

- **PASS.** The agent completed the task with one clear documented path and no guessing. Minor page-hopping is fine.
- **PARTIAL.** The agent completed the task but had to guess, resolve an ambiguity, or reconcile pages that disagree. Internal contradictions are automatically at most PARTIAL.
- **FAIL.** The docs could not get the agent to a confident answer.

Panelists are instructed to grade generously. The series leads with what docs get right, and a contradiction between two live pages is reported as drift, a systems problem every docs site accumulates, not as a team being sloppy.

## The Agent-Ready Grade

The headline result is a US school letter grade. Reading points: the fifteen votes, three judges across five tasks, at PASS 2, PARTIAL 1, FAIL 0, for a maximum of 30. Readiness points: the five surface checks at 10 each, for a maximum of 50. The total out of 80 becomes a percentage, and the percentage becomes the grade, A+ at 97 and above down to F below 60.

The matrix is the breakdown: every vote appears as its own tile, and no aggregate tally is shown. Consensus tiles mark each row’s majority, play no scoring role, and a true three-way split reads SPLIT rather than pretending to be a verdict. Disagreement is data: the same pages reading three different ways is often the most useful finding in the report. Single runs are noise, and the panel, never a single run, is the unit of truth.

## Evidence and verification

Every verdict requires one verbatim evidence quote and the URL it came from, plus a log of every URL the panelist consulted. A verdict without a quote is invalid.

Before anything is published, every quote destined for a scorecard or report is independently re-fetched and confirmed verbatim on the live page. Anything that fails is dropped or replaced. Each report prints its verification dates, because pages change.

## The readiness survey

Alongside the reading test, each report records the agent-facing surface the product ships. These are verifiable facts with evidence, not judgments, so they are never put to a panel vote. Each of the five checks is worth 10 points of the grade; the docs platform is recorded as metadata and never graded.

- **Docs platform.** The tool serving the docs, identified from page markers, asset hosts, or generator tags.
- **llms.txt and llms-full.txt.** Checked by HTTP request at the docs roots. A 200 with plausible content counts.
- **Markdown mirror.** Confirmed by appending .md to a real docs page URL, or by following llms.txt links, and checking that it serves markdown.
- **MCP server.** Probed with a JSON-RPC initialize POST against candidate endpoints. A 200 on GET may be a marketing page, and a 405 on GET may still be a live server, so only a valid MCP response counts.
- **Docs AI.** An embedded assistant users can interact with on the docs site, identified from widget markers.

## Limits of the method

Strong models can infer around documentation gaps, and popular products’ docs may appear in training data, so the no-prior-knowledge instruction cannot be fully enforced. The design compensates in three ways: panelists must ground every verdict in a fetched quote, the panel spans model tiers, and disagreement is published rather than smoothed over.

The test reads documentation. It does not measure API design, SDK quality, support, or anything behind a login. A product with excellent docs and a difficult API will score well here, and that is the intended scope.

Canonical URL: https://docsforagents.com/methodology/

# Mailgun: Agent-Ready Grade C

**76.3% · 61/80 · Agent-Ready Grade · 21/30 reading points · 40/50 readiness points**

All three models passed the quickstart, the webhook pages never show the full payload, and the only FAIL vote of the run was about rate limits.

- Tested: 2026-08-05
- Verified: 2026-08-05 and 2026-08-06
- Published: 2026-08-06
- Battery: v1
- Scoring: reading 30 pts · readiness 50 pts
- Docs: https://documentation.mailgun.com

Three AI models, GPT 5.6 Sol, Claude Opus 5, and DeepSeek v4 Flash, each read Mailgun’s public documentation independently and attempted the same five first-hour developer jobs: send the first email, find the rate limits, recover from a 429, set up webhooks end to end, and use the Python SDK.

No accounts, no API calls, no code execution. A verdict means the docs got an agent to a confident answer, backed by a verbatim quote from a live page. Where the panel disagreed, the matrix shows the split.

## AI agent readiness · 40/50

| Check | Verdict | Points |
| --- | --- | --- |
| llms.txt | PASS | 10 |
| llms-full.txt | FAIL | 0 |
| Markdown mirror | PASS | 10 |
| MCP server | PASS | 10 |
| Docs AI | PASS | 10 |

## The Reading Test · 21/30

| Task | GPT 5.6 Sol | Opus 5 | DeepSeek v4F | Consensus |
| --- | --- | --- | --- | --- |
| Send the first email | PASS | PASS | PASS | PASS |
| Find the rate limits | FAIL | PARTIAL | PARTIAL | PARTIAL |
| Recover from a 429 | PASS | PARTIAL | PASS | PASS |
| Webhooks end to end | PARTIAL | PARTIAL | PARTIAL | PARTIAL |
| Use the Python path | PASS | PASS | PARTIAL | PASS |

Docs platform: Redocly (unscored) · verified 2026-08-05 and 2026-08-06

## What the docs get right

- **One quickstart, eight languages.** The sandbox quickstart walks from signup to a delivered email in five steps, shows the identical example in eight languages, and prints the exact 200 response body, so the send call needs no reconstruction.
- **A real markdown mirror.** Every page publishes a companion .md endpoint, and llms.txt indexes all 144 pages, so an agent can read the entire corpus verbatim instead of scraping a JavaScript shell.
- **One consolidated error table.** The API overview collects response codes 400 through 500 with per-code troubleshooting and defines the three X-RateLimit headers alongside them.
- **Production-grade webhook security.** Webhook verification is documented as a complete HMAC-SHA256 recipe with a runnable Node snippet and anti-replay guidance.

## Send the first email

**PASS**

The only unanimous PASS of the run. The path runs sign up, copy the private API key, authorize a recipient, send, check the inbox, and the same example appears in eight languages with the exact 200 response body.

Opus 5 logged two blemishes: a dangling cross-reference to a nonexistent Step 2B, and the page never says where to find your sandbox domain. DeepSeek hit a 404 on one quickstart URL variant, and the canonical path worked.

## Find the rate limits

**PARTIAL**

The docs document the mechanism thoroughly and the numbers almost not at all. The one concrete figure, 500 requests every 10 seconds, appears under a Rate Limits and Quotas heading nested inside the Metrics reporting page, and no page says whether it is Metrics-scoped or account-wide. No plan-quota table exists anywhere in the docs map. The only quota figures the panel found were 5 sandbox recipients and 300 emails per day for unverified domains.

GPT graded FAIL, Opus 5 and DeepSeek PARTIAL. DeepSeek added a structural note: the dedicated rate-limits page is a JavaScript-rendered OpenAPI spec that would not fetch, so its one number came from a search snippet rather than a readable page.

## Recover from a 429

**PASS**

The API overview’s response-code table covers 429 and defines X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset, which is enough to build a retry loop, and two panelists passed it.

Opus 5 graded PARTIAL for two reasons: the reset header’s definition conflates an absolute epoch timestamp with a relative duration, and the send endpoint’s own page documents only 200 and 400 responses, so an agent holding a 429 from that endpoint finds nothing about it there.

## Webhooks end to end

**PARTIAL**

All three legs exist: a create endpoint whose event list includes delivered, a delivered-event payload example, and the HMAC verification recipe. All three panelists still graded PARTIAL, for converging reasons. The security page shows the signature object and says it is posted alongside the webhook’s event data. The payloads page shows the event body and never mentions a signature. The combined POST envelope an agent must actually parse appears on no page.

Opus 5 also flagged that two pages disagree on retry intervals, and DeepSeek could not fetch the account-webhooks API reference at all, leaving the programmatic create step to guesswork.

## Use the Python path

**PASS**

The page is labelled Official Mailgun Python SDK, installs with pip install mailgun-python, links a repository that resolves, and its send example matches the REST shape of the messages endpoint.

DeepSeek graded PARTIAL: the page defers to the repository readme for full details, and the SDK’s client wrapper differs from the quickstart’s raw requests calls with nothing reconciling the two.

## The receipt

> API calls are subject to a rate limit of 500 requests every 10 seconds

The only concrete rate-limit number in the docs, nested under a Metrics reporting page. No plan-quota table exists in the docs map.

- [documentation.mailgun.com/docs/mailgun/api-reference/send/mailgun/metrics](https://documentation.mailgun.com/docs/mailgun/api-reference/send/mailgun/metrics)

## Readiness notes

Mailgun’s MCP server rejects a plain GET with a 405 and answered a JSON-RPC initialize on POST at documentation.mailgun.com/mcp. A status-code check alone would have read it as absent.

The markdown mirror is real: one panelist navigated the entire test through the llms.txt map and the .md endpoints.

## Method note

This is a reading test of public documentation, not an execution test. No accounts were created and no API calls were run. The Agent-Ready Grade counts the fifteen reading votes at PASS 2, PARTIAL 1, FAIL 0 (30 points) and the five readiness checks at 10 points each (50 points), out of 80, shown as a US school letter grade with its percentage. Consensus chips show each row’s majority and play no scoring role. The panel split on three of the five tasks, and the matrix shows every vote. Quotes shown here were re-fetched and confirmed verbatim on the dates above.

Methodology: https://docsforagents.com/methodology/

Canonical URL: https://docsforagents.com/reports/mailgun/


# Resend: Agent-Ready Grade B+

**88.8% · 71/80 · Agent-Ready Grade · 21/30 reading points · 50/50 readiness points**

The most deliberately agent-engineered docs in the pilot, with a webhook surface whose pages disagree with each other.

- Tested: 2026-08-05
- Verified: 2026-08-05 and 2026-08-06
- Published: 2026-08-06
- Battery: v1
- Scoring: reading 30 pts · readiness 50 pts
- Docs: https://resend.com/docs

Three AI models, GPT 5.6 Sol, Claude Opus 5, and DeepSeek v4 Flash, each read Resend’s public documentation independently and attempted the same five first-hour developer jobs: send the first email, find the rate limits, recover from a 429, set up webhooks end to end, and use the Python SDK.

No accounts, no API calls, no code execution. A verdict means the docs got an agent to a confident answer, backed by a verbatim quote from a live page. Where the panel disagreed, the matrix shows the split.

## AI agent readiness · 50/50

| Check | Verdict | Points |
| --- | --- | --- |
| llms.txt | PASS | 10 |
| llms-full.txt | PASS | 10 |
| Markdown mirror | PASS | 10 |
| MCP server | PASS | 10 |
| Docs AI | PASS | 10 |

## The Reading Test · 21/30

| Task | GPT 5.6 Sol | Opus 5 | DeepSeek v4F | Consensus |
| --- | --- | --- | --- | --- |
| Send the first email | PARTIAL | PARTIAL | PASS | PARTIAL |
| Find the rate limits | FAIL | PASS | PARTIAL | SPLIT |
| Recover from a 429 | PARTIAL | PASS | PASS | PASS |
| Webhooks end to end | PARTIAL | PARTIAL | PARTIAL | PARTIAL |
| Use the Python SDK | PASS | PASS | PASS | PASS |

Docs platform: Mintlify (unscored) · verified 2026-08-05 and 2026-08-06

## What the docs get right

- **Markdown everywhere.** Every docs page is served as raw markdown at the same URL plus .md, alongside llms.txt and llms-full.txt indexes, so an agent reads authored source text instead of scraping rendered HTML.
- **A 429 that explains itself.** A dedicated errors reference lists every error code with its status, message, and suggested action, and splits 429 into rate_limit_exceeded, daily_quota_exceeded, and monthly_quota_exceeded, each with its own remedy.
- **Consistent numbers.** The 10 requests per second per team limit is stated identically on three separate pages.
- **Agents as first-class users.** The MCP server ships a submit_feedback tool that invites agents to report documentation problems.

## Send the first email

**PARTIAL**

The install, key, send sequence is short and copyable. The split came from the prerequisites: the quickstart lists a verified domain as a requirement while its runnable example sends from onboarding@resend.dev, and no page states that the resend.dev test addresses are exempt from that restriction. An agent cannot tell whether the copy-paste example succeeds on a fresh account.

DeepSeek read the same pages as one unambiguous path and passed it.

## Find the rate limits

**SPLIT**

The pilot’s only three-way split: the same pages drew FAIL from GPT, PASS from Opus 5, and PARTIAL from DeepSeek. The per-second limit is consistent everywhere it appears. The disagreement is about plan quotas: the docs say paid tiers have monthly quotas dictated by plan tier, and the panel split on whether the numbers themselves are stated in the docs or only on the pricing page.

A true three-way split gets no consensus verdict. The chip reads SPLIT, and all three votes stand in the tally.

## Recover from a 429

**PASS**

The errors reference gives each 429 subtype its own suggested action, and the rate-limit page documents the response headers, Retry-After, and backoff guidance, so the fix is mechanical.

GPT’s PARTIAL: the API introduction describes every 429 as a rate-limit failure, while the error reference splits it three ways.

## Webhooks end to end

**PARTIAL**

Every piece exists: creation returns the signing secret, the event payload pages agree with each other, and a verification page documents two verification paths. The five-step receiving guide never mentions signature verification or links to the page that explains it.

The verification page warns: “Some frameworks parse the request as JSON and then stringify it, and this will also break the signature verification.” The receiving introduction demonstrates verification with JSON.stringify(req.body).

The verify call itself is named inconsistently. The parameter for the signing secret is webhookSecret on the verification page and secret on the Agent Email Inbox page, Resend’s own page for AI agents.

## Use the Python SDK

**PASS**

Unanimous. The Python SDK is listed under Official SDKs, and the send example is character-for-character the same call and parameter shape as the API reference, whose keys match the documented REST body parameters exactly.

## The receipt

> The same verify call takes webhookSecret: on the verification guide and secret: on the Agent Email Inbox skill page.

Two live pages give the same parameter two different names, and the second page is Resend’s own guide for AI agents.

- [resend.com/docs/webhooks/verify-webhooks-requests](https://resend.com/docs/webhooks/verify-webhooks-requests)
- [resend.com/docs/agent-email-inbox-skill](https://resend.com/docs/agent-email-inbox-skill)

## Readiness notes

The only clean sweep of the survey in the pilot: all five surfaces are present.

resend.com/mcp is a page about MCP. The live server answers JSON-RPC at resend.com/docs/mcp, identifies itself as Resend 1.0.0, and ships the submit_feedback tool.

## Method note

This is a reading test of public documentation, not an execution test. No accounts were created and no API calls were run. The Agent-Ready Grade counts the fifteen reading votes at PASS 2, PARTIAL 1, FAIL 0 (30 points) and the five readiness checks at 10 points each (50 points), out of 80, shown as a US school letter grade with its percentage. Consensus chips show each row’s majority and play no scoring role, and a true three-way split reads SPLIT. The panel split on three of the five tasks, and the matrix shows every vote. Quotes shown here were re-fetched and confirmed verbatim on the dates above.

Methodology: https://docsforagents.com/methodology/

Canonical URL: https://docsforagents.com/reports/resend/


# Stripe: Agent-Ready Grade C+

**78.8% · 63/80 · Agent-Ready Grade · 23/30 reading points · 40/50 readiness points**

Stripe now writes documentation to the AI agent directly. The panel measured what that steering does to the classic integration path.

- Tested: 2026-08-05
- Verified: 2026-08-05 and 2026-08-06
- Published: 2026-08-06
- Battery: v1
- Scoring: reading 30 pts · readiness 50 pts
- Docs: https://docs.stripe.com

Three AI models, GPT 5.6 Sol, Claude Opus 5, and DeepSeek v4 Flash, each read Stripe’s public documentation independently and attempted five first-hour developer jobs: accept a test payment, find the rate limits, recover from a 429, set up webhooks end to end, and use the Python SDK.

No accounts, no API calls, no code execution. A verdict means the docs got an agent to a confident answer, backed by a verbatim quote from a live page. Where the panel disagreed, the matrix shows the split.

## AI agent readiness · 40/50

| Check | Verdict | Points |
| --- | --- | --- |
| llms.txt | PASS | 10 |
| llms-full.txt | FAIL | 0 |
| Markdown mirror | PASS | 10 |
| MCP server | PASS | 10 |
| Docs AI | PASS | 10 |

## The Reading Test · 23/30

| Task | GPT 5.6 Sol | Opus 5 | DeepSeek v4F | Consensus |
| --- | --- | --- | --- | --- |
| Accept a test payment | PARTIAL | PARTIAL | PASS | PARTIAL |
| Find the rate limits | PASS | PARTIAL | PASS | PASS |
| Recover from a 429 | PASS | PASS | PASS | PASS |
| Webhooks end to end | PARTIAL | PARTIAL | PASS | PARTIAL |
| Use the Python SDK | PARTIAL | PARTIAL | PASS | PARTIAL |

Docs platform: custom in-house (unscored) · verified 2026-08-05 and 2026-08-06

## What the docs get right

- **The most specific rate-limit page in the pilot.** Live and sandbox numbers, per-endpoint and concurrency limits, read-request allocations, and a header taxonomy that says which limit was hit.
- **429 recovery with no dead ends.** Four pages chain cleanly from the status table to a typed stripe.RateLimitError with remediation, and a 429 without the rate-limit header is distinguished as a lock timeout.
- **Markdown mirrors of every page.** Each page serves a clean .md variant, so tabbed code samples survive fetching.
- **Test keys in every sample.** Sample test API keys are embedded in the examples, so an agent can check a call shape immediately.

## Accept a test payment

**PARTIAL**

Credentials are unambiguous. The friction is where the docs steer: the Python quickstart’s first SDK request creates a Product and a Price rather than a payment, and reaching an actual test payment means assembling create and confirm steps from separate pages. The Payment Intents variant of accept-a-payment opens by telling the reading agent not to use that API.

DeepSeek followed the embedded test keys through one path and passed it.

## Find the rate limits

**PASS**

The dedicated page states exact numbers: 100 requests per second in live mode, 25 in sandbox, per-endpoint and resource-specific limits, and read-request allocations, consistent across the pages that reference them.

Opus 5 graded PARTIAL because the canonical table omits limits that exist only on product pages, so an agent reading the canonical page infers a lower limit than some endpoints actually have.

## Recover from a 429

**PASS**

Unanimous. The chain runs from the API status table to the rate_limit error code to the typed exception with remediation, and the Stripe-Rate-Limited-Reason header decodes which limit fired. A 429 without that header is identified as a lock timeout, a distinction none of the other pilot products draw.

## Webhooks end to end

**PARTIAL**

The main guide is unusually complete on one page: handler code, stripe listen forwarding, stripe trigger for the success event, endpoint registration, and both library and manual HMAC verification. Every page agrees the success event is payment_intent.succeeded.

Opus 5’s PARTIAL: the Python samples disagree on the verification API, catching stripe.error.SignatureVerificationError in the quickstart while the error-handling reference documents stripe.SignatureVerificationError. GPT’s PARTIAL traces to the payment-events guide, which labels signature checking optional and opens with a handler example that parses the event without verifying it.

## Use the Python SDK

**PARTIAL**

The SDK is trivially findable, with a pinned version and a repository that resolves. No single Python page shows a minimal payment: the quickstart creates a Product and a Price, and the payment call has to come from the API reference, which uses a different call signature for the same client.

DeepSeek passed it on the strength of the SDK pattern matching the REST shape.

## The receipt

> Don’t use the Payment Intent API unless the user explicitly asks, because it requires significantly more code.

An instruction addressed to the AI agent reading the page, on the Payment Intents variant of accept-a-payment.

- [docs.stripe.com/payments/accept-a-payment](https://docs.stripe.com/payments/accept-a-payment)

## Readiness notes

The MCP server at mcp.stripe.com is live and auth-gated, answering unauthenticated requests with a 401.

Docs pages open with agent-facing setup blocks, including the lines “Integrate with Stripe using skills and plugins” and “run stripe agent setup”. The quickstart PARTIALs trace to this steering layer making the classic path ambiguous.

## Method note

This is a reading test of public documentation, not an execution test. No accounts were created and no API calls were run. The Agent-Ready Grade counts the fifteen reading votes at PASS 2, PARTIAL 1, FAIL 0 (30 points) and the five readiness checks at 10 points each (50 points), out of 80, shown as a US school letter grade with its percentage. Consensus chips show each row’s majority and play no scoring role. The panel split on four of the five tasks, and the matrix shows every vote, including a judge spread from five PASS votes to one. Quotes shown here were re-fetched and confirmed verbatim on the dates above.

Methodology: https://docsforagents.com/methodology/

Canonical URL: https://docsforagents.com/reports/stripe/


Content may be quoted with attribution and a link to https://docsforagents.com/.