explainer · 2026-10-12
Agent-ready developer docs: what to test
Website standards, documentation reading, and integration execution each test a different part of readiness.
An agent can fetch every page of a documentation site and still fail the job it was given. It retrieves a setup guide, finds two install commands that disagree, sees no stated rate limit, and answers from the mismatch while the log records successful requests. Agent-ready developer docs are documentation that answers a particular developer job for the agent reading it: the pages are retrievable, the representation the agent consumes carries the content, the instructions for that job are complete and unambiguous, and a separate authorized check confirms the documented path against the live product. Each part is tested separately, and each can fail alone. Readiness is claimed for a particular job, so documentation can stay behind reader authentication or publish one interface and still be ready for the work it promises.
What a standards scan checks
Cloudflare’s Agent Readiness guidance scores four categories: Discoverability, Content, Bot Access Control, and Capabilities. Discoverability covers robots.txt, sitemap.xml, and Link response headers. Content covers machine-readable page forms, including Markdown content negotiation. Bot Access Control covers signals such as Content Signals and AI bot rules, with Web Bot Auth authenticating clients. Capabilities covers interfaces an agent can call, including Agent Skills, Model Context Protocol (MCP) Server Cards, and WebMCP.
A scan of these conventions reports what a site advertises and which interfaces it publishes. It does not guarantee that a particular client reaches a particular page, and it does not grant access to protected content. A site can follow every convention in the framework and still bury the sentence that defines a required header.
Cloudflare’s Markdown for Agents page describes a conversion path. When a zone has the feature enabled, a client that sends Accept: text/markdown receives Content-Type: text/markdown, and Vary includes Accept, so caches keep the HTML and Markdown variants separate. The practical check is to request the Markdown version of an important page and confirm that the instructions, tables, code examples, and warnings it carries survived the conversion. Conversion cannot supply a fact that the original page never stated, so a converted page that drops the only statement of a limit loses more than formatting.
Cloudflare’s documentation for agents adds site-wide and per-product llms.txt indexes, llms-full.txt content bundles, scoped indexes that keep product context separate, and OpenAPI specifications that describe API endpoints, parameters, and responses. These are usable interfaces for different jobs, and an owner selects the ones that support the product’s actual workflow. An API catalog, an MCP server, or a Markdown mirror can be the right surface for one job and irrelevant to the next.
What the reading test adds
The AI Agent Readiness Test is the program our series runs, and it reports the AI Agent Readiness Score, a separate metric whose arithmetic is on the score page. The test uses a fixed panel of three models: GPT 5.6 Sol, Claude Opus 5, and DeepSeek v4 Flash. Each model runs independently against public documentation only and works through five product-specific first-hour jobs: a first working result, exact limits, error recovery, a webhook or an authentication path, and the primary software development kit. Each job produces a vote, and fifteen votes give a reading score out of 30. Five agent-surface checks at 10 points each give 50 surface points: llms.txt, llms-full.txt, a Markdown mirror, an MCP server, and embedded documentation AI. The total is 80 points.
The test creates no accounts, makes no API calls, and runs no integration code. A PASS means the public documentation supplied a confident, unambiguous answer to that job. It is an assessment of the page by a model, not proof of correctness, and not evidence that anything executed. Published reports quote a source and a URL behind each verdict, so disagreement inside the panel stays visible.
The released corpus snapshot retrieved on October 12, 2026 holds 97 product reports. Ninety-five list an llms.txt file, 91 publish a Markdown mirror, and 42 carry all five surface checks. No report reaches a perfect reading score, and the highest is 28 out of 30. Each report carries its own test date, so October 12 is when the snapshot was pulled, not when every product was retested. The set is a sample of products someone chose to test, not an estimate of all developer documentation. The full list is at grades.
The Mem0 report, with its reading test and quotes verified on August 20 and its agent surface rechecked on September 21, 2026, records a complete agent surface at 50 out of 50 and a reading score of 9 out of 30, for a total of 59 out of 80, 73.8 percent, a grade of C. For writing fixes that close that gap, our earlier piece on writing documentation for AI agents draws on a 93-product analysis.
What a readiness score cannot prove
The strongest objection to a readiness score is that models miss information present on a page and sometimes infer answers the page does not support. Published verdicts quote their sources for that reason. A fixed panel of three models cannot show that every agent, or a later model release, reads a page the same way. It cannot show real-world integration success either, because the reading test executes nothing. The corpus has limits: ninety-seven reports are products someone chose to test, not a survey of the industry, each carries a historical test date, and the pages may have changed since. We do not treat every new protocol, or all five surface checks, as a prerequisite for documentation that works for agents.
Combining access, reading, and execution
| Method | What it establishes | What it leaves unmeasured |
|---|---|---|
| Website standards scan | That the site publishes its discovery, access, and interaction conventions | Whether a given client can load a given page, and whether the prose answers a job |
| Docs for Agents reading and surface test | Whether public documentation gives confident answers to the five jobs, and which agent surfaces exist | Whether the product works when called, and anything inside private or authenticated scopes |
| Authorized integration execution in a sandbox | Whether the documented path succeeds against the live product | Documentation coverage across the site, and standing readiness between releases |
Run the three methods and keep the findings separate. Exercise representative developer jobs through the actual representation an agent consumes: if agents read the Markdown mirror, test the mirror; if they call an API through an MCP server, test that path. Keep reader authentication in place. Public discoverability does not authorize an agent to reach private content, and removing a login to satisfy a public scanner gives up real protection without improving readiness.
Four observations belong in separate records: whether the HTTP request succeeded, whether the returned text was readable, whether the instructions were unambiguous, and whether an authorized execution of the documented path succeeded against the live product. Merging two of them into one verdict hides the layer that failed. A standards scanner, a task reading test, and an authorized sandbox run are complementary, and no score from one layer proves the others.
Choose the interface that matches the workflow: an API catalog serves a different job than a command-line quickstart. Documentation and versions change, so a report describes the pages on its test date. After a release that touches a documented path, repeat the affected checks and record the new date.
Owners who separate these checks can fix the layer that failed and retest it after the next release. The published evidence sits in the reports index at reports.
Put another docs site through the battery.
Nominate a docs site