Published 2026-08-20 · Tested 2026-08-19

Firecrawl

B-

Firecrawl received 10 PASS votes and passed 4 of five agent surface checks. The clearest finding came from the verify a crawl webhook task.

Panel: GPT 5.6 Sol, Opus 5, DeepSeek v4F Battery: v1 Read as markdown (opens in a new tab)

Three AI models, GPT 5.6 Sol, Claude Opus 5, and DeepSeek v4 Flash, each read Firecrawl’s public documentation independently and attempted five first-hour developer jobs: scrape a page, find the exact limits, recover from a 429, verify a crawl webhook, use the Python SDK.

No accounts, API calls, or code execution were used. Every verdict came from public pages and every published quotation passed a live verification check.

Freshness

How rechecks work
Tested
Quotes verified
Surface rechecked

No change since the test.

5 of 5 quoted passages still appear on the live pages.

Firecrawl Mintlify · published
B-

81.2% · 65/80 · AI Agent Readiness Score · reading 30 pts · surface 50 pts

llms.txt PASS
llms-full.txt PASS
markdown mirror PASS
MCP server PASS
docs AI FAIL
Task GPT 5.6 SolOpus 5DeepSeek v4F Consensus
Scrape a page PARTIALPARTIALPASS PARTIAL
Find the exact limits PASSPARTIALPASS PASS
Recover from a 429 PASSPASSPASS PASS
Verify a crawl webhook PARTIALPASSPARTIAL PARTIAL
Use the Python SDK PASSPASSPASS PASS

docs platform: Mintlify (unscored) · verified 2026-08-19

What to fix first

These 3 fixes could add up to 14 points to the AI Agent Readiness Score. The list ranks each fix by the points it would add. How the ranking works

  1. 1
    +10 points Docs AI check failed

    Found: The live docs expose no Ask AI or assistant control.

    Fix: Add an assistant to the docs site that answers questions from the docs and links to its sources.

  2. 2
    +2 points Scrape a page PARTIAL

    Found: The v2 API reference requires a key on every request, while the Introduction and Scrape pages show a keyless scrape.

    Fix: State on the v2 API reference which requests work without a key, matching the Scrape and Rate Limits pages.

    Evidence: docs.firecrawl.dev/api-reference/v2-introduction (opens in a new tab)

  3. 3
    +2 points Verify a crawl webhook PARTIAL

    Found: GPT found that Webhooks Overview omits crawl.failed, which the Crawl page lists. DeepSeek found that the Webhooks feature page never mentions or links signature verification.

    Fix: List the same crawl events on Webhooks Overview and Crawl, and link the Webhooks feature page to the security page.

    Evidence: docs.firecrawl.dev/features/crawl (opens in a new tab)

What the docs get right

  • Recover from a 429: 3 PASS votes. The Errors page distinguishes both 429 messages, gives a remedy for each, explains Retry-After, and includes backoff code.
  • Use the Python SDK: 3 PASS votes. The Python quickstart installs firecrawl-py, constructs Firecrawl, and calls app.scrape(), matching the v2 API reference's SDK form.
  • 4 of 5 agent surface checks. Present: llms.txt, llms-full.txt, markdown mirrors, an MCP server. Missing: docs AI.

Scrape a page

PARTIAL

PARTIAL consensus from 1 PASS, 2 PARTIAL.

The API introduction requires bearer authentication, while the product introduction and Scrape guide document the same v2 scrape request without a key. The authenticated path is complete and unambiguous (base URL https://api.firecrawl.dev, Bearer fc- key, POST /v2/scrape, formats ["markdown", "html"]), but the API reference states a key is required on every request while the Scrape feature page, the Python SDK page, the Build with AI page, and the Rate Limits page all document a working keyless REST scrape. The main page offers a keyless cURL example immediately; adding an API key for higher limits is clearly explained one sentence later.

Find the exact limits

PASS

PASS consensus from 2 PASS, 1 PARTIAL.

Billing and Rate Limits provide monthly credits, endpoint RPM, concurrent-browser caps, queue caps, and Browser Sandbox RPM with matching plan figures. Per-plan requests per minute, concurrent browsers, max queued jobs, and browser sandbox limits are all exact and the concurrency figures agree between Rate Limits and Billing, but the keyless daily request and credit caps are described without any numbers and the keyless MCP page points back to that same page for the value. A single page lists every plan's per-endpoint RPM, concurrent browsers, and max queued jobs in two readable tables with no contradictions across pages.

Recover from a 429

PASS

PASS consensus from 3 PASS.

The Errors page distinguishes both 429 messages, gives a remedy for each, explains Retry-After, and includes backoff code. The error catalog gives both 429 conditions separate rows with distinct causes and remedies (Rate limit exceeded says back off and retry after Retry-After; Concurrency limit reached says wait for in-flight jobs, lower concurrency, or upgrade), plus a runnable backoff snippet in Python, Node, and cURL. Both 429 conditions (rate-limit exceeded and concurrency limit reached) are documented with distinct error messages and remedies, and the errors page includes a copy-pasteable exponential-backoff snippet.

Verify a crawl webhook

PARTIAL

PARTIAL consensus from 1 PASS, 2 PARTIAL.

The configuration, payload, and HMAC steps are complete, but Overview omits failed from Crawl while the Crawl page lists crawl.failed. The four webhook pages cover the full loop end to end: the webhook object fields with a crawl cURL example, the crawl.started/crawl.page/crawl.completed payloads, the sha256= prefixed X-Firecrawl-Signature with raw-body HMAC verification in Node and Python using timing-safe comparison, where the secret lives, the 10-second timeout with a 1/5/15 minute retry ladder, and a troubleshooting page naming the parsed-body mistake. Webhook configuration and lifecycle events are fully documented on the main webhooks page, but that page contains no link to the security page, no mention of X-Firecrawl-Signature, and no mention of HMAC verification, forcing an agent to discover the security page through search or prior knowledge.

Use the Python SDK

PASS

PASS consensus from 3 PASS.

The Python quickstart installs firecrawl-py, constructs Firecrawl, and calls app.scrape(), matching the v2 API reference's SDK form. The quickstart, the SDK reference, and the API reference Python tab agree on pip install firecrawl-py, from firecrawl import Firecrawl, and firecrawl.scrape(url) against the current v2 API, so the minimum scrape example matches the current API form. The official Python SDK (firecrawl-py) and a minimal scrape example are documented on both the dedicated SDK page and the advanced scraping guide, using the current v2/scrape endpoint.

The receipt

Every webhook request from Firecrawl includes an X-Firecrawl-Signature header containing an HMAC-SHA256 signature.

The configuration, payload, and HMAC steps are complete, but Overview omits failed from Crawl while the Crawl page lists crawl.failed.

Agent surface notes

Initialize returned JSON-RPC protocol 2025-06-18 and server name Firecrawl Docs.

The live docs expose no Ask AI or assistant control.

Show the score

AI Agent Readiness Score 81.2%, grade B-

Paste this into a readme:

[![AI Agent Readiness Score 81.2%](https://docsforagents.com/badge/firecrawl.svg)](https://docsforagents.com/reports/firecrawl-docs-ai-agent-readiness/)

Method note

This is a reading test of public documentation, not an execution test. No accounts were created and no API calls were run. The AI Agent Readiness Score counts fifteen reading votes at PASS 2, PARTIAL 1, and FAIL 0, for 30 possible points. Five agent surface checks add 10 points each. The total is 80. Consensus chips show each row majority and do not affect scoring. The panel split on 3 of five tasks. Quotes shown here were re-fetched and confirmed verbatim on 2026-08-19.

Read the full methodology

Put another docs site through the battery.

Nominate a docs site