Published 2026-08-20 · Tested 2026-08-19

CARTO

B-

CARTO received 4 PASS votes and passed 5 of five readiness checks. The clearest finding came from the find the exact limits task.

Panel: GPT 5.6 Sol, Opus 5, DeepSeek v4F Battery: v1 Verified: 2026-08-19 Read as markdown (opens in a new tab)

Three AI models, GPT 5.6 Sol, Claude Opus 5, and DeepSeek v4 Flash, each read CARTO’s public documentation independently and attempted five first-hour developer jobs: make the first API call, find the exact limits, recover from a 429, authenticate with OAuth, use the JavaScript module.

No accounts, API calls, or code execution were used. Every verdict came from public pages and every published quotation passed a live verification check. Preflight substitutions: T4: Machine-to-machine OAuth authentication replaced webhooks because the public docs expose no general CARTO product-webhook interface.

CARTO GitBook · published 2026-08-20
B-

82.5% · 66/80 · AI Agent Readiness Score · reading 30 pts · surface 50 pts

llms.txt PASS
llms-full.txt PASS
markdown mirror PASS
MCP server PASS
docs AI PASS
Task GPT 5.6 SolOpus 5DeepSeek v4F Consensus
Make the first api call PARTIALPARTIALPASS PARTIAL
Find the exact limits PARTIALPARTIALPARTIAL PARTIAL
Recover from a 429 FAILFAILFAIL FAIL
Authenticate with oauth PARTIALPASSPASS PASS
Use the javascript module PARTIALPARTIALPASS PARTIAL

docs platform: GitBook (unscored) · verified 2026-08-19

What the docs get right

  • Authenticate with oauth: 2 PASS votes. The exchange and bearer call are documented, but M2M setup pages disagree on whether creation needs only a name or also two URLs.
  • Make the first api call: 1 PASS votes. The token and SQL steps are complete, but the quickstart hard-codes the US-East direct host while the base-URL page requires the organization-specific host.
  • 5 of 5 readiness checks. Present: llms.txt, llms-full.txt, markdown mirrors, an MCP server, docs AI.

Make the first api call

PARTIAL

PARTIAL consensus from 1 PASS, 2 PARTIAL.

The token and SQL steps are complete, but the quickstart hard-codes the US-East direct host while the base-URL page requires the organization-specific host. The quickstart gives a complete runnable path (token scopes, connection carto_dw, allow all sources, then a copy-pasteable curl), but its SQL example calls the host direct-gcp-us-east1.api.carto.com while the API reference and the API Base URL page both give gcp-us-east1.api.carto.com for United States East, and the direct- prefix is defined nowhere in the docs. Creating an API Access Token and running a SQL API query is fully documented with step-by-step UI guidance and a curl example.

Find the exact limits

PARTIAL

PARTIAL consensus from 3 PARTIAL.

The docs give matching per-IP ceilings and annual API weights, but they publish only a 35-300 monthly M2M range and no numeric API Access Token entitlement by plan. Scopes and values are consistent , the per-IP per-minute table (Maps 3500, LDS 3000, SQL 2500, Imports 500, Tokens & Accounts 300, Connections/Workflows/Org Maps 300, AI 300, all others 1000) appears identically on two pages, and annual quota weights are given as Maps 0.2, SQL 10, SQL Widgets 1, Import 10, LDS 0.1, all other APIs 0 , but the answer must be assembled from three separate pages, and the organization-level API Access Token quota is described as plan-dependent with no per-plan value stated anywhere. Per-IP request limits and API quota weights are documented precisely and consistently. Plan-dependent token quotas are vague (an example '100 out of 100 tokens' with no per-plan breakdown; M2M token minting stated as '35-300 tokens/month, depending on your plan' without plan-specific numbers).

Recover from a 429

FAIL

FAIL consensus from 3 FAIL.

The docs identify the 429 trigger but state no block duration, retry timing, or rate-safe retry procedure. The docs state the trigger and the 429 response but never state how long the temporary block lasts, never mention a Retry-After header, and give no retry or backoff timing; prevention guidance exists (stay under the per-minute ceiling, let the CDN serve repeats, reuse M2M tokens, email support@carto.com for a higher limit), and CARTO's own documentation ask endpoint confirmed the gap rather than inventing a value, replying that the docs "do not specify how long the block lasts". The docs state that a 429 is returned and the IP is 'automatically and temporarily blocked', but do not specify the block duration, any Retry-After header, recommended retry timing, or a safe correction strategy.

Authenticate with oauth

PASS

PASS consensus from 2 PASS, 1 PARTIAL.

The exchange and bearer call are documented, but M2M setup pages disagree on whether creation needs only a name or also two URLs. The full path is documented end to end and agrees across pages: create an M2M OAuth Client and copy its Client ID and Secret, exchange them at auth.carto.com/oauth/token with grant_type=client_credentials and audience=carto-cloud-native-api (identical parameters shown again as working code in the integration guide), receive a token valid for 24 hours that is meant to be cached and reused, then call any API with Authorization: Bearer; the Enterprise Large plan requirement is disclosed up front. Creating an M2M OAuth Client, exchanging credentials for an access token, and using it with CARTO APIs is fully documented end-to-end across three consistent pages.

Use the javascript module

PARTIAL

PARTIAL consensus from 1 PASS, 2 PARTIAL.

The example builds a dataset layer, but it imports vectorTableSource from @deck.gl/carto while current release guidance directs data-source imports to @carto/api-client. A minimum dataset visualization exists and its authentication matches current guidance (apiBaseUrl plus API Access Token plus connectionName), but the guide imports vectorTableSource from @deck.gl/carto while the data-sources reference and key-concepts pages both import it from @carto/api-client, and the guide's install line reads npm install maplibre-gl @deck.gl/core @deck.gl/carto with no @carto/api-client at all, so following it verbatim conflicts with the current documented package split. The Hello World example uses @deck.gl/carto ^9.2.0 with vectorTableSource and VectorTileLayer. It authenticates with VITE_API_ACCESS_TOKEN and VITE_API_BASE_URL, matching the documented API Access Token approach and API Base URL guidance.

The receipt

Each Machine-to-Machine (M2M) OAuth Client can mint a limited number of OAuth access tokens per month (in the range of 35-300 tokens/month, depending on your plan).

The docs give matching per-IP ceilings and annual API weights, but they publish only a 35-300 monthly M2M range and no numeric API Access Token entitlement by plan.

Agent surface notes

Initialize returned JSON-RPC protocol 2025-06-18 with server information and tool capabilities.

The live docs expose a public GitBook Ask or search assistant control.

Method note

This is a reading test of public documentation, not an execution test. No accounts were created and no API calls were run. The AI Agent Readiness Score counts fifteen reading votes at PASS 2, PARTIAL 1, and FAIL 0, for 30 possible points. Five readiness checks add 10 points each. The total is 80. Consensus chips show each row majority and do not affect scoring. The panel split on 3 of five tasks. Quotes shown here were re-fetched and confirmed verbatim on 2026-08-19.

Read the full methodology

Put another docs site through the battery.

Nominate a docs site