Baseten

PASS consensus from 3 PASS.

The quickstart covers account creation, key export, SDK installation, complete chat calls, and a sample successful response. The quickstart gives the API key export, the SDK install, the base URL https://inference.baseten.co/v1, and a complete runnable chat completion in Python, JavaScript, and cURL on one page with no cross-referencing. The docs provide a complete, copy-pasteable Python example with the OpenAI SDK, the base URL, the BASETEN_API_KEY environment variable, and a model slug, plus a separate page explaining how to create that API key.

PASS consensus from 2 PASS, 1 PARTIAL.

One table gives every account state, and no fetched Model API page gives different account-tier values. The four account tiers carry exact numbers in one table (Basic unverified 15 RPM / 100,000 TPM, Basic verified 120 / 500,000, Pro 120 / 1,000,000, Enterprise custom), but the Fast tier limits are declared to exist and never stated, and the frontier gateway page contradicts their existence by saying Model APIs ceilings apply regardless of which model you call. A single, consistent table lists RPM and TPM for every account tier including verified vs unverified Basic, with no contradictions across pages.

PASS consensus from 2 PASS, 1 PARTIAL.

The 429 guide says status and cancel endpoints allow 100 requests per second, but their API references say 20. The 429 section separates the three surfaces that can produce it (Model APIs RPM/TPM, async endpoints at 12,000 requests per minute, and dedicated deployment capacity carrying a CAPACITY_EXCEEDED code), prescribes backoff, and states when backoff is the wrong fix and replicas should be raised instead. The docs distinguish rate-limit 429s from capacity 429s, prescribe exponential backoff, link to the tier-specific limits table, and explain how to distinguish transient bursts from persistent over-limit traffic.

PASS consensus from 3 PASS.

One page covers HTTPS setup, submission, payload receipt, delivery retries, and raw-body HMAC-SHA256 verification. The page covers the full loop without gaps: the /async_predict call with webhook_endpoint, the delivered headers and JSON body, where to create the webhook secret, a timing-safe verify_signature implementation, the 24-hour dual-validity rotation window, and the retry policy of two total attempts. The end-to-end flow is documented: submit to /async_predict with a webhook endpoint, receive a POST with X-BASETEN-SIGNATURE (when a webhook secret is configured), and verify using the provided timing-safe HMAC-SHA256 code.

PARTIAL consensus from 1 PASS, 2 PARTIAL.

The SDK lacks runnable Python deployment code, and two Qwen deployment guides require different builder GPU counts. The SDK is official and its five reference pages document every truss.push parameter consistently with the CLI flags and GitHub Action inputs, but no minimum deployment example exists: across the entire 2.4 MB llms-full.txt corpus there is no import of the deploy SDK and no truss.push() or truss.login() call in any code block, so the example must be synthesized from parameter tables. The official Truss Python SDK is on PyPI. The docs show a complete minimum deployment (Qwen 2.5 3B via config.yaml with truss push), and the resulting endpoint uses the same OpenAI-compatible API pattern shown elsewhere for Model APIs.