Published 2026-09-21 · Tested 2026-08-19

Orkes

F

Orkes received 8 PASS votes and passed 2 of five agent surface checks. The clearest finding came from the find the exact limits task.

Panel: GPT 5.6 Sol, Opus 5, DeepSeek v4F Battery: v1 Read as markdown (opens in a new tab)

Three AI models, GPT 5.6 Sol, Claude Opus 5, and DeepSeek v4 Flash, each read Orkes’s public documentation independently and attempted five first-hour developer jobs: run the first workflow, find the exact limits, recover from a 429, verify an incoming webhook, use the Java SDK.

No accounts, API calls, or code execution were used. Every verdict came from public pages and every published quotation passed a live verification check.

Freshness

How rechecks work
Tested
Quotes verified
Surface rechecked

No confirmed change since the test.

4 of 5 quoted passages still appear on the live pages. The quoted page changed for Recover from a 429.

Orkes MkDocs Material 9.7.5 · published
F

51.2% · 41/80 · AI Agent Readiness Score · reading 30 pts · surface 50 pts

llms.txt PASS
llms-full.txt PASS
markdown mirror FAIL
MCP server FAIL
docs AI FAIL
Task GPT 5.6 SolOpus 5DeepSeek v4F Consensus
Run the first workflow PASSPASSPASS PASS
Find the exact limits PARTIALPARTIALPASS PARTIAL
Recover from a 429 FAILPARTIALFAIL FAIL
Verify an incoming webhook PASSPASSPASS PASS
Use the Java SDK PARTIALPARTIALPASS PARTIAL

docs platform: MkDocs Material 9.7.5 (unscored) · verified 2026-08-19

What to fix first

These 6 fixes could add up to 39 points to the AI Agent Readiness Score. The list ranks each fix by the points it would add. How the ranking works

  1. 1
    +10 points Markdown mirror check failed

    Found: Appending .md to live docs pages returned HTTP 404 HTML.

    Fix: Serve every docs page as Markdown at its URL plus .md, and return the same Markdown for Accept: text/markdown.

    Evidence: orkes.io/content/quickstart/workers.md (opens in a new tab)

  2. 2
    +10 points MCP server check failed

    Found: Initialize returned HTTP 404 HTML at tested candidate endpoints.

    Fix: Run a public MCP server for the docs that answers a JSON-RPC initialize call and offers a docs search tool.

    Evidence: orkes.io/content/mcp (opens in a new tab)

  3. 3
    +10 points Docs AI check failed

    Found: The live docs expose standard search but no Ask AI control.

    Fix: Add an assistant to the docs site that answers questions from the docs and links to its sources.

  4. 4
    +5 points Recover from a 429 FAIL

    Found: The Workflow Message Queue (WMQ) docs give no backoff schedule. The architecture page says a full queue returns 429 or 400, while the WMQ guide lists only 429.

    Fix: Add a full-queue backoff schedule to the WMQ guide, and list one set of full-queue status codes across WMQ pages.

    Evidence: orkes.io/content/wmq/workflow-message-queue-architecture (opens in a new tab)

  5. 5
    +2 points Find the exact limits PARTIAL

    Found: The task definition schema states zero behavior for rateLimitPerFrequency but not for concurrentExecLimit. Opus 5 also found that the field's scope appears only on another page.

    Fix: Add concurrentExecLimit's zero behavior and scope to the task definition schema.

    Evidence: orkes.io/content/developer-guides/rate-limits (opens in a new tab)

  6. 6
    +2 points Use the Java SDK PARTIAL

    Found: The Java SDK page leaves the dependency version as the placeholder VERSION. Opus 5 also found that it sets up the client differently from the quickstart.

    Fix: Replace VERSION with a release number on the Java SDK page, and share one client setup with the quickstart.

    Evidence: orkes.io/content/sdks/java (opens in a new tab)

What the docs get right

  • Run the first workflow: 3 PASS votes. The guide provides setup, one complete Python file, a run command, expected output, verification, and troubleshooting.
  • Verify an incoming webhook: 3 PASS votes. The GitHub path connects a shared secret to verified delivery, automatic workflow start, and execution inspection.
  • 2 of 5 agent surface checks. Present: llms.txt, llms-full.txt. Missing: markdown mirrors, an MCP server, docs AI.

Run the first workflow

PASS

PASS consensus from 3 PASS.

The guide provides setup, one complete Python file, a run command, expected output, verification, and troubleshooting. The quickstart hub routes to Connect to Conductor and then to first-worker, which gives the greetings/greet example in six languages plus a durability check and a troubleshooting list, so the path to Hello Conductor is unambiguous when entered from /content/quickstart. The Hello Conductor quickstart is a single page with prerequisites, install, worker code, workflow code, execution, and verification steps in one sequential guide.

Find the exact limits

PARTIAL

PARTIAL consensus from 1 PASS, 2 PARTIAL.

Task scope and rate-limit zero behavior are documented, but concurrentExecLimit=0 behavior is not stated. Rate-limit semantics are exact and zero-valued behavior is stated for rateLimitPerFrequency, timeoutSeconds, and pollTimeoutSeconds, but concurrentExecLimit is the one field on the same table with no documented zero behavior even though API responses return concurrentExecLimit: 0, and its scope is only described as across all workers on a different page. The task-definition schema clearly documents rateLimitPerFrequency (default 0 = no limit), rateLimitFrequencyInSeconds, and concurrentExecLimit (default 0 = no limit). Values are consistent across task-definition and task-lifecycle pages; zero disables the constraint.

Recover from a 429

FAIL

FAIL consensus from 1 PARTIAL, 2 FAIL.

The docs identify saturation but omit a client backoff schedule and disagree on whether the response can be 400. The cause, the maxQueueSize default of 1000, and the PULL_WORKFLOW_MESSAGES drain path are documented, but no page gives a backoff algorithm, a Retry-After header, a retry ceiling, or a delay, and the architecture page contradicts this one by saying the same condition can return HTTP 400. The error-handling page covers retry strategies (FIXED, EXPONENTIAL_BACKOFF, LINEAR_BACKOFF) for failed tasks but never mentions what happens when a task queue reaches capacity or how to detect/backoff from a 429. The rate-limits page shows how to prevent overload but not how to recover from it.

Verify an incoming webhook

PASS

PASS consensus from 3 PASS.

The GitHub path connects a shared secret to verified delivery, automatic workflow start, and execution inspection. The guide gives the seven routes, a verifier table naming the exact verification input and challenge behavior per provider, secret-reference configuration, and both the start and WAIT_FOR_WEBHOOK resume delivery modes, and a separate cURL tutorial walks the full create-invoke-verify-resume loop end to end. The webhook integration page documents creating webhooks (UI + API), six verifier choices (HEADER_BASED, SIGNATURE_BASED, HMAC_BASED, SLACK_BASED, STRIPE, TWITTER, SENDGRID), idempotency-key configuration, and both start-workflow and resume-WAIT_FOR_WEBHOOK delivery modes.

Use the Java SDK

PARTIAL

PARTIAL consensus from 1 PASS, 2 PARTIAL.

The example matches name, version, input, and returned ID, but the dependency keeps an unresolved VERSION placeholder. The official Java SDK page ships a structurally complete worker, workflow, and execution example, but its dependency coordinate is the literal placeholder VERSION, and its client model (ApiClient plus OrkesClients, WorkflowExecutor(apiClient, 100), startWorkflow with a StartWorkflowRequest) does not match the quickstart's Java tab for the same greetings workflow (ConductorClient.builder(), WorkflowExecutor(client), four-argument positional startWorkflow), so completing it requires picking between two incompatible bootstraps. The Java SDK page provides a 60-Second Quickstart with install coordinates (Maven/Gradle), a GreetWorker implementing the Worker interface, a Main.java that builds and registers the greetings workflow, and execution instructions.

The receipt

No rate limit if set to 0.

Task scope and rate-limit zero behavior are documented, but concurrentExecLimit=0 behavior is not stated.

Agent surface notes

Initialize returned HTTP 404 HTML at tested candidate endpoints.

The live docs expose standard search but no Ask AI control.

Show the score

AI Agent Readiness Score 51.2%, grade F

Paste this into a readme:

[![AI Agent Readiness Score 51.2%](https://docsforagents.com/badge/orkes.svg)](https://docsforagents.com/reports/orkes-docs-ai-agent-readiness/)

Method note

This is a reading test of public documentation, not an execution test. No accounts were created and no API calls were run. The AI Agent Readiness Score counts fifteen reading votes at PASS 2, PARTIAL 1, and FAIL 0, for 30 possible points. Five agent surface checks add 10 points each. The total is 80. Consensus chips show each row majority and do not affect scoring. The panel split on 3 of five tasks. Quotes shown here were re-fetched and confirmed verbatim on 2026-08-19.

Read the full methodology

Put another docs site through the battery.

Nominate a docs site