Published 2026-09-21 · Tested 2026-08-19

Union.ai

B-

Union.ai received 11 PASS votes and passed 4 of five agent surface checks. The clearest finding came from the verify a workflow webhook task.

Panel: GPT 5.6 Sol, Opus 5, DeepSeek v4F Battery: v1 Read as markdown (opens in a new tab)

Three AI models, GPT 5.6 Sol, Claude Opus 5, and DeepSeek v4 Flash, each read Union.ai’s public documentation independently and attempted five first-hour developer jobs: run the first workflow, find the exact limits, recover from a lease error, verify a workflow webhook, use the Python SDK.

No accounts, API calls, or code execution were used. Every verdict came from public pages and every published quotation passed a live verification check. Preflight substitutions: T3: A documented lease error replaced 429 because it is the closest common product error.

Freshness

How rechecks work
Tested
Quotes verified
Surface rechecked

No confirmed change since the test.

3 of 5 quoted passages still appear on the live pages. The quoted page changed for Run the first workflow and Verify a workflow webhook.

Union.ai Hugo with custom Union theme · published
B-

82.5% · 66/80 · AI Agent Readiness Score · reading 30 pts · surface 50 pts

llms.txt PASS
llms-full.txt PASS
markdown mirror PASS
MCP server FAIL
docs AI PASS
Task GPT 5.6 SolOpus 5DeepSeek v4F Consensus
Run the first workflow PASSPASSPASS PASS
Find the exact limits PASSPARTIALPASS PASS
Recover from a lease error PASSPASSPASS PASS
Verify a workflow webhook PARTIALPARTIALPARTIAL PARTIAL
Use the Python SDK PASSPASSPASS PASS

docs platform: Hugo with custom Union theme (unscored) · verified 2026-08-19

What to fix first

These 2 fixes could add up to 13 points to the AI Agent Readiness Score. The list ranks each fix by the points it would add. How the ranking works

  1. 1
    +10 points MCP server check failed

    Found: Initialize returned redirects or access errors, not a valid MCP response.

    Fix: Run a public MCP server for the docs that answers a JSON-RPC initialize call and offers a docs search tool.

    Evidence: www.union.ai/docs/mcp (opens in a new tab)

  2. 2
    +3 points Verify a workflow webhook PARTIAL

    Found: The docs show sending a notification webhook with an Authorization header, but no receiver that checks it.

    Fix: Add a receiver example to the notifications page that checks the Authorization header before handling the payload.

    Evidence: www.union.ai/docs/v2/union/user-guide/tasks/task-deployment/run-with-notifications (opens in a new tab)

What the docs get right

  • Run the first workflow: 3 PASS votes. One page gives the prerequisite, install command, local config, complete hello.py, run command, and result inspection.
  • Recover from a lease error: 3 PASS votes. A dedicated Leases page shows the literal error string lease expired for action a0 ..., explains heartbeat, grace period, expiration and failover, states that no correction is required in most cases, and gives the one condition that does warrant action, namely repeated lease expirations on the same run or cluster.
  • 4 of 5 agent surface checks. Present: llms.txt, llms-full.txt, markdown mirrors, docs AI. Missing: an MCP server.

Run the first workflow

PASS

PASS consensus from 3 PASS.

One page gives the prerequisite, install command, local config, complete hello.py, run command, and result inspection. One page carries the whole path with no guessing: Python 3.10+, pip install 'flyte[tui]', flyte --version, flyte create config --local-persistence, a complete hello.py, and flyte run --local hello.py main; however all four Next steps links on that page 404 because they resolve against a quickstart/ base that does not exist (for example /get-started/quickstart/core-concepts/_index). The Quickstart provides an unambiguous four-step path (install SDK, configure, write hello.py with TaskEnvironment + @env.task, run locally) with working code and expected output.

Find the exact limits

PASS

PASS consensus from 2 PASS, 1 PARTIAL.

The 10 MiB default appears consistently, and resource ceilings and quota budgets use project-domain scopes. Three pages agree the inline-I/O ceiling is 10 MiB set by max_inline_io_bytes on @env.task, but the Data flow page states 10MB and names a parameter max_inline_io on flyte.TaskEnvironment that appears nowhere else in the SDK reference; the project-by-domain side is exact and consistent, with a full task_resource.min/max key list, an org to domain to project inheritance chain, and the rule that task_resource.max.* is a hard limit that caps rather than rejects an oversized request. Inline I/O ceiling is explicitly 10 MiB default with a configurable override. Project-by-domain quota rules are documented via the Settings hierarchy (org → domain → project) with specific task_resource.max.* keys that are enforced as hard per-task ceilings.

Recover from a lease error

PASS

PASS consensus from 3 PASS.

The page says automatic failover needs no correction unless expirations recur on the same run or cluster. A dedicated Leases page shows the literal error string lease expired for action a0 ..., explains heartbeat, grace period, expiration and failover, states that no correction is required in most cases, and gives the one condition that does warrant action, namely repeated lease expirations on the same run or cluster. The Leases page explains the error's cause (heartbeat failure → grace period → lease reaped), why no action is normally needed, and what to check if the error recurs (cluster health, network, or resource provisioning).

Verify a workflow webhook

PARTIAL

PARTIAL consensus from 3 PARTIAL.

The sender is documented, but receiver token checking appears only in a separate webhook app example with different semantics. The send half is exact, including a headers={"Authorization": "Bearer my-token"} example, the four terminal ActionPhase values, and attachment per run via flyte.with_runcontext(notifications=...), but the receive and verify half is absent: the corpus has no delivery log, retry semantics, signature or HMAC, default payload when body is omitted, or local receiver recipe, even though the same page family gives a local aiosmtpd recipe for email, and the run-with-notifications page's only link to the webhook example is itself a 404. Sending a terminal-phase webhook with an Authorization header is documented via notify.Webhook on the Triggers page. However, no documentation shows how to receive and verify such a notification webhook at the receiver; a different webhook pattern (webhook app triggering tasks) shows a receiver with auth but is conceptually distinct, requiring cross-referencing.

Use the Python SDK

PASS

PASS consensus from 3 PASS.

The SDK quickstart and migration guide use flyte.TaskEnvironment and @env.task, matching Flyte 2's task model. The SDK reference is versioned (Flyte SDK version 2.0) and its TaskEnvironment entry matches the quickstart's minimum working workflow of flyte.TaskEnvironment plus @env.task with required type hints, though the flyte.notify package page's own quick start uses a @flyte.task(trigger=...) decorator that the SDK function index does not list. The official SDK is the flyte package installed via pip install 'flyte[tui]'. The Quickstart shows a complete working workflow using TaskEnvironment and @env.task, which matches the Flyte 2 task model used throughout the docs.

The receipt

headers={"Authorization": "Bearer my-token"},

The sender is documented, but receiver token checking appears only in a separate webhook app example with different semantics.

Agent surface notes

Initialize returned redirects or access errors, not a valid MCP response.

The live docs expose a Search and Ask AI control.

Show the score

AI Agent Readiness Score 82.5%, grade B-

Paste this into a readme:

[![AI Agent Readiness Score 82.5%](https://docsforagents.com/badge/union-ai.svg)](https://docsforagents.com/reports/union-ai-docs-ai-agent-readiness/)

Method note

This is a reading test of public documentation, not an execution test. No accounts were created and no API calls were run. The AI Agent Readiness Score counts fifteen reading votes at PASS 2, PARTIAL 1, and FAIL 0, for 30 possible points. Five agent surface checks add 10 points each. The total is 80. Consensus chips show each row majority and do not affect scoring. The panel split on 1 of five tasks. Quotes shown here were re-fetched and confirmed verbatim on 2026-08-19.

Read the full methodology

Put another docs site through the battery.

Nominate a docs site