skills/agent-swarm/references/orchestration.md

Workflow orchestration

The workflow tool runs one JavaScript body with top-level await and returns a JSON value. It is the only way to fan out dozens of agents without flooding the lead's context.

Call shape

{
  "meta": {"name": "swarm-wave-1", "description": "Build wave", "phases": [{"title": "wave 1"}]},
  "script": "<script body>",
  "args": {
    "wave": 1,
    "tasks": [{"id": "T01", "write": ["src/a.ts"]}],
    "schema": "<paste the contents of assets/report.schema.json>",
    "concurrency": 6
  },
  "run_in_background": false
}

• meta.phases is optional; use one phase per wave and phase() inside the script.
• args is data, never evaluated. The script cannot read files; pass paths and small structured facts.
• Pass args.schema as the parsed schema object, not a path or a string.
• Pass args.concurrency explicitly. It must be ≤ the free active-child slots; do not rely on the engine to queue.
• The tool blocks until every child settles. Use run_in_background for long waves; poll with job_output, cancel with job_kill.

agent()

agent(prompt, opts) starts one child and resolves with:

• its final text when no schema is passed;
• a validated object when schema is passed;
• null when the child fails, returns a non-completed stop reason, or produces no structured value.

Options: label, phase, schema, provider, model. Unknown or deferred options kill the script.

agent() can also throw when the runtime cannot start a child (for example active child limit reached). That throw is not a null; if it happens inside parallel, it can abort the whole run. Wrap each agent() call in try/catch inside the batched runner and record a scheduling failure for that task instead of losing the batch.

The child is disposed after the call; there is no follow-up message inside a workflow. Design tasks as one-shot packets.

parallel vs pipeline

• parallel([thunks]) — start every thunk together; await all. It is not a queue: it does not limit concurrency. Use it only for a chunk whose size fits the free slots.
• pipeline(items, ...stages) — per-item chain with no barrier. Use only when each item must pass through stages without waiting for the wave (for example, extract → summarize per source). It multiplies agent count by the number of stages.
• For more items than slots, run batches sequentially and use parallel inside each batch (assets/wave.workflow.js).

Do not use parallel for dependent tasks; use separate waves and let the lead inspect results.

Schema subset

The engine validates a strict subset. Keep schemas boring:

• Object root ("type": "object").
• properties, required (names must exist in properties), additionalProperties boolean.
• Arrays with items.
• Strings/numbers/integers/booleans/null, enum of scalars, const.
• oneOf only with exactly-one semantics and no sibling keywords.
• No minItems, maxLength, pattern, format, $ref, anyOf, allOf.

A schema miss returns null. Verifiers should use the minimal schema in assets/verdict.schema.json: {task_id, verdict, summary, review_file}. Put the detailed findings in .swarm/reviews/Vxx.md; do not make a long findings array required in the workflow schema. If a schema miss happens, read the review file on disk before deciding that the verification failed, then retry with the smaller schema.

Concurrency and caps

• Do not hardcode a concurrency default. The active-child cap is host/policy-specific. Some runtimes allow 14; this environment rejected 8 workers when other children were active. Treat the cap as runtime data.
• Preflight: read the runtime diagnostics or list_agents; record slot_cap and active_at_dispatch in .swarm/state.json. Recon, verifiers, integrators and any other active children all count.
• Batch size: min(8, slot_cap - active_at_dispatch - 1), at least 2. If you do not know the cap, use 6. Leave at least one free slot for retries.
• Each child holds a slot for its whole life. A parallel call starts every thunk immediately; if the batch exceeds free slots, the spawns throw.
• Slots are FIFO; a slow task delays later ones. Keep tasks in a batch similar in size.
• maxTotalAgents: 1000 per run. maxItemsPerCall: 4096 per parallel/pipeline.

Batched worker runner

Use assets/wave.workflow.js. It splits args.tasks into chunks of args.concurrency, runs one parallel chunk at a time, catches agent() throws per task, and returns a structured failure for scheduling errors instead of killing the run. If a chunk reports a limit error, rerun only the unstarted tasks with a smaller concurrency.

Retry pattern

Full template: assets/wave.workflow.js. Retry only when the result is null or status === "failed". Do not retry blocked — that needs a lead decision. Do not retry a spawn/limit error immediately; shrink the batch or let the lead dispatch the remainder.

async function runOne(t) {
  for (let attempt = 1; attempt <= 2; attempt++) {
    try {
      const r = await agent(promptFor(t, attempt), {
        label: t.id,
        phase: "wave 1",
        schema: REPORT
      });
      if (r && r.status !== "failed") return r;
      log(`${t.id}: attempt ${attempt} produced no usable report`);
    } catch (error) {
      const message = String(error && error.message ? error.message : error);
      if (!/(limit|spawn|could not start|active child|capacity|start a child)/i.test(message)) {
        throw error; // invalid options/schema must fail loudly
      }
      return failed(t, `scheduling error: ${message}`);
    }
  }
  return failed(t);
}

On the second attempt, add one line: what failed and what to check. Do not repeat the whole brief. Catch only spawn/scheduling failures around agent(); invalid hook options or an unsupported schema should still fail loudly so you fix the script.

Prompt assembly

Build the prompt from args, not from prose in the script:

const promptFor = (t, attempt) => [
  `You are agent ${t.id} in a coordinated swarm.`,
  `Read .swarm/brief.md, .swarm/tasks/${t.id}.md, and .swarm/contracts.md first.`,
  `Write only within: ${(t.write || []).join(", ") || "(read-only task)"}, plus your own report, review, and scratch under .swarm/reports/, .swarm/reviews/, and .swarm/tmp/${t.id}/.`,
  `Do not edit other tasks' files or reports; do not run installs, repo-wide format/lint/codegen, migrations, dev servers, or git writes.`,
  `Run the acceptance checks and record exact commands and results.`,
  `Write .swarm/reports/${t.id}.json and return the same object.`,
  `If a load-bearing unknown or a needed file outside scope blocks you, return status "blocked" with blocked_question; do not guess.`,
  attempt > 1
    ? `A previous attempt produced no usable report. Re-read the task and fix the cause or report the exact blocker.`
    : ""
].filter(Boolean).join("\n");

The task card carries the details; the prompt carries the role, scope, and protocol. Do not duplicate the card in the prompt.

Verification runner

Use assets/verify.workflow.js for the verify wave (after freezing the tree). It embeds the minimal verdict schema and fetches reviews in batches:

{
  "script": "<contents of assets/verify.workflow.js>",
  "args": {
    "wave": 3,
    "tasks": [{"id": "V01", "review": ".swarm/reviews/V01.md"}],
    "concurrency": 4
  },
  "run_in_background": true
}

The verifier writes the review file first and returns only {task_id, verdict, summary, review_file}. If the workflow reports null/no usable verdict, check the review file; then re-run with the minimal schema. Code is frozen during this wave.

Background runs

Use run_in_background: true when a wave may exceed ~10 minutes or you want live progress in the job list. The call returns a job id; job_output with wait: true blocks until settlement. If a wave hangs, job_kill cancels the run and disposes children. A background run's result arrives once, at settlement.

Archetypes

Code

• Units: module/feature/file cluster, each with its own tests.
• Ownership: source + test files per unit; shared types and barrels are integrator-only.
• Acceptance: targeted test command, typecheck, lint on changed files.
• Verify: a fresh agent runs the tests and reads the diff; no self-review.

Research

• Units: one question or one source cluster per agent; output files under research/.
• Ownership: one file per agent; no shared notes file.
• Acceptance: source URLs resolve, claims are dated, every criterion covered.
• Verify: citation check and a consistency pass; a second agent samples claims.

Content / documents

• Units: one section per agent, plus one editor-integrator.
• Ownership: sections are separate files; the editor owns the assembled document.
• Acceptance: section brief, word budget, terminology list, citations.
• Verify: one editor pass for voice and transitions; a fact-checker for claims.

Verification runs

Use assets/verify.workflow.js with assets/verdict.schema.json after freezing the tree. Each verifier:

1. writes its full review to .swarm/reviews/Vxx.md first;
2. returns only {task_id, verdict, summary, review_file};
3. reports pass, pass_with_risks, or fail plus evidence in the file.

If the workflow returns null/no usable verdict, check the review file before treating it as a product failure; then retry with the minimal schema or message the verifier for a concise verdict. After any fix, re-run the affected verifier and mark the old review stale.

Failure handling

• null per item: ordinary child failure; retry or mark failed in the result. Do not fail the whole wave.
• Spawn/limit error inside agent(): catch per task, keep completed results, and rerun only unstarted tasks with a smaller batch. This is a scheduling failure, not a worker failure.
• A thrown hook error (bad option, unsupported schema, unknown hook) kills the run: fix the script. Do not hide it in a broad try/catch.
• Cancellation: stop the run, inspect partial reports on disk; do not trust the tool result.

Cost

Each child carries the system prompt, tool schemas, packet, code reads, and its output: typically 10–40k tokens. A 12-agent wave is roughly 150–400k tokens, but a host capped at 8 cannot run 12 at once; batches reduce peak concurrency, not total cost. Budget before dispatch and count retries/re-verifications; stop at max_agents.