242 lines
19 KiB
Markdown
242 lines
19 KiB
Markdown
# Swarm playbook
|
||
|
||
Detailed runbook for `agent-swarm`. Read this after the fit check passes. The runbook is domain-neutral; choose an adapter from `domains.md` and put its details in `.swarm/profile.md`. Keep `.swarm/state.json` current; it is the resume point if the session restarts.
|
||
|
||
## 0. Fit check and budget
|
||
|
||
Answer these before spawning:
|
||
- **Mode.** Is this really a swarm, or does a direct run / paired critic cover the risk with less overhead?
|
||
- **Units.** Can you list 8+ deliverables that different agents can finish without waiting for each other?
|
||
- **Writes.** Can every unit get a disjoint write scope? Name the shared files now.
|
||
- **Checks.** Does every unit have a command or criterion a verifier can re-run?
|
||
- **Interfaces.** Can you freeze signatures, data shapes, and file names before dispatch?
|
||
- **Breadth.** Is the time dominated by many independent pieces, not one deep chain?
|
||
|
||
Budget:
|
||
- `agents ≈ units + ceil(units / 5) verifiers + 1–2 integrators + 2–4 recon`; add retries and re-verifications explicitly. Set `max_agents` only after checking available tools and slot cap.
|
||
- Tokens ≈ `agents × fixed overhead (10–25k) + context reads + output + reviews`. Treat this as an estimate, then calibrate from measured usage. A 3k-line file read by several agents can cost more than the code they write.
|
||
- Reserve **20–30%** for verification, integration and one fix round. Set a soft cap at ~70% and a hard cap; stop at the hard cap with a partial report.
|
||
- Waves ≈ `ceil(units / batch_size)` + verify + integrate, where `batch_size` comes from the **real** active-child cap, not from a documented default.
|
||
- Stop at `max_agents`, `max_waves` or the token cap, whichever comes first. Record all caps, `slot_cap`, `active_at_dispatch`, `batch_size`, measured/estimated usage per phase, retries and re-verifications in `state.json`.
|
||
- See `references/token-economy.md`; do not invent precise token numbers without measurement.
|
||
|
||
Go: a 40-file migration with per-module tests, a 30-source research report, a 20-section document with a style guide, a multi-dataset analysis, a multi-screen visual redesign, a multi-service runbook.
|
||
No-go: one failing test, one architectural decision, a detail pass where every unit touches the same artifact.
|
||
|
||
## 1. Prerequisite gate
|
||
|
||
Do this before recon and before writing task cards. A missing compiler, runtime, browser, display,
|
||
database, credential or package manager found mid-wave costs an entire wave and can leave the tree
|
||
half-changed. Full policy and the approval template: `references/prerequisites.md`.
|
||
|
||
1. Inventory required vs optional items: toolchain + pinned version, build/test/lint commands, runtime,
|
||
services, display/GPU, credentials (presence only), agent tools (`workflow`/`subagent`), disk/ports.
|
||
2. Run the documented checks yourself. Do not trust a worker's "should work".
|
||
3. Prefer existing/project-local toolchains. **Never** run `sudo`, system package managers, global
|
||
installs or `curl | sh` without explicit user approval; never change the user's global environment
|
||
as a side effect.
|
||
4. If a required item is missing and cannot be installed safely: **stop before work**, ask the user
|
||
with the exact item, command, scope (project/user/system), alternatives and impact. Do not dispatch
|
||
workers while the answer is pending.
|
||
5. After approval/install, re-run the checks and the baseline suite. Record `.swarm/prerequisites.md`
|
||
and `state.json.prerequisites` as `passed`, `needs_user` or `degraded`.
|
||
6. Optional items may be skipped only with a recorded reduced scope; mark affected acceptance checks
|
||
`not_run`. Workers never install; they return `blocked` with the exact command/output.
|
||
|
||
Do not start with "I'll install it later if needed." The gate is before recon, not after the first
|
||
failure.
|
||
|
||
## 2. Recon (2–4 agents)
|
||
|
||
Give each recon agent one question, not the whole task:
|
||
- Structure: entry points, modules, build/test commands.
|
||
- Conventions: naming, error handling, test style, formatting.
|
||
- Hazards: generated files, migrations, flaky tests, shared caches, secrets.
|
||
- Prior work: existing branches, TODOs, partial implementations.
|
||
|
||
Require findings as `path:line — fact`, no proposals. Merge into `.swarm/brief.md` and `.swarm/map.md`. Cap recon at ~10% of the budget and **10–15 minutes per recon agent**; on a small repo, do it yourself.
|
||
|
||
Recon lifecycle matters: each recon agent writes its report and stops. It must kill GUI/probe processes it started, must not run the full test suite, and must not keep a slot while the build wave starts. If a recon agent is still active after its report exists, interrupt it; otherwise it silently consumes one of the active-child slots.
|
||
|
||
## 3. Plan gate
|
||
|
||
Resolve `<skill>` to the base directory reported by the skill loader. If the workspace is a git repository, add `.swarm/` to `.gitignore` before the baseline so coordination files never reach the diff. Bootstrap once:
|
||
|
||
```sh
|
||
mkdir -p .swarm/tasks .swarm/reports .swarm/reviews .swarm/tmp
|
||
cp <skill>/assets/brief.md.tmpl .swarm/brief.md
|
||
cp <skill>/assets/prerequisites.md.tmpl .swarm/prerequisites.md
|
||
cp <skill>/assets/report.schema.json .swarm/report.schema.json
|
||
cp <skill>/assets/verdict.schema.json .swarm/verdict.schema.json
|
||
node <skill>/scripts/audit-scope.mjs --init
|
||
```
|
||
|
||
Write the plan before any worker starts. Keep the plan proportionate: a small domain task may need only a brief and a verifier, not a full `.swarm` tree.
|
||
|
||
**Decomposition.** One task = one deliverable, one writer, one acceptance check, at most one page. The deliverable may be a module, a research question, a document section, a dataset, a design artifact, or a runbook. Budget **5–25 minutes for S/M** and **40–60 minutes for L**; if a task is expected to produce a single source file over ~800 lines (or a very large document/dataset), split it or accept an integration-heavy task. If it needs two writers, split it; if it needs the whole repo, it is not a task.
|
||
|
||
**Vertical slices.** Prefer one end-to-end capability at a time (selection → transform → undo → save) over a big-bang layer-by-layer build. A slice has its own checkpoint, tests and independent review; it limits late integration and repeated full reviews.
|
||
|
||
**Token shape.** Estimate the fixed cost per child, the context each task must read, and the review cost. Merge tiny tasks that pay fixed overhead for little work; split giant files that force repeated large reads. The goal is quality per token, not the smallest number of agents.
|
||
|
||
**Ownership.** Every artifact/path in exactly one `write` list: files, sections, datasets, source notes, design files. Add an explicit `LEAD` task whose list contains lead/integrator-owned shared artifacts, screenshots, licenses and the final report; without it, audit-scope reports the lead's own integration edits as unowned. Shared files are integrator-only: manifests, lockfiles, indexes/barrels, schemas, shared style/token files, migrations, i18n catalogs, snapshots, generated code. If two tasks must touch one artifact, serialize them with `depends_on` or merge them.
|
||
|
||
**Contracts.** Freeze what crosses a boundary within the chosen domain: function signatures/types/file names/CLI behavior for software; source/date/citation format for research; section structure/terminology/render target for content; schema/units/definitions for data; breakpoints/states/tokens for design; approval/rollback boundaries for operations. The invariant is the same: a verifier can check the contract independently. Put it in `.swarm/contracts.md`; workers may not change it without returning `blocked`.
|
||
|
||
**Coverage matrix.** Each success criterion → task(s) → check. A criterion with no task is a hole; a task with no criterion is scope creep.
|
||
|
||
**Visual acceptance.** For a GUI or visual product, define a deterministic screenshot/artifact command before dispatch (fixed window sizes, output path, clean pointer state) and put it in the acceptance matrix. Do not rely on desktop portal capture; it is unreliable on shared sessions and under Wayland. Visual agents must use isolated displays/configs — see `references/visual-testing.md`; never let two agents share one desktop.
|
||
|
||
**Waves.** Topologically sort by `depends_on`, then balance effort across batches. Do **not** plan 8–12 workers from a documented default: discover the runtime active-child cap first. Compute `batch_size = clamp(slot_cap - active_children - 1, 2, 8)`; recon and any other subagents count as active children. If the cap is unknown, use 6. Put foundational or risky tasks in batch 1; verify after the last build batch.
|
||
|
||
**Gate checklist:**
|
||
- [ ] Required prerequisites are `passed` (or user-approved `degraded`), recorded in `.swarm/prerequisites.md`; no `needs_user`/`blocked` item remains.
|
||
- [ ] Baseline build/test command has been run and its result recorded.
|
||
- [ ] 8+ tasks, each with objective, inputs, write scope, acceptance, stop condition, and a token/effort budget.
|
||
- [ ] Files/modules over ~800 lines are split, or the task explicitly owns the integration cost.
|
||
- [ ] Token budget, soft cap and verification reserve are recorded; units are merged/split to avoid fixed-overhead waste.
|
||
- [ ] The plan uses vertical slices/checkpoints for a multi-milestone product.
|
||
- [ ] No path in two write scopes.
|
||
- [ ] Contracts frozen for every cross-task interface.
|
||
- [ ] Coverage matrix complete.
|
||
- [ ] `slot_cap`, `active_at_dispatch`, and `batch_size` are recorded and `batch_size` fits the available slots.
|
||
- [ ] Recon agents have stopped (or have been interrupted); no extra children hold slots.
|
||
- [ ] A `LEAD`/`INTEGRATOR` task owns shared files, screenshots, licenses and the final report.
|
||
- [ ] For a visual/GUI product, a deterministic screenshot command and sizes are in the acceptance matrix.
|
||
- [ ] `.swarm/brief.md`, `plan.json`, `plan.md`, `ownership.md`, `contracts.md`, `tasks/*.md` exist.
|
||
- [ ] `audit-scope.mjs --init` recorded the baseline.
|
||
|
||
If any box is empty, fix the plan; do not dispatch.
|
||
|
||
Minimal `plan.json` shape:
|
||
|
||
```json
|
||
{
|
||
"goal": "one line",
|
||
"budget": { "tokens": 600000, "soft_cap": 0.7, "reserved_for_verify": 0.25 },
|
||
"tasks": [
|
||
{
|
||
"id": "LEAD",
|
||
"title": "lead-owned shared files, screenshots, licenses, final report",
|
||
"wave": 0,
|
||
"status": "in_progress",
|
||
"depends_on": [],
|
||
"write": ["package.json", "docs/**", "LICENSE*"],
|
||
"read": []
|
||
},
|
||
{
|
||
"id": "T01",
|
||
"title": "short title",
|
||
"wave": 1,
|
||
"status": "pending",
|
||
"depends_on": [],
|
||
"write": ["src/feature/**"],
|
||
"read": ["src/types.ts"],
|
||
"acceptance": ["pnpm vitest run src/feature"]
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
`audit-scope.mjs` reads only `id` and `write`. Keep those accurate. Use `path/**` for directories, including directories the task will create.
|
||
|
||
## 4. Build wave
|
||
|
||
Dispatch one `workflow` call per build wave (see `references/orchestration.md`). Use `assets/wave.workflow.js`, which splits tasks into batches, catches spawn errors instead of discarding the whole wave, and carries `args.concurrency`.
|
||
|
||
Before the call:
|
||
- [ ] Task cards are on disk and reference real paths/ranges, not pasted context.
|
||
- [ ] `args.tasks` carries `{id, write}` for each task.
|
||
- [ ] The report schema is passed as `args.schema`, and `args.concurrency` is at most the free slot count.
|
||
- [ ] Every task has a stop condition and a size/effort budget.
|
||
- [ ] Workers run targeted checks, not the full suite; the lead owns the full gate.
|
||
- [ ] Baseline exists and `state.json` records the cap and token budget.
|
||
|
||
After the call:
|
||
- [ ] Record measured/estimated token usage per phase if available.
|
||
- [ ] Triage structured reports: done / partial / blocked / failed.
|
||
- [ ] Treat `spawn/runtime error` and `active child limit` results as scheduling failures, not worker failures: keep completed results and dispatch only the unstarted remainder in a smaller batch.
|
||
- [ ] Run `node <skill>/scripts/audit-scope.mjs` (the lead runs it; workers do not); revert or reassign out-of-scope changes.
|
||
- [ ] Run the build/test gate yourself; do not ask workers to fix shared breakage.
|
||
- [ ] Update `plan.json` and `state.json`.
|
||
- [ ] Decide: continue, answer blocked, re-scope, or stop.
|
||
|
||
Do not paste reports into the next prompt. Summarize decisions into the brief or contracts. Open a full report only for `blocked` and `failed`.
|
||
|
||
### Answering a blocked task
|
||
|
||
A `blocked` report is a request for a lead decision, not a failure. Fast path:
|
||
1. Read the exact `blocked_question` and the relevant contract section.
|
||
2. Make the smallest possible shared/contract change yourself (or via the integrator), and update `.swarm/contracts.md` and `state.json`.
|
||
3. `send_message` to the **same** worker with the decision and any exact signatures; ask it to continue under the same write scope.
|
||
4. Do not spawn a replacement until the original worker cannot continue, and do not retry a blocked task with the same unanswered prompt.
|
||
|
||
If the block reveals an underspecified interface shared by several tasks, update the contract once and message all affected workers. If two blocked tasks need conflicting contracts, stop and re-plan.
|
||
|
||
## 5. Verify wave
|
||
|
||
Choose verifiers by risk cluster, not one per task: 4–6 verifiers for 8–12 tasks, fewer for a small domain task. A verifier gets the task cards, the changed files (or diff), and the acceptance commands. It does not get the worker's rationale and does not fix anything.
|
||
|
||
**Cheap gates first.** Run compile/typecheck/targeted tests/artifact checks before spending verifier tokens. A verifier does not review a red build; it reviews a green diff and evidence. Keep reviews to findings + commands + evidence (target 5–8k tokens); raw logs stay in artifacts.
|
||
|
||
For a GUI product, give each visual verifier its own Xvfb/display and project copy (see `references/visual-testing.md`). Capture at least the default size, the minimum supported size, empty/error states and every modal; a shared-desktop screenshot is not evidence. Computer-use is a final, single-agent exploratory tier on a disposable desktop/VM — never a parallel test mechanism and never the only evidence.
|
||
|
||
**Freeze before verify.** Once you start this wave, do not edit source, tests or docs. If you already started and then fixed something, mark the in-flight review stale and re-run that verifier on the new build. A verifier's verdict is only valid for the exact tree it saw.
|
||
|
||
Use `assets/verify.workflow.js` and `assets/verdict.schema.json`, not the full worker report schema. The verifier's detailed review **must be written to `.swarm/reviews/Vxx.md` before it returns**; the workflow return is only `{task_id, verdict, summary, review_file}`. Do not treat a workflow `null`/`no usable verdict` as a product failure if the review file exists — read the file, then re-run with the minimal schema if needed.
|
||
|
||
Evidence standard: for every criterion, a command/artifact/citation/metric and its observed result. `pass` without evidence is `not_run`. Accept `pass_with_risks` only with named risks and a follow-up task. Verifiers should copy or hash-check generated files rather than rewriting the tree; if a required gate script rewrites generated artifacts, report the hash delta.
|
||
|
||
On failure, create a fix task with the verifier's repro; do not send the verifier back to fix its own finding. After fixes, re-run only the affected **delta** (diff + findings + acceptance commands), not the whole repository, and mark the previous review stale; repeat until the remaining findings are minor/documented. Then run one final integrated verification pass on the frozen build.
|
||
|
||
## 6. Integrate
|
||
|
||
One integrator (two only for disjoint areas) owns:
|
||
- shared files and cross-cutting edits;
|
||
- merge conflicts and import/barrel fixes;
|
||
- one formatting/lint pass over changed files;
|
||
- the full acceptance suite.
|
||
|
||
No parallel workers during integration. The integrator may read all task cards and reviews; it writes `.swarm/integration.md` with what changed and why. Freeze the tree while the final verification runs; any post-verification edit invalidates the affected evidence.
|
||
|
||
## 7. Close
|
||
|
||
Final review is against the **original request**, not the plan: a fresh agent checks the success criteria, the diff, and the evidence. Then the lead writes `.swarm/final-report.md` with these sections:
|
||
|
||
- Shipped — what exists now, with paths.
|
||
- Not shipped / deferred — and why.
|
||
- Evidence — commands and observed results.
|
||
- Risks and open questions.
|
||
- Cost — agents, waves, wall-clock.
|
||
- Artifacts — `.swarm/...` paths.
|
||
|
||
## Failure recovery
|
||
|
||
| Signal | Likely cause | Action |
|
||
|---|---|---|
|
||
| Workers edit the same file | ownership map missed a shared file | stop, reassign, serialize, re-audit |
|
||
| Many `blocked` on the same interface | contract underspecified | answer once in `contracts.md`, retry those tasks only |
|
||
| Many schema `null`s | schema too strict or prompt unclear | loosen schema, add one example to the task card |
|
||
| Wave slower than serial | units too small or too coupled | merge units, reduce batch size |
|
||
| Workflow dies with `active child limit reached` | batch sized from wrong cap; recon still active | shrink batch, interrupt finished recon, dispatch only unstarted tasks |
|
||
| `no usable verdict` but review files exist | verifier schema miss / prose return | read review files; switch to `verdict.schema.json`; re-run if stale |
|
||
| Verifier reviewed an older build | source edited during verify | freeze, re-run affected verifier, mark old review stale |
|
||
| `blocked` task with a shared-file need | contract omits an integration point | lead applies the change, messages the same worker, no replacement |
|
||
| Verifiers find the same defect | brief missing a convention | fix the brief, add a check, re-run affected tasks |
|
||
| Lead context filling up | reading full reports | read summary + evidence; open only exceptions |
|
||
| A worker hangs | no per-agent timeout exists | kill the job, split the task, add a tighter stop condition |
|
||
| Missing tool/runtime discovered mid-wave | prerequisite gate skipped | stop, run `prerequisites.md`, ask user, do not spawn replacements |
|
||
| Version mismatch with lockfile | unpinned global toolchain | use project-local pinned version; record it and re-run baseline |
|
||
| Required credential/display unavailable | not checked before dispatch | ask user for exact item/scope; degrade only with recorded reduced scope |
|
||
| Token spend explodes | huge files, too many agents, full re-reads/reviews | split files, right-size batch, delta-review, record usage |
|
||
| Verifier repeats a full review after a small fix | no delta procedure | review diff + findings + affected checks only |
|
||
| Reports/reviews dominate input | oversized prose | cap to evidence lines; logs in artifacts |
|
||
| Visual agents produce wrong/identical screenshots | shared desktop or portal capture | isolate per-agent Xvfb/config/project; use in-app screenshots |
|
||
| A visual "pass" has no artifact | verifier skipped the visual tier | treat as `not_run`; require screenshot path + command |
|
||
|
||
## Time and cost
|
||
|
||
Parallel speedup is bounded by the sequential share, the real slot cap and token cost, not by the number of tasks. Push sequential work down: recon once, contracts before dispatch, verification in parallel. A 36-unit task with 10-minute units and a 6-slot batch is ~6 batches × 10 min ≈ 60 min of agent work, plus plan, verify and integration; do not promise a 3× speedup when the host caps active children at 8 or fewer.
|
||
|
||
Token cost is dominated by fixed per-child overhead and repeated context reads. Eight substantial workers are often cheaper and better than twenty tiny ones; one delta re-check is much cheaper than a second full review. Track tokens per accepted task and per confirmed defect; optimize the ratio, not raw spend. See `references/token-economy.md`.
|