Add token budgets, telemetry and delta verification

This commit is contained in:
Emil Shanaty
2026-10-27 19:36:02 +03:00
parent 74fdfa458c
commit c3f58ceb40
9 changed files with 152 additions and 17 deletions
+2 -1
View File
@@ -14,6 +14,7 @@
- [Runbook](agent-swarm/references/playbook.md)
- [Cookbook по `workflow`](agent-swarm/references/orchestration.md)
- [Domain-адаптеры](agent-swarm/references/domains.md)
- [Token economy](agent-swarm/references/token-economy.md)
- [Изолированное визуальное тестирование](agent-swarm/references/visual-testing.md)
### Установка
@@ -32,7 +33,7 @@ cp -r skills/agent-swarm ~/.dsh/skills/
agent-swarm/
├── SKILL.md # основной протокол
├── README.ru.md # краткое описание
├── references/ # playbook, packets, orchestration, domains, visual testing
├── references/ # playbook, packets, orchestration, domains, visual testing, token economy
├── assets/ # шаблоны, схемы, workflow-скрипты
└── scripts/audit-scope.mjs # проверка ownership
```
+1
View File
@@ -33,6 +33,7 @@
- Верификаторы работают на замороженном дереве; полный разбор пишется в `.swarm/reviews/Vxx.md`, а workflow возвращает только `{task_id, verdict, summary, review_file}`.
- `blocked`-воркер не заменяется: лид правит контракт/shared-файл и продолжает того же агента через `send_message`.
- Evidence зависит от домена: команда и тест для кода, источник и дата для research, отрендеренный артефакт для дизайна, reproducible metric для данных.
- Токены — такой же бюджет, как время: 20–30% резервируется на verify/integration, после фиксов делается delta-review, а не полная перепроверка. Детали — в `references/token-economy.md`.
- Визуальные агенты работают в отдельных Xvfb/display-окружениях, со своим HOME/config/копией проекта; общий рабочий стол для скриншотов не используется. Детали — в `references/visual-testing.md`.
- Computer-use — крайний tier: один агент, одноразовый desktop/VM, только после детерминированных проверок и никогда как единственное evidence.
- Стоимость: 30–40 агентов ≈ 0.5–1.5M токенов.
+6 -4
View File
@@ -40,12 +40,12 @@ Otherwise: one agent for small work; 2–5 `subagent` calls for a few long-lived
## Protocol
0. **Fit check & preflight** — go/no-go, budget (`max_agents`, `max_waves`), and **discover the available tools and real active-child cap** before sizing a wave. Record in `state.json`: tool availability (`workflow`, `subagent`, display/artifact command, build commands), slot cap, active children, planned batch size, and the chosen domain adapter.
0. **Fit check & preflight** — go/no-go, budget (`max_agents`, `max_waves`, **tokens**), and **discover the available tools and real active-child cap** before sizing a wave. Record in `state.json`: tool availability (`workflow`, `subagent`, display/artifact command, build commands), slot cap, active children, planned batch size, chosen domain adapter, token budget/soft cap/reserve, and whether usage is measured or estimated (see `references/token-economy.md`).
1. **Recon** — 2–4 parallel agents return findings with paths and line numbers. Timebox them, merge into `.swarm/brief.md` and `.swarm/map.md`, then stop them; recon must not hold slots during the build wave.
2. **Plan gate** — write `.swarm/plan.json`, `plan.md`, `ownership.md`, `contracts.md`, and one `tasks/Txx.md` per unit. Every success criterion maps to a task and a check. No two tasks write the same path. Include a `LEAD`/`INTEGRATOR` task owning lead-owned shared files, screenshots, licenses and the final report. Freeze interfaces. **Do not dispatch until this gate passes.**
2. **Plan gate** — write `.swarm/plan.json`, `plan.md`, `ownership.md`, `contracts.md`, and one `tasks/Txx.md` per unit. Every success criterion maps to a task and a check. No two tasks write the same path. Include a `LEAD`/`INTEGRATOR` task owning lead-owned shared files, screenshots, licenses and the final report. Freeze interfaces. Prefer vertical slices and files under ~800 lines; a task that requires reading a 3k-line file is too broad. **Do not dispatch until this gate passes.**
3. **Build wave** — one `workflow` call per build wave when the tool exists, using the batched runner in `assets/wave.workflow.js`; its `concurrency` must fit the available slots (see Sizing). If `workflow`/`subagent` is unavailable, the lead executes the same packets sequentially. Retry `null`/`failed` once; if the runtime reports an active-child limit, halve the next batch and dispatch the remaining tasks — never relaunch the same oversubscribed fan-out.
4. **Audit** — run `scripts/audit-scope.mjs` from the skill base directory; revert or reassign out-of-scope changes. Run the build/test gate.
5. **Verify wave** — **freeze source changes**, then run 4–6 fresh verifiers by risk cluster (or 1–2 for a small domain task). Use `assets/verdict.schema.json` and, when available, `assets/verify.workflow.js`; the review file on disk is the source of truth, not the workflow return. Verifiers never fix. Match evidence to the domain adapter: rerun commands, read rendered artifacts, sample citations, recompute metrics. For GUI products, use per-agent isolated displays, never a shared desktop (see `references/visual-testing.md`). If you fix findings, re-run the affected verifier and mark the old review stale.
5. **Verify wave** — **freeze source changes**, run automated gates first (compile/tests/artifact), then 4–6 fresh verifiers by risk cluster (or 1–2 for a small domain task). Use `assets/verdict.schema.json` and, when available, `assets/verify.workflow.js`; the review file on disk is the source of truth, not the workflow return. Verifiers never fix. Match evidence to the domain adapter: rerun commands, read rendered artifacts, sample citations, recompute metrics. For GUI products, use per-agent isolated displays, never a shared desktop (see `references/visual-testing.md`). If you fix findings, re-run only the affected **delta** and mark the old review stale.
6. **Integrate** — one integrator for shared files and cross-cutting fixes; re-run the full acceptance suite; then a final verification pass on the integrated build.
7. **Close** — freeze the build and run a final independent review against the original request (not the plan), then write `final-report.md`.
@@ -60,6 +60,7 @@ Otherwise: one agent for small work; 2–5 `subagent` calls for a few long-lived
7. **Freeze during verification.** Once the verify wave starts, do not edit source/tests/docs except to fix a verified finding. After a fix, re-run the affected verifier; an old review is stale.
8. **Stop rules.** Two waves with no net progress → stop and re-plan. Verification failure rate > 30% → fix briefs/contracts, not agent count. Budget exhausted → report partial and list unverified areas.
9. **No shared visual desktop.** Visual agents get their own display/config/project copy; see `references/visual-testing.md`. A screenshot from a shared desktop is not evidence. Computer-use is a last-resort, single-agent tier on a disposable desktop/VM; it is never the sole evidence and never runs in parallel with other visual agents.
10. **Token budget is a quality constraint.** Record spend per phase, keep a 20–30% reserve for verification/integration, and stop at the hard cap with a partial report. Cut redundant context, agents, full-file reads and full re-reviews — never cut tests or independent review. See `references/token-economy.md`.
## Sizing
@@ -90,7 +91,7 @@ Otherwise: one agent for small work; 2–5 `subagent` calls for a few long-lived
baseline.json pre-swarm file manifest (`--init`)
report.schema.json report contract for workers
verdict.schema.json minimal contract for verifiers
state.json wave, budget spent, open issues
state.json wave, token budget/spend per phase, open issues
final-report.md
tmp/T01/ worker scratch, never shared
```
@@ -107,6 +108,7 @@ For a `blocked` task: apply the smallest contract/shared-file change, update `co
- `references/packets.md` — how to write briefs, task cards, and verifier cards; examples and common mistakes.
- `references/orchestration.md` — `workflow` cookbook: script patterns, schema subset, retries, background runs, archetypes (code / research / content).
- `references/domains.md` — adapters for software, research, content, data, design and operations.
- `references/token-economy.md` — measurement, budgets, context discipline, delta verification, vertical slices.
- `references/visual-testing.md` — isolated Xvfb/container visual tests, deterministic screenshots, verifier procedure (GUI adapter).
- `assets/` — copyable brief, task, and verifier templates; `report.schema.json`, `verdict.schema.json`; `wave.workflow.js` (batched workers), `verify.workflow.js` (frozen verification).
- `scripts/audit-scope.mjs` — compares changed files to the ownership map.
+5
View File
@@ -41,6 +41,11 @@ Scope is checked by the lead with the skill's `scripts/audit-scope.mjs`; workers
- {{Generated files, migrations, flaky tests, shared ports, secrets, caches.}}
- {{Generated or vendored paths that must not be edited.}}
## Token / effort budget
- Hard budget: {{tokens or n/a}}; reserve {{20–30%}} for verification and integration.
- Prefer vertical slices and files under ~800 lines.
- Workers read the listed inputs/ranges, not the whole repository.
## Frozen contracts
See `.swarm/contracts.md`. Do not change an interface without returning `blocked`.
+3
View File
@@ -28,6 +28,8 @@ Role: worker | Wave: {{n}} | Depends on: {{ids or —}} | Effort: {{S/M/L}}
## Constraints
- {{No new dependencies, style, performance, compatibility.}}
- Context budget: read only the inputs above; if you need more than {{N}} additional files, return `blocked` instead of exploring the whole repo.
- Do not run the full test suite unless the card explicitly asks; run targeted checks.
## Acceptance (re-runnable)
1. `{{command or artifact path}}` → {{expected result/state}}
@@ -36,6 +38,7 @@ Role: worker | Wave: {{n}} | Depends on: {{ids or —}} | Effort: {{S/M/L}}
## Report
`.swarm/reports/{{ID}}.json` per `.swarm/report.schema.json`; status `done|partial|blocked|failed`.
Keep the report to summary + evidence lines; raw logs go to `.swarm/tmp/{{ID}}/`.
## Stop conditions
- Time box: 25 minutes for S/M, 45–60 minutes for L, or 3 failed attempts at the same step.
+1
View File
@@ -34,6 +34,7 @@ Computer-use (mouse/keyboard) is an optional last-resort tier: one agent, dispos
`pass` / `pass_with_risks` / `fail` — with the top risks and exact repro commands.
## Report
Keep the review to findings + commands + evidence (target 5–8k tokens); raw logs stay in `.swarm/tmp/`. Review the diff and affected checks, not the whole repository.
FIRST write the full review to `.swarm/reviews/{{ID}}.md`. Then return only the minimal structured verdict:
`{task_id, verdict, summary, review_file}` per `.swarm/verdict.schema.json`.
The review file is the source of truth; it must exist even if the structured reply is lost. Do not return a long findings array in the workflow schema.
+1 -1
View File
@@ -208,4 +208,4 @@ If the workflow returns `null`/`no usable verdict`, check the review file before
## Cost
Each child carries the system prompt, tool schemas, packet, code reads, and its output: typically 10–40k tokens. A 12-agent wave is roughly 150–400k tokens, but a host capped at 8 cannot run 12 at once; batches reduce peak concurrency, not total cost. Budget before dispatch and count retries/re-verifications; stop at `max_agents`.
Each child carries the system prompt, tool schemas, packet, code reads, and its output: typically 10–25k tokens of fixed overhead before any real work. A 12-agent wave can exceed 300k tokens, and a host capped at 8 cannot run 12 at once; batching reduces peak concurrency, not total cost. Budget before dispatch and count retries, delta re-checks and reviews; stop at the token cap. Eight substantial workers are often cheaper and better than twenty tiny ones. See `references/token-economy.md`.
+30 -11
View File
@@ -14,9 +14,11 @@ Answer these before spawning:
Budget:
- `agents ≈ units + ceil(units / 5) verifiers + 1–2 integrators + 2–4 recon`; add retries and re-verifications explicitly. Set `max_agents` only after checking available tools and slot cap.
- Tokens ≈ `agents × 20k` (10–40k per agent depending on code reads).
- Tokens ≈ `agents × fixed overhead (10–25k) + context reads + output + reviews`. Treat this as an estimate, then calibrate from measured usage. A 3k-line file read by several agents can cost more than the code they write.
- Reserve **20–30%** for verification, integration and one fix round. Set a soft cap at ~70% and a hard cap; stop at the hard cap with a partial report.
- Waves ≈ `ceil(units / batch_size)` + verify + integrate, where `batch_size` comes from the **real** active-child cap, not from a documented default.
- Stop at `max_agents` or `max_waves`, whichever comes first. Record both, plus `slot_cap`, `active_at_dispatch`, `batch_size`, retries and re-verifications in `state.json`.
- Stop at `max_agents`, `max_waves` or the token cap, whichever comes first. Record all caps, `slot_cap`, `active_at_dispatch`, `batch_size`, measured/estimated usage per phase, retries and re-verifications in `state.json`.
- See `references/token-economy.md`; do not invent precise token numbers without measurement.
Go: a 40-file migration with per-module tests, a 30-source research report, a 20-section document with a style guide, a multi-dataset analysis, a multi-screen visual redesign, a multi-service runbook.
No-go: one failing test, one architectural decision, a detail pass where every unit touches the same artifact.
@@ -47,7 +49,11 @@ node <skill>/scripts/audit-scope.mjs --init
Write the plan before any worker starts. Keep the plan proportionate: a small domain task may need only a brief and a verifier, not a full `.swarm` tree.
**Decomposition.** One task = one deliverable, one writer, one acceptance check, at most one page. The deliverable may be a module, a research question, a document section, a dataset, a design artifact, or a runbook. Budget **5–25 minutes for S/M** and **40–60 minutes for L**; if a task is expected to produce a single source file over ~1,500 lines (or a very large document/dataset), split it or accept an integration-heavy task. If it needs two writers, split it; if it needs the whole repo, it is not a task.
**Decomposition.** One task = one deliverable, one writer, one acceptance check, at most one page. The deliverable may be a module, a research question, a document section, a dataset, a design artifact, or a runbook. Budget **5–25 minutes for S/M** and **40–60 minutes for L**; if a task is expected to produce a single source file over ~800 lines (or a very large document/dataset), split it or accept an integration-heavy task. If it needs two writers, split it; if it needs the whole repo, it is not a task.
**Vertical slices.** Prefer one end-to-end capability at a time (selection → transform → undo → save) over a big-bang layer-by-layer build. A slice has its own checkpoint, tests and independent review; it limits late integration and repeated full reviews.
**Token shape.** Estimate the fixed cost per child, the context each task must read, and the review cost. Merge tiny tasks that pay fixed overhead for little work; split giant files that force repeated large reads. The goal is quality per token, not the smallest number of agents.
**Ownership.** Every artifact/path in exactly one `write` list: files, sections, datasets, source notes, design files. Add an explicit `LEAD` task whose list contains lead/integrator-owned shared artifacts, screenshots, licenses and the final report; without it, audit-scope reports the lead's own integration edits as unowned. Shared files are integrator-only: manifests, lockfiles, indexes/barrels, schemas, shared style/token files, migrations, i18n catalogs, snapshots, generated code. If two tasks must touch one artifact, serialize them with `depends_on` or merge them.
@@ -60,7 +66,10 @@ Write the plan before any worker starts. Keep the plan proportionate: a small do
**Waves.** Topologically sort by `depends_on`, then balance effort across batches. Do **not** plan 8–12 workers from a documented default: discover the runtime active-child cap first. Compute `batch_size = clamp(slot_cap - active_children - 1, 2, 8)`; recon and any other subagents count as active children. If the cap is unknown, use 6. Put foundational or risky tasks in batch 1; verify after the last build batch.
**Gate checklist:**
- [ ] 8+ tasks, each with objective, inputs, write scope, acceptance, stop condition.
- [ ] 8+ tasks, each with objective, inputs, write scope, acceptance, stop condition, and a token/effort budget.
- [ ] Files/modules over ~800 lines are split, or the task explicitly owns the integration cost.
- [ ] Token budget, soft cap and verification reserve are recorded; units are merged/split to avoid fixed-overhead waste.
- [ ] The plan uses vertical slices/checkpoints for a multi-milestone product.
- [ ] No path in two write scopes.
- [ ] Contracts frozen for every cross-task interface.
- [ ] Coverage matrix complete.
@@ -78,6 +87,7 @@ Minimal `plan.json` shape:
```json
{
"goal": "one line",
"budget": { "tokens": 600000, "soft_cap": 0.7, "reserved_for_verify": 0.25 },
"tasks": [
{
"id": "LEAD",
@@ -109,13 +119,15 @@ Minimal `plan.json` shape:
Dispatch one `workflow` call per build wave (see `references/orchestration.md`). Use `assets/wave.workflow.js`, which splits tasks into batches, catches spawn errors instead of discarding the whole wave, and carries `args.concurrency`.
Before the call:
- [ ] Task cards are on disk and reference real paths.
- [ ] Task cards are on disk and reference real paths/ranges, not pasted context.
- [ ] `args.tasks` carries `{id, write}` for each task.
- [ ] The report schema is passed as `args.schema`, and `args.concurrency` is at most the free slot count.
- [ ] Every task has a stop condition.
- [ ] Baseline exists and `state.json` records the cap.
- [ ] Every task has a stop condition and a size/effort budget.
- [ ] Workers run targeted checks, not the full suite; the lead owns the full gate.
- [ ] Baseline exists and `state.json` records the cap and token budget.
After the call:
- [ ] Record measured/estimated token usage per phase if available.
- [ ] Triage structured reports: done / partial / blocked / failed.
- [ ] Treat `spawn/runtime error` and `active child limit` results as scheduling failures, not worker failures: keep completed results and dispatch only the unstarted remainder in a smaller batch.
- [ ] Run `node <skill>/scripts/audit-scope.mjs` (the lead runs it; workers do not); revert or reassign out-of-scope changes.
@@ -137,7 +149,9 @@ If the block reveals an underspecified interface shared by several tasks, update
## 4. Verify wave
Choose verifiers by risk cluster, not one per task: 4–6 verifiers for 8–12 tasks. A verifier gets the task cards, the changed files (or diff), and the acceptance commands. It does not get the worker's rationale and does not fix anything.
Choose verifiers by risk cluster, not one per task: 4–6 verifiers for 8–12 tasks, fewer for a small domain task. A verifier gets the task cards, the changed files (or diff), and the acceptance commands. It does not get the worker's rationale and does not fix anything.
**Cheap gates first.** Run compile/typecheck/targeted tests/artifact checks before spending verifier tokens. A verifier does not review a red build; it reviews a green diff and evidence. Keep reviews to findings + commands + evidence (target 5–8k tokens); raw logs stay in artifacts.
For a GUI product, give each visual verifier its own Xvfb/display and project copy (see `references/visual-testing.md`). Capture at least the default size, the minimum supported size, empty/error states and every modal; a shared-desktop screenshot is not evidence. Computer-use is a final, single-agent exploratory tier on a disposable desktop/VM — never a parallel test mechanism and never the only evidence.
@@ -145,9 +159,9 @@ For a GUI product, give each visual verifier its own Xvfb/display and project co
Use `assets/verify.workflow.js` and `assets/verdict.schema.json`, not the full worker report schema. The verifier's detailed review **must be written to `.swarm/reviews/Vxx.md` before it returns**; the workflow return is only `{task_id, verdict, summary, review_file}`. Do not treat a workflow `null`/`no usable verdict` as a product failure if the review file exists — read the file, then re-run with the minimal schema if needed.
Evidence standard: for every criterion, a command and its observed result. `pass` without evidence is `not_run`. Accept `pass_with_risks` only with named risks and a follow-up task. Verifiers should copy or hash-check generated files rather than rewriting the tree; if a required gate script rewrites generated artifacts, report the hash delta.
Evidence standard: for every criterion, a command/artifact/citation/metric and its observed result. `pass` without evidence is `not_run`. Accept `pass_with_risks` only with named risks and a follow-up task. Verifiers should copy or hash-check generated files rather than rewriting the tree; if a required gate script rewrites generated artifacts, report the hash delta.
On failure, create a fix task with the verifier's repro; do not send the verifier back to fix its own finding. After fixes, re-run the affected verifier and mark the previous review stale; repeat until the remaining findings are minor/documented. Then run a final integrated verification pass.
On failure, create a fix task with the verifier's repro; do not send the verifier back to fix its own finding. After fixes, re-run only the affected **delta** (diff + findings + acceptance commands), not the whole repository, and mark the previous review stale; repeat until the remaining findings are minor/documented. Then run one final integrated verification pass on the frozen build.
## 5. Integrate
@@ -185,9 +199,14 @@ Final review is against the **original request**, not the plan: a fresh agent ch
| Verifiers find the same defect | brief missing a convention | fix the brief, add a check, re-run affected tasks |
| Lead context filling up | reading full reports | read summary + evidence; open only exceptions |
| A worker hangs | no per-agent timeout exists | kill the job, split the task, add a tighter stop condition |
| Token spend explodes | huge files, too many agents, full re-reads/reviews | split files, right-size batch, delta-review, record usage |
| Verifier repeats a full review after a small fix | no delta procedure | review diff + findings + affected checks only |
| Reports/reviews dominate input | oversized prose | cap to evidence lines; logs in artifacts |
| Visual agents produce wrong/identical screenshots | shared desktop or portal capture | isolate per-agent Xvfb/config/project; use in-app screenshots |
| A visual "pass" has no artifact | verifier skipped the visual tier | treat as `not_run`; require screenshot path + command |
## Time and cost
Parallel speedup is bounded by the sequential share and by the real slot cap, not by the number of tasks. Push sequential work down: recon once, contracts before dispatch, verification in parallel. A 36-unit task with 10-minute units and a 6-slot batch is ~6 batches × 10 min ≈ 60 min of agent work, plus plan, verify and integration; do not promise a 3× speedup when the host caps active children at 8 or fewer.
Parallel speedup is bounded by the sequential share, the real slot cap and token cost, not by the number of tasks. Push sequential work down: recon once, contracts before dispatch, verification in parallel. A 36-unit task with 10-minute units and a 6-slot batch is ~6 batches × 10 min ≈ 60 min of agent work, plus plan, verify and integration; do not promise a 3× speedup when the host caps active children at 8 or fewer.
Token cost is dominated by fixed per-child overhead and repeated context reads. Eight substantial workers are often cheaper and better than twenty tiny ones; one delta re-check is much cheaper than a second full review. Track tokens per accepted task and per confirmed defect; optimize the ratio, not raw spend. See `references/token-economy.md`.
+103
View File
@@ -0,0 +1,103 @@
# Token economy
Quality per token beats raw token spend. The expensive part of a swarm is usually **input context and
repeated reading**, not the code a worker writes. Measure, budget, then cut redundancy — never cut
the independent checks that catch real defects.
## Measure before optimizing
Record usage per phase and per agent in `.swarm/state.json`:
```json
{
"budget": { "tokens": 600000, "soft_cap": 0.7, "reserved_for_verify": 0.2 },
"usage": [
{ "phase": "recon", "agent": "R01", "tokens_in": 18000, "tokens_out": 4000 },
{ "phase": "build", "agent": "T03", "tokens_in": 24000, "tokens_out": 9000 }
]
}
```
If the runtime does not expose usage, estimate from context size and output length and mark the value
as `estimated`. The point is a feedback loop, not accounting precision.
Useful ratios:
- tokens per accepted task;
- verifier tokens per confirmed defect;
- rework tokens caused by a missed contract;
- share of spend on recon / build / verify / integration.
## Budgets and reserves
- Set a **hard budget** for the milestone and a **soft cap** at ~70%. At the soft cap, stop expanding
scope; finish the current slice and verify.
- Reserve **20–30%** for verification, integration and one round of fixes. Do not spend the whole
budget on generation.
- Budget per phase, not only per wave: recon, build, verify, integrate.
- Stop at the cap and report partial results plus unverified areas. Silent overspend is worse than a
partial result.
- Calibrate from the first measured milestone; do not copy invented numbers from a template.
## Context discipline
The biggest savings without quality loss:
1. **Small files.** Split modules above ~800 lines (UI panels, viewport, inspector). A worker should
read 300–800 relevant lines, not 3,000. Smaller modules also compile and verify faster.
2. **Read ranges, not repositories.** Use `path:line`, symbol search and grep first. Do not read a
whole crate to change one function.
3. **Point to artifacts, do not paste them.** Task cards reference `.swarm/brief.md`,
`.swarm/contracts.md` and exact ranges; they do not duplicate them in the prompt.
4. **Cap reports and reviews.** Worker report: summary, files/artifacts, evidence, issues,
assumptions. Review: findings + commands + evidence; target 5–8k tokens, raw logs stay in files.
5. **One page per task.** If a card needs more, the task is too broad.
6. **No full-suite runs per worker.** Workers run targeted checks; the lead runs the full gate once
per wave.
7. **Fresh context only where independence matters.** The final verifier needs a clean read. A fix
re-check can be a **delta review**: diff + findings + acceptance commands, not the whole repo.
## Verification ladders
Do not spend LLM review tokens on a red build:
1. compile / typecheck / lint;
2. targeted tests;
3. artifact/render/source check;
4. independent verifier on the diff and evidence;
5. final full review once, on a frozen integrated build.
Freeze before step 4. If code changes, re-check only the affected delta; mark the old review stale.
This preserves quality while avoiding repeated full reviews.
## Orchestration shape
- **Fixed overhead dominates small tasks.** Each child carries system prompt, tools, brief and task
context. 30 small agents can cost more than 8 substantial ones.
- **Right-size:** 2–4 implementers + 1 verifier for a vertical slice; 6–10 agents for a broad
milestone; 30 only for genuinely independent breadth.
- **Model routing:** strong model for architecture, difficult fixes and final review; cheaper/faster
model for tests, docs, screenshots and mechanical edits.
- **Vertical slices:** ship one end-to-end feature at a time (selection → transform → undo → save),
with a checkpoint and verification after each. Do not generate the whole product and verify at the
end.
- **Reuse:** do not regenerate a working core, format or codegen. Extend it.
## Anti-patterns
| Pattern | Cost | Fix |
|---|---|---|
| 3k-line file edited by several agents | repeated reads, merge conflicts | split before dispatch |
| Full repo read for a one-function change | huge input | path:line + grep |
| Full re-review after every fix | duplicate verifier spend | delta review |
| Verifier runs on a failing build | wasted review | automated gate first |
| 20 tiny tasks | fixed overhead per child | merge into coherent units |
| Reports/reviews as essays | output + lead input | schema + evidence lines |
| Whole product in one wave | late integration, huge rework | vertical slices + checkpoints |
| Vague "max quality" with no budget | unbounded spend | hard/soft caps + reserve |
## Minimal operating rule
> Measure per phase, reserve for verification, keep files and packets small, verify the diff after
> fixes, and spend the saved tokens on one strong independent review rather than many redundant
> agents.