Simplify agent-swarm to a lite core with optional modules and evals
This commit is contained in:
Emil Shanaty
@@ -4,10 +4,10 @@
|
||||
|
||||
## agent-swarm
|
||||
|
||||
Универсальные большие задачи, которые делятся на 8+ независимых кусков: код и миграции,
|
||||
аудиты, research, многосекционные документы, данные, дизайн, инфраструктура. Лид планирует
|
||||
и интегрирует, воркеры работают по пакетам с непересекающимися файлами/артефактами, а
|
||||
независимые верификаторы проверяют артефакт, а не отчёт воркера.
|
||||
Тонкий оркестрационный скилл для больших или рискованных задач: код и миграции, аудиты,
|
||||
research, документы, данные, дизайн, инфраструктура. Режимы direct / paired critic / swarm;
|
||||
процесс опциональный, инварианты — обязательные: ownership, независимая проверка, evidence,
|
||||
product acceptance. Тяжёлые темы вынесены в опциональные references.
|
||||
|
||||
- [Описание на русском](agent-swarm/README.ru.md)
|
||||
- [Инструкции скилла](agent-swarm/SKILL.md)
|
||||
@@ -18,6 +18,7 @@
|
||||
- [Prerequisite gate](agent-swarm/references/prerequisites.md)
|
||||
- [Product quality gates](agent-swarm/references/product-quality.md)
|
||||
- [Изолированное визуальное тестирование](agent-swarm/references/visual-testing.md)
|
||||
- [Evals самого скилла](agent-swarm/evals/README.md)
|
||||
|
||||
### Установка
|
||||
|
||||
@@ -33,9 +34,10 @@ cp -r skills/agent-swarm ~/.dsh/skills/
|
||||
|
||||
```
|
||||
agent-swarm/
|
||||
├── SKILL.md # основной протокол
|
||||
├── SKILL.md # тонкий core: режимы + инварианты
|
||||
├── README.ru.md # краткое описание
|
||||
├── references/ # playbook, packets, orchestration, domains, visual testing, token economy, prerequisites, product quality
|
||||
├── evals/ # baseline vs with-skill, 3 задачи, scorecard
|
||||
├── references/ # опциональные модули: playbook, packets, domains, quality, tokens...
|
||||
├── assets/ # шаблоны, схемы, workflow-скрипты
|
||||
└── scripts/audit-scope.mjs # проверка ownership
|
||||
```
|
||||
|
||||
+61
-29
@@ -1,6 +1,8 @@
|
||||
# agent-swarm
|
||||
|
||||
Универсальный скилл для больших задач, которые делятся на 8+ независимых кусков: код и миграции, аудиты, research, документы, данные, дизайн, инфраструктура. Лид планирует, пакует контекст, запускает волны (через `workflow`, `subagent` или последовательно — если инструмента нет) и интегрирует. Воркеры не пересекаются по файлам/артефактам, каждый получает пакет и возвращает структурный отчёт.
|
||||
Тонкий оркестрационный скилл для больших или рискованных задач: код и миграции, аудиты, research,
|
||||
документы, данные, дизайн, инфраструктура. Это **не обязательный пайплайн**, а набор режимов и
|
||||
инвариантов: direct / paired critic / swarm.
|
||||
|
||||
## Вызов
|
||||
|
||||
@@ -10,46 +12,76 @@
|
||||
|
||||
или просто опишите задачу — скилл подключится по описанию.
|
||||
|
||||
Скилл не обязателен для каждой задачи: маленькую задачу можно делать напрямую, среднюю — с независимым критиком, и только широкую — через swarm. Пользователь выбирает стек, объём и темп; скилл добавляет координацию, когда она оправдана.
|
||||
## Режимы
|
||||
|
||||
Процесс — опциональный, quality gates — нет. Для продуктовой работы заранее выбирается профиль `prototype` или `product`. В `product` обязательны release-замеры производительности, реальный UX-сценарий, сравнение с baseline и явный вердикт пользователя. Технический `pass` verifier'а не заменяет приёмку продукта.
|
||||
| Режим | Когда | Что добавляет |
|
||||
|---|---|---|
|
||||
| Direct | одна задача, быстрый фидбек, низкий риск | ничего |
|
||||
| Paired critic | средняя задача или рискованное решение | независимую проверку |
|
||||
| Swarm | 8+ независимых кусков или ширина | ownership, параллельных воркеров, verifier |
|
||||
|
||||
Если куски делят один экран, один дизайн-язык или один mutable-артефакт — не swarm. Один
|
||||
implementer + один critic обычно дают более цельный UI, чем параллельные владельцы панелей.
|
||||
|
||||
## Инварианты
|
||||
|
||||
1. Сначала outcome и quality bar, потом процесс.
|
||||
2. Один владелец на артефакт; shared-файлы — у лида/интегратора.
|
||||
3. Воркер получает пакет, а не всю историю чата.
|
||||
4. Автор не проверяет сам себя; для рискованной работы нужен свежий контекст.
|
||||
5. «Готово» — это runnable artifact, команда, цитата, метрика или рендер, а не уверенная фраза.
|
||||
6. Технический `pass` — не product acceptance. Для product/UI: release-замеры, реальный сценарий,
|
||||
baseline и явный вердикт пользователя.
|
||||
7. Если лагает или выглядит хуже baseline — чинить, а не добавлять фичи.
|
||||
8. Для product — один milestone за раз; следующий не начинать без приёмки.
|
||||
9. Проверять prerequisites; глобальные установки/деструктив — только с разрешения.
|
||||
10. Инструкции пользователя важнее скилла: попросил быстро/прототип — убрать церемонии, оставить
|
||||
safety и evidence.
|
||||
|
||||
## Что происходит
|
||||
|
||||
1. **Fit-check.** Если задача не делится на 8+ независимых юнитов, скилл скажет не использовать swarm.
|
||||
2. **Prerequisite gate.** Скилл проверяет toolchain, runtime, сервисы, display, credentials и baseline. Ставит только project-local/pinned и только с разрешения; если обязательный prerequisite не закрывается — спрашивает пользователя **до** запуска работы. Воркеры ничего не устанавливают.
|
||||
3. **Recon.** 2–4 агента картографируют репозиторий/источники/данные; лид пишет `.swarm/brief.md`, `.swarm/profile.md` и выбирает domain-адаптер из `references/domains.md`.
|
||||
4. **План.** Задачи, владельцы файлов, контракты, критерии приёмки. Гейт: без этого волны не стартуют.
|
||||
5. **Волны.** Воркеры идут батчами по 6–8 (реальный лимит активных детей узнаётся перед запуском и записывается в `state.json`; recon не должен занимать слоты). Затем 4–6 независимых верификаторов на «замороженном» дереве, затем интегратор.
|
||||
6. **Аудит.** `scripts/audit-scope.mjs` сравнивает изменения с картой владения.
|
||||
7. **Интеграция.** Один интегратор собирает общие файлы и прогоняет полный acceptance suite.
|
||||
8. **Product gate.** Для `product`: release-замеры, реальный сценарий, baseline и вердикт пользователя. Без этого следующий milestone не начинается.
|
||||
9. **Отчёт.** `.swarm/final-report.md`.
|
||||
1. **Понять** outcome, аудиторию, quality bar и ограничения.
|
||||
2. **Проверить** окружение: toolchain, команды, сервисы, display, credentials.
|
||||
3. **Спланировать только необходимое.** Для swarm — brief, disjoint scopes, contracts, acceptance.
|
||||
Для direct/paired — без `.swarm`-бюрократии.
|
||||
4. **Сделать** напрямую, с критиком или батчами воркеров.
|
||||
5. **Проверить** технически, затем независимым критиком; для product — performance/UX/baseline.
|
||||
6. **Принять и интегрировать.** Лид владеет shared-файлами и финальной проверкой.
|
||||
|
||||
## Артефакты
|
||||
|
||||
Всё лежит в `.swarm/` в корне рабочей папки: бриф, профиль домена, план, карточки задач, отчёты, ревью, baseline, финальный отчёт. `.swarm/` можно удалить после задачи или добавить в `.gitignore`; структура подходит и для кода, и для research/контента/данных/дизайна.
|
||||
Для swarm **опционально**: `.swarm/brief.md`, `tasks/`, `reports/`, `reviews/`. Они нужны для
|
||||
координации и восстановления, а не для отчётности ради отчётности. Для direct/paired их можно не
|
||||
создавать вообще.
|
||||
|
||||
## Ограничения
|
||||
|
||||
- Это инструкция, а не плагин: запуск идёт через инструмент `workflow`.
|
||||
- Одновременно выполняется столько агентов, сколько разрешает runtime (часто 8, не 14). Лимит проверяется preflight-ом; `wave.workflow.js` запускает задачи батчами, а не одним большим fan-out.
|
||||
- Дети не общаются между собой и не делегируют дальше; координация — через файлы и лида.
|
||||
- Верификаторы работают на замороженном дереве; полный разбор пишется в `.swarm/reviews/Vxx.md`, а workflow возвращает только `{task_id, verdict, summary, review_file}`.
|
||||
- `blocked`-воркер не заменяется: лид правит контракт/shared-файл и продолжает того же агента через `send_message`.
|
||||
- Evidence зависит от домена: команда и тест для кода, источник и дата для research, отрендеренный артефакт для дизайна, reproducible metric для данных.
|
||||
- Токены — потолок, а не цель: 20–30% резервируется на verify/integration и polish, после фиксов делается delta-review, а не полная перепроверка. Детали — в `references/token-economy.md`.
|
||||
- Визуальные агенты работают в отдельных Xvfb/display-окружениях, со своим HOME/config/копией проекта; общий рабочий стол для скриншотов не используется. Детали — в `references/visual-testing.md`.
|
||||
- Computer-use — крайний tier: один агент, одноразовый desktop/VM, только после детерминированных проверок и никогда как единственное evidence.
|
||||
- Стоимость: 30–40 агентов ≈ 0.5–1.5M токенов.
|
||||
- `workflow` не обязателен: есть `subagent` — используем его, нет — работаем последовательно.
|
||||
- Число одновременных агентов ограничено runtime; не запускать oversubscribed fan-out.
|
||||
- Воркеры не устанавливают зависимости и не трогают live-системы.
|
||||
- Визуальные проверки — в изолированных display; общий рабочий стол не evidence.
|
||||
- Computer-use — крайний tier: один агент, одноразовый desktop/VM, не единственное evidence.
|
||||
- Токены — потолок, а не цель: резерв на verify/integration/polish; после фиксов — delta-review.
|
||||
- Для задачи, которую один агент делает за 15 минут, swarm не нужен.
|
||||
|
||||
## Опциональные модули
|
||||
|
||||
Читать только по необходимости:
|
||||
|
||||
- `references/playbook.md` — подробный runbook для больших swarm.
|
||||
- `references/packets.md` — шаблоны brief/task/verifier.
|
||||
- `references/orchestration.md` — `workflow`, батчи, ретраи.
|
||||
- `references/domains.md` — адаптеры для кода/research/контента/данных/дизайна/ops.
|
||||
- `references/prerequisites.md` — preflight и политика установки.
|
||||
- `references/token-economy.md` — бюджеты, контекст, delta-review.
|
||||
- `references/visual-testing.md` — изолированные display, скриншоты, computer-use.
|
||||
- `references/product-quality.md` — performance/UX/baseline/human acceptance.
|
||||
- `evals/` — как проверять сам скилл (baseline vs with-skill).
|
||||
|
||||
## Настройка
|
||||
|
||||
- Шаблоны: `assets/`, включая `prerequisites.md.tmpl`.
|
||||
- Prerequisites: `references/prerequisites.md`.
|
||||
- Product quality: `references/product-quality.md`.
|
||||
- Подробный runbook: `references/playbook.md`.
|
||||
- Оркестрация: `references/orchestration.md`.
|
||||
- Шаблоны: `assets/`.
|
||||
- Проверка скоупа: `node scripts/audit-scope.mjs --init` перед стартом, затем `node scripts/audit-scope.mjs`.
|
||||
- Для директорий в `plan.json` используйте glob `path/**`, особенно для ещё не созданных папок.
|
||||
- Для директорий в `plan.json` используйте glob `path/**`, включая ещё не созданные папки.
|
||||
- Перед изменением скилла прочитайте `evals/README.md`; не считайте, что новая версия лучше старой
|
||||
без сравнения.
|
||||
|
||||
+79
-114
@@ -1,138 +1,103 @@
|
||||
---
|
||||
name: agent-swarm
|
||||
description: Use for large tasks that split into 8+ independent work units across software, research, content, data, design or operations (codebase-wide changes, audits, migrations, multi-source research, multi-section documents, dataset work, visual redesigns) to run coordinated subagents in waves under one lead, with ownership, packed context, and independent verification. Not for single-file, sequential, or highly coupled work.
|
||||
whenToUse: The task is broad, parallelizable, and has executable acceptance checks; one agent would take hours and separate agents can finish units without touching the same files.
|
||||
description: Use when a large task splits into 8+ independent units, or when a risky deliverable needs an independent critic across software, research, content, data, design or operations. Not for small, sequential or tightly coupled work.
|
||||
whenToUse: The task is broad or risky enough that parallel workers or a fresh verifier change the outcome; one agent would take hours and the units can be described without stepping on each other.
|
||||
---
|
||||
|
||||
# Agent Swarm
|
||||
|
||||
Run a large task as waves of coordinated subagents. The lead plans and integrates; workers execute one work packet each; verifiers check independently. A swarm is only as fast as its briefs are clear and its write scopes are disjoint.
|
||||
A small orchestration core, not a mandatory methodology. Use the lightest mode that meets the
|
||||
risk; the user keeps the stack, scope and pace.
|
||||
|
||||
The protocol is **domain-neutral**: software, research, content, data, design and operations all use the same roles and evidence rules. Pick an adapter from `references/domains.md`; do not force a software pipeline onto non-code work. Tools are optional too: if `workflow` is unavailable, use `subagent` or direct sequential work with the same packets.
|
||||
## Choose the mode
|
||||
|
||||
The skill is a **capability, not a mandatory process**. Apply the lightest mode that meets the risk:
|
||||
|
||||
| Mode | When | What it adds |
|
||||
| Mode | Use when | What it adds |
|
||||
|---|---|---|
|
||||
| Direct | one unit, fast feedback, low blast radius | none |
|
||||
| Paired critic | medium task or risky design decision | independent review, same artifact |
|
||||
| Swarm | 8+ independent units, breadth dominates | plan, ownership, parallel workers, verifier wave |
|
||||
| **Direct** | one unit, fast feedback, low blast radius | nothing |
|
||||
| **Paired critic** | medium task, design decision, risky change | one independent review |
|
||||
| **Swarm** | 8+ independent units or breadth dominates | ownership, parallel workers, verifier |
|
||||
|
||||
Do not force task cards, waves or `.swarm/` artifacts onto a small request. Conversely, do not skip independent verification on a large or risky one. The user keeps the choice of stack, scope and pace; the skill only adds coordination when breadth or risk justifies it.
|
||||
If the units share one screen, one design language or one mutable artifact, do not swarm. Use one
|
||||
implementer and one critic; parallel panel writers produce incoherent UI.
|
||||
|
||||
**Process is optional; quality gates are not.** For anything the user will actually use, read
|
||||
`references/product-quality.md` and agree on a **prototype** or **product** profile. Product work
|
||||
requires a release-build performance check, a real UX scenario, baseline comparison and an explicit
|
||||
user verdict. A verifier's `pass` is technical evidence, not product acceptance.
|
||||
## Invariants
|
||||
|
||||
## Fit check — before spawning anything
|
||||
1. **Outcome before process.** Agree on what “done” means and how it will be checked. Do not create
|
||||
`.swarm/` files, task cards or reports just to look organized.
|
||||
2. **One owner per artifact.** In a swarm, every path/artifact belongs to one writer. Shared files
|
||||
and contracts belong to the lead/integrator.
|
||||
3. **Packets, not transcripts.** A worker gets a self-contained brief, its scope, the contract and
|
||||
the acceptance checks — never the lead's full conversation.
|
||||
4. **Independent verification.** The author does not verify their own work. A fresh context checks
|
||||
the artifact, not the worker's summary. Verification is required for risky or product work; a
|
||||
trivial direct change may only need a command and a read-back.
|
||||
5. **Evidence before claims.** “Done” means a runnable artifact, a passing command, a citation, a
|
||||
metric or a rendered result — not a confident sentence.
|
||||
6. **A technical pass is not product acceptance.** For product/UI work, measure the release build,
|
||||
run a real scenario, compare with the baseline and get the user's verdict. If performance/UX was
|
||||
not measured, the product gate is `not_run`, not `pass`.
|
||||
7. **Do not add features to hide bad quality.** If it is laggy, visually worse or harder to use than
|
||||
the baseline, fix the slice before expanding scope.
|
||||
8. **One milestone at a time.** For product work, do not start M1/M2 while M0 is still rough or
|
||||
explicitly “not usable”.
|
||||
9. **Safety.** Check prerequisites before work; ask before global installs, destructive actions,
|
||||
deployments or irreversible changes. Workers never install or touch live systems.
|
||||
10. **User overrides.** Direct user instructions beat this skill. If the user asks for speed, a
|
||||
prototype or “just do it”, drop the ceremony — keep only the safety and evidence that still
|
||||
matter.
|
||||
|
||||
Swarm only when all are true:
|
||||
- The task splits into **8+ independent units** with disjoint write scopes.
|
||||
- Each unit has a **re-runnable acceptance check**: a command, a named artifact, a source/citation check, a metric, or a rendered result. There is no universal evidence type; match it to the domain adapter.
|
||||
- Wall-clock is dominated by **breadth**, not by one deep sequential chain.
|
||||
- You can state the interfaces between units **before** dispatch.
|
||||
- The units do **not** share one visual language or one screen. UI coherence is usually a single-owner
|
||||
job; parallel panel writers produce a Frankenstein interface.
|
||||
## Loop
|
||||
|
||||
Otherwise: one agent for small work; 2–5 `subagent` calls for a few long-lived roles you need to steer. Never swarm one bug, one design decision, or work where every unit edits the same files.
|
||||
1. **Understand** the outcome, audience, quality bar and constraints. Ask only when the answer
|
||||
changes the result.
|
||||
2. **Check** the environment: toolchain, commands, services, display, credentials. If a required
|
||||
item is missing and cannot be resolved safely, ask before starting.
|
||||
3. **Plan only as much as the risk requires.** For a swarm, write a one-page brief, disjoint scopes,
|
||||
frozen contracts and acceptance checks. For direct/paired work, skip the ceremony.
|
||||
4. **Work.** Direct, paired or batched workers. Keep files small enough that each worker reads only
|
||||
what it needs; use available concurrency, not a fixed wave number.
|
||||
5. **Verify.** Technical checks first, then a fresh critic. For product work, add release performance,
|
||||
UX scenario, baseline comparison and the user's explicit verdict.
|
||||
6. **Accept and integrate.** The lead owns shared files, final checks and the report. No next
|
||||
milestone without acceptance (or an explicit waiver).
|
||||
|
||||
## Roles
|
||||
## If you swarm
|
||||
|
||||
- **Lead** — you. Owns the plan, packets, contracts, gates, integration, and the final report. Does not implement during waves.
|
||||
- **Recon** (2–4) — maps files, commands, conventions, and hazards. Does not design the solution.
|
||||
- **Worker** — one task card, one write scope, one structured report.
|
||||
- **Verifier** — fresh context; checks the artifact against acceptance criteria, never the worker's summary.
|
||||
- **Integrator** (1–2) — owns shared files and cross-cutting fixes after the parallel waves. During a wave, the lead/integrator may make a small contract/shared-file change to unblock a task, then resume that same worker with `send_message`; do not spawn a replacement.
|
||||
Optional, for resumability and coordination — not a requirement for every task:
|
||||
|
||||
## Protocol
|
||||
- `.swarm/brief.md` — goal, quality bar, commands, conventions, hazards;
|
||||
- `.swarm/contracts.md` — only interfaces that cross a task boundary;
|
||||
- `.swarm/tasks/Txx.md` — one page per unit;
|
||||
- `.swarm/reports/Txx.json` — structured evidence, if the lead needs to triage many workers;
|
||||
- `.swarm/reviews/Vxx.md` — independent verdict and findings.
|
||||
|
||||
0. **Fit check & prerequisite gate** — go/no-go, budget (`max_agents`, `max_waves`, **tokens**), and **discover the available tools and real active-child cap** before sizing a wave. Check every required toolchain/runtime/service/display/credential with a real command; record the result in `.swarm/prerequisites.md` and `state.json`. Prefer project-local pinned setup; **never install globally/system-wide or run `sudo`/`curl | sh` without explicit user approval**. If a required prerequisite is missing and cannot be resolved safely, ask the user with the exact item, command, scope and alternatives **before any worker starts**. Workers never install; they return `blocked`.
|
||||
1. **Resolve prerequisites** — after approval/install, re-run the checks and the baseline suite. No build or verify wave starts while a required item is `needs_user` or `blocked`; optional items may be skipped only with a recorded reduced scope. See `references/prerequisites.md`.
|
||||
2. **Recon** — 2–4 parallel agents return findings with paths and line numbers. Timebox them, merge into `.swarm/brief.md` and `.swarm/map.md`, then stop them; recon must not hold slots during the build wave.
|
||||
3. **Plan gate** — write `.swarm/plan.json`, `plan.md`, `ownership.md`, `contracts.md`, and one `tasks/Txx.md` per unit. Every success criterion maps to a task and a check. No two tasks write the same path. Include a `LEAD`/`INTEGRATOR` task owning lead-owned shared files, screenshots, licenses and the final report. Freeze interfaces. Prefer vertical slices and files under ~800 lines; a task that requires reading a 3k-line file is too broad. Record the **quality profile**, performance budget, baseline scenarios and user-acceptance owner. **Do not dispatch until this gate passes.**
|
||||
4. **Build wave** — one `workflow` call per build wave when the tool exists, using the batched runner in `assets/wave.workflow.js`; its `concurrency` must fit the available slots (see Sizing). If `workflow`/`subagent` is unavailable, the lead executes the same packets sequentially. Retry `null`/`failed` once; if the runtime reports an active-child limit, halve the next batch and dispatch the remaining tasks — never relaunch the same oversubscribed fan-out.
|
||||
5. **Audit** — run `scripts/audit-scope.mjs` from the skill base directory; revert or reassign out-of-scope changes. Run the build/test gate.
|
||||
6. **Verify wave** — **freeze source changes**, run automated gates first (compile/tests/artifact), then 4–6 fresh verifiers by risk cluster (or 1–2 for a small domain task). Use `assets/verdict.schema.json` and, when available, `assets/verify.workflow.js`; the review file on disk is the source of truth, not the workflow return. Verifiers never fix. Match evidence to the domain adapter: rerun commands, read rendered artifacts, sample citations, recompute metrics. For GUI products, use per-agent isolated displays, never a shared desktop (see `references/visual-testing.md`). If you fix findings, re-run only the affected **delta** and mark the old review stale.
|
||||
7. **Product gate** — for product-profile work, measure the release build against the performance budget, run the real UX scenario, compare with the baseline, and get the user's explicit verdict. A technical verifier pass is not enough. If the product is slower or worse than the baseline, stop and fix the slice; do not start the next milestone. See `references/product-quality.md`.
|
||||
8. **Integrate** — one integrator for shared files and cross-cutting fixes; re-run the full acceptance suite; then a final verification pass on the integrated build.
|
||||
9. **Close** — freeze the build and run a final independent review against the original request (not the plan), then write `final-report.md`.
|
||||
Use the batched runner in `assets/wave.workflow.js` when `workflow` exists; otherwise use `subagent`
|
||||
or direct sequential work. Never relaunch an oversubscribed fan-out. Read `references/playbook.md`
|
||||
for the detailed runbook, `references/packets.md` for cards and `references/orchestration.md` for
|
||||
tool specifics — only when the task needs them.
|
||||
|
||||
## Hard rules
|
||||
## Optional modules
|
||||
|
||||
1. **One writer per path.** A path appears in at most one task's `write` list. Shared files (`package.json`, lockfiles, barrels, schemas, migrations, CI) belong to the integrator only.
|
||||
2. **Packet, not transcript.** Each worker reads `.swarm/brief.md`, its `tasks/Txx.md`, `.swarm/contracts.md`, and the domain adapter in `.swarm/profile.md` (if any). Never fork the lead's conversation for dozens of workers.
|
||||
3. **Structured reports.** Workers return `{task_id, status, summary, files_changed, evidence, acceptance, issues, assumptions}` and write the same object to `.swarm/reports/Txx.json`. Verifiers write the full review to `.swarm/reviews/Vxx.md` and return the minimal `{task_id, verdict, summary, review_file}`. Prose is not a report.
|
||||
4. **No self-verification.** The worker's own checks are a smoke test; a different agent verifies the artifact.
|
||||
5. **No global side effects.** Workers do not run installs, repo-wide format/lint/codegen, migrations, dev servers, deployments, external publishes, or any git write command without explicit scope. They do not edit other tasks' files/reports or touch live systems. The lead owns irreversible operations and only after the final review.
|
||||
6. **Respect the runtime slot cap.** Active children include recon, verifiers, integrators and any other subagents. A spawn-limit error is a scheduling bug, not a worker failure: shrink the batch, keep completed results, and dispatch the remainder.
|
||||
7. **Freeze during verification.** Once the verify wave starts, do not edit source/tests/docs except to fix a verified finding. After a fix, re-run the affected verifier; an old review is stale.
|
||||
8. **Stop rules.** Two waves with no net progress → stop and re-plan. Verification failure rate > 30% → fix briefs/contracts, not agent count. Budget exhausted → report partial and list unverified areas.
|
||||
9. **No shared visual desktop.** Visual agents get their own display/config/project copy; see `references/visual-testing.md`. A screenshot from a shared desktop is not evidence. Computer-use is a last-resort, single-agent tier on a disposable desktop/VM; it is never the sole evidence and never runs in parallel with other visual agents.
|
||||
10. **Prerequisites before dispatch.** A required tool/runtime/service/credential is checked, installed with explicit approval if needed, and re-checked before workers start. Ask the user before work if it cannot be resolved; never let a worker install or silently skip a required check.
|
||||
11. **Token budget is a ceiling, not a target.** Record spend per phase, keep a 20–30% reserve for verification/integration/**polish**, and do not stop a product task at the soft cap if the quality gate is unmet; stop at the hard cap or when the gate is met. Cut redundant context, agents, full-file reads and full re-reviews — never cut tests, independent review or user-driven fixes. See `references/token-economy.md`.
|
||||
12. **A technical pass is not product acceptance.** Product-profile work needs measured performance, a real UX scenario, baseline comparison and an explicit user verdict; otherwise the product gate is `not_run`. See `references/product-quality.md`.
|
||||
13. **One milestone at a time.** Do not start M1/M2 while M0 is still laggy, visually rough or explicitly “not usable”. Scope adds must wait for acceptance.
|
||||
Read only what applies:
|
||||
|
||||
## Sizing
|
||||
- `references/playbook.md` — detailed swarm runbook, gates, recovery.
|
||||
- `references/packets.md` — brief/task/verifier templates and examples.
|
||||
- `references/orchestration.md` — `workflow` patterns, retries, batching.
|
||||
- `references/domains.md` — software, research, content, data, design, operations.
|
||||
- `references/prerequisites.md` — preflight, install/approval policy.
|
||||
- `references/token-economy.md` — budgets, context discipline, delta review.
|
||||
- `references/visual-testing.md` — isolated displays, screenshots, computer-use limits.
|
||||
- `references/product-quality.md` — performance/UX/baseline/human acceptance.
|
||||
- `evals/` — how to test this skill itself (baseline vs with-skill, trigger checks).
|
||||
|
||||
| Independent units | Batch size | Batches | Agents | Notes |
|
||||
|---|---|---|---|---|
|
||||
| < 8 | — | — | — | Do not swarm |
|
||||
| 8–11 | 6–8 | 1 + verify | ~12–14 | Small swarm; one batch |
|
||||
| 12–24 | 6–8 | 2 | ~20 | Default; 2 batches per build wave |
|
||||
| 25–48 | 6–8 | 3–4 | ~40 | Large; watch the lead's context |
|
||||
| 49+ | 6–8 | 5+ | 60+ | Only if truly independent; the lead becomes the bottleneck |
|
||||
|
||||
**The wave size is not fixed by this table.** The runtime active-child cap is host/policy-specific and may be as low as 8; do not assume 14 or any default. Discover it before dispatch (runtime diagnostics or a tiny trial), subtract active recon/other children, and leave at least one slot free for retries. If the cap is unknown, start with 6 workers. The batched runner in `assets/wave.workflow.js` starts tasks in chunks, so a failure to spawn one chunk does not discard completed work.
|
||||
|
||||
## Workspace
|
||||
|
||||
```
|
||||
.swarm/
|
||||
brief.md shared context: goal, domain, commands, conventions, hazards
|
||||
prerequisites.md tool/runtime/display/credential gate + approvals
|
||||
map.md repository/source map from recon: paths, entry points, hazards
|
||||
contracts.md frozen interfaces between tasks
|
||||
profile.md domain adapter: artifacts, commands, evidence types (optional)
|
||||
plan.json tasks, deps, write scopes, waves, status
|
||||
plan.md human-readable plan + coverage matrix
|
||||
ownership.md path -> task map
|
||||
tasks/T01.md one work packet per task
|
||||
reports/T01.json worker report (worker writes)
|
||||
reviews/V01.md verifier report (verifier writes)
|
||||
baseline.json pre-swarm file manifest (`--init`)
|
||||
report.schema.json report contract for workers
|
||||
verdict.schema.json minimal contract for verifiers
|
||||
state.json wave, token budget/spend, prerequisites, open issues
|
||||
final-report.md
|
||||
tmp/T01/ worker scratch, never shared
|
||||
```
|
||||
|
||||
## Coordination loop
|
||||
|
||||
Between waves the lead does only four things: **continue**, **answer blocked tasks and retry**, **re-scope**, or **stop**. Read structured reports and review files, not raw logs. Keep `state.json` current, including the real slot cap, retries and re-verifications. The lead is the only agent that talks to the user.
|
||||
|
||||
For a `blocked` task: apply the smallest contract/shared-file change, update `contracts.md`, record it in `state.json`, then `send_message` to the **same** worker with the answer and let it continue. Do not spawn a replacement for a worker that only needed a decision.
|
||||
|
||||
## References
|
||||
|
||||
- `references/playbook.md` — phase-by-phase runbook, gates, checklists, failure recovery.
|
||||
- `references/packets.md` — how to write briefs, task cards, and verifier cards; examples and common mistakes.
|
||||
- `references/orchestration.md` — `workflow` cookbook: script patterns, schema subset, retries, background runs, archetypes (code / research / content).
|
||||
- `references/domains.md` — adapters for software, research, content, data, design and operations.
|
||||
- `references/token-economy.md` — measurement, budgets, context discipline, delta verification, vertical slices.
|
||||
- `references/prerequisites.md` — preflight checks, install/approval policy, degraded-scope record.
|
||||
- `references/visual-testing.md` — isolated Xvfb/container visual tests, deterministic screenshots, verifier procedure (GUI adapter).
|
||||
- `references/product-quality.md` — prototype/product profiles, performance and UX gates, baseline comparison, human acceptance.
|
||||
- `assets/` — copyable brief, task, and verifier templates; `report.schema.json`, `verdict.schema.json`; `wave.workflow.js` (batched workers), `verify.workflow.js` (frozen verification).
|
||||
- `scripts/audit-scope.mjs` — compares changed files to the ownership map.
|
||||
|
||||
## Signals that the swarm is failing
|
||||
## Signals
|
||||
|
||||
- Workers ask the same question or edit the same file → the plan or contracts are underspecified.
|
||||
- Reports are long on prose and short on evidence → the task card lacks executable checks.
|
||||
- The lead spends more tokens reading reports than the workers spend working → waves are too large or reports too verbose.
|
||||
- Verification finds the same class of defect repeatedly → fix the brief, not the worker.
|
||||
- A workflow ends with `no usable verdict` although reviews exist on disk → the verifier schema is too strict; read the review files and switch to `assets/verdict.schema.json`.
|
||||
- A wave dies with `active child limit reached` → the batch was sized from the wrong cap; halve it, subtract recon, and dispatch only unstarted tasks.
|
||||
- Reports are long on prose and short on evidence → the task card lacks a check.
|
||||
- The lead reads more tokens than the workers produce → waves or reports are too large.
|
||||
- The same defect repeats → fix the brief, not the worker count.
|
||||
- The product compiles and passes tests but feels worse than the previous version → stop adding
|
||||
features and run the product gate.
|
||||
- You are creating process artifacts that nobody will read → delete them and do the work.
|
||||
- You are changing this skill without evidence → run `evals/README.md` against a no-skill baseline;
|
||||
keep the change only if it improves quality/time without over-orchestration.
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
# Testing the skill itself
|
||||
|
||||
Do not assume the skill improves outcomes. Measure it against a no-skill baseline and against the
|
||||
previous version. This is the same idea as Anthropic's `skill-creator`: run baseline and with-skill
|
||||
runs on realistic prompts, compare quality and cost, then revise.
|
||||
|
||||
## What to measure
|
||||
|
||||
| Metric | How |
|
||||
|---|---|
|
||||
| Acceptance | Did the artifact pass the task's own checks? |
|
||||
| Human quality (1–5) | Blind review by someone who did not run the task |
|
||||
| Time | Wall-clock from prompt to accepted artifact |
|
||||
| Tokens / requests | Harness usage; record input/output/cache if available |
|
||||
| Agents spawned | Count; detect over-orchestration |
|
||||
| Process artifacts | Files under `.swarm/`; are they read or just produced? |
|
||||
| Regression | Tests, UX, performance, baseline comparison |
|
||||
| Performance | For UI: release p95 frame time, input latency, startup |
|
||||
| Trigger | Did the skill activate when appropriate, and stay out of small work? |
|
||||
|
||||
## Runs
|
||||
|
||||
For each task, run at least:
|
||||
|
||||
1. **baseline** — no skill;
|
||||
2. **candidate** — current `agent-swarm`;
|
||||
3. optionally, the previous version.
|
||||
|
||||
Use fresh contexts, the same model/settings, the same repository commit and the same time limit.
|
||||
Run twice if variance is high. Do not let the baseline see the skill's artifacts.
|
||||
|
||||
## Tasks
|
||||
|
||||
- [`tasks/small-fix.md`](tasks/small-fix.md) — should **not** trigger a swarm.
|
||||
- [`tasks/product-ui.md`](tasks/product-ui.md) — should trigger a small team and product gate.
|
||||
- [`tasks/broad-migration.md`](tasks/broad-migration.md) — should trigger a real swarm.
|
||||
|
||||
## Pass condition
|
||||
|
||||
The skill is useful only if it improves at least one of:
|
||||
|
||||
- acceptance/quality without a large time or token penalty;
|
||||
- time/tokens without reducing quality;
|
||||
- product quality (performance/UX/baseline) on the product task.
|
||||
|
||||
It fails if it adds agents/artifacts without improving outcomes, or if it triggers on the small fix.
|
||||
Record results in a copy of [`scorecard.md`](scorecard.md).
|
||||
@@ -0,0 +1,36 @@
|
||||
# Skill eval scorecard
|
||||
|
||||
Date:
|
||||
Model/settings:
|
||||
Skill version/commit:
|
||||
Evaluator:
|
||||
|
||||
## small-fix
|
||||
|
||||
| Run | Accepted | Quality 1–5 | Time | Tokens | Agents | `.swarm` artifacts | Notes |
|
||||
|---|---|---:|---:|---:|---:|---|---|
|
||||
| baseline | | | | | | | |
|
||||
| candidate | | | | | | | |
|
||||
|
||||
Trigger verdict: `correct / over-triggered / under-triggered`
|
||||
|
||||
## product-ui
|
||||
|
||||
| Run | Accepted | Quality 1–5 | Time | Tokens | p95 frame | Input latency | Baseline comparison | User verdict |
|
||||
|---|---|---:|---:|---:|---:|---:|---|---|
|
||||
| baseline | | | | | | | | |
|
||||
| candidate | | | | | | | | |
|
||||
|
||||
## broad-migration
|
||||
|
||||
| Run | Accepted | Quality 1–5 | Time | Tokens | Agents | Conflicts | Rework | Notes |
|
||||
|---|---|---:|---:|---:|---:|---:|---|---|
|
||||
| baseline | | | | | | | | |
|
||||
| candidate | | | | | | | | |
|
||||
|
||||
## Decision
|
||||
|
||||
- Keep / revise / revert:
|
||||
- What improved:
|
||||
- What regressed:
|
||||
- Next change:
|
||||
@@ -0,0 +1,26 @@
|
||||
# Broad migration — swarm test
|
||||
|
||||
Use a repository with 8+ independent call sites or modules that need the same mechanical change
|
||||
(e.g. adding pagination to independent endpoints, or migrating independent modules to a new API).
|
||||
|
||||
Prompt:
|
||||
|
||||
```text
|
||||
Apply the migration across the affected modules. Each unit must have its own scope and tests. Do not
|
||||
touch unrelated code. Verify the integrated result and report conflicts or skipped units.
|
||||
```
|
||||
|
||||
Expected with a good skill:
|
||||
|
||||
- a short plan with disjoint write scopes and frozen interfaces;
|
||||
- parallel or batched workers, or a deliberate sequential choice with a reason;
|
||||
- independent verification of the integrated result;
|
||||
- no overlapping edits, no silent skipped units;
|
||||
- time/tokens improve over baseline, or quality improves enough to justify the cost.
|
||||
|
||||
Failure signals:
|
||||
|
||||
- 20 agents for 8 small edits;
|
||||
- workers editing the same file;
|
||||
- no integration check;
|
||||
- reports without evidence.
|
||||
@@ -0,0 +1,28 @@
|
||||
# Product UI — quality-gate test
|
||||
|
||||
Use a small app with an existing UI (or a fixture). Ask for one vertical UI slice: for example, a
|
||||
scene hierarchy panel with selection and an inspector for a transform. The artifact must be runnable.
|
||||
|
||||
Prompt:
|
||||
|
||||
```text
|
||||
Add the requested UI slice to this app. It is a product task: the result must be usable, not just
|
||||
compile. Use the existing stack. Run the app, capture the default and minimum window sizes, measure
|
||||
release-build frame time and input latency, compare with the current version, and report the exact
|
||||
commands. Do not expand into unrelated features.
|
||||
```
|
||||
|
||||
Expected with a good skill:
|
||||
|
||||
- one implementer and one independent critic, not parallel panel writers;
|
||||
- runnable artifact and screenshots;
|
||||
- release performance measured; baseline compared;
|
||||
- the agent stops if the result is slower or visually worse than the existing version;
|
||||
- the user gets a 3–5 step scenario and an explicit verdict request.
|
||||
|
||||
Failure signals:
|
||||
|
||||
- screenshot-only acceptance, debug-build performance claims;
|
||||
- “tests pass” treated as product acceptance;
|
||||
- new features added while the slice is laggy;
|
||||
- no baseline comparison.
|
||||
@@ -0,0 +1,24 @@
|
||||
# Small fix — trigger test
|
||||
|
||||
Use a small repository with one failing test (or one clearly scoped bug). Provide the repo path,
|
||||
the failing command and the expected behavior.
|
||||
|
||||
Prompt:
|
||||
|
||||
```text
|
||||
Fix the failing test in this repository. Make the smallest correct change, run the failing test and
|
||||
the relevant suite, and report the exact commands and results. Do not refactor unrelated code.
|
||||
```
|
||||
|
||||
Expected with a good skill:
|
||||
|
||||
- no swarm, no `.swarm/` tree, one worker or direct work;
|
||||
- the failing test now passes;
|
||||
- no unrelated changes;
|
||||
- time and tokens close to baseline.
|
||||
|
||||
Failure signals:
|
||||
|
||||
- the skill spawns multiple agents or writes a full plan for a one-line fix;
|
||||
- process artifacts exceed the code change;
|
||||
- the agent claims success without running the test.
|
||||
Reference in New Issue
Block a user