From 4e1cc29874d9fdeaeaa5002f367c1ba196ad7388 Mon Sep 17 00:00:00 2001 From: Emil Shanaty Date: Wed, 28 Oct 2026 00:17:37 +0300 Subject: [PATCH] Add product quality gates and milestone acceptance --- README.md | 3 +- agent-swarm/README.ru.md | 9 ++- agent-swarm/SKILL.md | 19 ++++- agent-swarm/assets/brief.md.tmpl | 7 ++ agent-swarm/assets/task.md.tmpl | 5 ++ agent-swarm/assets/verifier.md.tmpl | 13 +++ agent-swarm/references/packets.md | 1 + agent-swarm/references/playbook.md | 27 ++++++- agent-swarm/references/product-quality.md | 99 +++++++++++++++++++++++ agent-swarm/references/token-economy.md | 9 ++- 10 files changed, 180 insertions(+), 12 deletions(-) create mode 100644 agent-swarm/references/product-quality.md diff --git a/README.md b/README.md index d3cc9ca..c35ecf6 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,7 @@ - [Domain-адаптеры](agent-swarm/references/domains.md) - [Token economy](agent-swarm/references/token-economy.md) - [Prerequisite gate](agent-swarm/references/prerequisites.md) +- [Product quality gates](agent-swarm/references/product-quality.md) - [Изолированное визуальное тестирование](agent-swarm/references/visual-testing.md) ### Установка @@ -34,7 +35,7 @@ cp -r skills/agent-swarm ~/.dsh/skills/ agent-swarm/ ├── SKILL.md # основной протокол ├── README.ru.md # краткое описание -├── references/ # playbook, packets, orchestration, domains, visual testing, token economy, prerequisites +├── references/ # playbook, packets, orchestration, domains, visual testing, token economy, prerequisites, product quality ├── assets/ # шаблоны, схемы, workflow-скрипты └── scripts/audit-scope.mjs # проверка ownership ``` diff --git a/agent-swarm/README.ru.md b/agent-swarm/README.ru.md index 28ff74c..3dc4bf1 100644 --- a/agent-swarm/README.ru.md +++ b/agent-swarm/README.ru.md @@ -12,6 +12,8 @@ Скилл не обязателен для каждой задачи: маленькую задачу можно делать напрямую, среднюю — с независимым критиком, и только широкую — через swarm. Пользователь выбирает стек, объём и темп; скилл добавляет координацию, когда она оправдана. +Процесс — опциональный, quality gates — нет. Для продуктовой работы заранее выбирается профиль `prototype` или `product`. В `product` обязательны release-замеры производительности, реальный UX-сценарий, сравнение с baseline и явный вердикт пользователя. Технический `pass` verifier'а не заменяет приёмку продукта. + ## Что происходит 1. **Fit-check.** Если задача не делится на 8+ независимых юнитов, скилл скажет не использовать swarm. @@ -20,7 +22,9 @@ 4. **План.** Задачи, владельцы файлов, контракты, критерии приёмки. Гейт: без этого волны не стартуют. 5. **Волны.** Воркеры идут батчами по 6–8 (реальный лимит активных детей узнаётся перед запуском и записывается в `state.json`; recon не должен занимать слоты). Затем 4–6 независимых верификаторов на «замороженном» дереве, затем интегратор. 6. **Аудит.** `scripts/audit-scope.mjs` сравнивает изменения с картой владения. -7. **Отчёт.** `.swarm/final-report.md`. +7. **Интеграция.** Один интегратор собирает общие файлы и прогоняет полный acceptance suite. +8. **Product gate.** Для `product`: release-замеры, реальный сценарий, baseline и вердикт пользователя. Без этого следующий milestone не начинается. +9. **Отчёт.** `.swarm/final-report.md`. ## Артефакты @@ -34,7 +38,7 @@ - Верификаторы работают на замороженном дереве; полный разбор пишется в `.swarm/reviews/Vxx.md`, а workflow возвращает только `{task_id, verdict, summary, review_file}`. - `blocked`-воркер не заменяется: лид правит контракт/shared-файл и продолжает того же агента через `send_message`. - Evidence зависит от домена: команда и тест для кода, источник и дата для research, отрендеренный артефакт для дизайна, reproducible metric для данных. -- Токены — такой же бюджет, как время: 20–30% резервируется на verify/integration, после фиксов делается delta-review, а не полная перепроверка. Детали — в `references/token-economy.md`. +- Токены — потолок, а не цель: 20–30% резервируется на verify/integration и polish, после фиксов делается delta-review, а не полная перепроверка. Детали — в `references/token-economy.md`. - Визуальные агенты работают в отдельных Xvfb/display-окружениях, со своим HOME/config/копией проекта; общий рабочий стол для скриншотов не используется. Детали — в `references/visual-testing.md`. - Computer-use — крайний tier: один агент, одноразовый desktop/VM, только после детерминированных проверок и никогда как единственное evidence. - Стоимость: 30–40 агентов ≈ 0.5–1.5M токенов. @@ -44,6 +48,7 @@ - Шаблоны: `assets/`, включая `prerequisites.md.tmpl`. - Prerequisites: `references/prerequisites.md`. +- Product quality: `references/product-quality.md`. - Подробный runbook: `references/playbook.md`. - Оркестрация: `references/orchestration.md`. - Проверка скоупа: `node scripts/audit-scope.mjs --init` перед стартом, затем `node scripts/audit-scope.mjs`. diff --git a/agent-swarm/SKILL.md b/agent-swarm/SKILL.md index 0fc4be7..f7ed6a3 100644 --- a/agent-swarm/SKILL.md +++ b/agent-swarm/SKILL.md @@ -20,6 +20,11 @@ The skill is a **capability, not a mandatory process**. Apply the lightest mode Do not force task cards, waves or `.swarm/` artifacts onto a small request. Conversely, do not skip independent verification on a large or risky one. The user keeps the choice of stack, scope and pace; the skill only adds coordination when breadth or risk justifies it. +**Process is optional; quality gates are not.** For anything the user will actually use, read +`references/product-quality.md` and agree on a **prototype** or **product** profile. Product work +requires a release-build performance check, a real UX scenario, baseline comparison and an explicit +user verdict. A verifier's `pass` is technical evidence, not product acceptance. + ## Fit check — before spawning anything Swarm only when all are true: @@ -27,6 +32,8 @@ Swarm only when all are true: - Each unit has a **re-runnable acceptance check**: a command, a named artifact, a source/citation check, a metric, or a rendered result. There is no universal evidence type; match it to the domain adapter. - Wall-clock is dominated by **breadth**, not by one deep sequential chain. - You can state the interfaces between units **before** dispatch. +- The units do **not** share one visual language or one screen. UI coherence is usually a single-owner + job; parallel panel writers produce a Frankenstein interface. Otherwise: one agent for small work; 2–5 `subagent` calls for a few long-lived roles you need to steer. Never swarm one bug, one design decision, or work where every unit edits the same files. @@ -43,12 +50,13 @@ Otherwise: one agent for small work; 2–5 `subagent` calls for a few long-lived 0. **Fit check & prerequisite gate** — go/no-go, budget (`max_agents`, `max_waves`, **tokens**), and **discover the available tools and real active-child cap** before sizing a wave. Check every required toolchain/runtime/service/display/credential with a real command; record the result in `.swarm/prerequisites.md` and `state.json`. Prefer project-local pinned setup; **never install globally/system-wide or run `sudo`/`curl | sh` without explicit user approval**. If a required prerequisite is missing and cannot be resolved safely, ask the user with the exact item, command, scope and alternatives **before any worker starts**. Workers never install; they return `blocked`. 1. **Resolve prerequisites** — after approval/install, re-run the checks and the baseline suite. No build or verify wave starts while a required item is `needs_user` or `blocked`; optional items may be skipped only with a recorded reduced scope. See `references/prerequisites.md`. 2. **Recon** — 2–4 parallel agents return findings with paths and line numbers. Timebox them, merge into `.swarm/brief.md` and `.swarm/map.md`, then stop them; recon must not hold slots during the build wave. -3. **Plan gate** — write `.swarm/plan.json`, `plan.md`, `ownership.md`, `contracts.md`, and one `tasks/Txx.md` per unit. Every success criterion maps to a task and a check. No two tasks write the same path. Include a `LEAD`/`INTEGRATOR` task owning lead-owned shared files, screenshots, licenses and the final report. Freeze interfaces. Prefer vertical slices and files under ~800 lines; a task that requires reading a 3k-line file is too broad. **Do not dispatch until this gate passes.** +3. **Plan gate** — write `.swarm/plan.json`, `plan.md`, `ownership.md`, `contracts.md`, and one `tasks/Txx.md` per unit. Every success criterion maps to a task and a check. No two tasks write the same path. Include a `LEAD`/`INTEGRATOR` task owning lead-owned shared files, screenshots, licenses and the final report. Freeze interfaces. Prefer vertical slices and files under ~800 lines; a task that requires reading a 3k-line file is too broad. Record the **quality profile**, performance budget, baseline scenarios and user-acceptance owner. **Do not dispatch until this gate passes.** 4. **Build wave** — one `workflow` call per build wave when the tool exists, using the batched runner in `assets/wave.workflow.js`; its `concurrency` must fit the available slots (see Sizing). If `workflow`/`subagent` is unavailable, the lead executes the same packets sequentially. Retry `null`/`failed` once; if the runtime reports an active-child limit, halve the next batch and dispatch the remaining tasks — never relaunch the same oversubscribed fan-out. 5. **Audit** — run `scripts/audit-scope.mjs` from the skill base directory; revert or reassign out-of-scope changes. Run the build/test gate. 6. **Verify wave** — **freeze source changes**, run automated gates first (compile/tests/artifact), then 4–6 fresh verifiers by risk cluster (or 1–2 for a small domain task). Use `assets/verdict.schema.json` and, when available, `assets/verify.workflow.js`; the review file on disk is the source of truth, not the workflow return. Verifiers never fix. Match evidence to the domain adapter: rerun commands, read rendered artifacts, sample citations, recompute metrics. For GUI products, use per-agent isolated displays, never a shared desktop (see `references/visual-testing.md`). If you fix findings, re-run only the affected **delta** and mark the old review stale. -7. **Integrate** — one integrator for shared files and cross-cutting fixes; re-run the full acceptance suite; then a final verification pass on the integrated build. -8. **Close** — freeze the build and run a final independent review against the original request (not the plan), then write `final-report.md`. +7. **Product gate** — for product-profile work, measure the release build against the performance budget, run the real UX scenario, compare with the baseline, and get the user's explicit verdict. A technical verifier pass is not enough. If the product is slower or worse than the baseline, stop and fix the slice; do not start the next milestone. See `references/product-quality.md`. +8. **Integrate** — one integrator for shared files and cross-cutting fixes; re-run the full acceptance suite; then a final verification pass on the integrated build. +9. **Close** — freeze the build and run a final independent review against the original request (not the plan), then write `final-report.md`. ## Hard rules @@ -62,7 +70,9 @@ Otherwise: one agent for small work; 2–5 `subagent` calls for a few long-lived 8. **Stop rules.** Two waves with no net progress → stop and re-plan. Verification failure rate > 30% → fix briefs/contracts, not agent count. Budget exhausted → report partial and list unverified areas. 9. **No shared visual desktop.** Visual agents get their own display/config/project copy; see `references/visual-testing.md`. A screenshot from a shared desktop is not evidence. Computer-use is a last-resort, single-agent tier on a disposable desktop/VM; it is never the sole evidence and never runs in parallel with other visual agents. 10. **Prerequisites before dispatch.** A required tool/runtime/service/credential is checked, installed with explicit approval if needed, and re-checked before workers start. Ask the user before work if it cannot be resolved; never let a worker install or silently skip a required check. -11. **Token budget is a quality constraint.** Record spend per phase, keep a 20–30% reserve for verification/integration, and stop at the hard cap with a partial report. Cut redundant context, agents, full-file reads and full re-reviews — never cut tests or independent review. See `references/token-economy.md`. +11. **Token budget is a ceiling, not a target.** Record spend per phase, keep a 20–30% reserve for verification/integration/**polish**, and do not stop a product task at the soft cap if the quality gate is unmet; stop at the hard cap or when the gate is met. Cut redundant context, agents, full-file reads and full re-reviews — never cut tests, independent review or user-driven fixes. See `references/token-economy.md`. +12. **A technical pass is not product acceptance.** Product-profile work needs measured performance, a real UX scenario, baseline comparison and an explicit user verdict; otherwise the product gate is `not_run`. See `references/product-quality.md`. +13. **One milestone at a time.** Do not start M1/M2 while M0 is still laggy, visually rough or explicitly “not usable”. Scope adds must wait for acceptance. ## Sizing @@ -114,6 +124,7 @@ For a `blocked` task: apply the smallest contract/shared-file change, update `co - `references/token-economy.md` — measurement, budgets, context discipline, delta verification, vertical slices. - `references/prerequisites.md` — preflight checks, install/approval policy, degraded-scope record. - `references/visual-testing.md` — isolated Xvfb/container visual tests, deterministic screenshots, verifier procedure (GUI adapter). +- `references/product-quality.md` — prototype/product profiles, performance and UX gates, baseline comparison, human acceptance. - `assets/` — copyable brief, task, and verifier templates; `report.schema.json`, `verdict.schema.json`; `wave.workflow.js` (batched workers), `verify.workflow.js` (frozen verification). - `scripts/audit-scope.mjs` — compares changed files to the ownership map. diff --git a/agent-swarm/assets/brief.md.tmpl b/agent-swarm/assets/brief.md.tmpl index 83dd3e2..d277d2a 100644 --- a/agent-swarm/assets/brief.md.tmpl +++ b/agent-swarm/assets/brief.md.tmpl @@ -12,6 +12,13 @@ Scope is checked by the lead with the skill's `scripts/audit-scope.mjs`; workers ## Non-goals - {{What must not change or be attempted.}} +## Quality profile +- Profile: {{prototype / product}} +- Performance budget: {{p95 frame time / latency / startup / memory, or n/a}} +- Baseline version/scenarios: {{version + 3–5 step scenario, or none}} +- User acceptance owner: {{name / pending}} +- Release build command: {{command or n/a}} + ## Domain profile - Domain: {{software / research / content / data / design / operations}} - Primary artifacts: {{paths, sections, datasets, renders, services}} diff --git a/agent-swarm/assets/task.md.tmpl b/agent-swarm/assets/task.md.tmpl index 6bab1cd..c4a4e73 100644 --- a/agent-swarm/assets/task.md.tmpl +++ b/agent-swarm/assets/task.md.tmpl @@ -37,6 +37,11 @@ Role: worker | Wave: {{n}} | Depends on: {{ids or —}} | Effort: {{S/M/L}} 2. `{{citation check / metric / render at sizes}}` → {{expected result}} 3. Every criterion must have evidence a fresh verifier can reproduce. +If the quality profile is `product` and this task touches UI/interaction, also record: +- release-build p95 frame time and input latency at the target size; +- the 3–5 step user scenario; +- baseline comparison (new must not be worse). + ## Report `.swarm/reports/{{ID}}.json` per `.swarm/report.schema.json`; status `done|partial|blocked|failed`. Keep the report to summary + evidence lines; raw logs go to `.swarm/tmp/{{ID}}/`. diff --git a/agent-swarm/assets/verifier.md.tmpl b/agent-swarm/assets/verifier.md.tmpl index 2bbe2f7..5198511 100644 --- a/agent-swarm/assets/verifier.md.tmpl +++ b/agent-swarm/assets/verifier.md.tmpl @@ -32,8 +32,21 @@ For GUI products, run the app in your own isolated Xvfb/display (never a shared Computer-use (mouse/keyboard) is an optional last-resort tier: one agent, disposable desktop/VM, after deterministic checks, never parallel, never the sole evidence. Prefer app screenshot mode or scripted input. +## Product quality (product profile only) +Verify the release/optimized build, not a debug build: +- p50/p95 frame time at the target size, input-to-visible latency, startup, idle CPU/memory; +- the 3–5 step real user scenario, not just a screenshot; +- baseline comparison on the same scenario (regression = fail); +- report `not_run` if these were not measured. + +A technical pass is **not** product acceptance. The user still runs the artifact and gives an explicit +verdict (usable / usable with issues / not usable). Do not mark the product gate `pass` on the user's +behalf. + ## Verdict `pass` / `pass_with_risks` / `fail` — with the top risks and exact repro commands. +For product-profile work, `pass` requires functional + measured performance + UX scenario + baseline; +otherwise use `pass_with_risks` with missing gates named, or `not_run`. ## Report Keep the review to findings + commands + evidence (target 5–8k tokens); raw logs stay in `.swarm/tmp/`. Review the diff and affected checks, not the whole repository. diff --git a/agent-swarm/references/packets.md b/agent-swarm/references/packets.md index d460632..da7746b 100644 --- a/agent-swarm/references/packets.md +++ b/agent-swarm/references/packets.md @@ -45,6 +45,7 @@ Copy `assets/verifier.md.tmpl`. Rules: - Ask for scope, contract, and test-quality checks. - Require the full review file **first**, then a minimal `{task_id, verdict, summary, review_file}` return (`assets/verdict.schema.json`). The review file is the source of truth; a schema miss must not erase the verification work. - GUI: require an isolated Xvfb/display, screenshots at default and minimum sizes, and the exact commands used; shared-desktop screenshots are not evidence. +- Product profile: require release-build performance numbers (p95 frame time, input latency, startup), the real UX scenario and a baseline comparison. A technical pass is not product acceptance; the user still gives the final verdict. ## Report diff --git a/agent-swarm/references/playbook.md b/agent-swarm/references/playbook.md index aa60368..1c0bb71 100644 --- a/agent-swarm/references/playbook.md +++ b/agent-swarm/references/playbook.md @@ -12,6 +12,8 @@ Answer these before spawning: - **Interfaces.** Can you freeze signatures, data shapes, and file names before dispatch? - **Breadth.** Is the time dominated by many independent pieces, not one deep chain? +First decide the **quality profile** with the user: `prototype` (runnable spike, no performance promise) or `product` (performance, UX, baseline and human acceptance are mandatory). See `references/product-quality.md`. For UI/product work, prefer one implementer + one critic over parallel panel owners. + Budget: - `agents ≈ units + ceil(units / 5) verifiers + 1–2 integrators + 2–4 recon`; add retries and re-verifications explicitly. Set `max_agents` only after checking available tools and slot cap. - Tokens ≈ `agents × fixed overhead (10–25k) + context reads + output + reviews`. Treat this as an estimate, then calibrate from measured usage. A 3k-line file read by several agents can cost more than the code they write. @@ -96,6 +98,8 @@ Write the plan before any worker starts. Keep the plan proportionate: a small do - [ ] Files/modules over ~800 lines are split, or the task explicitly owns the integration cost. - [ ] Token budget, soft cap and verification reserve are recorded; units are merged/split to avoid fixed-overhead waste. - [ ] The plan uses vertical slices/checkpoints for a multi-milestone product. +- [ ] Quality profile (`prototype`/`product`), performance budget, baseline scenarios and user-acceptance owner are recorded. +- [ ] For product-profile GUI work, the plan names release-build measurements and the exact user scenario. - [ ] No path in two write scopes. - [ ] Contracts frozen for every cross-task interface. - [ ] Coverage matrix complete. @@ -183,13 +187,28 @@ For a GUI product, give each visual verifier its own Xvfb/display and project co **Freeze before verify.** Once you start this wave, do not edit source, tests or docs. If you already started and then fixed something, mark the in-flight review stale and re-run that verifier on the new build. A verifier's verdict is only valid for the exact tree it saw. +**Technical verification is not product acceptance.** Verifiers check evidence; for product-profile work they must also measure release-build frame time/input latency, run the real UX scenario and compare with the baseline. If performance/UX was not measured, the product gate is `not_run`, not `pass`. + Use `assets/verify.workflow.js` and `assets/verdict.schema.json`, not the full worker report schema. The verifier's detailed review **must be written to `.swarm/reviews/Vxx.md` before it returns**; the workflow return is only `{task_id, verdict, summary, review_file}`. Do not treat a workflow `null`/`no usable verdict` as a product failure if the review file exists — read the file, then re-run with the minimal schema if needed. Evidence standard: for every criterion, a command/artifact/citation/metric and its observed result. `pass` without evidence is `not_run`. Accept `pass_with_risks` only with named risks and a follow-up task. Verifiers should copy or hash-check generated files rather than rewriting the tree; if a required gate script rewrites generated artifacts, report the hash delta. On failure, create a fix task with the verifier's repro; do not send the verifier back to fix its own finding. After fixes, re-run only the affected **delta** (diff + findings + acceptance commands), not the whole repository, and mark the previous review stale; repeat until the remaining findings are minor/documented. Then run one final integrated verification pass on the frozen build. -## 6. Integrate +## 6. Product gate + +For `product`-profile work, run this before integration: + +1. Build the release/optimized artifact. Debug numbers are provisional only. +2. Measure p50/p95 frame time at the target size, input-to-visible latency, startup and idle CPU/memory on a realistic scene. +3. Run the 3–5 step real user scenario, not just a screenshot. +4. Compare with the baseline version on the same scenario; a regression is a failure. +5. Give the user the artifact and ask for an explicit verdict: usable / usable with issues / not usable. +6. Record measurements and the user verdict. Do not start the next milestone until accepted or explicitly waived. + +If the result is functional but laggy or visually worse than the baseline, stop and fix the slice. Do not add features to compensate. See `references/product-quality.md`. + +## 7. Integrate One integrator (two only for disjoint areas) owns: - shared files and cross-cutting edits; @@ -199,7 +218,7 @@ One integrator (two only for disjoint areas) owns: No parallel workers during integration. The integrator may read all task cards and reviews; it writes `.swarm/integration.md` with what changed and why. Freeze the tree while the final verification runs; any post-verification edit invalidates the affected evidence. -## 7. Close +## 8. Close Final review is against the **original request**, not the plan: a fresh agent checks the success criteria, the diff, and the evidence. Then the lead writes `.swarm/final-report.md` with these sections: @@ -233,6 +252,10 @@ Final review is against the **original request**, not the plan: a fresh agent ch | Reports/reviews dominate input | oversized prose | cap to evidence lines; logs in artifacts | | Visual agents produce wrong/identical screenshots | shared desktop or portal capture | isolate per-agent Xvfb/config/project; use in-app screenshots | | A visual "pass" has no artifact | verifier skipped the visual tier | treat as `not_run`; require screenshot path + command | +| Everything passes but the app is laggy | no performance budget; debug build used as evidence | measure release p95/input latency; fix or cut scope before features | +| New version works but feels worse than the old one | no baseline comparison | define baseline scenarios; regression = failure | +| Verifier passes, user says "not usable" | technical pass treated as product acceptance | product gate + explicit user verdict; fix before next milestone | +| Agents add M1/M2 while M0 is rough | milestone gate skipped | freeze scope; accept M0 or explicitly waive before expanding | ## Time and cost diff --git a/agent-swarm/references/product-quality.md b/agent-swarm/references/product-quality.md new file mode 100644 index 0000000..3899d99 --- /dev/null +++ b/agent-swarm/references/product-quality.md @@ -0,0 +1,99 @@ +# Product quality gates + +Functional correctness is necessary, not sufficient. A build can compile, pass tests and produce a +screenshot while still being too slow, visually incoherent or unpleasant to use. Do not let +`pass` on technical acceptance stand in for product acceptance. + +## Prototype vs product + +Choose the quality profile with the user before dispatch: + +| Profile | Use when | Mandatory gates | +|---|---|---| +| **Prototype** | the user explicitly wants a rough idea, spike or throwaway demo | runnable artifact, honest limits, no performance promise | +| **Product** | the artifact will be used, shipped, demoed to others or compared with an existing version | functional + performance + UX/visual + baseline + human acceptance | + +Do not silently treat a prototype as a product. Do not demand product polish for a disposable spike. +State the profile in the brief. + +## Gates + +### 1. Functional + +- acceptance commands and edge cases pass; +- the artifact runs, not just compiles; +- the verifier can reproduce the checks; +- no known crash/data-loss path. + +### 2. Performance + +Performance is part of correctness for interactive products. Define a budget in the brief before +code, measure it on a **release/optimized** build, and record the environment. + +Suggested starting points (adjust to the product, do not treat as universal): + +- p95 frame time at the target window size: **≤16.7 ms** for 60 FPS, or a documented lower target; +- input-to-visible response: **≤50 ms** for click/selection, **≤100 ms** for heavier actions; +- startup to usable UI: **≤2 s** for a local desktop editor; +- idle CPU and memory recorded; no runaway process when idle; +- test with a realistic scene, not only an empty one (e.g. 100/1 000 entities). + +Measure with a frame-time diagnostic/overlay, a scripted interaction, or an external profiler. +Record p50/p95, resolution, GPU/CPU, build profile and commit. **Debug builds are not evidence of +release performance**; if only debug is available, report the numbers as provisional and do not claim +the budget is met. + +If the budget is not met: reduce scope or fix the hot path before adding features. Do not hide a +frame-time regression behind more UI. + +### 3. UX / visual + +For GUI and visual work: + +- run a real 5-minute task, not just a screenshot; +- check hierarchy, typography, spacing, contrast, focus, hover/active/disabled states; +- check default, minimum and error/empty states; +- check that the viewport is not blocked by panels and that resizing does not clip controls; +- compare with the existing version/baseline on the same scenarios; +- capture artifacts and record the exact commands. + +“Looks fine” is not a verdict. Name what is readable, reachable, clipped or missing. + +### 4. Baseline + +If a previous version exists, define the scenarios where the new version must be **no worse**: +startup, time to first edit, selection, drag, save, visual density. A regression against the baseline +is a failure even if the new implementation has more features. + +### 5. Human acceptance + +A verifier can check evidence; it cannot decide that the product feels usable. After each milestone +or vertical slice: + +1. the lead gives the user a short runnable artifact and a 3–5 step scenario; +2. the user runs it and answers: usable / usable with issues / not usable; +3. issues become the next milestone, not a footnote in a report; +4. **do not start the next milestone until the current one is accepted** (or explicitly waived). + +Record the verdict in `state.json`/`final-report.md`. A detached verifier `pass` is not a substitute. + +### 6. Scope + +One milestone per session. Do not implement M1/M2 while M0 is still unpleasant. If the current slice is +not usable, adding assets, play mode or export will not fix it. + +## Stop rules + +Stop and re-plan when any of these is true: + +- functional acceptance passes but the product is visibly slower or worse than the baseline; +- performance was not measured on an optimized build for a product task; +- the user says “not usable” and the next step is more features instead of fixing the current slice; +- the work has consumed half the milestone budget/time without a runnable artifact; +- a UI slice has multiple parallel writers and the visual language is drifting; +- the verifier can only produce a `pass` by ignoring performance/UX. + +## Minimal product gate + +> Release build, measured p95 frame time and input latency, 5-minute real scenario, baseline +> comparison, explicit user verdict. If any is missing, the product gate is `not_run`, not `pass`. diff --git a/agent-swarm/references/token-economy.md b/agent-swarm/references/token-economy.md index a977452..1c177d7 100644 --- a/agent-swarm/references/token-economy.md +++ b/agent-swarm/references/token-economy.md @@ -32,9 +32,11 @@ Useful ratios: - Set a **hard budget** for the milestone and a **soft cap** at ~70%. At the soft cap, stop expanding scope; finish the current slice and verify. -- Reserve **20–30%** for verification, integration and one round of fixes. Do not spend the whole - budget on generation. -- Budget per phase, not only per wave: recon, build, verify, integrate. +- Reserve **20–30%** for verification, integration and one round of fixes, plus a **polish reserve** + for product-profile GUI/UX work. Do not spend the whole budget on generation. +- A budget is a ceiling, not a target. Do not stop a product task at the soft cap if the quality gate + is unmet; stop at the hard cap or when the product gate passes. Cheapness is not quality. +- Budget per phase, not only per wave: recon, build, verify, product gate, integrate. - Stop at the cap and report partial results plus unverified areas. Silent overspend is worse than a partial result. - Calibrate from the first measured milestone; do not copy invented numbers from a template. @@ -94,6 +96,7 @@ This preserves quality while avoiding repeated full reviews. | 20 tiny tasks | fixed overhead per child | merge into coherent units | | Reports/reviews as essays | output + lead input | schema + evidence lines | | Whole product in one wave | late integration, huge rework | vertical slices + checkpoints | +| Cutting polish to stay under budget | functional but unusable product | protect a polish reserve; stop at the hard cap, not the soft cap | | Vague "max quality" with no budget | unbounded spend | hard/soft caps + reserve | ## Minimal operating rule