Add product quality gates and milestone acceptance
This commit is contained in:
@@ -12,6 +12,8 @@
|
||||
|
||||
Скилл не обязателен для каждой задачи: маленькую задачу можно делать напрямую, среднюю — с независимым критиком, и только широкую — через swarm. Пользователь выбирает стек, объём и темп; скилл добавляет координацию, когда она оправдана.
|
||||
|
||||
Процесс — опциональный, quality gates — нет. Для продуктовой работы заранее выбирается профиль `prototype` или `product`. В `product` обязательны release-замеры производительности, реальный UX-сценарий, сравнение с baseline и явный вердикт пользователя. Технический `pass` verifier'а не заменяет приёмку продукта.
|
||||
|
||||
## Что происходит
|
||||
|
||||
1. **Fit-check.** Если задача не делится на 8+ независимых юнитов, скилл скажет не использовать swarm.
|
||||
@@ -20,7 +22,9 @@
|
||||
4. **План.** Задачи, владельцы файлов, контракты, критерии приёмки. Гейт: без этого волны не стартуют.
|
||||
5. **Волны.** Воркеры идут батчами по 6–8 (реальный лимит активных детей узнаётся перед запуском и записывается в `state.json`; recon не должен занимать слоты). Затем 4–6 независимых верификаторов на «замороженном» дереве, затем интегратор.
|
||||
6. **Аудит.** `scripts/audit-scope.mjs` сравнивает изменения с картой владения.
|
||||
7. **Отчёт.** `.swarm/final-report.md`.
|
||||
7. **Интеграция.** Один интегратор собирает общие файлы и прогоняет полный acceptance suite.
|
||||
8. **Product gate.** Для `product`: release-замеры, реальный сценарий, baseline и вердикт пользователя. Без этого следующий milestone не начинается.
|
||||
9. **Отчёт.** `.swarm/final-report.md`.
|
||||
|
||||
## Артефакты
|
||||
|
||||
@@ -34,7 +38,7 @@
|
||||
- Верификаторы работают на замороженном дереве; полный разбор пишется в `.swarm/reviews/Vxx.md`, а workflow возвращает только `{task_id, verdict, summary, review_file}`.
|
||||
- `blocked`-воркер не заменяется: лид правит контракт/shared-файл и продолжает того же агента через `send_message`.
|
||||
- Evidence зависит от домена: команда и тест для кода, источник и дата для research, отрендеренный артефакт для дизайна, reproducible metric для данных.
|
||||
- Токены — такой же бюджет, как время: 20–30% резервируется на verify/integration, после фиксов делается delta-review, а не полная перепроверка. Детали — в `references/token-economy.md`.
|
||||
- Токены — потолок, а не цель: 20–30% резервируется на verify/integration и polish, после фиксов делается delta-review, а не полная перепроверка. Детали — в `references/token-economy.md`.
|
||||
- Визуальные агенты работают в отдельных Xvfb/display-окружениях, со своим HOME/config/копией проекта; общий рабочий стол для скриншотов не используется. Детали — в `references/visual-testing.md`.
|
||||
- Computer-use — крайний tier: один агент, одноразовый desktop/VM, только после детерминированных проверок и никогда как единственное evidence.
|
||||
- Стоимость: 30–40 агентов ≈ 0.5–1.5M токенов.
|
||||
@@ -44,6 +48,7 @@
|
||||
|
||||
- Шаблоны: `assets/`, включая `prerequisites.md.tmpl`.
|
||||
- Prerequisites: `references/prerequisites.md`.
|
||||
- Product quality: `references/product-quality.md`.
|
||||
- Подробный runbook: `references/playbook.md`.
|
||||
- Оркестрация: `references/orchestration.md`.
|
||||
- Проверка скоупа: `node scripts/audit-scope.mjs --init` перед стартом, затем `node scripts/audit-scope.mjs`.
|
||||
|
||||
+15
-4
@@ -20,6 +20,11 @@ The skill is a **capability, not a mandatory process**. Apply the lightest mode
|
||||
|
||||
Do not force task cards, waves or `.swarm/` artifacts onto a small request. Conversely, do not skip independent verification on a large or risky one. The user keeps the choice of stack, scope and pace; the skill only adds coordination when breadth or risk justifies it.
|
||||
|
||||
**Process is optional; quality gates are not.** For anything the user will actually use, read
|
||||
`references/product-quality.md` and agree on a **prototype** or **product** profile. Product work
|
||||
requires a release-build performance check, a real UX scenario, baseline comparison and an explicit
|
||||
user verdict. A verifier's `pass` is technical evidence, not product acceptance.
|
||||
|
||||
## Fit check — before spawning anything
|
||||
|
||||
Swarm only when all are true:
|
||||
@@ -27,6 +32,8 @@ Swarm only when all are true:
|
||||
- Each unit has a **re-runnable acceptance check**: a command, a named artifact, a source/citation check, a metric, or a rendered result. There is no universal evidence type; match it to the domain adapter.
|
||||
- Wall-clock is dominated by **breadth**, not by one deep sequential chain.
|
||||
- You can state the interfaces between units **before** dispatch.
|
||||
- The units do **not** share one visual language or one screen. UI coherence is usually a single-owner
|
||||
job; parallel panel writers produce a Frankenstein interface.
|
||||
|
||||
Otherwise: one agent for small work; 2–5 `subagent` calls for a few long-lived roles you need to steer. Never swarm one bug, one design decision, or work where every unit edits the same files.
|
||||
|
||||
@@ -43,12 +50,13 @@ Otherwise: one agent for small work; 2–5 `subagent` calls for a few long-lived
|
||||
0. **Fit check & prerequisite gate** — go/no-go, budget (`max_agents`, `max_waves`, **tokens**), and **discover the available tools and real active-child cap** before sizing a wave. Check every required toolchain/runtime/service/display/credential with a real command; record the result in `.swarm/prerequisites.md` and `state.json`. Prefer project-local pinned setup; **never install globally/system-wide or run `sudo`/`curl | sh` without explicit user approval**. If a required prerequisite is missing and cannot be resolved safely, ask the user with the exact item, command, scope and alternatives **before any worker starts**. Workers never install; they return `blocked`.
|
||||
1. **Resolve prerequisites** — after approval/install, re-run the checks and the baseline suite. No build or verify wave starts while a required item is `needs_user` or `blocked`; optional items may be skipped only with a recorded reduced scope. See `references/prerequisites.md`.
|
||||
2. **Recon** — 2–4 parallel agents return findings with paths and line numbers. Timebox them, merge into `.swarm/brief.md` and `.swarm/map.md`, then stop them; recon must not hold slots during the build wave.
|
||||
3. **Plan gate** — write `.swarm/plan.json`, `plan.md`, `ownership.md`, `contracts.md`, and one `tasks/Txx.md` per unit. Every success criterion maps to a task and a check. No two tasks write the same path. Include a `LEAD`/`INTEGRATOR` task owning lead-owned shared files, screenshots, licenses and the final report. Freeze interfaces. Prefer vertical slices and files under ~800 lines; a task that requires reading a 3k-line file is too broad. **Do not dispatch until this gate passes.**
|
||||
3. **Plan gate** — write `.swarm/plan.json`, `plan.md`, `ownership.md`, `contracts.md`, and one `tasks/Txx.md` per unit. Every success criterion maps to a task and a check. No two tasks write the same path. Include a `LEAD`/`INTEGRATOR` task owning lead-owned shared files, screenshots, licenses and the final report. Freeze interfaces. Prefer vertical slices and files under ~800 lines; a task that requires reading a 3k-line file is too broad. Record the **quality profile**, performance budget, baseline scenarios and user-acceptance owner. **Do not dispatch until this gate passes.**
|
||||
4. **Build wave** — one `workflow` call per build wave when the tool exists, using the batched runner in `assets/wave.workflow.js`; its `concurrency` must fit the available slots (see Sizing). If `workflow`/`subagent` is unavailable, the lead executes the same packets sequentially. Retry `null`/`failed` once; if the runtime reports an active-child limit, halve the next batch and dispatch the remaining tasks — never relaunch the same oversubscribed fan-out.
|
||||
5. **Audit** — run `scripts/audit-scope.mjs` from the skill base directory; revert or reassign out-of-scope changes. Run the build/test gate.
|
||||
6. **Verify wave** — **freeze source changes**, run automated gates first (compile/tests/artifact), then 4–6 fresh verifiers by risk cluster (or 1–2 for a small domain task). Use `assets/verdict.schema.json` and, when available, `assets/verify.workflow.js`; the review file on disk is the source of truth, not the workflow return. Verifiers never fix. Match evidence to the domain adapter: rerun commands, read rendered artifacts, sample citations, recompute metrics. For GUI products, use per-agent isolated displays, never a shared desktop (see `references/visual-testing.md`). If you fix findings, re-run only the affected **delta** and mark the old review stale.
|
||||
7. **Integrate** — one integrator for shared files and cross-cutting fixes; re-run the full acceptance suite; then a final verification pass on the integrated build.
|
||||
8. **Close** — freeze the build and run a final independent review against the original request (not the plan), then write `final-report.md`.
|
||||
7. **Product gate** — for product-profile work, measure the release build against the performance budget, run the real UX scenario, compare with the baseline, and get the user's explicit verdict. A technical verifier pass is not enough. If the product is slower or worse than the baseline, stop and fix the slice; do not start the next milestone. See `references/product-quality.md`.
|
||||
8. **Integrate** — one integrator for shared files and cross-cutting fixes; re-run the full acceptance suite; then a final verification pass on the integrated build.
|
||||
9. **Close** — freeze the build and run a final independent review against the original request (not the plan), then write `final-report.md`.
|
||||
|
||||
## Hard rules
|
||||
|
||||
@@ -62,7 +70,9 @@ Otherwise: one agent for small work; 2–5 `subagent` calls for a few long-lived
|
||||
8. **Stop rules.** Two waves with no net progress → stop and re-plan. Verification failure rate > 30% → fix briefs/contracts, not agent count. Budget exhausted → report partial and list unverified areas.
|
||||
9. **No shared visual desktop.** Visual agents get their own display/config/project copy; see `references/visual-testing.md`. A screenshot from a shared desktop is not evidence. Computer-use is a last-resort, single-agent tier on a disposable desktop/VM; it is never the sole evidence and never runs in parallel with other visual agents.
|
||||
10. **Prerequisites before dispatch.** A required tool/runtime/service/credential is checked, installed with explicit approval if needed, and re-checked before workers start. Ask the user before work if it cannot be resolved; never let a worker install or silently skip a required check.
|
||||
11. **Token budget is a quality constraint.** Record spend per phase, keep a 20–30% reserve for verification/integration, and stop at the hard cap with a partial report. Cut redundant context, agents, full-file reads and full re-reviews — never cut tests or independent review. See `references/token-economy.md`.
|
||||
11. **Token budget is a ceiling, not a target.** Record spend per phase, keep a 20–30% reserve for verification/integration/**polish**, and do not stop a product task at the soft cap if the quality gate is unmet; stop at the hard cap or when the gate is met. Cut redundant context, agents, full-file reads and full re-reviews — never cut tests, independent review or user-driven fixes. See `references/token-economy.md`.
|
||||
12. **A technical pass is not product acceptance.** Product-profile work needs measured performance, a real UX scenario, baseline comparison and an explicit user verdict; otherwise the product gate is `not_run`. See `references/product-quality.md`.
|
||||
13. **One milestone at a time.** Do not start M1/M2 while M0 is still laggy, visually rough or explicitly “not usable”. Scope adds must wait for acceptance.
|
||||
|
||||
## Sizing
|
||||
|
||||
@@ -114,6 +124,7 @@ For a `blocked` task: apply the smallest contract/shared-file change, update `co
|
||||
- `references/token-economy.md` — measurement, budgets, context discipline, delta verification, vertical slices.
|
||||
- `references/prerequisites.md` — preflight checks, install/approval policy, degraded-scope record.
|
||||
- `references/visual-testing.md` — isolated Xvfb/container visual tests, deterministic screenshots, verifier procedure (GUI adapter).
|
||||
- `references/product-quality.md` — prototype/product profiles, performance and UX gates, baseline comparison, human acceptance.
|
||||
- `assets/` — copyable brief, task, and verifier templates; `report.schema.json`, `verdict.schema.json`; `wave.workflow.js` (batched workers), `verify.workflow.js` (frozen verification).
|
||||
- `scripts/audit-scope.mjs` — compares changed files to the ownership map.
|
||||
|
||||
|
||||
@@ -12,6 +12,13 @@ Scope is checked by the lead with the skill's `scripts/audit-scope.mjs`; workers
|
||||
## Non-goals
|
||||
- {{What must not change or be attempted.}}
|
||||
|
||||
## Quality profile
|
||||
- Profile: {{prototype / product}}
|
||||
- Performance budget: {{p95 frame time / latency / startup / memory, or n/a}}
|
||||
- Baseline version/scenarios: {{version + 3–5 step scenario, or none}}
|
||||
- User acceptance owner: {{name / pending}}
|
||||
- Release build command: {{command or n/a}}
|
||||
|
||||
## Domain profile
|
||||
- Domain: {{software / research / content / data / design / operations}}
|
||||
- Primary artifacts: {{paths, sections, datasets, renders, services}}
|
||||
|
||||
@@ -37,6 +37,11 @@ Role: worker | Wave: {{n}} | Depends on: {{ids or —}} | Effort: {{S/M/L}}
|
||||
2. `{{citation check / metric / render at sizes}}` → {{expected result}}
|
||||
3. Every criterion must have evidence a fresh verifier can reproduce.
|
||||
|
||||
If the quality profile is `product` and this task touches UI/interaction, also record:
|
||||
- release-build p95 frame time and input latency at the target size;
|
||||
- the 3–5 step user scenario;
|
||||
- baseline comparison (new must not be worse).
|
||||
|
||||
## Report
|
||||
`.swarm/reports/{{ID}}.json` per `.swarm/report.schema.json`; status `done|partial|blocked|failed`.
|
||||
Keep the report to summary + evidence lines; raw logs go to `.swarm/tmp/{{ID}}/`.
|
||||
|
||||
@@ -32,8 +32,21 @@ For GUI products, run the app in your own isolated Xvfb/display (never a shared
|
||||
|
||||
Computer-use (mouse/keyboard) is an optional last-resort tier: one agent, disposable desktop/VM, after deterministic checks, never parallel, never the sole evidence. Prefer app screenshot mode or scripted input.
|
||||
|
||||
## Product quality (product profile only)
|
||||
Verify the release/optimized build, not a debug build:
|
||||
- p50/p95 frame time at the target size, input-to-visible latency, startup, idle CPU/memory;
|
||||
- the 3–5 step real user scenario, not just a screenshot;
|
||||
- baseline comparison on the same scenario (regression = fail);
|
||||
- report `not_run` if these were not measured.
|
||||
|
||||
A technical pass is **not** product acceptance. The user still runs the artifact and gives an explicit
|
||||
verdict (usable / usable with issues / not usable). Do not mark the product gate `pass` on the user's
|
||||
behalf.
|
||||
|
||||
## Verdict
|
||||
`pass` / `pass_with_risks` / `fail` — with the top risks and exact repro commands.
|
||||
For product-profile work, `pass` requires functional + measured performance + UX scenario + baseline;
|
||||
otherwise use `pass_with_risks` with missing gates named, or `not_run`.
|
||||
|
||||
## Report
|
||||
Keep the review to findings + commands + evidence (target 5–8k tokens); raw logs stay in `.swarm/tmp/`. Review the diff and affected checks, not the whole repository.
|
||||
|
||||
@@ -45,6 +45,7 @@ Copy `assets/verifier.md.tmpl`. Rules:
|
||||
- Ask for scope, contract, and test-quality checks.
|
||||
- Require the full review file **first**, then a minimal `{task_id, verdict, summary, review_file}` return (`assets/verdict.schema.json`). The review file is the source of truth; a schema miss must not erase the verification work.
|
||||
- GUI: require an isolated Xvfb/display, screenshots at default and minimum sizes, and the exact commands used; shared-desktop screenshots are not evidence.
|
||||
- Product profile: require release-build performance numbers (p95 frame time, input latency, startup), the real UX scenario and a baseline comparison. A technical pass is not product acceptance; the user still gives the final verdict.
|
||||
|
||||
## Report
|
||||
|
||||
|
||||
@@ -12,6 +12,8 @@ Answer these before spawning:
|
||||
- **Interfaces.** Can you freeze signatures, data shapes, and file names before dispatch?
|
||||
- **Breadth.** Is the time dominated by many independent pieces, not one deep chain?
|
||||
|
||||
First decide the **quality profile** with the user: `prototype` (runnable spike, no performance promise) or `product` (performance, UX, baseline and human acceptance are mandatory). See `references/product-quality.md`. For UI/product work, prefer one implementer + one critic over parallel panel owners.
|
||||
|
||||
Budget:
|
||||
- `agents ≈ units + ceil(units / 5) verifiers + 1–2 integrators + 2–4 recon`; add retries and re-verifications explicitly. Set `max_agents` only after checking available tools and slot cap.
|
||||
- Tokens ≈ `agents × fixed overhead (10–25k) + context reads + output + reviews`. Treat this as an estimate, then calibrate from measured usage. A 3k-line file read by several agents can cost more than the code they write.
|
||||
@@ -96,6 +98,8 @@ Write the plan before any worker starts. Keep the plan proportionate: a small do
|
||||
- [ ] Files/modules over ~800 lines are split, or the task explicitly owns the integration cost.
|
||||
- [ ] Token budget, soft cap and verification reserve are recorded; units are merged/split to avoid fixed-overhead waste.
|
||||
- [ ] The plan uses vertical slices/checkpoints for a multi-milestone product.
|
||||
- [ ] Quality profile (`prototype`/`product`), performance budget, baseline scenarios and user-acceptance owner are recorded.
|
||||
- [ ] For product-profile GUI work, the plan names release-build measurements and the exact user scenario.
|
||||
- [ ] No path in two write scopes.
|
||||
- [ ] Contracts frozen for every cross-task interface.
|
||||
- [ ] Coverage matrix complete.
|
||||
@@ -183,13 +187,28 @@ For a GUI product, give each visual verifier its own Xvfb/display and project co
|
||||
|
||||
**Freeze before verify.** Once you start this wave, do not edit source, tests or docs. If you already started and then fixed something, mark the in-flight review stale and re-run that verifier on the new build. A verifier's verdict is only valid for the exact tree it saw.
|
||||
|
||||
**Technical verification is not product acceptance.** Verifiers check evidence; for product-profile work they must also measure release-build frame time/input latency, run the real UX scenario and compare with the baseline. If performance/UX was not measured, the product gate is `not_run`, not `pass`.
|
||||
|
||||
Use `assets/verify.workflow.js` and `assets/verdict.schema.json`, not the full worker report schema. The verifier's detailed review **must be written to `.swarm/reviews/Vxx.md` before it returns**; the workflow return is only `{task_id, verdict, summary, review_file}`. Do not treat a workflow `null`/`no usable verdict` as a product failure if the review file exists — read the file, then re-run with the minimal schema if needed.
|
||||
|
||||
Evidence standard: for every criterion, a command/artifact/citation/metric and its observed result. `pass` without evidence is `not_run`. Accept `pass_with_risks` only with named risks and a follow-up task. Verifiers should copy or hash-check generated files rather than rewriting the tree; if a required gate script rewrites generated artifacts, report the hash delta.
|
||||
|
||||
On failure, create a fix task with the verifier's repro; do not send the verifier back to fix its own finding. After fixes, re-run only the affected **delta** (diff + findings + acceptance commands), not the whole repository, and mark the previous review stale; repeat until the remaining findings are minor/documented. Then run one final integrated verification pass on the frozen build.
|
||||
|
||||
## 6. Integrate
|
||||
## 6. Product gate
|
||||
|
||||
For `product`-profile work, run this before integration:
|
||||
|
||||
1. Build the release/optimized artifact. Debug numbers are provisional only.
|
||||
2. Measure p50/p95 frame time at the target size, input-to-visible latency, startup and idle CPU/memory on a realistic scene.
|
||||
3. Run the 3–5 step real user scenario, not just a screenshot.
|
||||
4. Compare with the baseline version on the same scenario; a regression is a failure.
|
||||
5. Give the user the artifact and ask for an explicit verdict: usable / usable with issues / not usable.
|
||||
6. Record measurements and the user verdict. Do not start the next milestone until accepted or explicitly waived.
|
||||
|
||||
If the result is functional but laggy or visually worse than the baseline, stop and fix the slice. Do not add features to compensate. See `references/product-quality.md`.
|
||||
|
||||
## 7. Integrate
|
||||
|
||||
One integrator (two only for disjoint areas) owns:
|
||||
- shared files and cross-cutting edits;
|
||||
@@ -199,7 +218,7 @@ One integrator (two only for disjoint areas) owns:
|
||||
|
||||
No parallel workers during integration. The integrator may read all task cards and reviews; it writes `.swarm/integration.md` with what changed and why. Freeze the tree while the final verification runs; any post-verification edit invalidates the affected evidence.
|
||||
|
||||
## 7. Close
|
||||
## 8. Close
|
||||
|
||||
Final review is against the **original request**, not the plan: a fresh agent checks the success criteria, the diff, and the evidence. Then the lead writes `.swarm/final-report.md` with these sections:
|
||||
|
||||
@@ -233,6 +252,10 @@ Final review is against the **original request**, not the plan: a fresh agent ch
|
||||
| Reports/reviews dominate input | oversized prose | cap to evidence lines; logs in artifacts |
|
||||
| Visual agents produce wrong/identical screenshots | shared desktop or portal capture | isolate per-agent Xvfb/config/project; use in-app screenshots |
|
||||
| A visual "pass" has no artifact | verifier skipped the visual tier | treat as `not_run`; require screenshot path + command |
|
||||
| Everything passes but the app is laggy | no performance budget; debug build used as evidence | measure release p95/input latency; fix or cut scope before features |
|
||||
| New version works but feels worse than the old one | no baseline comparison | define baseline scenarios; regression = failure |
|
||||
| Verifier passes, user says "not usable" | technical pass treated as product acceptance | product gate + explicit user verdict; fix before next milestone |
|
||||
| Agents add M1/M2 while M0 is rough | milestone gate skipped | freeze scope; accept M0 or explicitly waive before expanding |
|
||||
|
||||
## Time and cost
|
||||
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
# Product quality gates
|
||||
|
||||
Functional correctness is necessary, not sufficient. A build can compile, pass tests and produce a
|
||||
screenshot while still being too slow, visually incoherent or unpleasant to use. Do not let
|
||||
`pass` on technical acceptance stand in for product acceptance.
|
||||
|
||||
## Prototype vs product
|
||||
|
||||
Choose the quality profile with the user before dispatch:
|
||||
|
||||
| Profile | Use when | Mandatory gates |
|
||||
|---|---|---|
|
||||
| **Prototype** | the user explicitly wants a rough idea, spike or throwaway demo | runnable artifact, honest limits, no performance promise |
|
||||
| **Product** | the artifact will be used, shipped, demoed to others or compared with an existing version | functional + performance + UX/visual + baseline + human acceptance |
|
||||
|
||||
Do not silently treat a prototype as a product. Do not demand product polish for a disposable spike.
|
||||
State the profile in the brief.
|
||||
|
||||
## Gates
|
||||
|
||||
### 1. Functional
|
||||
|
||||
- acceptance commands and edge cases pass;
|
||||
- the artifact runs, not just compiles;
|
||||
- the verifier can reproduce the checks;
|
||||
- no known crash/data-loss path.
|
||||
|
||||
### 2. Performance
|
||||
|
||||
Performance is part of correctness for interactive products. Define a budget in the brief before
|
||||
code, measure it on a **release/optimized** build, and record the environment.
|
||||
|
||||
Suggested starting points (adjust to the product, do not treat as universal):
|
||||
|
||||
- p95 frame time at the target window size: **≤16.7 ms** for 60 FPS, or a documented lower target;
|
||||
- input-to-visible response: **≤50 ms** for click/selection, **≤100 ms** for heavier actions;
|
||||
- startup to usable UI: **≤2 s** for a local desktop editor;
|
||||
- idle CPU and memory recorded; no runaway process when idle;
|
||||
- test with a realistic scene, not only an empty one (e.g. 100/1 000 entities).
|
||||
|
||||
Measure with a frame-time diagnostic/overlay, a scripted interaction, or an external profiler.
|
||||
Record p50/p95, resolution, GPU/CPU, build profile and commit. **Debug builds are not evidence of
|
||||
release performance**; if only debug is available, report the numbers as provisional and do not claim
|
||||
the budget is met.
|
||||
|
||||
If the budget is not met: reduce scope or fix the hot path before adding features. Do not hide a
|
||||
frame-time regression behind more UI.
|
||||
|
||||
### 3. UX / visual
|
||||
|
||||
For GUI and visual work:
|
||||
|
||||
- run a real 5-minute task, not just a screenshot;
|
||||
- check hierarchy, typography, spacing, contrast, focus, hover/active/disabled states;
|
||||
- check default, minimum and error/empty states;
|
||||
- check that the viewport is not blocked by panels and that resizing does not clip controls;
|
||||
- compare with the existing version/baseline on the same scenarios;
|
||||
- capture artifacts and record the exact commands.
|
||||
|
||||
“Looks fine” is not a verdict. Name what is readable, reachable, clipped or missing.
|
||||
|
||||
### 4. Baseline
|
||||
|
||||
If a previous version exists, define the scenarios where the new version must be **no worse**:
|
||||
startup, time to first edit, selection, drag, save, visual density. A regression against the baseline
|
||||
is a failure even if the new implementation has more features.
|
||||
|
||||
### 5. Human acceptance
|
||||
|
||||
A verifier can check evidence; it cannot decide that the product feels usable. After each milestone
|
||||
or vertical slice:
|
||||
|
||||
1. the lead gives the user a short runnable artifact and a 3–5 step scenario;
|
||||
2. the user runs it and answers: usable / usable with issues / not usable;
|
||||
3. issues become the next milestone, not a footnote in a report;
|
||||
4. **do not start the next milestone until the current one is accepted** (or explicitly waived).
|
||||
|
||||
Record the verdict in `state.json`/`final-report.md`. A detached verifier `pass` is not a substitute.
|
||||
|
||||
### 6. Scope
|
||||
|
||||
One milestone per session. Do not implement M1/M2 while M0 is still unpleasant. If the current slice is
|
||||
not usable, adding assets, play mode or export will not fix it.
|
||||
|
||||
## Stop rules
|
||||
|
||||
Stop and re-plan when any of these is true:
|
||||
|
||||
- functional acceptance passes but the product is visibly slower or worse than the baseline;
|
||||
- performance was not measured on an optimized build for a product task;
|
||||
- the user says “not usable” and the next step is more features instead of fixing the current slice;
|
||||
- the work has consumed half the milestone budget/time without a runnable artifact;
|
||||
- a UI slice has multiple parallel writers and the visual language is drifting;
|
||||
- the verifier can only produce a `pass` by ignoring performance/UX.
|
||||
|
||||
## Minimal product gate
|
||||
|
||||
> Release build, measured p95 frame time and input latency, 5-minute real scenario, baseline
|
||||
> comparison, explicit user verdict. If any is missing, the product gate is `not_run`, not `pass`.
|
||||
@@ -32,9 +32,11 @@ Useful ratios:
|
||||
|
||||
- Set a **hard budget** for the milestone and a **soft cap** at ~70%. At the soft cap, stop expanding
|
||||
scope; finish the current slice and verify.
|
||||
- Reserve **20–30%** for verification, integration and one round of fixes. Do not spend the whole
|
||||
budget on generation.
|
||||
- Budget per phase, not only per wave: recon, build, verify, integrate.
|
||||
- Reserve **20–30%** for verification, integration and one round of fixes, plus a **polish reserve**
|
||||
for product-profile GUI/UX work. Do not spend the whole budget on generation.
|
||||
- A budget is a ceiling, not a target. Do not stop a product task at the soft cap if the quality gate
|
||||
is unmet; stop at the hard cap or when the product gate passes. Cheapness is not quality.
|
||||
- Budget per phase, not only per wave: recon, build, verify, product gate, integrate.
|
||||
- Stop at the cap and report partial results plus unverified areas. Silent overspend is worse than a
|
||||
partial result.
|
||||
- Calibrate from the first measured milestone; do not copy invented numbers from a template.
|
||||
@@ -94,6 +96,7 @@ This preserves quality while avoiding repeated full reviews.
|
||||
| 20 tiny tasks | fixed overhead per child | merge into coherent units |
|
||||
| Reports/reviews as essays | output + lead input | schema + evidence lines |
|
||||
| Whole product in one wave | late integration, huge rework | vertical slices + checkpoints |
|
||||
| Cutting polish to stay under budget | functional but unusable product | protect a polish reserve; stop at the hard cap, not the soft cap |
|
||||
| Vague "max quality" with no budget | unbounded spend | hard/soft caps + reserve |
|
||||
|
||||
## Minimal operating rule
|
||||
|
||||
Reference in New Issue
Block a user