Add isolated visual testing guidance

2026-10-27 19:17:49 +03:00
parent 54a5c849d4
commit 464b610bf1
7 changed files with 137 additions and 2 deletions
@@ -54,7 +54,7 @@ Write the plan before any worker starts.

 **Coverage matrix.** Each success criterion → task(s) → check. A criterion with no task is a hole; a task with no criterion is scope creep.

-**Visual acceptance.** For a GUI or visual product, define a deterministic screenshot/artifact command before dispatch (fixed window sizes, output path, clean pointer state) and put it in the acceptance matrix. Do not rely on desktop portal capture; it is unreliable on shared sessions and under Wayland.
+**Visual acceptance.** For a GUI or visual product, define a deterministic screenshot/artifact command before dispatch (fixed window sizes, output path, clean pointer state) and put it in the acceptance matrix. Do not rely on desktop portal capture; it is unreliable on shared sessions and under Wayland. Visual agents must use isolated displays/configs — see `references/visual-testing.md`; never let two agents share one desktop.

 **Waves.** Topologically sort by `depends_on`, then balance effort across batches. Do **not** plan 8–12 workers from a documented default: discover the runtime active-child cap first. Compute `batch_size = clamp(slot_cap - active_children - 1, 2, 8)`; recon and any other subagents count as active children. If the cap is unknown, use 6. Put foundational or risky tasks in batch 1; verify after the last build batch.

@@ -138,6 +138,8 @@ If the block reveals an underspecified interface shared by several tasks, update

 Choose verifiers by risk cluster, not one per task: 4–6 verifiers for 8–12 tasks. A verifier gets the task cards, the changed files (or diff), and the acceptance commands. It does not get the worker's rationale and does not fix anything.

+For a GUI product, give each visual verifier its own Xvfb/display and project copy (see `references/visual-testing.md`). Capture at least the default size, the minimum supported size, empty/error states and every modal; a shared-desktop screenshot is not evidence.
+
 **Freeze before verify.** Once you start this wave, do not edit source, tests or docs. If you already started and then fixed something, mark the in-flight review stale and re-run that verifier on the new build. A verifier's verdict is only valid for the exact tree it saw.

 Use `assets/verify.workflow.js` and `assets/verdict.schema.json`, not the full worker report schema. The verifier's detailed review **must be written to `.swarm/reviews/Vxx.md` before it returns**; the workflow return is only `{task_id, verdict, summary, review_file}`. Do not treat a workflow `null`/`no usable verdict` as a product failure if the review file exists — read the file, then re-run with the minimal schema if needed.
@@ -182,6 +184,8 @@ Final review is against the **original request**, not the plan: a fresh agent ch
 | Verifiers find the same defect | brief missing a convention | fix the brief, add a check, re-run affected tasks |
 | Lead context filling up | reading full reports | read summary + evidence; open only exceptions |
 | A worker hangs | no per-agent timeout exists | kill the job, split the task, add a tighter stop condition |
+| Visual agents produce wrong/identical screenshots | shared desktop or portal capture | isolate per-agent Xvfb/config/project; use in-app screenshots |
+| A visual "pass" has no artifact | verifier skipped the visual tier | treat as `not_run`; require screenshot path + command |

 ## Time and cost