Add isolated visual testing guidance

README.md

@@ -12,6 +12,7 @@
 - [Инструкции скилла](agent-swarm/SKILL.md)
 - [Runbook](agent-swarm/references/playbook.md)
 - [Cookbook по `workflow`](agent-swarm/references/orchestration.md)
+- [Изолированное визуальное тестирование](agent-swarm/references/visual-testing.md)
 
 ### Установка
 

agent-swarm/README.ru.md

@@ -30,6 +30,7 @@
 - Дети не общаются между собой и не делегируют дальше; координация — через файлы и лида.
 - Верификаторы работают на замороженном дереве; полный разбор пишется в `.swarm/reviews/Vxx.md`, а workflow возвращает только `{task_id, verdict, summary, review_file}`.
 - `blocked`-воркер не заменяется: лид правит контракт/shared-файл и продолжает того же агента через `send_message`.
+- Визуальные агенты работают в отдельных Xvfb/display-окружениях, со своим HOME/config/копией проекта; общий рабочий стол для скриншотов не используется. Детали — в `references/visual-testing.md`.
 - Стоимость: 30–40 агентов ≈ 0.5–1.5M токенов.
 - Для задачи, которую один агент делает за 15 минут, swarm не нужен.
 

agent-swarm/SKILL.md

@@ -33,7 +33,7 @@ Otherwise: one agent for small work; 2–5 `subagent` calls for a few long-lived
 2. **Plan gate** — write `.swarm/plan.json`, `plan.md`, `ownership.md`, `contracts.md`, and one `tasks/Txx.md` per unit. Every success criterion maps to a task and a check. No two tasks write the same path. Include a `LEAD`/`INTEGRATOR` task owning lead-owned shared files, screenshots, licenses and the final report. Freeze interfaces. **Do not dispatch until this gate passes.**
 3. **Build wave** — one `workflow` call per build wave, using the batched runner in `assets/wave.workflow.js`; its `concurrency` must fit the available slots (see Sizing). Retry `null`/`failed` once; if the runtime reports an active-child limit, halve the next batch and dispatch the remaining tasks — never relaunch the same oversubscribed fan-out.
 4. **Audit** — run `scripts/audit-scope.mjs` from the skill base directory; revert or reassign out-of-scope changes. Run the build/test gate.
-5. **Verify wave** — **freeze source changes**, then run 4–6 fresh verifiers by risk cluster. Use `assets/verdict.schema.json` and `assets/verify.workflow.js`; the review file on disk is the source of truth, not the workflow return. Verifiers never fix. If you fix findings, re-run the affected verifier and mark the old review stale.
+5. **Verify wave** — **freeze source changes**, then run 4–6 fresh verifiers by risk cluster. Use `assets/verdict.schema.json` and `assets/verify.workflow.js`; the review file on disk is the source of truth, not the workflow return. Verifiers never fix. For GUI products, verify in **per-agent isolated displays**, never a shared desktop (see `references/visual-testing.md`). If you fix findings, re-run the affected verifier and mark the old review stale.
 6. **Integrate** — one integrator for shared files and cross-cutting fixes; re-run the full acceptance suite; then a final verification pass on the integrated build.
 7. **Close** — freeze the build and run a final independent review against the original request (not the plan), then write `final-report.md`.
 
@@ -47,6 +47,7 @@ Otherwise: one agent for small work; 2–5 `subagent` calls for a few long-lived
 6. **Respect the runtime slot cap.** Active children include recon, verifiers, integrators and any other subagents. A spawn-limit error is a scheduling bug, not a worker failure: shrink the batch, keep completed results, and dispatch the remainder.
 7. **Freeze during verification.** Once the verify wave starts, do not edit source/tests/docs except to fix a verified finding. After a fix, re-run the affected verifier; an old review is stale.
 8. **Stop rules.** Two waves with no net progress → stop and re-plan. Verification failure rate > 30% → fix briefs/contracts, not agent count. Budget exhausted → report partial and list unverified areas.
+9. **No shared visual desktop.** Visual agents get their own display/config/project copy; see `references/visual-testing.md`. A screenshot from a shared desktop is not evidence.
 
 ## Sizing
 
@@ -92,6 +93,7 @@ For a `blocked` task: apply the smallest contract/shared-file change, update `co
 - `references/playbook.md` — phase-by-phase runbook, gates, checklists, failure recovery.
 - `references/packets.md` — how to write briefs, task cards, and verifier cards; examples and common mistakes.
 - `references/orchestration.md` — `workflow` cookbook: script patterns, schema subset, retries, background runs, archetypes (code / research / content).
+- `references/visual-testing.md` — isolated Xvfb/container visual tests, deterministic screenshots, verifier procedure.
 - `assets/` — copyable brief, task, and verifier templates; `report.schema.json`, `verdict.schema.json`; `wave.workflow.js` (batched workers), `verify.workflow.js` (frozen verification).
 - `scripts/audit-scope.mjs` — compares changed files to the ownership map.
 

agent-swarm/assets/verifier.md.tmpl

@@ -21,6 +21,9 @@ You did not write this work. Do not fix it. Do not trust worker summaries.
 ## Freeze
 The tree is frozen during verification. Do not edit product source, tests, docs or manifests. If you need a gate script that rewrites generated files, copy it/hash-check it instead of changing the tree.
 
+## Visual checks
+For GUI products, run the app in your own isolated Xvfb/display (never a shared desktop). Capture the default size, the minimum size, empty/error states and every modal. Record the screenshot path and exact command alongside each finding; a screenshot without a command is not evidence.
+
 ## Verdict
 `pass` / `pass_with_risks` / `fail` — with the top risks and exact repro commands.
 

agent-swarm/references/packets.md

@@ -31,6 +31,7 @@ Copy `assets/task.md.tmpl`. Rules:
 - Write scope: globs. Everything else is read-only.
 - Contract: exact signature or output shape. If it is not written, it is not frozen.
 - Acceptance: 1–3 commands a verifier can re-run. "Code is clean" is not a check.
+- GUI: acceptance includes a deterministic screenshot command, fixed window sizes and an isolated display; see `references/visual-testing.md`.
 - Stop conditions: time/attempt limit (25 min for S/M, 45–60 for L) plus "ask, do not guess" for load-bearing unknowns.
 - Never include another task's internal details; workers do not need the whole plan.
 
@@ -43,6 +44,7 @@ Copy `assets/verifier.md.tmpl`. Rules:
 - Require a table: criterion, command, observed, pass/fail.
 - Ask for scope, contract, and test-quality checks.
 - Require the full review file **first**, then a minimal `{task_id, verdict, summary, review_file}` return (`assets/verdict.schema.json`). The review file is the source of truth; a schema miss must not erase the verification work.
+- GUI: require an isolated Xvfb/display, screenshots at default and minimum sizes, and the exact commands used; shared-desktop screenshots are not evidence.
 
 ## Report
 

agent-swarm/references/playbook.md

@@ -54,7 +54,7 @@ Write the plan before any worker starts.
 
 **Coverage matrix.** Each success criterion → task(s) → check. A criterion with no task is a hole; a task with no criterion is scope creep.
 
-**Visual acceptance.** For a GUI or visual product, define a deterministic screenshot/artifact command before dispatch (fixed window sizes, output path, clean pointer state) and put it in the acceptance matrix. Do not rely on desktop portal capture; it is unreliable on shared sessions and under Wayland.
+**Visual acceptance.** For a GUI or visual product, define a deterministic screenshot/artifact command before dispatch (fixed window sizes, output path, clean pointer state) and put it in the acceptance matrix. Do not rely on desktop portal capture; it is unreliable on shared sessions and under Wayland. Visual agents must use isolated displays/configs — see `references/visual-testing.md`; never let two agents share one desktop.
 
 **Waves.** Topologically sort by `depends_on`, then balance effort across batches. Do **not** plan 8–12 workers from a documented default: discover the runtime active-child cap first. Compute `batch_size = clamp(slot_cap - active_children - 1, 2, 8)`; recon and any other subagents count as active children. If the cap is unknown, use 6. Put foundational or risky tasks in batch 1; verify after the last build batch.
 
@@ -138,6 +138,8 @@ If the block reveals an underspecified interface shared by several tasks, update
 
 Choose verifiers by risk cluster, not one per task: 4–6 verifiers for 8–12 tasks. A verifier gets the task cards, the changed files (or diff), and the acceptance commands. It does not get the worker's rationale and does not fix anything.
 
+For a GUI product, give each visual verifier its own Xvfb/display and project copy (see `references/visual-testing.md`). Capture at least the default size, the minimum supported size, empty/error states and every modal; a shared-desktop screenshot is not evidence.
+
 **Freeze before verify.** Once you start this wave, do not edit source, tests or docs. If you already started and then fixed something, mark the in-flight review stale and re-run that verifier on the new build. A verifier's verdict is only valid for the exact tree it saw.
 
 Use `assets/verify.workflow.js` and `assets/verdict.schema.json`, not the full worker report schema. The verifier's detailed review **must be written to `.swarm/reviews/Vxx.md` before it returns**; the workflow return is only `{task_id, verdict, summary, review_file}`. Do not treat a workflow `null`/`no usable verdict` as a product failure if the review file exists — read the file, then re-run with the minimal schema if needed.
@@ -182,6 +184,8 @@ Final review is against the **original request**, not the plan: a fresh agent ch
 | Verifiers find the same defect | brief missing a convention | fix the brief, add a check, re-run affected tasks |
 | Lead context filling up | reading full reports | read summary + evidence; open only exceptions |
 | A worker hangs | no per-agent timeout exists | kill the job, split the task, add a tighter stop condition |
+| Visual agents produce wrong/identical screenshots | shared desktop or portal capture | isolate per-agent Xvfb/config/project; use in-app screenshots |
+| A visual "pass" has no artifact | verifier skipped the visual tier | treat as `not_run`; require screenshot path + command |
 
 ## Time and cost
 

agent-swarm/references/visual-testing.md

@@ -0,0 +1,122 @@
+# Visual testing for GUI agents
+
+GUI products fail verification for reasons that unit tests cannot see: clipped panels, focus-stealing
+modals, dialogs that do not close, wrong DPI, a viewport that renders a blank background, or a
+screenshot that captures a different agent's window. Treat visual verification as a separate test
+tier with its own infrastructure.
+
+## Rule zero: never share one desktop
+
+Do not run several visual agents against the same X11/Wayland session. They steal focus, overlap
+windows, race on the clipboard/configuration, and screenshot portals may return another window or
+the desktop wallpaper. A common failure mode is "the screenshot looks fine" while it is a different
+build's window.
+
+Each visual agent gets:
+
+- its own display (`DISPLAY=:N` / Wayland socket),
+- its own `HOME`, `XDG_CONFIG_HOME`, `XDG_RUNTIME_DIR`, `TMPDIR`,
+- its own project copy/worktree and artifact directory,
+- its own `CARGO_TARGET_DIR` (or a read-only prebuilt cache + writable per-agent target),
+- its own service port (BRP, dev server, websocket),
+- a cleanup trap that kills the app, the X server and any helper processes.
+
+## Three tiers
+
+| Tier | Display | Renderer | Parallelism | Use |
+|---|---|---|---|---|
+| 1. Logic/headless | none | none | high | unit tests, egui `Context::run`, data/actions |
+| 2. Visual UI | per-agent Xvfb | llvmpipe/lavapipe | 2–4 agents | layout, dialogs, screenshots, 2D UI |
+| 3. Real GPU | one host/display/GPU | real GPU | **1** | PBR, real Bevy render, performance, final screenshot |
+
+Do not run Tier 3 in parallel with other visual agents. Do not use a shared desktop as Tier 2.
+
+## Per-agent Xvfb recipe
+
+Requirements in the image/host: `Xvfb`, `import` (ImageMagick), optionally `xdotool`,
+`openbox`/`fluxbox`, Mesa llvmpipe and Vulkan lavapipe.
+
+```sh
+# Pick a free display; never hardcode one that another agent may use.
+for n in $(seq 90 140); do
+  [ ! -e "/tmp/.X11-unix/X$n" ] && DISPLAY_NUM=$n && break
+done
+SIZE=1440x900
+ART=/tmp/agent-$DISPLAY_NUM
+
+mkdir -p "$ART"/{home,config,runtime,tmp}
+Xvfb ":$DISPLAY_NUM" -screen 0 "${SIZE}x24" -nolisten tcp >"$ART/xvfb.log" 2>&1 &
+XVFB_PID=$!
+APP_PID=""
+cleanup() {
+  [ -n "$APP_PID" ] && kill "$APP_PID" 2>/dev/null || true
+  kill "$XVFB_PID" 2>/dev/null || true
+  wait "$XVFB_PID" 2>/dev/null || true
+}
+trap cleanup EXIT
+
+export DISPLAY=":$DISPLAY_NUM"
+export HOME="$ART/home"
+export XDG_CONFIG_HOME="$ART/config"
+export XDG_RUNTIME_DIR="$ART/runtime"
+export TMPDIR="$ART/tmp"
+export LIBGL_ALWAYS_SOFTWARE=1
+export VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/lvp_icd.x86_64.json
+
+# Start the app *after* the X socket exists. A small WM helps focus/IME.
+sleep 0.5
+# openbox &   # optional
+timeout 120s cargo run -p app -- --screenshot-mode "$ART/app.png" &
+APP_PID=$!
+
+# Prefer an in-app deterministic screenshot. Fallback: capture the root window.
+wait "$APP_PID" || true
+[ -f "$ART/app.png" ] || import -display ":$DISPLAY_NUM" -window root "$ART/fallback.png"
+```
+
+If the app has no screenshot mode, add one before the UI wave: fixed size, wait N frames, hide
+tooltips/animations/cursor, write PNG, exit 0. External root capture is a last resort.
+
+## Containers
+
+Containers are useful for reproducibility and cleanup, not as a default for every iteration.
+
+- Build one pinned image with Rust, Xvfb, Mesa/lavapipe, ImageMagick, xdotool and a minimal WM.
+- Run one container per visual agent: `--rm --network=none --userns=keep-id`, no host `$HOME`, a
+  read-only repo mount plus a writable worktree/artifact mount.
+- Keep cargo registry/cache in a named read-only volume; use a per-container target dir.
+- GPU: `--device /dev/dri` for Intel/AMD; NVIDIA needs the container toolkit/CDI. Prefer software
+  rendering for UI tests and reserve GPU containers for one final Tier 3 run.
+- Limit visual containers to 2–4 concurrently; llvmpipe uses CPU aggressively.
+
+## Deterministic screenshots
+
+- Fixed window size and scale factor (`1.0`) for both small and target sizes, e.g. 1000×640 and
+  1440×900.
+- Wait for the UI to settle (textures loaded, panels laid out) before capture.
+- Disable tooltips, hover highlights, animations, autosave toasts and pointer-dependent chrome in
+  screenshot mode; move the pointer out of the window.
+- Capture the app's own framebuffer, not the compositor. Portal capture is unreliable on Wayland and
+  shared sessions.
+- Compare hashes for determinism and pixel-diff with a tolerance for antialiasing/GPU differences.
+- Store the exact command, window size and build commit next to each artifact.
+
+## Visual verifier procedure
+
+A visual verifier should:
+
+1. Launch the exact build in its own isolated display; never reuse another agent's window.
+2. Capture at least the default size, the minimum supported size, empty/error states, and every
+   modal that blocks input.
+3. Check clipping/overflow, focus after closing a modal, keyboard shortcuts behind modals, selection
+   visibility, contrast, and scroll reachability.
+4. Record a screenshot path and the observed issue for every finding; "looks off" is not evidence.
+5. For animation/drag interactions, use a scripted headless harness or a video/screenshot sequence;
+   dragging through a portal may collapse press/move/release into one frame.
+
+## Failure signs
+
+- Two visual jobs produce identical screenshots → they shared a display or captured the root.
+- A screenshot shows the wrong title/theme/build → focus or portal capture, not a UI bug.
+- Only the first visual job is reliable → the display/config isolation is incomplete.
+- A "pass" has no screenshot or command → treat it as `not_run`.