P2c: in-app choir agent with proposals and approval

This commit is contained in:
Emil
2026-11-01 02:27:44 +03:00
parent 0df07577a1
commit 2ecb8822de
20 changed files with 2236 additions and 30 deletions
+9 -3
View File
@@ -2,7 +2,7 @@
Sound is terrain. A sampling instrument where material is cut in a quarry, laid as stones in a nave, and played by a hand or an agent through the same verbs.
This repository is **P2 in progress**: the engine, the wire, the descent, the quarry, a persistent project, recording, manual faces, undo across restarts, a native host, an **8-bar arrangement of four sections**, and an **offline WAV render**. A found or recorded sound is cut into sixteen faces, laid on a vein, arranged across A–D, heard as an eight-bar loop and rendered to a 16.5-second WAV; the agent is the remaining P2 slice. The visual demo in [`reference/demo`](reference/demo) remains a design reference, not the codebase. The roadmap is in [`docs/PLAN.md`](docs/PLAN.md).
This repository is **P2 complete**: the engine, the wire, the descent, the quarry, a persistent project, recording, manual faces, undo across restarts, a native host, an **8-bar arrangement of four sections**, an **offline WAV render**, and the **choir** — an in-app LLM agent on the same verbs. A found or recorded sound is cut into sixteen faces, laid on a vein, arranged across A–D, heard as an eight-bar loop, rendered to a 16.5-second WAV, and moved by a performer that hears the levels and asks the hand to approve its edits. The visual demo in [`reference/demo`](reference/demo) remains a design reference, not the codebase. The roadmap is in [`docs/PLAN.md`](docs/PLAN.md).
![The quarry](preview-p1-quarry.png)
@@ -20,6 +20,8 @@ This repository is **P2 in progress**: the engine, the wire, the descent, the qu
![P2 · an offline render](preview-p2-render.png)
![P2 · the choir](preview-p2-agent.png)
## What is real
- **Rust engine.** `cpal` output at 48 kHz, five veins, a sixteen-step phrase, a fixed voice pool, per-vein levels and gains, a bar clock. Procedural voices for empty pattern rows; real sample voices for rows with a face. No Web Audio.
@@ -35,6 +37,7 @@ This repository is **P2 in progress**: the engine, the wire, the descent, the qu
- **Native host.** The Tauri app starts the same host inside its own process: engine, session, SQLite project, assets and loopback wire. The built window is self-contained; the browser path uses the standalone daemon. One process owns `127.0.0.1:4317` at a time.
- **Arrangement.** Four 2-bar sections (A–D) make an 8-bar loop. The engine plays the sections in order; the descent draws all eight bars with section seams and letters; the quarry edits the active section, selects, copies and clears. The sections, the active section and their edits are persisted and undoable; the pattern rows and faces stay per section.
- **Render.** The `render` button (or the `render` command) renders the full 8-bar arrangement offline with the same voices, gains, sample faces and master as the live engine, then writes a 32-bit float stereo WAV to `<project>/renders/`. The `rendered` event carries the path and duration; no device is opened for the render.
- **Choir.** An in-app LLM performer on the same command verbs. It receives the project, the sections and the live levels, speaks in short concrete moves, performs safe verbs immediately and turns project edits into proposals the hand approves or rejects. The `choir` room carries the conversation, the proposals and the section/level readout. Without an API key it runs a deterministic mock; set `ZHILA_AGENT_KEY` (or `DEEPSEEK_API_KEY`) and optionally `ZHILA_AGENT_URL`/`ZHILA_AGENT_MODEL` for the live provider.
- **Verification.** Rust unit tests, a live wire test, a material import test, a browser quarry test and the P0 regression smoke, plus `svelte-check` and a production build.
The Tauri 2 app is **built and running with its own engine**: the native window loads the embedded surface and the host starts the engine, session, project and wire in the app process. It is built with Tauri 2.12.5 / CLI 2.12.4 against WebKitGTK 2.54.2, GTK 3.24.52 and dbus 1.16.2. The binary is `app/src-tauri/target/release/zhila-app` (14 MB); installers are a later step.
@@ -110,6 +113,8 @@ npm --prefix ui run faces-smoke # daemon + Vite: drag a seam, trim, save,
npm --prefix ui run arrangement-smoke # daemon + Vite: sections, edit B, 8-bar descent
node scripts/render-smoke.mjs # daemon running: render, parse the WAV
npm --prefix ui run render-ui-smoke # daemon + Vite: render button, wire, screenshot
node scripts/agent-smoke.mjs # daemon running: ask, proposal, approve, tempo
npm --prefix ui run agent-ui-smoke # daemon + Vite: choir ask/approve, screenshot
cd app && ./node_modules/.bin/tauri build --no-bundle # native app with the embedded host
npm --prefix ui run restored-smoke # after restarting the daemon on the saved project
```
@@ -120,7 +125,8 @@ npm --prefix ui run restored-smoke # after restarting the daemon on the saved
- The arrangement is exactly four 2-bar sections; no section length, reorder, insert or per-section mute yet.
- Render is 48 kHz stereo 32-bit float with a 1.5 s tail; no stems, normalisation or per-section export.
- The agent is the remaining P2 slice.
- The choir is in-app only; no MCP server yet. Without a key it is the deterministic mock. The provider path is OpenAI-compatible, one tool round per ask, and the conversation lives in daemon memory (not the project).
- The agent's proposals are applied with `Actor::Agent` and saved; the chat is not persisted across host restarts.
## P1 limits (kept honest)
@@ -135,4 +141,4 @@ npm --prefix ui run restored-smoke # after restarting the daemon on the saved
- No agent/MCP in this phase.
- The Tauri shell now builds and runs; the engine is still the separate daemon (embedding is next).
P2 is the arrangement, the offline WAV render and the in-app agent. P2a (the arrangement) and P2b (the render) are in this tree; P2c follows. The plan is in [`docs/PLAN.md`](docs/PLAN.md); the phase notes are in [`docs/P2.md`](docs/P2.md). The plan is in [`docs/PLAN.md`](docs/PLAN.md); the phase notes are in [`docs/P1.md`](docs/P1.md).
P2 is complete: the arrangement, the offline WAV render and the choir. P2a, P2b and P2c are in this tree. The plan is in [`docs/PLAN.md`](docs/PLAN.md); the phase notes are in [`docs/P2.md`](docs/P2.md). The next phase is P3: the arrangement across scenes, MCP for external agents, and the deeper mix. The plan is in [`docs/PLAN.md`](docs/PLAN.md); the phase notes are in [`docs/P1.md`](docs/P1.md).