P2c: in-app choir agent with proposals and approval
This commit is contained in:
Generated
+819
-12
File diff suppressed because it is too large
Load Diff
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
Sound is terrain. A sampling instrument where material is cut in a quarry, laid as stones in a nave, and played by a hand or an agent through the same verbs.
|
Sound is terrain. A sampling instrument where material is cut in a quarry, laid as stones in a nave, and played by a hand or an agent through the same verbs.
|
||||||
|
|
||||||
This repository is **P2 in progress**: the engine, the wire, the descent, the quarry, a persistent project, recording, manual faces, undo across restarts, a native host, an **8-bar arrangement of four sections**, and an **offline WAV render**. A found or recorded sound is cut into sixteen faces, laid on a vein, arranged across A–D, heard as an eight-bar loop and rendered to a 16.5-second WAV; the agent is the remaining P2 slice. The visual demo in [`reference/demo`](reference/demo) remains a design reference, not the codebase. The roadmap is in [`docs/PLAN.md`](docs/PLAN.md).
|
This repository is **P2 complete**: the engine, the wire, the descent, the quarry, a persistent project, recording, manual faces, undo across restarts, a native host, an **8-bar arrangement of four sections**, an **offline WAV render**, and the **choir** — an in-app LLM agent on the same verbs. A found or recorded sound is cut into sixteen faces, laid on a vein, arranged across A–D, heard as an eight-bar loop, rendered to a 16.5-second WAV, and moved by a performer that hears the levels and asks the hand to approve its edits. The visual demo in [`reference/demo`](reference/demo) remains a design reference, not the codebase. The roadmap is in [`docs/PLAN.md`](docs/PLAN.md).
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
@@ -20,6 +20,8 @@ This repository is **P2 in progress**: the engine, the wire, the descent, the qu
|
|||||||
|
|
||||||

|

|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
## What is real
|
## What is real
|
||||||
|
|
||||||
- **Rust engine.** `cpal` output at 48 kHz, five veins, a sixteen-step phrase, a fixed voice pool, per-vein levels and gains, a bar clock. Procedural voices for empty pattern rows; real sample voices for rows with a face. No Web Audio.
|
- **Rust engine.** `cpal` output at 48 kHz, five veins, a sixteen-step phrase, a fixed voice pool, per-vein levels and gains, a bar clock. Procedural voices for empty pattern rows; real sample voices for rows with a face. No Web Audio.
|
||||||
@@ -35,6 +37,7 @@ This repository is **P2 in progress**: the engine, the wire, the descent, the qu
|
|||||||
- **Native host.** The Tauri app starts the same host inside its own process: engine, session, SQLite project, assets and loopback wire. The built window is self-contained; the browser path uses the standalone daemon. One process owns `127.0.0.1:4317` at a time.
|
- **Native host.** The Tauri app starts the same host inside its own process: engine, session, SQLite project, assets and loopback wire. The built window is self-contained; the browser path uses the standalone daemon. One process owns `127.0.0.1:4317` at a time.
|
||||||
- **Arrangement.** Four 2-bar sections (A–D) make an 8-bar loop. The engine plays the sections in order; the descent draws all eight bars with section seams and letters; the quarry edits the active section, selects, copies and clears. The sections, the active section and their edits are persisted and undoable; the pattern rows and faces stay per section.
|
- **Arrangement.** Four 2-bar sections (A–D) make an 8-bar loop. The engine plays the sections in order; the descent draws all eight bars with section seams and letters; the quarry edits the active section, selects, copies and clears. The sections, the active section and their edits are persisted and undoable; the pattern rows and faces stay per section.
|
||||||
- **Render.** The `render` button (or the `render` command) renders the full 8-bar arrangement offline with the same voices, gains, sample faces and master as the live engine, then writes a 32-bit float stereo WAV to `<project>/renders/`. The `rendered` event carries the path and duration; no device is opened for the render.
|
- **Render.** The `render` button (or the `render` command) renders the full 8-bar arrangement offline with the same voices, gains, sample faces and master as the live engine, then writes a 32-bit float stereo WAV to `<project>/renders/`. The `rendered` event carries the path and duration; no device is opened for the render.
|
||||||
|
- **Choir.** An in-app LLM performer on the same command verbs. It receives the project, the sections and the live levels, speaks in short concrete moves, performs safe verbs immediately and turns project edits into proposals the hand approves or rejects. The `choir` room carries the conversation, the proposals and the section/level readout. Without an API key it runs a deterministic mock; set `ZHILA_AGENT_KEY` (or `DEEPSEEK_API_KEY`) and optionally `ZHILA_AGENT_URL`/`ZHILA_AGENT_MODEL` for the live provider.
|
||||||
- **Verification.** Rust unit tests, a live wire test, a material import test, a browser quarry test and the P0 regression smoke, plus `svelte-check` and a production build.
|
- **Verification.** Rust unit tests, a live wire test, a material import test, a browser quarry test and the P0 regression smoke, plus `svelte-check` and a production build.
|
||||||
|
|
||||||
The Tauri 2 app is **built and running with its own engine**: the native window loads the embedded surface and the host starts the engine, session, project and wire in the app process. It is built with Tauri 2.12.5 / CLI 2.12.4 against WebKitGTK 2.54.2, GTK 3.24.52 and dbus 1.16.2. The binary is `app/src-tauri/target/release/zhila-app` (14 MB); installers are a later step.
|
The Tauri 2 app is **built and running with its own engine**: the native window loads the embedded surface and the host starts the engine, session, project and wire in the app process. It is built with Tauri 2.12.5 / CLI 2.12.4 against WebKitGTK 2.54.2, GTK 3.24.52 and dbus 1.16.2. The binary is `app/src-tauri/target/release/zhila-app` (14 MB); installers are a later step.
|
||||||
@@ -110,6 +113,8 @@ npm --prefix ui run faces-smoke # daemon + Vite: drag a seam, trim, save,
|
|||||||
npm --prefix ui run arrangement-smoke # daemon + Vite: sections, edit B, 8-bar descent
|
npm --prefix ui run arrangement-smoke # daemon + Vite: sections, edit B, 8-bar descent
|
||||||
node scripts/render-smoke.mjs # daemon running: render, parse the WAV
|
node scripts/render-smoke.mjs # daemon running: render, parse the WAV
|
||||||
npm --prefix ui run render-ui-smoke # daemon + Vite: render button, wire, screenshot
|
npm --prefix ui run render-ui-smoke # daemon + Vite: render button, wire, screenshot
|
||||||
|
node scripts/agent-smoke.mjs # daemon running: ask, proposal, approve, tempo
|
||||||
|
npm --prefix ui run agent-ui-smoke # daemon + Vite: choir ask/approve, screenshot
|
||||||
cd app && ./node_modules/.bin/tauri build --no-bundle # native app with the embedded host
|
cd app && ./node_modules/.bin/tauri build --no-bundle # native app with the embedded host
|
||||||
npm --prefix ui run restored-smoke # after restarting the daemon on the saved project
|
npm --prefix ui run restored-smoke # after restarting the daemon on the saved project
|
||||||
```
|
```
|
||||||
@@ -120,7 +125,8 @@ npm --prefix ui run restored-smoke # after restarting the daemon on the saved
|
|||||||
|
|
||||||
- The arrangement is exactly four 2-bar sections; no section length, reorder, insert or per-section mute yet.
|
- The arrangement is exactly four 2-bar sections; no section length, reorder, insert or per-section mute yet.
|
||||||
- Render is 48 kHz stereo 32-bit float with a 1.5 s tail; no stems, normalisation or per-section export.
|
- Render is 48 kHz stereo 32-bit float with a 1.5 s tail; no stems, normalisation or per-section export.
|
||||||
- The agent is the remaining P2 slice.
|
- The choir is in-app only; no MCP server yet. Without a key it is the deterministic mock. The provider path is OpenAI-compatible, one tool round per ask, and the conversation lives in daemon memory (not the project).
|
||||||
|
- The agent's proposals are applied with `Actor::Agent` and saved; the chat is not persisted across host restarts.
|
||||||
|
|
||||||
## P1 limits (kept honest)
|
## P1 limits (kept honest)
|
||||||
|
|
||||||
@@ -135,4 +141,4 @@ npm --prefix ui run restored-smoke # after restarting the daemon on the saved
|
|||||||
- No agent/MCP in this phase.
|
- No agent/MCP in this phase.
|
||||||
- The Tauri shell now builds and runs; the engine is still the separate daemon (embedding is next).
|
- The Tauri shell now builds and runs; the engine is still the separate daemon (embedding is next).
|
||||||
|
|
||||||
P2 is the arrangement, the offline WAV render and the in-app agent. P2a (the arrangement) and P2b (the render) are in this tree; P2c follows. The plan is in [`docs/PLAN.md`](docs/PLAN.md); the phase notes are in [`docs/P2.md`](docs/P2.md). The plan is in [`docs/PLAN.md`](docs/PLAN.md); the phase notes are in [`docs/P1.md`](docs/P1.md).
|
P2 is complete: the arrangement, the offline WAV render and the choir. P2a, P2b and P2c are in this tree. The plan is in [`docs/PLAN.md`](docs/PLAN.md); the phase notes are in [`docs/P2.md`](docs/P2.md). The next phase is P3: the arrangement across scenes, MCP for external agents, and the deeper mix. The plan is in [`docs/PLAN.md`](docs/PLAN.md); the phase notes are in [`docs/P1.md`](docs/P1.md).
|
||||||
|
|||||||
@@ -587,7 +587,11 @@ impl Session {
|
|||||||
Command::SelectSection { index } => self.select_section(index, actor),
|
Command::SelectSection { index } => self.select_section(index, actor),
|
||||||
Command::ClearSection { index } => self.clear_section(index, actor),
|
Command::ClearSection { index } => self.clear_section(index, actor),
|
||||||
Command::CopySection { from, to } => self.copy_section(from, to, actor),
|
Command::CopySection { from, to } => self.copy_section(from, to, actor),
|
||||||
Command::Render => Ok((
|
Command::Render
|
||||||
|
| Command::AgentAsk { .. }
|
||||||
|
| Command::AgentApprove { .. }
|
||||||
|
| Command::AgentReject { .. }
|
||||||
|
| Command::AgentReset => Ok((
|
||||||
CommandResult::State {
|
CommandResult::State {
|
||||||
state: self.snapshot(),
|
state: self.snapshot(),
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -27,3 +27,4 @@ clap.workspace = true
|
|||||||
tracing.workspace = true
|
tracing.workspace = true
|
||||||
tracing-subscriber.workspace = true
|
tracing-subscriber.workspace = true
|
||||||
hound = "3"
|
hound = "3"
|
||||||
|
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
|
||||||
|
|||||||
@@ -0,0 +1,440 @@
|
|||||||
|
//! The choir: an LLM performer on the same verbs as the hand.
|
||||||
|
//!
|
||||||
|
//! The model never touches the session directly. It returns tool calls; safe
|
||||||
|
//! performance verbs execute immediately, project edits become proposals that
|
||||||
|
//! the hand approves. With no key (or ZHILA_AGENT_MOCK=1) a deterministic mock
|
||||||
|
//! keeps the whole path testable.
|
||||||
|
|
||||||
|
use std::collections::HashMap;
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use serde_json::{Value, json};
|
||||||
|
use zhila_protocol::{AgentMessage, AgentRole, Command, MeterFrame, State, VeinId};
|
||||||
|
|
||||||
|
#[derive(Clone)]
|
||||||
|
pub struct AgentConfig {
|
||||||
|
key: Option<String>,
|
||||||
|
base_url: String,
|
||||||
|
model: String,
|
||||||
|
mock: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl AgentConfig {
|
||||||
|
pub fn from_env() -> Self {
|
||||||
|
let key = std::env::var("ZHILA_AGENT_KEY")
|
||||||
|
.ok()
|
||||||
|
.or_else(|| std::env::var("DEEPSEEK_API_KEY").ok())
|
||||||
|
.filter(|value| !value.trim().is_empty());
|
||||||
|
let mock = std::env::var("ZHILA_AGENT_MOCK").is_ok() || key.is_none();
|
||||||
|
Self {
|
||||||
|
key,
|
||||||
|
base_url: std::env::var("ZHILA_AGENT_URL")
|
||||||
|
.unwrap_or_else(|_| "https://api.deepseek.com".into()),
|
||||||
|
model: std::env::var("ZHILA_AGENT_MODEL").unwrap_or_else(|_| "deepseek-chat".into()),
|
||||||
|
mock,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn is_mock(&self) -> bool {
|
||||||
|
self.mock
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone)]
|
||||||
|
pub struct Proposal {
|
||||||
|
pub id: u64,
|
||||||
|
pub description: String,
|
||||||
|
pub commands: Vec<Command>,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct Turn {
|
||||||
|
pub reply: String,
|
||||||
|
pub immediate: Vec<Command>,
|
||||||
|
pub proposal: Option<Proposal>,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct Agent {
|
||||||
|
config: AgentConfig,
|
||||||
|
client: reqwest::Client,
|
||||||
|
messages: Vec<AgentMessage>,
|
||||||
|
pending: HashMap<u64, Proposal>,
|
||||||
|
next_id: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Agent {
|
||||||
|
pub fn new(config: AgentConfig) -> Self {
|
||||||
|
let client = reqwest::Client::builder()
|
||||||
|
.timeout(Duration::from_secs(60))
|
||||||
|
.build()
|
||||||
|
.expect("agent http client");
|
||||||
|
Self {
|
||||||
|
config,
|
||||||
|
client,
|
||||||
|
messages: Vec::new(),
|
||||||
|
pending: HashMap::new(),
|
||||||
|
next_id: 0,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn messages(&self) -> &[AgentMessage] {
|
||||||
|
&self.messages
|
||||||
|
}
|
||||||
|
|
||||||
|
pub async fn ask(&mut self, text: &str, context: &str) -> Turn {
|
||||||
|
self.messages.push(AgentMessage {
|
||||||
|
role: AgentRole::User,
|
||||||
|
text: text.to_string(),
|
||||||
|
});
|
||||||
|
let (reply, calls) = if self.config.mock {
|
||||||
|
mock_turn(text)
|
||||||
|
} else {
|
||||||
|
match self.llm(text, context).await {
|
||||||
|
Ok(result) => result,
|
||||||
|
Err(error) => (
|
||||||
|
format!("The provider is not answering: {error}. The room is still yours."),
|
||||||
|
Vec::new(),
|
||||||
|
),
|
||||||
|
}
|
||||||
|
};
|
||||||
|
self.messages.push(AgentMessage {
|
||||||
|
role: AgentRole::Agent,
|
||||||
|
text: reply.clone(),
|
||||||
|
});
|
||||||
|
let mut immediate = Vec::new();
|
||||||
|
let mut project = Vec::new();
|
||||||
|
for command in calls {
|
||||||
|
if is_safe(&command) {
|
||||||
|
immediate.push(command);
|
||||||
|
} else {
|
||||||
|
project.push(command);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let proposal = if project.is_empty() {
|
||||||
|
None
|
||||||
|
} else {
|
||||||
|
self.next_id += 1;
|
||||||
|
let proposal = Proposal {
|
||||||
|
id: self.next_id,
|
||||||
|
description: reply.clone(),
|
||||||
|
commands: project,
|
||||||
|
};
|
||||||
|
self.pending.insert(proposal.id, proposal.clone());
|
||||||
|
Some(proposal)
|
||||||
|
};
|
||||||
|
Turn {
|
||||||
|
reply,
|
||||||
|
immediate,
|
||||||
|
proposal,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn take(&mut self, id: u64) -> Option<Proposal> {
|
||||||
|
self.pending.remove(&id)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn note(&mut self, text: impl Into<String>) {
|
||||||
|
self.messages.push(AgentMessage {
|
||||||
|
role: AgentRole::Agent,
|
||||||
|
text: text.into(),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn reset(&mut self) {
|
||||||
|
self.messages.clear();
|
||||||
|
self.pending.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn llm(&self, text: &str, context: &str) -> anyhow::Result<(String, Vec<Command>)> {
|
||||||
|
let key = self
|
||||||
|
.config
|
||||||
|
.key
|
||||||
|
.as_ref()
|
||||||
|
.ok_or_else(|| anyhow::anyhow!("no agent key"))?;
|
||||||
|
let mut messages = vec![json!({
|
||||||
|
"role": "system",
|
||||||
|
"content": system_prompt(context),
|
||||||
|
})];
|
||||||
|
for message in self.messages.iter().rev().take(12).rev() {
|
||||||
|
let role = match message.role {
|
||||||
|
AgentRole::User => "user",
|
||||||
|
AgentRole::Agent => "assistant",
|
||||||
|
AgentRole::System => "system",
|
||||||
|
};
|
||||||
|
messages.push(json!({ "role": role, "content": message.text }));
|
||||||
|
}
|
||||||
|
let body = json!({
|
||||||
|
"model": self.config.model,
|
||||||
|
"messages": messages,
|
||||||
|
"tools": tools(),
|
||||||
|
"tool_choice": "auto",
|
||||||
|
"temperature": 0.7,
|
||||||
|
"max_tokens": 500,
|
||||||
|
});
|
||||||
|
let response = self
|
||||||
|
.client
|
||||||
|
.post(format!(
|
||||||
|
"{}/chat/completions",
|
||||||
|
self.config.base_url.trim_end_matches('/')
|
||||||
|
))
|
||||||
|
.bearer_auth(key)
|
||||||
|
.json(&body)
|
||||||
|
.send()
|
||||||
|
.await?;
|
||||||
|
if !response.status().is_success() {
|
||||||
|
let status = response.status();
|
||||||
|
let text = response.text().await.unwrap_or_default();
|
||||||
|
anyhow::bail!("{status}: {text}");
|
||||||
|
}
|
||||||
|
let value: Value = response.json().await?;
|
||||||
|
let message = &value["choices"][0]["message"];
|
||||||
|
let mut content = message["content"].as_str().unwrap_or("").trim().to_string();
|
||||||
|
let mut commands = Vec::new();
|
||||||
|
if let Some(calls) = message["tool_calls"].as_array() {
|
||||||
|
for call in calls {
|
||||||
|
if let Some(command) = command_from_tool(
|
||||||
|
call["function"]["name"].as_str().unwrap_or_default(),
|
||||||
|
call["function"]["arguments"].as_str().unwrap_or("{}"),
|
||||||
|
) {
|
||||||
|
commands.push(command);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if content.is_empty() {
|
||||||
|
content = if commands.is_empty() {
|
||||||
|
"I am listening.".into()
|
||||||
|
} else {
|
||||||
|
"I would move these.".into()
|
||||||
|
};
|
||||||
|
}
|
||||||
|
let _ = text;
|
||||||
|
Ok((content, commands))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn is_safe(command: &Command) -> bool {
|
||||||
|
matches!(
|
||||||
|
command,
|
||||||
|
Command::Play
|
||||||
|
| Command::Stop
|
||||||
|
| Command::Strike { .. }
|
||||||
|
| Command::StrikeFace { .. }
|
||||||
|
| Command::Render
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn mock_turn(text: &str) -> (String, Vec<Command>) {
|
||||||
|
let lower = text.to_lowercase();
|
||||||
|
if lower.contains("tempo") || lower.contains("faster") || lower.contains("быстр") {
|
||||||
|
(
|
||||||
|
"A little more forward. I would take the room to 132 bpm and let the low end breathe."
|
||||||
|
.into(),
|
||||||
|
vec![Command::SetTempo { bpm: 132.0 }],
|
||||||
|
)
|
||||||
|
} else if lower.contains("play") {
|
||||||
|
("Opening the loop.".into(), vec![Command::Play])
|
||||||
|
} else if lower.contains("stop") {
|
||||||
|
("Stopping on the stone.".into(), vec![Command::Stop])
|
||||||
|
} else if lower.contains("render") {
|
||||||
|
(
|
||||||
|
"Rendering the eight bars to disk.".into(),
|
||||||
|
vec![Command::Render],
|
||||||
|
)
|
||||||
|
} else if lower.contains("listen") || lower.contains("hear") || lower.contains("слыш") {
|
||||||
|
(
|
||||||
|
"The low end holds the room; the high seam is thin. I would strike the bass face once and let it ring."
|
||||||
|
.into(),
|
||||||
|
vec![Command::Strike {
|
||||||
|
vein: VeinId::Bass,
|
||||||
|
velocity: 0.82,
|
||||||
|
}],
|
||||||
|
)
|
||||||
|
} else if lower.contains("bass") || lower.contains("бас") {
|
||||||
|
(
|
||||||
|
"The bass wants a little more weight; I would lift it to 0.9.".into(),
|
||||||
|
vec![Command::SetVeinGain {
|
||||||
|
vein: VeinId::Bass,
|
||||||
|
gain: 0.9,
|
||||||
|
}],
|
||||||
|
)
|
||||||
|
} else if lower.contains("section") || lower.contains("секци") {
|
||||||
|
(
|
||||||
|
"Section B is where the room turns; I would open it.".into(),
|
||||||
|
vec![Command::SelectSection { index: 1 }],
|
||||||
|
)
|
||||||
|
} else if lower.contains("render") {
|
||||||
|
("Rendering the arrangement.".into(), vec![Command::Render])
|
||||||
|
} else {
|
||||||
|
(
|
||||||
|
"I am here, listening to the room. Ask me to move something, or press listen.".into(),
|
||||||
|
Vec::new(),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn system_prompt(context: &str) -> String {
|
||||||
|
format!(
|
||||||
|
"You are the choir of ZHILA: a second performer in a sampling instrument where sound is terrain. \
|
||||||
|
You speak in short, concrete sentences and you listen to the current room, not to a generic idea of it. \
|
||||||
|
You have the same verbs as the hand.\n\
|
||||||
|
Project edits (tempo, gains, pattern steps, faces, sections, cuts, undo, redo) are proposals: call the tool, and the hand will approve or reject. \
|
||||||
|
Performance (play, stop, strike, strike_face, render) is immediate. Never invent cores, faces or sections that are not in the context. \
|
||||||
|
Do not ask for a plan; move one or two things at a time. If the user asks to listen, answer with what you hear in the levels and propose a small move.\n\nContext:\n{context}"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn describe(state: &State, meter: &MeterFrame) -> String {
|
||||||
|
let mut text = format!(
|
||||||
|
"project {} · {:.0} bpm · revision {} · {:.0} bars\nsections: ",
|
||||||
|
state.project_name, state.transport.bpm, state.revision, state.transport.loop_bars
|
||||||
|
);
|
||||||
|
for (index, section) in state.arrangement.iter().enumerate() {
|
||||||
|
let notes: usize = section
|
||||||
|
.rows
|
||||||
|
.iter()
|
||||||
|
.map(|row| row.velocities.iter().filter(|value| **value > 0).count())
|
||||||
|
.sum();
|
||||||
|
let faces: usize = section.rows.iter().filter(|row| row.face.is_some()).count();
|
||||||
|
let letter = (b'A' + index as u8) as char;
|
||||||
|
let active = if index == state.active_section as usize {
|
||||||
|
"*"
|
||||||
|
} else {
|
||||||
|
""
|
||||||
|
};
|
||||||
|
text.push_str(&format!("{letter}{active}({notes} notes, {faces} faces) "));
|
||||||
|
}
|
||||||
|
text.push('\n');
|
||||||
|
text.push_str("veins: ");
|
||||||
|
for vein in &state.veins {
|
||||||
|
let level = meter.veins.get(vein.id.index()).copied().unwrap_or(0.0);
|
||||||
|
text.push_str(&format!(
|
||||||
|
"{} gain {:.2} level {:.2}; ",
|
||||||
|
vein.name, vein.gain, level
|
||||||
|
));
|
||||||
|
}
|
||||||
|
text.push_str(&format!(
|
||||||
|
"\nmaster {:.3} · bar {:.2} · playing {}\n",
|
||||||
|
meter.master, meter.bar, state.transport.playing
|
||||||
|
));
|
||||||
|
for core in &state.cores {
|
||||||
|
let top = core
|
||||||
|
.faces
|
||||||
|
.iter()
|
||||||
|
.find(|face| face.rank == 0)
|
||||||
|
.map(|face| face.index + 1)
|
||||||
|
.unwrap_or(0);
|
||||||
|
text.push_str(&format!(
|
||||||
|
"core {} (seal {:?}): {} faces, top f{}, duration {:.2}s\n",
|
||||||
|
core.name,
|
||||||
|
core.seal,
|
||||||
|
core.faces.len(),
|
||||||
|
top,
|
||||||
|
core.duration_sec
|
||||||
|
));
|
||||||
|
}
|
||||||
|
text
|
||||||
|
}
|
||||||
|
|
||||||
|
fn tools() -> Value {
|
||||||
|
json!([
|
||||||
|
{"type":"function","function":{"name":"play","description":"Start the arrangement.","parameters":{"type":"object","properties":{},"additionalProperties":false}}},
|
||||||
|
{"type":"function","function":{"name":"stop","description":"Stop the arrangement.","parameters":{"type":"object","properties":{},"additionalProperties":false}}},
|
||||||
|
{"type":"function","function":{"name":"render","description":"Render the current arrangement to a WAV.","parameters":{"type":"object","properties":{},"additionalProperties":false}}},
|
||||||
|
{"type":"function","function":{"name":"strike","description":"Strike a vein once.","parameters":{"type":"object","properties":{"vein":{"type":"string","enum":["bass","crack","voice","noise","silt"]},"velocity":{"type":"number"}},"required":["vein"]}}},
|
||||||
|
{"type":"function","function":{"name":"strike_face","description":"Audition a face of a core.","parameters":{"type":"object","properties":{"core":{"type":"string"},"face":{"type":"integer"},"velocity":{"type":"number"}},"required":["core","face"]}}},
|
||||||
|
{"type":"function","function":{"name":"set_tempo","description":"Propose a tempo (40–220 bpm).","parameters":{"type":"object","properties":{"bpm":{"type":"number"}},"required":["bpm"]}}},
|
||||||
|
{"type":"function","function":{"name":"set_vein_gain","description":"Propose a vein gain (0–1.2).","parameters":{"type":"object","properties":{"vein":{"type":"string","enum":["bass","crack","voice","noise","silt"]},"gain":{"type":"number"}},"required":["vein","gain"]}}},
|
||||||
|
{"type":"function","function":{"name":"set_pattern_step","description":"Propose a step velocity (0–127) on a pattern row (0–4) of the active section.","parameters":{"type":"object","properties":{"row":{"type":"integer"},"step":{"type":"integer"},"velocity":{"type":"integer"}},"required":["row","step","velocity"]}}},
|
||||||
|
{"type":"function","function":{"name":"set_pattern_face","description":"Propose laying a core face on a row of the active section.","parameters":{"type":"object","properties":{"row":{"type":"integer"},"core":{"type":"string"},"face":{"type":"integer"}},"required":["row","core","face"]}}},
|
||||||
|
{"type":"function","function":{"name":"select_section","description":"Propose selecting a section (0–3).","parameters":{"type":"object","properties":{"index":{"type":"integer"}},"required":["index"]}}},
|
||||||
|
{"type":"function","function":{"name":"copy_section","description":"Propose copying a section.","parameters":{"type":"object","properties":{"from":{"type":"integer"},"to":{"type":"integer"}},"required":["from","to"]}}},
|
||||||
|
{"type":"function","function":{"name":"clear_section","description":"Propose clearing a section.","parameters":{"type":"object","properties":{"index":{"type":"integer"}},"required":["index"]}}},
|
||||||
|
{"type":"function","function":{"name":"set_core_faces","description":"Propose new face boundaries (17 seconds, ascending) for a core.","parameters":{"type":"object","properties":{"core":{"type":"string"},"boundaries":{"type":"array","items":{"type":"number"}}},"required":["core","boundaries"]}}},
|
||||||
|
{"type":"function","function":{"name":"undo","description":"Propose undo.","parameters":{"type":"object","properties":{},"additionalProperties":false}}},
|
||||||
|
{"type":"function","function":{"name":"redo","description":"Propose redo.","parameters":{"type":"object","properties":{},"additionalProperties":false}}}
|
||||||
|
])
|
||||||
|
}
|
||||||
|
|
||||||
|
fn command_from_tool(name: &str, arguments: &str) -> Option<Command> {
|
||||||
|
let args: Value = serde_json::from_str(arguments).unwrap_or(Value::Null);
|
||||||
|
let vein = |value: &Value| serde_json::from_value::<VeinId>(value.clone()).ok();
|
||||||
|
let subject = |value: &Value| serde_json::from_value::<VeinId>(value.clone()).ok();
|
||||||
|
let _ = subject;
|
||||||
|
match name {
|
||||||
|
"play" => Some(Command::Play),
|
||||||
|
"stop" => Some(Command::Stop),
|
||||||
|
"render" => Some(Command::Render),
|
||||||
|
"strike" => Some(Command::Strike {
|
||||||
|
vein: vein(&args["vein"])?,
|
||||||
|
velocity: args["velocity"].as_f64().unwrap_or(0.8) as f32,
|
||||||
|
}),
|
||||||
|
"strike_face" => Some(Command::StrikeFace {
|
||||||
|
core: args["core"].as_str()?.to_string(),
|
||||||
|
face: args["face"].as_u64()? as u8,
|
||||||
|
velocity: args["velocity"].as_f64().unwrap_or(0.8) as f32,
|
||||||
|
}),
|
||||||
|
"set_tempo" => Some(Command::SetTempo {
|
||||||
|
bpm: args["bpm"].as_f64()? as f32,
|
||||||
|
}),
|
||||||
|
"set_vein_gain" => Some(Command::SetVeinGain {
|
||||||
|
vein: vein(&args["vein"])?,
|
||||||
|
gain: args["gain"].as_f64()? as f32,
|
||||||
|
}),
|
||||||
|
"set_pattern_step" => Some(Command::SetPatternStep {
|
||||||
|
row: args["row"].as_u64()? as u8,
|
||||||
|
step: args["step"].as_u64()? as u8,
|
||||||
|
velocity: args["velocity"].as_u64()? as u8,
|
||||||
|
}),
|
||||||
|
"set_pattern_face" => Some(Command::SetPatternFace {
|
||||||
|
row: args["row"].as_u64()? as u8,
|
||||||
|
core: args["core"].as_str()?.to_string(),
|
||||||
|
face: args["face"].as_u64()? as u8,
|
||||||
|
}),
|
||||||
|
"select_section" => Some(Command::SelectSection {
|
||||||
|
index: args["index"].as_u64()? as u8,
|
||||||
|
}),
|
||||||
|
"copy_section" => Some(Command::CopySection {
|
||||||
|
from: args["from"].as_u64()? as u8,
|
||||||
|
to: args["to"].as_u64()? as u8,
|
||||||
|
}),
|
||||||
|
"clear_section" => Some(Command::ClearSection {
|
||||||
|
index: args["index"].as_u64()? as u8,
|
||||||
|
}),
|
||||||
|
"set_core_faces" => Some(Command::SetCoreFaces {
|
||||||
|
core: args["core"].as_str()?.to_string(),
|
||||||
|
boundaries: serde_json::from_value(args["boundaries"].clone()).ok()?,
|
||||||
|
}),
|
||||||
|
"undo" => Some(Command::Undo),
|
||||||
|
"redo" => Some(Command::Redo),
|
||||||
|
_ => None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn mock_proposes_tempo_and_plays() {
|
||||||
|
let (reply, commands) = mock_turn("raise the tempo");
|
||||||
|
assert!(reply.to_lowercase().contains("132"));
|
||||||
|
assert!(matches!(commands.as_slice(), [Command::SetTempo { .. }]));
|
||||||
|
let (_, commands) = mock_turn("play");
|
||||||
|
assert!(matches!(commands.as_slice(), [Command::Play]));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn tool_calls_become_commands() {
|
||||||
|
assert!(matches!(
|
||||||
|
command_from_tool("set_tempo", r#"{"bpm":132}"#),
|
||||||
|
Some(Command::SetTempo { bpm }) if bpm == 132.0
|
||||||
|
));
|
||||||
|
assert!(matches!(
|
||||||
|
command_from_tool("strike", r#"{"vein":"bass","velocity":0.7}"#),
|
||||||
|
Some(Command::Strike {
|
||||||
|
vein: VeinId::Bass,
|
||||||
|
..
|
||||||
|
})
|
||||||
|
));
|
||||||
|
assert!(command_from_tool("nonsense", "{}").is_none());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,8 +1,10 @@
|
|||||||
//! ZHILA local daemon: one WebSocket endpoint, one session, one engine.
|
//! ZHILA local host: one WebSocket endpoint, one session, one engine, one choir.
|
||||||
//!
|
//!
|
||||||
//! Text frames are JSON control messages; meter frames are binary. The daemon
|
//! Text frames are JSON control messages; meter frames are binary. The daemon
|
||||||
//! binds to loopback only and requires a token in the hello.
|
//! binds to loopback only and requires a token in the hello.
|
||||||
|
|
||||||
|
mod agent;
|
||||||
|
|
||||||
use std::path::PathBuf;
|
use std::path::PathBuf;
|
||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
use std::time::{Duration, SystemTime, UNIX_EPOCH};
|
use std::time::{Duration, SystemTime, UNIX_EPOCH};
|
||||||
@@ -21,8 +23,8 @@ use zhila_core::{CoreError, Engine, MaterialSnapshot, Session, default_pattern};
|
|||||||
use zhila_engine::{AudioEngine, NullEngine, RenderRequest, render_arrangement};
|
use zhila_engine::{AudioEngine, NullEngine, RenderRequest, render_arrangement};
|
||||||
use zhila_project::{ProjectSnapshot, ProjectStore};
|
use zhila_project::{ProjectSnapshot, ProjectStore};
|
||||||
use zhila_protocol::{
|
use zhila_protocol::{
|
||||||
Actor, ClientMessage, Command, CommandResult, ErrorCode, Event, PROTOCOL_VERSION, Pattern,
|
Actor, AgentMessage, AgentRole, ClientMessage, Command, CommandResult, ErrorCode, Event,
|
||||||
Seal, ServerMessage, VEIN_COUNT,
|
PROTOCOL_VERSION, Pattern, Seal, ServerMessage, State, VEIN_COUNT,
|
||||||
};
|
};
|
||||||
|
|
||||||
const HELLO_TIMEOUT: Duration = Duration::from_secs(5);
|
const HELLO_TIMEOUT: Duration = Duration::from_secs(5);
|
||||||
@@ -48,6 +50,8 @@ enum Outbound {
|
|||||||
struct Shared {
|
struct Shared {
|
||||||
session: Arc<Mutex<Session>>,
|
session: Arc<Mutex<Session>>,
|
||||||
store: Arc<std::sync::Mutex<ProjectStore>>,
|
store: Arc<std::sync::Mutex<ProjectStore>>,
|
||||||
|
engine: Arc<dyn Engine>,
|
||||||
|
agent: Arc<Mutex<agent::Agent>>,
|
||||||
out: broadcast::Sender<Outbound>,
|
out: broadcast::Sender<Outbound>,
|
||||||
token: Arc<String>,
|
token: Arc<String>,
|
||||||
session_id: Arc<String>,
|
session_id: Arc<String>,
|
||||||
@@ -106,11 +110,20 @@ pub async fn run(config: HostConfig) -> anyhow::Result<()> {
|
|||||||
};
|
};
|
||||||
let session = Arc::new(Mutex::new(session));
|
let session = Arc::new(Mutex::new(session));
|
||||||
let store = Arc::new(std::sync::Mutex::new(store));
|
let store = Arc::new(std::sync::Mutex::new(store));
|
||||||
|
let agent_config = agent::AgentConfig::from_env();
|
||||||
|
let agent_mode = if agent_config.is_mock() {
|
||||||
|
"mock"
|
||||||
|
} else {
|
||||||
|
"live"
|
||||||
|
};
|
||||||
|
let agent = Arc::new(Mutex::new(agent::Agent::new(agent_config)));
|
||||||
let (out, _) = broadcast::channel::<Outbound>(128);
|
let (out, _) = broadcast::channel::<Outbound>(128);
|
||||||
let session_id = format!("zhila-{:x}", now_ms());
|
let session_id = format!("zhila-{:x}", now_ms());
|
||||||
let shared = Shared {
|
let shared = Shared {
|
||||||
session,
|
session,
|
||||||
store,
|
store,
|
||||||
|
engine: control.clone(),
|
||||||
|
agent,
|
||||||
out: out.clone(),
|
out: out.clone(),
|
||||||
token: Arc::new(token.clone()),
|
token: Arc::new(token.clone()),
|
||||||
session_id: Arc::new(session_id.clone()),
|
session_id: Arc::new(session_id.clone()),
|
||||||
@@ -139,6 +152,7 @@ pub async fn run(config: HostConfig) -> anyhow::Result<()> {
|
|||||||
&session_id,
|
&session_id,
|
||||||
&project_name,
|
&project_name,
|
||||||
&project_path,
|
&project_path,
|
||||||
|
agent_mode,
|
||||||
);
|
);
|
||||||
if token == "dev" {
|
if token == "dev" {
|
||||||
warn!("using the development token; pass --token or ZHILA_TOKEN to change it");
|
warn!("using the development token; pass --token or ZHILA_TOKEN to change it");
|
||||||
@@ -173,11 +187,13 @@ fn print_banner(
|
|||||||
session_id: &str,
|
session_id: &str,
|
||||||
project_name: &str,
|
project_name: &str,
|
||||||
project_path: &std::path::Path,
|
project_path: &std::path::Path,
|
||||||
|
agent_mode: &str,
|
||||||
) {
|
) {
|
||||||
println!();
|
println!();
|
||||||
println!(" ZHILA engine");
|
println!(" ZHILA engine");
|
||||||
println!(" ws://127.0.0.1:{port} · token {token}");
|
println!(" ws://127.0.0.1:{port} · token {token}");
|
||||||
println!(" project {project_name} · {}", project_path.display());
|
println!(" project {project_name} · {}", project_path.display());
|
||||||
|
println!(" choir {agent_mode}");
|
||||||
if info.running {
|
if info.running {
|
||||||
let buffer = if info.buffer_frames == 0 {
|
let buffer = if info.buffer_frames == 0 {
|
||||||
"default".to_string()
|
"default".to_string()
|
||||||
@@ -249,6 +265,211 @@ fn sanitize_name(name: &str) -> String {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async fn handle_agent(
|
||||||
|
shared: &Shared,
|
||||||
|
direct_tx: &mpsc::UnboundedSender<Outbound>,
|
||||||
|
id: u64,
|
||||||
|
command: Command,
|
||||||
|
) {
|
||||||
|
match command {
|
||||||
|
Command::AgentAsk { text } => {
|
||||||
|
let (context, revision) = {
|
||||||
|
let session = shared.session.lock().await;
|
||||||
|
let state = session.snapshot();
|
||||||
|
(
|
||||||
|
agent::describe(&state, &shared.engine.meter()),
|
||||||
|
state.revision,
|
||||||
|
)
|
||||||
|
};
|
||||||
|
broadcast_event(
|
||||||
|
shared,
|
||||||
|
revision,
|
||||||
|
Event::AgentMessage {
|
||||||
|
message: AgentMessage {
|
||||||
|
role: AgentRole::User,
|
||||||
|
text: text.clone(),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
);
|
||||||
|
let turn = {
|
||||||
|
let mut agent = shared.agent.lock().await;
|
||||||
|
agent.ask(&text, &context).await
|
||||||
|
};
|
||||||
|
for immediate in &turn.immediate {
|
||||||
|
apply_agent_immediate(shared, immediate).await;
|
||||||
|
}
|
||||||
|
let revision = shared.session.lock().await.revision();
|
||||||
|
broadcast_event(
|
||||||
|
shared,
|
||||||
|
revision,
|
||||||
|
Event::AgentMessage {
|
||||||
|
message: AgentMessage {
|
||||||
|
role: AgentRole::Agent,
|
||||||
|
text: turn.reply.clone(),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
);
|
||||||
|
if let Some(proposal) = turn.proposal {
|
||||||
|
broadcast_event(
|
||||||
|
shared,
|
||||||
|
revision,
|
||||||
|
Event::AgentProposal {
|
||||||
|
id: proposal.id,
|
||||||
|
description: proposal.description,
|
||||||
|
commands: proposal.commands,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
send_agent_result(shared, direct_tx, id).await;
|
||||||
|
}
|
||||||
|
Command::AgentApprove { id: proposal_id } => {
|
||||||
|
let proposal = {
|
||||||
|
let mut agent = shared.agent.lock().await;
|
||||||
|
agent.take(proposal_id)
|
||||||
|
};
|
||||||
|
if let Some(proposal) = proposal {
|
||||||
|
for command in &proposal.commands {
|
||||||
|
let outcome = {
|
||||||
|
let mut session = shared.session.lock().await;
|
||||||
|
session.apply(command.clone(), Actor::Agent)
|
||||||
|
};
|
||||||
|
if let Ok((result, events)) = outcome {
|
||||||
|
let revision = match result {
|
||||||
|
CommandResult::State { state } | CommandResult::Snapshot { state } => {
|
||||||
|
state.revision
|
||||||
|
}
|
||||||
|
};
|
||||||
|
for event in events {
|
||||||
|
broadcast_event(shared, revision, event);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
save_now(shared).await;
|
||||||
|
let note = "Applied. The room has moved.";
|
||||||
|
{
|
||||||
|
let mut agent = shared.agent.lock().await;
|
||||||
|
agent.note(note);
|
||||||
|
}
|
||||||
|
let revision = shared.session.lock().await.revision();
|
||||||
|
broadcast_event(
|
||||||
|
shared,
|
||||||
|
revision,
|
||||||
|
Event::AgentMessage {
|
||||||
|
message: AgentMessage {
|
||||||
|
role: AgentRole::Agent,
|
||||||
|
text: note.into(),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
);
|
||||||
|
broadcast_event(
|
||||||
|
shared,
|
||||||
|
revision,
|
||||||
|
Event::AgentResolved {
|
||||||
|
id: proposal_id,
|
||||||
|
applied: true,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
send_agent_result(shared, direct_tx, id).await;
|
||||||
|
}
|
||||||
|
Command::AgentReject { id: proposal_id } => {
|
||||||
|
let note = "Rejected. I will leave the room as it is.";
|
||||||
|
{
|
||||||
|
let mut agent = shared.agent.lock().await;
|
||||||
|
agent.take(proposal_id);
|
||||||
|
agent.note(note);
|
||||||
|
}
|
||||||
|
let revision = shared.session.lock().await.revision();
|
||||||
|
broadcast_event(
|
||||||
|
shared,
|
||||||
|
revision,
|
||||||
|
Event::AgentMessage {
|
||||||
|
message: AgentMessage {
|
||||||
|
role: AgentRole::Agent,
|
||||||
|
text: note.into(),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
);
|
||||||
|
broadcast_event(
|
||||||
|
shared,
|
||||||
|
revision,
|
||||||
|
Event::AgentResolved {
|
||||||
|
id: proposal_id,
|
||||||
|
applied: false,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
send_agent_result(shared, direct_tx, id).await;
|
||||||
|
}
|
||||||
|
Command::AgentReset => {
|
||||||
|
{
|
||||||
|
let mut agent = shared.agent.lock().await;
|
||||||
|
agent.reset();
|
||||||
|
}
|
||||||
|
send_agent_result(shared, direct_tx, id).await;
|
||||||
|
}
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn apply_agent_immediate(shared: &Shared, command: &Command) {
|
||||||
|
if matches!(command, Command::Render) {
|
||||||
|
let input = render_input(shared).await;
|
||||||
|
let root = shared
|
||||||
|
.store
|
||||||
|
.lock()
|
||||||
|
.expect("project store lock poisoned")
|
||||||
|
.root()
|
||||||
|
.to_path_buf();
|
||||||
|
let rendered = tokio::task::spawn_blocking(move || render_to_wav(input, &root)).await;
|
||||||
|
if let Ok(Ok((path, duration_sec))) = rendered {
|
||||||
|
let revision = shared.session.lock().await.revision();
|
||||||
|
broadcast_event(shared, revision, Event::Rendered { path, duration_sec });
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let outcome = {
|
||||||
|
let mut session = shared.session.lock().await;
|
||||||
|
session.apply(command.clone(), Actor::Agent)
|
||||||
|
};
|
||||||
|
if let Ok((result, events)) = outcome {
|
||||||
|
let revision = match result {
|
||||||
|
CommandResult::State { state } | CommandResult::Snapshot { state } => state.revision,
|
||||||
|
};
|
||||||
|
for event in events {
|
||||||
|
broadcast_event(shared, revision, event);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn render_input(shared: &Shared) -> RenderInput {
|
||||||
|
let session = shared.session.lock().await;
|
||||||
|
RenderInput {
|
||||||
|
sections: session.sections().to_vec(),
|
||||||
|
material: session.material_snapshot(),
|
||||||
|
gains: session.gains(),
|
||||||
|
bpm: session.bpm(),
|
||||||
|
project: session.project_name().to_string(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn broadcast_event(shared: &Shared, revision: u64, event: Event) {
|
||||||
|
let message = ServerMessage::Event { revision, event };
|
||||||
|
let _ = shared.out.send(Outbound::Text(
|
||||||
|
serde_json::to_string(&message).expect("event serializes"),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn send_agent_result(shared: &Shared, direct_tx: &mpsc::UnboundedSender<Outbound>, id: u64) {
|
||||||
|
let state: State = shared.session.lock().await.snapshot();
|
||||||
|
let message = ServerMessage::Result {
|
||||||
|
id,
|
||||||
|
result: CommandResult::State { state },
|
||||||
|
};
|
||||||
|
let _ = direct_tx.send(Outbound::Text(
|
||||||
|
serde_json::to_string(&message).expect("agent result serializes"),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
async fn save_now(shared: &Shared) {
|
async fn save_now(shared: &Shared) {
|
||||||
let session = shared.session.lock().await;
|
let session = shared.session.lock().await;
|
||||||
let state = session.snapshot();
|
let state = session.snapshot();
|
||||||
@@ -392,6 +613,17 @@ async fn handle(stream: TcpStream, shared: Shared) -> anyhow::Result<()> {
|
|||||||
),
|
),
|
||||||
)
|
)
|
||||||
.await?;
|
.await?;
|
||||||
|
let agent_messages = shared.agent.lock().await.messages().to_vec();
|
||||||
|
send_out(
|
||||||
|
&mut sink,
|
||||||
|
Outbound::Text(
|
||||||
|
serde_json::to_string(&ServerMessage::AgentHistory {
|
||||||
|
messages: agent_messages,
|
||||||
|
})
|
||||||
|
.expect("agent history serializes"),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
|
||||||
let (direct_tx, mut direct_rx) = mpsc::unbounded_channel::<Outbound>();
|
let (direct_tx, mut direct_rx) = mpsc::unbounded_channel::<Outbound>();
|
||||||
let mut events = shared.out.subscribe();
|
let mut events = shared.out.subscribe();
|
||||||
@@ -588,6 +820,16 @@ async fn handle(stream: TcpStream, shared: Shared) -> anyhow::Result<()> {
|
|||||||
}
|
}
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
if matches!(
|
||||||
|
&command,
|
||||||
|
Command::AgentAsk { .. }
|
||||||
|
| Command::AgentApprove { .. }
|
||||||
|
| Command::AgentReject { .. }
|
||||||
|
| Command::AgentReset
|
||||||
|
) {
|
||||||
|
handle_agent(&shared, &direct_tx, id, command).await;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
let should_save = !matches!(
|
let should_save = !matches!(
|
||||||
&command,
|
&command,
|
||||||
Command::Play
|
Command::Play
|
||||||
|
|||||||
@@ -75,6 +75,7 @@ impl VeinId {
|
|||||||
#[serde(rename_all = "snake_case")]
|
#[serde(rename_all = "snake_case")]
|
||||||
pub enum Actor {
|
pub enum Actor {
|
||||||
Hand,
|
Hand,
|
||||||
|
Agent,
|
||||||
Api,
|
Api,
|
||||||
Mcp,
|
Mcp,
|
||||||
System,
|
System,
|
||||||
@@ -165,6 +166,9 @@ pub enum ServerMessage {
|
|||||||
id: u64,
|
id: u64,
|
||||||
core: CoreSummary,
|
core: CoreSummary,
|
||||||
},
|
},
|
||||||
|
AgentHistory {
|
||||||
|
messages: Vec<AgentMessage>,
|
||||||
|
},
|
||||||
Error {
|
Error {
|
||||||
id: Option<u64>,
|
id: Option<u64>,
|
||||||
code: ErrorCode,
|
code: ErrorCode,
|
||||||
@@ -219,6 +223,16 @@ pub enum Command {
|
|||||||
to: u8,
|
to: u8,
|
||||||
},
|
},
|
||||||
Render,
|
Render,
|
||||||
|
AgentAsk {
|
||||||
|
text: String,
|
||||||
|
},
|
||||||
|
AgentApprove {
|
||||||
|
id: u64,
|
||||||
|
},
|
||||||
|
AgentReject {
|
||||||
|
id: u64,
|
||||||
|
},
|
||||||
|
AgentReset,
|
||||||
Undo,
|
Undo,
|
||||||
Redo,
|
Redo,
|
||||||
/// Journal-only: a core was imported through the binary path.
|
/// Journal-only: a core was imported through the binary path.
|
||||||
@@ -282,6 +296,32 @@ pub enum Event {
|
|||||||
path: String,
|
path: String,
|
||||||
duration_sec: f32,
|
duration_sec: f32,
|
||||||
},
|
},
|
||||||
|
AgentMessage {
|
||||||
|
message: AgentMessage,
|
||||||
|
},
|
||||||
|
AgentProposal {
|
||||||
|
id: u64,
|
||||||
|
description: String,
|
||||||
|
commands: Vec<Command>,
|
||||||
|
},
|
||||||
|
AgentResolved {
|
||||||
|
id: u64,
|
||||||
|
applied: bool,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "snake_case")]
|
||||||
|
pub enum AgentRole {
|
||||||
|
User,
|
||||||
|
Agent,
|
||||||
|
System,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||||
|
pub struct AgentMessage {
|
||||||
|
pub role: AgentRole,
|
||||||
|
pub text: String,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
|||||||
+20
-2
@@ -28,6 +28,24 @@ Verification: `cargo test -p zhila-engine` (non-silent 16.5 s buffer, correct fr
|
|||||||
|
|
||||||
Limits: WAV only, stereo float, fixed 48 kHz and 1.5 s tail; no stems, normalisation or section ranges.
|
Limits: WAV only, stereo float, fixed 48 kHz and 1.5 s tail; no stems, normalisation or section ranges.
|
||||||
|
|
||||||
## P2c · agent (next)
|
## P2c · the choir
|
||||||
|
|
||||||
An in-app LLM agent on the same verbs, with state/sound context and approval for project edits.
|
`crates/zhila-daemon/src/agent.rs` is the performer. It receives the project state and live meter as text, a system prompt with the ZHILA rules, and an OpenAI-compatible tool list. Safe verbs (play, stop, strike, strike_face, render) execute immediately; project verbs (tempo, gains, pattern, faces, sections, cuts, undo/redo) are collected into one proposal stored with an id. The hand approves or rejects; approval applies every command with `Actor::Agent`, saves the project, and emits the normal events plus an agent confirmation.
|
||||||
|
|
||||||
|
Wire: `agent_ask { text }`, `agent_approve { id }`, `agent_reject { id }`, `agent_reset`; events `agent_message { role, text }`, `agent_proposal { id, description, commands }`, `agent_resolved { id, applied }`; `agent_history { messages }` on hello. The host keeps the conversation and pending proposals in memory.
|
||||||
|
|
||||||
|
Config: `ZHILA_AGENT_KEY` or `DEEPSEEK_API_KEY` enables the live provider (`ZHILA_AGENT_URL`, default `https://api.deepseek.com`; `ZHILA_AGENT_MODEL`, default `deepseek-chat`). No key, or `ZHILA_AGENT_MOCK=1`, uses the deterministic mock: "tempo" proposes 132 bpm, "listen" strikes the bass, "play"/"stop"/"render" perform, "bass" proposes a gain.
|
||||||
|
|
||||||
|
Surface: the `choir` room (top rail) with the conversation, proposals (commands labelled, approve/reject), a listen button, an input, and a readout of sections, levels, bpm and cores. The agent's voice is Cormorant; the commands are mono.
|
||||||
|
|
||||||
|
Verification: `node scripts/agent-smoke.mjs` (mock: ask → proposal → approve → tempo 132, resolved, history); `npm --prefix ui run agent-ui-smoke` (choir ask/approve, readout 132, screenshot `preview-p2-agent.png`); `cargo test -p zhila-daemon` (mock turn, tool mapping).
|
||||||
|
|
||||||
|
Limits: in-app only, no MCP; one tool round per ask; the conversation is daemon memory; the mock is the default without a key; the host restart clears the chat (the project keeps the approved edits).
|
||||||
|
|
||||||
|
## P2 acceptance
|
||||||
|
|
||||||
|
- Arrangement: four 2-bar sections, 8-bar playback, per-section editing, persisted and undoable.
|
||||||
|
- Render: a valid 48 kHz stereo float WAV of the arrangement (16.5 s, non-silent, peak 0.442).
|
||||||
|
- Agent: ask → proposal → approve → a project edit applied with `Actor::Agent`, on the same wire.
|
||||||
|
- The native Tauri app hosts all of it in-process; the browser path uses the standalone daemon.
|
||||||
|
- `./scripts/check.sh`, the persistence smoke and the three P2 smokes pass.
|
||||||
|
|||||||
+1
-1
@@ -1,7 +1,7 @@
|
|||||||
# ZHILA — план проекта
|
# ZHILA — план проекта
|
||||||
|
|
||||||
Черновик v1.1 · 1 ноября 2026
|
Черновик v1.1 · 1 ноября 2026
|
||||||
Статус: **P2 в работе** (P0, P1a–P1f, P2a, P2b; см. `P0.md`, `P1.md`, `P2.md`). Демо `reference/demo/` — референс, не кодовая база. Осталось: агент.
|
Статус: **P2 полностью выполнен** (P0, P1a–P1f, P2a–P2c; см. `P0.md`, `P1.md`, `P2.md`). Демо `reference/demo/` — референс, не кодовая база. Следующая фаза — P3.
|
||||||
|
|
||||||
**Зафиксировано 1 ноября:** descent — основной вид; камень = паттерн; климат — на жилу (концепт погоды остаётся); платформа v1 — Linux; UI — Svelte 5. Журнал — §9.
|
**Зафиксировано 1 ноября:** descent — основной вид; камень = паттерн; климат — на жилу (концепт погоды остаётся); платформа v1 — Linux; UI — Svelte 5. Журнал — §9.
|
||||||
|
|
||||||
|
|||||||
@@ -176,3 +176,7 @@ The Tauri app starts the same host in-process, so the wire contract is unchanged
|
|||||||
## P2b additions
|
## P2b additions
|
||||||
|
|
||||||
`Command::render` (no arguments). The host renders the current arrangement to a WAV in the project, answers with the unchanged state, and broadcasts `rendered { path, duration_sec }`. The render is not journaled and does not push undo. The file is 48 kHz stereo 32-bit float.
|
`Command::render` (no arguments). The host renders the current arrangement to a WAV in the project, answers with the unchanged state, and broadcasts `rendered { path, duration_sec }`. The render is not journaled and does not push undo. The file is 48 kHz stereo 32-bit float.
|
||||||
|
|
||||||
|
## P2c additions
|
||||||
|
|
||||||
|
Agent commands: `agent_ask { text }`, `agent_approve { id }`, `agent_reject { id }`, `agent_reset`. Events: `agent_message { message: { role, text } }` (roles `user`/`agent`/`system`), `agent_proposal { id, description, commands }`, `agent_resolved { id, applied }`. On hello the host sends `agent_history { messages }`. Agent-approved project edits journal as `Actor::agent`; safe performance verbs execute immediately. The provider stays server-side; the key is never in the surface.
|
||||||
|
|||||||
Binary file not shown.
|
After Width: | Height: | Size: 246 KiB |
@@ -0,0 +1,53 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
// Mock choir: ask, get a proposal, approve it, see the tempo move.
|
||||||
|
const ws = new WebSocket('ws://127.0.0.1:4317');
|
||||||
|
const pending = new Map();
|
||||||
|
const waiters = [];
|
||||||
|
let proposal = null;
|
||||||
|
let resolved = null;
|
||||||
|
let history = null;
|
||||||
|
const waitFor = (predicate) => new Promise((resolve, reject) => {
|
||||||
|
const timer = setTimeout(() => reject(new Error('timeout')), 20000);
|
||||||
|
waiters.push({ predicate, resolve: (value) => { clearTimeout(timer); resolve(value); } });
|
||||||
|
const existing = messages.find(predicate);
|
||||||
|
if (existing) resolve(existing);
|
||||||
|
});
|
||||||
|
const messages = [];
|
||||||
|
ws.onopen = () => ws.send(JSON.stringify({ type: 'hello', protocol: 1, token: 'dev', client: 'agent-smoke' }));
|
||||||
|
ws.onmessage = (message) => {
|
||||||
|
if (typeof message.data !== 'string') return;
|
||||||
|
const parsed = JSON.parse(message.data);
|
||||||
|
messages.push(parsed);
|
||||||
|
if (parsed.type === 'agent_history') history = parsed.messages;
|
||||||
|
if (parsed.type === 'event' && parsed.event?.kind === 'agent_proposal') proposal = parsed.event;
|
||||||
|
if (parsed.type === 'event' && parsed.event?.kind === 'agent_resolved') resolved = parsed.event;
|
||||||
|
if (parsed.type === 'result') pending.set(parsed.id, parsed.result);
|
||||||
|
for (const waiter of [...waiters]) {
|
||||||
|
if (waiter.predicate(parsed)) { waiters.splice(waiters.indexOf(waiter), 1); waiter.resolve(parsed); }
|
||||||
|
}
|
||||||
|
};
|
||||||
|
const send = (id, command) => ws.send(JSON.stringify({ type: 'command', id, command }));
|
||||||
|
await waitFor((message) => message.type === 'snapshot');
|
||||||
|
send(1, { type: 'agent_ask', text: 'raise the tempo a little' });
|
||||||
|
await waitFor((message) => message.type === 'event' && message.event?.kind === 'agent_proposal');
|
||||||
|
if (!proposal) throw new Error('no proposal');
|
||||||
|
if (!proposal.commands.some((command) => command.type === 'set_tempo')) throw new Error('expected a tempo proposal');
|
||||||
|
const before = pending.get(1)?.state?.transport?.bpm;
|
||||||
|
send(2, { type: 'agent_approve', id: proposal.id });
|
||||||
|
await waitFor((message) => message.type === 'event' && message.event?.kind === 'agent_resolved');
|
||||||
|
const state = pending.get(2)?.state;
|
||||||
|
if (!resolved?.applied) throw new Error('proposal was not applied');
|
||||||
|
if (state?.transport?.bpm !== 132) throw new Error(`tempo did not move: ${state?.transport?.bpm}`);
|
||||||
|
const userLines = messages.filter((message) => message.type === 'event' && message.event?.kind === 'agent_message' && message.event.message.role === 'user').length;
|
||||||
|
const agentLines = messages.filter((message) => message.type === 'event' && message.event?.kind === 'agent_message' && message.event.message.role === 'agent').length;
|
||||||
|
console.log(JSON.stringify({
|
||||||
|
history: history?.length ?? 0,
|
||||||
|
proposal: { id: proposal.id, commands: proposal.commands, description: proposal.description },
|
||||||
|
resolved,
|
||||||
|
bpm: state.transport.bpm,
|
||||||
|
userLines,
|
||||||
|
agentLines,
|
||||||
|
journal: state.journal_len
|
||||||
|
}, null, 2));
|
||||||
|
ws.close();
|
||||||
|
console.log('agent smoke passed');
|
||||||
@@ -15,6 +15,7 @@
|
|||||||
"faces-smoke": "node scripts/faces-smoke.mjs",
|
"faces-smoke": "node scripts/faces-smoke.mjs",
|
||||||
"arrangement-smoke": "node scripts/arrangement-smoke.mjs",
|
"arrangement-smoke": "node scripts/arrangement-smoke.mjs",
|
||||||
"render-ui-smoke": "node scripts/render-ui-smoke.mjs",
|
"render-ui-smoke": "node scripts/render-ui-smoke.mjs",
|
||||||
|
"agent-ui-smoke": "node scripts/agent-ui-smoke.mjs",
|
||||||
"restored-smoke": "node scripts/restored-smoke.mjs"
|
"restored-smoke": "node scripts/restored-smoke.mjs"
|
||||||
},
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
|
|||||||
@@ -0,0 +1,40 @@
|
|||||||
|
import { chromium } from 'playwright-core';
|
||||||
|
const chrome = process.env.CHROME_PATH ?? '/home/emil/.cache/ms-playwright/chromium-1248/chrome-linux64/chrome';
|
||||||
|
const browser = await chromium.launch({ executablePath: chrome, args: ['--no-sandbox'] });
|
||||||
|
const page = await browser.newPage({ viewport: { width: 1600, height: 1000 }, deviceScaleFactor: 1 });
|
||||||
|
const errors = [];
|
||||||
|
page.on('pageerror', (error) => errors.push(error.message));
|
||||||
|
page.on('console', (message) => { if (message.type() === 'error') errors.push(message.text()); });
|
||||||
|
await page.goto('http://127.0.0.1:5173/', { waitUntil: 'load' });
|
||||||
|
await page.waitForTimeout(2200);
|
||||||
|
await page.locator('.room', { hasText: 'choir' }).click();
|
||||||
|
await page.waitForTimeout(400);
|
||||||
|
await page.locator('.choir-form input').fill('raise the tempo a little');
|
||||||
|
await page.locator('.choir-send').click();
|
||||||
|
await page.waitForSelector('.choir-proposal', { timeout: 20000 });
|
||||||
|
const proposal = (await page.locator('.choir-proposal').last().textContent())?.replace(/\s+/g, ' ').trim();
|
||||||
|
await page.locator('.choir-proposal-actions button').first().click();
|
||||||
|
await page.waitForFunction(
|
||||||
|
() => document.querySelector('.choir-readout')?.textContent?.includes('132'),
|
||||||
|
null,
|
||||||
|
{ timeout: 20000 }
|
||||||
|
);
|
||||||
|
await page.waitForTimeout(600);
|
||||||
|
await page.screenshot({ path: '../preview-p2-agent.png' });
|
||||||
|
const report = await page.evaluate(() => ({
|
||||||
|
readout: document.querySelector('.choir-readout')?.textContent?.replace(/\s+/g, ' ').trim(),
|
||||||
|
messages: [...document.querySelectorAll('.choir-message')].map((el) => el.textContent?.replace(/\s+/g, ' ').trim()),
|
||||||
|
proposals: document.querySelectorAll('.choir-proposal').length,
|
||||||
|
wire: [...document.querySelectorAll('.wire-label')].map((el) => el.textContent).slice(-8),
|
||||||
|
choir: document.querySelector('.choir-head h1')?.textContent
|
||||||
|
}));
|
||||||
|
console.log(JSON.stringify({ proposal, report, errors }, null, 2));
|
||||||
|
const failures = [];
|
||||||
|
if (!proposal?.includes('tempo')) failures.push('no tempo proposal');
|
||||||
|
if (!report.readout?.includes('132')) failures.push('tempo did not move in the readout');
|
||||||
|
if (!report.messages.some((line) => line?.includes('Applied'))) failures.push('no applied message');
|
||||||
|
if (report.proposals !== 0) failures.push('proposal was not resolved');
|
||||||
|
if (errors.length) failures.push('console errors');
|
||||||
|
await browser.close();
|
||||||
|
if (failures.length) { console.error('AGENT UI SMOKE FAILED: ' + failures.join('; ')); process.exit(1); }
|
||||||
|
console.log('agent UI smoke passed');
|
||||||
+58
-3
@@ -14,6 +14,8 @@
|
|||||||
CoreSummary,
|
CoreSummary,
|
||||||
Event,
|
Event,
|
||||||
HelloMessage,
|
HelloMessage,
|
||||||
|
AgentMessage,
|
||||||
|
AgentProposal,
|
||||||
MeterFrame,
|
MeterFrame,
|
||||||
Seal,
|
Seal,
|
||||||
State,
|
State,
|
||||||
@@ -23,6 +25,7 @@
|
|||||||
WireEntry
|
WireEntry
|
||||||
} from './lib/types';
|
} from './lib/types';
|
||||||
import { VEINS } from './lib/types';
|
import { VEINS } from './lib/types';
|
||||||
|
import Choir from './components/Choir.svelte';
|
||||||
|
|
||||||
const WIRE_URL = import.meta.env.VITE_ZHILA_WS ?? 'ws://127.0.0.1:4317';
|
const WIRE_URL = import.meta.env.VITE_ZHILA_WS ?? 'ws://127.0.0.1:4317';
|
||||||
const TOKEN = import.meta.env.VITE_ZHILA_TOKEN ?? 'dev';
|
const TOKEN = import.meta.env.VITE_ZHILA_TOKEN ?? 'dev';
|
||||||
@@ -33,13 +36,16 @@
|
|||||||
let selected: VeinId = $state('bass');
|
let selected: VeinId = $state('bass');
|
||||||
let entries: WireEntry[] = $state([]);
|
let entries: WireEntry[] = $state([]);
|
||||||
let hello: HelloMessage | null = $state(null);
|
let hello: HelloMessage | null = $state(null);
|
||||||
let view: 'nave' | 'quarry' = $state('nave');
|
let view: 'nave' | 'quarry' | 'choir' = $state('nave');
|
||||||
let selectedCoreId: string | null = $state(null);
|
let selectedCoreId: string | null = $state(null);
|
||||||
let selectedFace: number = $state(-1);
|
let selectedFace: number = $state(-1);
|
||||||
let selectedRow: number = $state(0);
|
let selectedRow: number = $state(0);
|
||||||
let importing: boolean = $state(false);
|
let importing: boolean = $state(false);
|
||||||
let rendering: boolean = $state(false);
|
let rendering: boolean = $state(false);
|
||||||
let lastRender: string | null = $state(null);
|
let lastRender: string | null = $state(null);
|
||||||
|
let agentMessages: AgentMessage[] = $state([]);
|
||||||
|
let agentProposals: AgentProposal[] = $state([]);
|
||||||
|
let agentBusy: boolean = $state(false);
|
||||||
let bridge: Bridge | null = null;
|
let bridge: Bridge | null = null;
|
||||||
let entryId = 0;
|
let entryId = 0;
|
||||||
let snapshotLogged = false;
|
let snapshotLogged = false;
|
||||||
@@ -114,6 +120,15 @@
|
|||||||
} else if (event.kind === 'rendered') {
|
} else if (event.kind === 'rendered') {
|
||||||
lastRender = event.path;
|
lastRender = event.path;
|
||||||
addEntry('recv', 'render', `${event.duration_sec.toFixed(1)} s · ${event.path}`);
|
addEntry('recv', 'render', `${event.duration_sec.toFixed(1)} s · ${event.path}`);
|
||||||
|
} else if (event.kind === 'agent_message') {
|
||||||
|
agentMessages = [...agentMessages.slice(-60), event.message];
|
||||||
|
} else if (event.kind === 'agent_proposal') {
|
||||||
|
agentProposals = [
|
||||||
|
...agentProposals,
|
||||||
|
{ id: event.id, description: event.description, commands: event.commands }
|
||||||
|
];
|
||||||
|
} else if (event.kind === 'agent_resolved') {
|
||||||
|
agentProposals = agentProposals.filter((proposal) => proposal.id !== event.id);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -221,6 +236,26 @@
|
|||||||
void run({ type: 'copy_section', from, to });
|
void run({ type: 'copy_section', from, to });
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function askAgent(text: string) {
|
||||||
|
if (!bridge || agentBusy) return;
|
||||||
|
agentBusy = true;
|
||||||
|
try {
|
||||||
|
await run({ type: 'agent_ask', text });
|
||||||
|
} finally {
|
||||||
|
agentBusy = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function approveAgent(id: number) {
|
||||||
|
if (!bridge) return;
|
||||||
|
await run({ type: 'agent_approve', id });
|
||||||
|
}
|
||||||
|
|
||||||
|
async function rejectAgent(id: number) {
|
||||||
|
if (!bridge) return;
|
||||||
|
await run({ type: 'agent_reject', id });
|
||||||
|
}
|
||||||
|
|
||||||
async function renderProject() {
|
async function renderProject() {
|
||||||
if (!bridge || rendering) return;
|
if (!bridge || rendering) return;
|
||||||
rendering = true;
|
rendering = true;
|
||||||
@@ -297,7 +332,10 @@
|
|||||||
onMeter,
|
onMeter,
|
||||||
onEvent,
|
onEvent,
|
||||||
onError,
|
onError,
|
||||||
onImported
|
onImported,
|
||||||
|
onAgentHistory: (messages) => {
|
||||||
|
agentMessages = messages;
|
||||||
|
}
|
||||||
});
|
});
|
||||||
bridge.connect();
|
bridge.connect();
|
||||||
window.addEventListener('keydown', onKey);
|
window.addEventListener('keydown', onKey);
|
||||||
@@ -324,10 +362,16 @@
|
|||||||
quarry
|
quarry
|
||||||
</button>
|
</button>
|
||||||
<span class="rail-rule" aria-hidden="true"></span>
|
<span class="rail-rule" aria-hidden="true"></span>
|
||||||
|
<button class="room" class:is-on={view === 'choir'} aria-pressed={view === 'choir'} onclick={() => (view = 'choir')}>
|
||||||
|
choir
|
||||||
|
</button>
|
||||||
|
<span class="rail-rule" aria-hidden="true"></span>
|
||||||
<span class="room is-on"><span class="mode-dot" aria-hidden="true"></span>hand</span>
|
<span class="room is-on"><span class="mode-dot" aria-hidden="true"></span>hand</span>
|
||||||
<span class="rail-note">
|
<span class="rail-note">
|
||||||
{view === 'quarry'
|
{view === 'quarry'
|
||||||
? `${project?.cores.length ?? 0} core${(project?.cores.length ?? 0) === 1 ? '' : 's'}`
|
? `${project?.cores.length ?? 0} core${(project?.cores.length ?? 0) === 1 ? '' : 's'}`
|
||||||
|
: view === 'choir'
|
||||||
|
? `${agentMessages.length} line${agentMessages.length === 1 ? '' : 's'}`
|
||||||
: project
|
: project
|
||||||
? `${project.veins.length} veins`
|
? `${project.veins.length} veins`
|
||||||
: 'five veins'}
|
: 'five veins'}
|
||||||
@@ -355,7 +399,7 @@
|
|||||||
<main class="hero">
|
<main class="hero">
|
||||||
{#if view === 'nave'}
|
{#if view === 'nave'}
|
||||||
<Descent state={project} {meter} {selected} onselect={select} {strike} />
|
<Descent state={project} {meter} {selected} onselect={select} {strike} />
|
||||||
{:else}
|
{:else if view === 'quarry'}
|
||||||
<Quarry
|
<Quarry
|
||||||
cores={project?.cores ?? []}
|
cores={project?.cores ?? []}
|
||||||
pattern={project?.pattern ?? null}
|
pattern={project?.pattern ?? null}
|
||||||
@@ -377,6 +421,17 @@
|
|||||||
onclearsection={clearSection}
|
onclearsection={clearSection}
|
||||||
oncopysection={copySection}
|
oncopysection={copySection}
|
||||||
/>
|
/>
|
||||||
|
{:else}
|
||||||
|
<Choir
|
||||||
|
messages={agentMessages}
|
||||||
|
proposals={agentProposals}
|
||||||
|
busy={agentBusy}
|
||||||
|
project={project}
|
||||||
|
{meter}
|
||||||
|
onask={askAgent}
|
||||||
|
onapprove={approveAgent}
|
||||||
|
onreject={rejectAgent}
|
||||||
|
/>
|
||||||
{/if}
|
{/if}
|
||||||
<Wire {entries} {status} onreconnect={reconnect} />
|
<Wire {entries} {status} onreconnect={reconnect} />
|
||||||
<EnginePanel state={project} {meter} {status} session={hello?.session ?? '—'} ongain={setGain} />
|
<EnginePanel state={project} {meter} {status} session={hello?.session ?? '—'} ongain={setGain} />
|
||||||
|
|||||||
+314
@@ -1731,6 +1731,281 @@ button.room {
|
|||||||
font-weight: 400;
|
font-weight: 400;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
/* — choir — */
|
||||||
|
|
||||||
|
.choir {
|
||||||
|
position: absolute;
|
||||||
|
inset: 0;
|
||||||
|
z-index: 2;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
padding: 22px calc(var(--panel-r) + 20px) 0 calc(var(--panel-l) + 22px);
|
||||||
|
background:
|
||||||
|
radial-gradient(ellipse at 50% 0%, rgba(232, 64, 42, 0.05), transparent 55%),
|
||||||
|
var(--resin);
|
||||||
|
animation: fade 0.7s 0.05s ease both;
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-head {
|
||||||
|
display: flex;
|
||||||
|
align-items: flex-end;
|
||||||
|
justify-content: space-between;
|
||||||
|
gap: 14px;
|
||||||
|
padding-bottom: 13px;
|
||||||
|
border-bottom: 1px solid rgba(155, 133, 121, 0.12);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-head h1 {
|
||||||
|
margin: 0;
|
||||||
|
font-family: var(--font-display);
|
||||||
|
font-size: 30px;
|
||||||
|
font-style: italic;
|
||||||
|
font-weight: 300;
|
||||||
|
line-height: 1;
|
||||||
|
color: var(--bone);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-head p {
|
||||||
|
margin: 5px 0 0;
|
||||||
|
font-size: 8.5px;
|
||||||
|
letter-spacing: 0.12em;
|
||||||
|
color: var(--ash-2);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-readout {
|
||||||
|
display: flex;
|
||||||
|
gap: 12px;
|
||||||
|
font-size: 8px;
|
||||||
|
letter-spacing: 0.08em;
|
||||||
|
color: var(--ash);
|
||||||
|
font-variant-numeric: tabular-nums;
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-log {
|
||||||
|
flex: 1;
|
||||||
|
min-height: 0;
|
||||||
|
overflow-y: auto;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 15px;
|
||||||
|
padding: 20px 4px 18px 0;
|
||||||
|
scrollbar-width: thin;
|
||||||
|
scrollbar-color: rgba(155, 133, 121, 0.3) transparent;
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-empty {
|
||||||
|
max-width: 530px;
|
||||||
|
margin: 7vh auto 0;
|
||||||
|
font-family: var(--font-display);
|
||||||
|
font-size: 22px;
|
||||||
|
font-style: italic;
|
||||||
|
font-weight: 300;
|
||||||
|
line-height: 1.5;
|
||||||
|
text-align: center;
|
||||||
|
color: var(--ash);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-message {
|
||||||
|
max-width: 760px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-role {
|
||||||
|
display: block;
|
||||||
|
margin-bottom: 4px;
|
||||||
|
font-size: 8px;
|
||||||
|
letter-spacing: 0.16em;
|
||||||
|
color: var(--ash-2);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-message p {
|
||||||
|
margin: 0;
|
||||||
|
font-family: var(--font-display);
|
||||||
|
font-size: 20px;
|
||||||
|
font-weight: 300;
|
||||||
|
line-height: 1.45;
|
||||||
|
color: var(--bone);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-message.is-user {
|
||||||
|
align-self: flex-end;
|
||||||
|
max-width: 560px;
|
||||||
|
text-align: right;
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-message.is-user .choir-role {
|
||||||
|
color: var(--ash-2);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-message.is-user p {
|
||||||
|
font-family: var(--font-mono);
|
||||||
|
font-size: 11px;
|
||||||
|
line-height: 1.7;
|
||||||
|
color: var(--ash);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-proposal {
|
||||||
|
max-width: 760px;
|
||||||
|
padding: 12px 0 0;
|
||||||
|
border-top: 1px solid rgba(232, 64, 42, 0.38);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-proposal-head {
|
||||||
|
display: flex;
|
||||||
|
justify-content: space-between;
|
||||||
|
font-size: 8px;
|
||||||
|
letter-spacing: 0.12em;
|
||||||
|
color: var(--ash-2);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-proposal p {
|
||||||
|
margin: 7px 0 9px;
|
||||||
|
font-family: var(--font-display);
|
||||||
|
font-size: 18px;
|
||||||
|
font-style: italic;
|
||||||
|
font-weight: 300;
|
||||||
|
line-height: 1.45;
|
||||||
|
color: var(--bone);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-moves {
|
||||||
|
display: flex;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
gap: 5px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-move {
|
||||||
|
padding: 3px 6px;
|
||||||
|
border: 1px solid rgba(155, 133, 121, 0.2);
|
||||||
|
font-size: 8.5px;
|
||||||
|
color: var(--ash);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-proposal-actions {
|
||||||
|
display: flex;
|
||||||
|
gap: 8px;
|
||||||
|
margin-top: 10px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-proposal-actions button,
|
||||||
|
.choir-form button {
|
||||||
|
padding: 6px 10px;
|
||||||
|
border: 1px solid rgba(155, 133, 121, 0.25);
|
||||||
|
font-size: 9px;
|
||||||
|
letter-spacing: 0.04em;
|
||||||
|
color: var(--bone);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-proposal-actions button:first-child {
|
||||||
|
border-color: rgba(232, 64, 42, 0.5);
|
||||||
|
color: var(--ember);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-proposal-actions button:hover:not(:disabled),
|
||||||
|
.choir-form button:hover:not(:disabled) {
|
||||||
|
border-color: var(--ember);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-proposal-actions button:disabled,
|
||||||
|
.choir-form button:disabled {
|
||||||
|
opacity: 0.35;
|
||||||
|
cursor: default;
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-live {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: space-between;
|
||||||
|
gap: 16px;
|
||||||
|
padding: 9px 0;
|
||||||
|
border-top: 1px solid rgba(155, 133, 121, 0.12);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-sections {
|
||||||
|
display: flex;
|
||||||
|
gap: 3px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-section {
|
||||||
|
display: inline-flex;
|
||||||
|
align-items: baseline;
|
||||||
|
gap: 5px;
|
||||||
|
padding: 3px 6px;
|
||||||
|
border: 1px solid rgba(155, 133, 121, 0.16);
|
||||||
|
font-size: 8px;
|
||||||
|
color: var(--ash-2);
|
||||||
|
font-variant-numeric: tabular-nums;
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-section b {
|
||||||
|
font-family: var(--font-display);
|
||||||
|
font-size: 14px;
|
||||||
|
font-style: italic;
|
||||||
|
font-weight: 300;
|
||||||
|
color: var(--ash);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-section.is-on {
|
||||||
|
border-color: var(--ember);
|
||||||
|
color: var(--bone);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-section.is-on b {
|
||||||
|
color: var(--ember);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-veins {
|
||||||
|
display: flex;
|
||||||
|
gap: 9px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-vein {
|
||||||
|
display: inline-flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 4px;
|
||||||
|
font-size: 8px;
|
||||||
|
color: var(--ash-2);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-vein i {
|
||||||
|
display: block;
|
||||||
|
width: 3px;
|
||||||
|
height: 16px;
|
||||||
|
background: linear-gradient(to top, rgba(232, 64, 42, 0.85) calc(var(--level) * 100%), rgba(155, 133, 121, 0.12) calc(var(--level) * 100%));
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-form {
|
||||||
|
display: flex;
|
||||||
|
gap: 7px;
|
||||||
|
padding: 11px 0 14px;
|
||||||
|
border-top: 1px solid rgba(155, 133, 121, 0.12);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-form input {
|
||||||
|
flex: 1;
|
||||||
|
min-width: 0;
|
||||||
|
padding: 8px 10px;
|
||||||
|
border: 1px solid rgba(155, 133, 121, 0.24);
|
||||||
|
background: transparent;
|
||||||
|
color: var(--bone);
|
||||||
|
font-family: var(--font-mono);
|
||||||
|
font-size: 11px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-form input:focus {
|
||||||
|
outline: none;
|
||||||
|
border-color: rgba(232, 64, 42, 0.55);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-form input::placeholder {
|
||||||
|
color: var(--ash-2);
|
||||||
|
}
|
||||||
|
|
||||||
|
.choir-listen {
|
||||||
|
border-color: rgba(232, 64, 42, 0.42) !important;
|
||||||
|
color: var(--ember) !important;
|
||||||
|
}
|
||||||
|
|
||||||
/* — responsive — */
|
/* — responsive — */
|
||||||
|
|
||||||
@media (max-width: 1240px) {
|
@media (max-width: 1240px) {
|
||||||
@@ -1780,6 +2055,16 @@ button.room {
|
|||||||
width: 100%;
|
width: 100%;
|
||||||
padding: 20px 18px 24px;
|
padding: 20px 18px 24px;
|
||||||
}
|
}
|
||||||
|
.choir {
|
||||||
|
position: relative;
|
||||||
|
inset: auto;
|
||||||
|
width: 100%;
|
||||||
|
min-height: 80vh;
|
||||||
|
padding: 20px 18px 0;
|
||||||
|
}
|
||||||
|
.choir-log {
|
||||||
|
min-height: 300px;
|
||||||
|
}
|
||||||
.quarry-grid {
|
.quarry-grid {
|
||||||
grid-template-columns: 1fr;
|
grid-template-columns: 1fr;
|
||||||
}
|
}
|
||||||
@@ -1940,6 +2225,35 @@ button.room {
|
|||||||
.quarry {
|
.quarry {
|
||||||
padding: 18px 14px 22px;
|
padding: 18px 14px 22px;
|
||||||
}
|
}
|
||||||
|
.choir {
|
||||||
|
min-height: 82vh;
|
||||||
|
padding: 18px 14px 0;
|
||||||
|
}
|
||||||
|
.choir-head {
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
.choir-readout {
|
||||||
|
display: none;
|
||||||
|
}
|
||||||
|
.choir-message p {
|
||||||
|
font-size: 18px;
|
||||||
|
}
|
||||||
|
.choir-message.is-user p {
|
||||||
|
font-size: 10.5px;
|
||||||
|
}
|
||||||
|
.choir-live {
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
.choir-veins {
|
||||||
|
gap: 6px;
|
||||||
|
}
|
||||||
|
.choir-form {
|
||||||
|
gap: 5px;
|
||||||
|
}
|
||||||
|
.choir-form button {
|
||||||
|
padding: 6px 8px;
|
||||||
|
font-size: 8.5px;
|
||||||
|
}
|
||||||
.quarry-head h1 {
|
.quarry-head h1 {
|
||||||
font-size: 26px;
|
font-size: 26px;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,146 @@
|
|||||||
|
<svelte:options runes={true} />
|
||||||
|
|
||||||
|
<script lang="ts">
|
||||||
|
import type { AgentMessage, AgentProposal, MeterFrame, Pattern, State } from '../lib/types';
|
||||||
|
import { VEINS } from '../lib/types';
|
||||||
|
import { commandLabel } from '../lib/format';
|
||||||
|
|
||||||
|
let {
|
||||||
|
messages,
|
||||||
|
proposals,
|
||||||
|
busy,
|
||||||
|
project,
|
||||||
|
meter,
|
||||||
|
onask,
|
||||||
|
onapprove,
|
||||||
|
onreject
|
||||||
|
}: {
|
||||||
|
messages: AgentMessage[];
|
||||||
|
proposals: AgentProposal[];
|
||||||
|
busy: boolean;
|
||||||
|
project: State | null;
|
||||||
|
meter: MeterFrame;
|
||||||
|
onask: (text: string) => void;
|
||||||
|
onapprove: (id: number) => void;
|
||||||
|
onreject: (id: number) => void;
|
||||||
|
} = $props();
|
||||||
|
|
||||||
|
let text = $state('');
|
||||||
|
let log: HTMLDivElement | null = $state(null);
|
||||||
|
|
||||||
|
$effect(() => {
|
||||||
|
messages.length;
|
||||||
|
proposals.length;
|
||||||
|
if (log) log.scrollTop = log.scrollHeight;
|
||||||
|
});
|
||||||
|
|
||||||
|
function notes(pattern: Pattern): number {
|
||||||
|
return pattern.rows.reduce(
|
||||||
|
(total, row) => total + row.velocities.filter((velocity) => velocity > 0).length,
|
||||||
|
0
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function send() {
|
||||||
|
const value = text.trim();
|
||||||
|
if (!value || busy) return;
|
||||||
|
text = '';
|
||||||
|
onask(value);
|
||||||
|
}
|
||||||
|
|
||||||
|
function listen() {
|
||||||
|
if (!busy) onask('Listen to the room. What do you hear?');
|
||||||
|
}
|
||||||
|
|
||||||
|
function handleKey(event: KeyboardEvent) {
|
||||||
|
if (event.key === 'Enter' && !event.shiftKey) {
|
||||||
|
event.preventDefault();
|
||||||
|
send();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<div class="choir">
|
||||||
|
<header class="choir-head">
|
||||||
|
<div>
|
||||||
|
<h1>choir</h1>
|
||||||
|
<p>the second performer · same verbs · proposals need a hand</p>
|
||||||
|
</div>
|
||||||
|
<div class="choir-readout">
|
||||||
|
<span>{project?.transport.playing ? 'playing' : 'still'}</span>
|
||||||
|
<span>{Math.round(project?.transport.bpm ?? 128)} bpm</span>
|
||||||
|
<span>{project?.transport.loop_bars ?? 8} bars</span>
|
||||||
|
<span>{project?.cores.length ?? 0} cores</span>
|
||||||
|
</div>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<div class="choir-log" bind:this={log}>
|
||||||
|
{#if messages.length === 0}
|
||||||
|
<p class="choir-empty">
|
||||||
|
Ask the choir to listen, to lift the bass, to open section B, or to render the eight bars.
|
||||||
|
It hears the levels; project edits arrive as proposals.
|
||||||
|
</p>
|
||||||
|
{/if}
|
||||||
|
{#each messages as message}
|
||||||
|
<div class="choir-message" class:is-user={message.role === 'user'}>
|
||||||
|
<span class="choir-role">{message.role === 'agent' ? 'choir' : message.role}</span>
|
||||||
|
<p>{message.text}</p>
|
||||||
|
</div>
|
||||||
|
{/each}
|
||||||
|
{#each proposals as proposal}
|
||||||
|
<div class="choir-proposal">
|
||||||
|
<div class="choir-proposal-head">
|
||||||
|
<span>proposal</span>
|
||||||
|
<span>{proposal.commands.length} move{proposal.commands.length === 1 ? '' : 's'}</span>
|
||||||
|
</div>
|
||||||
|
<p>{proposal.description}</p>
|
||||||
|
<div class="choir-moves">
|
||||||
|
{#each proposal.commands as command}
|
||||||
|
<span class="choir-move">{commandLabel(command)}</span>
|
||||||
|
{/each}
|
||||||
|
</div>
|
||||||
|
<div class="choir-proposal-actions">
|
||||||
|
<button onclick={() => onapprove(proposal.id)} disabled={busy}>approve</button>
|
||||||
|
<button onclick={() => onreject(proposal.id)} disabled={busy}>reject</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{/each}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="choir-live">
|
||||||
|
<div class="choir-sections" aria-label="sections">
|
||||||
|
{#each project?.arrangement ?? [] as section, index}
|
||||||
|
<span class="choir-section" class:is-on={index === project?.active_section}>
|
||||||
|
<b>{String.fromCharCode(65 + index)}</b>
|
||||||
|
{notes(section)}
|
||||||
|
</span>
|
||||||
|
{/each}
|
||||||
|
</div>
|
||||||
|
<div class="choir-veins" aria-label="levels">
|
||||||
|
{#each VEINS as vein, index}
|
||||||
|
<span class="choir-vein">
|
||||||
|
<i style={`--level:${meter.veins[index] ?? 0}`}></i>
|
||||||
|
{vein.slice(0, 3)}
|
||||||
|
</span>
|
||||||
|
{/each}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<form
|
||||||
|
class="choir-form"
|
||||||
|
onsubmit={(event) => {
|
||||||
|
event.preventDefault();
|
||||||
|
send();
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<button type="button" class="choir-listen" onclick={listen} disabled={busy}>listen</button>
|
||||||
|
<input
|
||||||
|
bind:value={text}
|
||||||
|
onkeydown={handleKey}
|
||||||
|
placeholder={busy ? 'the choir is listening…' : 'ask the choir…'}
|
||||||
|
disabled={busy}
|
||||||
|
aria-label="ask the choir"
|
||||||
|
/>
|
||||||
|
<button type="submit" class="choir-send" disabled={busy || !text.trim()}>send</button>
|
||||||
|
</form>
|
||||||
|
</div>
|
||||||
@@ -7,6 +7,7 @@ import type {
|
|||||||
MeterFrame,
|
MeterFrame,
|
||||||
Seal,
|
Seal,
|
||||||
ServerMessage,
|
ServerMessage,
|
||||||
|
AgentMessage,
|
||||||
State,
|
State,
|
||||||
Status
|
Status
|
||||||
} from './types';
|
} from './types';
|
||||||
@@ -20,6 +21,7 @@ export interface BridgeHandlers {
|
|||||||
onEvent: (event: Event) => void;
|
onEvent: (event: Event) => void;
|
||||||
onError: (message: string) => void;
|
onError: (message: string) => void;
|
||||||
onImported?: (core: CoreSummary) => void;
|
onImported?: (core: CoreSummary) => void;
|
||||||
|
onAgentHistory?: (messages: AgentMessage[]) => void;
|
||||||
}
|
}
|
||||||
|
|
||||||
interface Pending {
|
interface Pending {
|
||||||
@@ -190,6 +192,10 @@ export class Bridge {
|
|||||||
case 'event':
|
case 'event':
|
||||||
this.handlers.onEvent(message.event);
|
this.handlers.onEvent(message.event);
|
||||||
break;
|
break;
|
||||||
|
case 'agent_history': {
|
||||||
|
this.handlers.onAgentHistory?.(message.messages);
|
||||||
|
break;
|
||||||
|
}
|
||||||
case 'imported': {
|
case 'imported': {
|
||||||
const pending = this.imports.get(message.id);
|
const pending = this.imports.get(message.id);
|
||||||
if (!pending) break;
|
if (!pending) break;
|
||||||
|
|||||||
@@ -38,6 +38,10 @@ export function commandLabel(command: Command): string {
|
|||||||
case 'clear_section': return `clear section ${String.fromCharCode(65 + command.index)}`;
|
case 'clear_section': return `clear section ${String.fromCharCode(65 + command.index)}`;
|
||||||
case 'copy_section': return `copy ${String.fromCharCode(65 + command.from)} → ${String.fromCharCode(65 + command.to)}`;
|
case 'copy_section': return `copy ${String.fromCharCode(65 + command.from)} → ${String.fromCharCode(65 + command.to)}`;
|
||||||
case 'render': return 'render wav';
|
case 'render': return 'render wav';
|
||||||
|
case 'agent_ask': return `choir: ${command.text.slice(0, 42)}`;
|
||||||
|
case 'agent_approve': return `approve ${command.id}`;
|
||||||
|
case 'agent_reject': return `reject ${command.id}`;
|
||||||
|
case 'agent_reset': return 'reset choir';
|
||||||
case 'undo': return 'undo';
|
case 'undo': return 'undo';
|
||||||
case 'redo': return 'redo';
|
case 'redo': return 'redo';
|
||||||
case 'get_snapshot': return 'snapshot';
|
case 'get_snapshot': return 'snapshot';
|
||||||
|
|||||||
+27
-2
@@ -85,7 +85,10 @@ export type Event =
|
|||||||
| { kind: 'history'; can_undo: boolean; can_redo: boolean }
|
| { kind: 'history'; can_undo: boolean; can_redo: boolean }
|
||||||
| { kind: 'core_faces'; core: CoreSummary }
|
| { kind: 'core_faces'; core: CoreSummary }
|
||||||
| { kind: 'arrangement'; sections: Pattern[]; active: number }
|
| { kind: 'arrangement'; sections: Pattern[]; active: number }
|
||||||
| { kind: 'rendered'; path: string; duration_sec: number };
|
| { kind: 'rendered'; path: string; duration_sec: number }
|
||||||
|
| { kind: 'agent_message'; message: AgentMessage }
|
||||||
|
| { kind: 'agent_proposal'; id: number; description: string; commands: Command[] }
|
||||||
|
| { kind: 'agent_resolved'; id: number; applied: boolean };
|
||||||
|
|
||||||
export type Command =
|
export type Command =
|
||||||
| { type: 'get_snapshot' }
|
| { type: 'get_snapshot' }
|
||||||
@@ -102,6 +105,10 @@ export type Command =
|
|||||||
| { type: 'clear_section'; index: number }
|
| { type: 'clear_section'; index: number }
|
||||||
| { type: 'copy_section'; from: number; to: number }
|
| { type: 'copy_section'; from: number; to: number }
|
||||||
| { type: 'render' }
|
| { type: 'render' }
|
||||||
|
| { type: 'agent_ask'; text: string }
|
||||||
|
| { type: 'agent_approve'; id: number }
|
||||||
|
| { type: 'agent_reject'; id: number }
|
||||||
|
| { type: 'agent_reset' }
|
||||||
| { type: 'undo' }
|
| { type: 'undo' }
|
||||||
| { type: 'redo' };
|
| { type: 'redo' };
|
||||||
|
|
||||||
@@ -147,7 +154,25 @@ export interface ImportedMessage {
|
|||||||
core: CoreSummary;
|
core: CoreSummary;
|
||||||
}
|
}
|
||||||
|
|
||||||
export type ServerMessage = HelloMessage | SnapshotMessage | EventMessage | ResultMessage | ErrorMessage | ImportedMessage;
|
export type AgentRole = 'user' | 'agent' | 'system';
|
||||||
|
|
||||||
|
export interface AgentMessage {
|
||||||
|
role: AgentRole;
|
||||||
|
text: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface AgentProposal {
|
||||||
|
id: number;
|
||||||
|
description: string;
|
||||||
|
commands: Command[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface AgentHistoryMessage {
|
||||||
|
type: 'agent_history';
|
||||||
|
messages: AgentMessage[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export type ServerMessage = HelloMessage | SnapshotMessage | EventMessage | ResultMessage | ErrorMessage | ImportedMessage | AgentHistoryMessage;
|
||||||
|
|
||||||
export interface MeterFrame {
|
export interface MeterFrame {
|
||||||
bar: number;
|
bar: number;
|
||||||
|
|||||||
Reference in New Issue
Block a user