P1c: microphone recording through the material pipeline

This commit is contained in:
Emil
2026-11-01 01:44:36 +03:00
parent 0acf11ccb0
commit 88663a384e
14 changed files with 421 additions and 18 deletions
+11 -6
View File
@@ -2,7 +2,7 @@
Sound is terrain. A sampling instrument where material is cut in a quarry, laid as stones in a nave, and played by a hand or an agent through the same verbs. Sound is terrain. A sampling instrument where material is cut in a quarry, laid as stones in a nave, and played by a hand or an agent through the same verbs.
This repository is **P1b**: the engine, the wire, the descent, the material room, and a project that survives a restart. A found sound can be imported, cut into sixteen faces, laid on a vein and heard in the loop; tempo, gains and pattern edits are autosaved and reversible. The visual demo in [`reference/demo`](reference/demo) remains a design reference, not the codebase. The roadmap is in [`docs/PLAN.md`](docs/PLAN.md). This repository is **P1c**: the engine, the wire, the descent, the material room, a project that survives a restart, and a microphone. A found or recorded sound is cut into sixteen faces, laid on a vein and heard in the loop; tempo, gains and pattern edits are autosaved and reversible. The visual demo in [`reference/demo`](reference/demo) remains a design reference, not the codebase. The roadmap is in [`docs/PLAN.md`](docs/PLAN.md).
![The quarry](preview-p1-quarry.png) ![The quarry](preview-p1-quarry.png)
@@ -10,12 +10,15 @@ This repository is **P1b**: the engine, the wire, the descent, the material room
![P1b · restored project](preview-p1b-restored.png) ![P1b · restored project](preview-p1b-restored.png)
![P1c · a recorded take](preview-p1c-record.png)
## What is real ## What is real
- **Rust engine.** `cpal` output at 48 kHz, five veins, a sixteen-step phrase, a fixed voice pool, per-vein levels and gains, a bar clock. Procedural voices for empty pattern rows; real sample voices for rows with a face. No Web Audio. - **Rust engine.** `cpal` output at 48 kHz, five veins, a sixteen-step phrase, a fixed voice pool, per-vein levels and gains, a bar clock. Procedural voices for empty pattern rows; real sample voices for rows with a face. No Web Audio.
- **Material.** Import through the local wire; `symphonia` decodes WAV/FLAC/MP3/OGG; onset analysis finds seams and cuts sixteen faces with RMS, rank and waveform peaks; the archive keeps the mono f32 samples. The sample rate is preserved and the engine handles the playback ratio. - **Material.** Import through the local wire; `symphonia` decodes WAV/FLAC/MP3/OGG; onset analysis finds seams and cuts sixteen faces with RMS, rank and waveform peaks; the archive keeps the mono f32 samples. The sample rate is preserved and the engine handles the playback ratio.
- **Project.** A portable folder: `project.db` (SQLite) plus `assets/`. On daemon start it loads the cores, pattern, tempo, gains and journal; every project edit autosaves. The default is `~/.local/share/zhila/descent-04`; use `--project PATH` for another. The project name is shown in the top rail. - **Project.** A portable folder: `project.db` (SQLite) plus `assets/`. On daemon start it loads the cores, pattern, tempo, gains and journal; every project edit autosaves. The default is `~/.local/share/zhila/descent-04`; use `--project PATH` for another. The project name is shown in the top rail.
- **Undo/redo.** Tempo, gains and pattern steps/faces push a small history; `Ctrl/⌘+Z` and `Shift+Ctrl/⌘+Z`, or the rail buttons. The journal records every edit. History is per session: after a reload the project is restored, the undo stack starts empty. - **Undo/redo.** Tempo, gains and pattern steps/faces push a small history; `Ctrl/⌘+Z` and `Shift+Ctrl/⌘+Z`, or the rail buttons. The journal records every edit. History is per session: after a reload the project is restored, the undo stack starts empty.
- **Recording.** `record` in the quarry opens the microphone, captures mono PCM through an AudioWorklet, encodes a 16-bit WAV in the surface and sends it through the same import path. The take becomes a core with sixteen faces, seal `recorded`, and is autosaved in the project. Stops at 30 s; no monitoring.
- **Quarry.** Import, core strip, zoomed waveform with face boundaries, sixteen-face wall, face audition, seal and stats; assign a face to a pattern row; edit the sixteen steps; strike a face; all against the live session. - **Quarry.** Import, core strip, zoomed waveform with face boundaries, sixteen-face wall, face audition, seal and stats; assign a face to a pattern row; edit the sixteen steps; strike a face; all against the live session.
- **Protocol v1.** JSON control frames, a 31-byte binary meter frame, and a binary import path (an import header followed by the audio bytes). Snapshot, results, events, token and origin guard. - **Protocol v1.** JSON control frames, a 31-byte binary meter frame, and a binary import path (an import header followed by the audio bytes). Snapshot, results, events, token and origin guard.
- **Session + journal.** Validation, revisions, a journal for project edits (tempo, gains, pattern faces and steps); play, stop and strikes are performance, not history. - **Session + journal.** Validation, revisions, a journal for project edits (tempo, gains, pattern faces and steps); play, stop and strikes are performance, not history.
@@ -45,7 +48,7 @@ npm --prefix ui run dev
1. Open **quarry** in the top rail. 1. Open **quarry** in the top rail.
2. `take a core` and choose a WAV (a short field recording works; the fixture `fixtures/clicks-01.wav` is one). 2. `take a core` and choose a WAV (a short field recording works; the fixture `fixtures/clicks-01.wav` is one).
3. The file is decoded, the seams are found and sixteen faces are cut. Click a face, then `lay on bass`. 3. The file is decoded, the seams are found and sixteen faces are cut. Or press `record`, speak/play into the microphone, press `stop`; the take arrives the same way. Click a face, then `lay on bass`.
4. `space` plays the loop; the bass row now triggers the sample while the other veins keep their procedural voices. Edit steps in the pattern; each edit goes to the engine and the journal. 4. `space` plays the loop; the bass row now triggers the sample while the other veins keep their procedural voices. Edit steps in the pattern; each edit goes to the engine and the journal.
5. Go back to **nave** to see the pattern on the road and the levels in the room. 5. Go back to **nave** to see the pattern on the road and the levels in the room.
@@ -58,7 +61,7 @@ cd app
./src-tauri/target/release/zhila-app ./src-tauri/target/release/zhila-app
``` ```
The built window embeds the surface and connects to the daemon at `ws://127.0.0.1:4317`; start the daemon first. `tauri dev` uses the Vite server at `127.0.0.1:5173`. The engine is still the separate daemon in this phase. Rebuild the native app after any UI change; the binary embeds `ui/dist` at compile time. The built window embeds the surface and connects to the daemon at `ws://127.0.0.1:4317`; start the daemon first. `tauri dev` uses the Vite server at `127.0.0.1:5173`. The engine is still the separate daemon in this phase. Rebuild the native app after any UI change; the binary embeds `ui/dist` at compile time. The recording UI is in the shared surface; WebKitGTK microphone permission still needs a manual check on this machine.
Controls: `space` play/stop · `s` strike the selected face in the quarry or the selected vein in the nave · `1`–`5` select a vein (nave) or a pattern row (quarry) · click a face to select, double-click to strike · tempo `−` / `+`. Controls: `space` play/stop · `s` strike the selected face in the quarry or the selected vein in the nave · `1`–`5` select a vein (nave) or a pattern row (quarry) · click a face to select, double-click to strike · tempo `−` / `+`.
@@ -90,19 +93,21 @@ node scripts/persistence-smoke.mjs # spawned daemon: import/edit/undo/restart
npm --prefix ui run smoke # daemon + Vite: P0 regression, screenshots npm --prefix ui run smoke # daemon + Vite: P0 regression, screenshots
npm --prefix ui run quarry-smoke # daemon + Vite: import, assign, play, screenshots npm --prefix ui run quarry-smoke # daemon + Vite: import, assign, play, screenshots
npm --prefix ui run project-smoke # daemon + Vite: project, undo, import, screenshots npm --prefix ui run project-smoke # daemon + Vite: project, undo, import, screenshots
npm --prefix ui run record-smoke # daemon + Vite: fake microphone take, faces, seal
npm --prefix ui run restored-smoke # after restarting the daemon on the saved project npm --prefix ui run restored-smoke # after restarting the daemon on the saved project
``` ```
`npm --prefix ui run smoke` writes `preview-p0*.png`; `quarry-smoke` writes `preview-p1-*.png`. `npm --prefix ui run smoke` writes `preview-p0*.png`; `quarry-smoke` writes `preview-p1-*.png`.
## P1b limits (next, not hidden) ## P1c limits (next, not hidden)
- The project persists, but undo history does not: after a reload the undo stack starts empty. The journal is stored for audit, not reconstructed into undo. - The project persists, but undo history does not: after a reload the undo stack starts empty. The journal is stored for audit, not reconstructed into undo.
- No recording, no waveform trimming, no manual cut/loop points; the sixteen faces are automatic seams. - Recording is mono, 16-bit, requested 48 kHz, up to 30 s, no monitoring and no input device choice. WebKitGTK/Tauri microphone permission is untested; the browser path is verified with Chromium's fake capture.
- No waveform trimming, no manual cut/loop points; the sixteen faces are automatic seams.
- One fixed two-bar phrase and five veins; pattern rows are veins, one face per row, no per-step face or pitch. - One fixed two-bar phrase and five veins; pattern rows are veins, one face per row, no per-step face or pitch.
- The Tauri binary currently embeds an older surface build; rebuild it after UI changes (`npm --prefix ui run build`, then `tauri build`). The engine remains the separate daemon. - The Tauri binary currently embeds an older surface build; rebuild it after UI changes (`npm --prefix ui run build`, then `tauri build`). The engine remains the separate daemon.
- Formats depend on symphonia; WAV is the tested path. AIFF, multi-channel files and very long files are not specifically handled. - Formats depend on symphonia; WAV is the tested path. AIFF, multi-channel files and very long files are not specifically handled.
- No agent/MCP in this phase. - No agent/MCP in this phase.
- The Tauri shell now builds and runs; the engine is still the separate daemon (embedding is next). - The Tauri shell now builds and runs; the engine is still the separate daemon (embedding is next).
The next slice is P1c: recording, waveform trimming, reconstructed undo across reloads, and embedding the engine inside the Tauri backend. Then the arrangement and the agent. The plan is in [`docs/PLAN.md`](docs/PLAN.md); the phase notes are in [`docs/P1.md`](docs/P1.md). The next slice is P1d: waveform trimming and manual faces, reconstructed undo across reloads, and embedding the engine inside the Tauri backend. Then the arrangement and the agent. The plan is in [`docs/PLAN.md`](docs/PLAN.md); the phase notes are in [`docs/P1.md`](docs/P1.md).
+1
View File
@@ -83,6 +83,7 @@ pub enum Actor {
#[serde(rename_all = "snake_case")] #[serde(rename_all = "snake_case")]
pub enum Seal { pub enum Seal {
Found, Found,
Recorded,
Taken, Taken,
Gift, Gift,
Loan, Loan,
+45
View File
@@ -98,3 +98,48 @@ Screenshots: `preview-p1b-quarry.png`, `preview-p1b-restored.png`.
- Only the source bytes are archived; trimming/recording will add derived assets. - Only the source bytes are archived; trimming/recording will add derived assets.
- The engine is still the standalone daemon; the Tauri backend does not host it yet. - The engine is still the standalone daemon; the Tauri backend does not host it yet.
- A failed autosave logs a warning and leaves the in-memory session running; the surface does not yet show a save error. - A failed autosave logs a warning and leaves the in-memory session running; the surface does not yet show a save error.
---
# P1c · recording
The quarry can now take a microphone take and turn it into material.
## Path
```
record click
→ getUserMedia({ mono, no AGC/NS/echo })
→ AudioContext (requested 48 kHz) → AudioWorklet 'zhila-recorder'
→ Float32 chunks on the main thread
→ 16-bit PCM WAV encoder (in the surface)
→ the existing binary import path with seal 'recorded'
→ material decode + onset analysis + sixteen faces
→ project asset + autosave
```
The worklet copies each 128-frame render quantum; the main thread owns the growing chunk list so the audio thread never allocates the take. A mute GainNode keeps the input out of the speakers. The take name is `take-01.wav`, `take-02.wav`, …; the core name drops the extension. The recorder stops at 30 s; the user can stop earlier. Errors (permission denied, no device, worklet failure) surface under the quarry header.
The WAV is mono 16-bit PCM at the AudioContext rate (requested 48 kHz). The material crate already handles WAV; no daemon or protocol change was needed beyond `Seal::Recorded`.
## UI
- `record` in the quarry header and in the empty state: dot + label; `stop 0:12` while active with a pulsing dot.
- `take a core` is disabled while recording; import is disabled while recording.
- The take is imported with the recorded seal; the core strip and side facts show `recorded`.
- The timer is in the button; no modal, no separate recorder panel.
## Verification
- `npm --prefix ui run record-smoke` launches Chromium with `--use-fake-ui-for-media-stream --use-fake-device-for-media-stream --use-file-for-fake-audio-capture=fixtures/clicks-01.wav`, records 2.3 s through the real UI/worklet/encoder, stops, waits for `take-01`, and checks 16 faces, the recorded seal, and the wire import.
- The take is persisted in `~/.local/share/zhila/descent-04` (`core-4525165cd5.wav`, 16 faces, seal recorded) and shown by the state.
- `./scripts/check.sh` remains green.
Screenshot: `preview-p1c-record.png`.
## Limits
- Mono, 16-bit, 30 s max, no monitor, no device picker, no gain control.
- WebKitGTK/Tauri microphone support is untested; the Tauri binary also embeds an older surface build and must be rebuilt to see the record button.
- No trim, fade or manual loop points yet; the take is cut by the automatic seams.
- The take is stored as the encoded WAV, not the float PCM; trimming will later add derived assets.
+1 -1
View File
@@ -1,7 +1,7 @@
# ZHILA — план проекта # ZHILA — план проекта
Черновик v1.1 · 1 ноября 2026 Черновик v1.1 · 1 ноября 2026
Статус: **P0, P1a и P1b выполнены** (см. `P0.md`, `P1.md`). Демо `reference/demo/` — референс, не кодовая база. Статус: **P0, P1a, P1b и P1c выполнены** (см. `P0.md`, `P1.md`). Демо `reference/demo/` — референс, не кодовая база.
**Зафиксировано 1 ноября:** descent — основной вид; камень = паттерн; климат — на жилу (концепт погоды остаётся); платформа v1 — Linux; UI — Svelte 5. Журнал — §9. **Зафиксировано 1 ноября:** descent — основной вид; камень = паттерн; климат — на жилу (концепт погоды остаётся); платформа v1 — Linux; UI — Svelte 5. Журнал — §9.
+4
View File
@@ -152,3 +152,7 @@ Events: `pattern_step`, `pattern_face`, `face_struck`.
`State` adds `project_name` (string), `can_undo` and `can_redo` (bools). Commands `undo` and `redo` take no arguments; event `history { can_undo, can_redo }` is broadcast after them. `State` adds `project_name` (string), `can_undo` and `can_redo` (bools). Commands `undo` and `redo` take no arguments; event `history { can_undo, can_redo }` is broadcast after them.
The project is loaded by the daemon from its `--project` folder and saved after every project command or import. The protocol only exposes the project name; the path, SQLite file and assets stay server-side. There is no explicit save command: autosave is the contract. The journal is persisted, but undo history is not reconstructed on load. The project is loaded by the daemon from its `--project` folder and saved after every project command or import. The protocol only exposes the project name; the path, SQLite file and assets stay server-side. There is no explicit save command: autosave is the contract. The journal is persisted, but undo history is not reconstructed on load.
## P1c addition
`Seal` gains `recorded`. Recording introduces no new wire messages: the surface encodes a WAV and uses the existing `import_core` header + binary payload. The recorded seal is persisted with the core row and is visible in the snapshot.
Binary file not shown.

After

Width:  |  Height:  |  Size: 321 KiB

+1
View File
@@ -11,6 +11,7 @@
"smoke": "node scripts/smoke.mjs", "smoke": "node scripts/smoke.mjs",
"quarry-smoke": "node scripts/quarry-smoke.mjs", "quarry-smoke": "node scripts/quarry-smoke.mjs",
"project-smoke": "node scripts/project-smoke.mjs", "project-smoke": "node scripts/project-smoke.mjs",
"record-smoke": "node scripts/record-smoke.mjs",
"restored-smoke": "node scripts/restored-smoke.mjs" "restored-smoke": "node scripts/restored-smoke.mjs"
}, },
"devDependencies": { "devDependencies": {
+60
View File
@@ -0,0 +1,60 @@
import { chromium } from 'playwright-core';
import path from 'node:path';
const chrome = process.env.CHROME_PATH ?? '/home/emil/.cache/ms-playwright/chromium-1248/chrome-linux64/chrome';
const fixture = path.resolve('../fixtures/clicks-01.wav');
const browser = await chromium.launch({
executablePath: chrome,
args: [
'--no-sandbox',
'--autoplay-policy=no-user-gesture-required',
'--use-fake-ui-for-media-stream',
'--use-fake-device-for-media-stream',
`--use-file-for-fake-audio-capture=${fixture}`
]
});
const page = await browser.newPage({ viewport: { width: 1600, height: 1000 }, deviceScaleFactor: 1 });
const errors = [];
page.on('pageerror', (error) => errors.push(`pageerror: ${error.message}`));
page.on('console', (message) => { if (message.type() === 'error') errors.push(`console: ${message.text()}`); });
await page.goto('http://127.0.0.1:5173/', { waitUntil: 'load' });
await page.waitForTimeout(2200);
await page.locator('.room', { hasText: 'quarry' }).click();
await page.waitForTimeout(500);
const record = page.locator('.quarry-actions .record');
await record.click();
await page.waitForSelector('.quarry-actions .record.is-on', { timeout: 5000 });
await page.waitForTimeout(2300);
const during = await record.textContent();
await record.click();
await page.waitForFunction(
() => [...document.querySelectorAll('.core-name')].some((el) => el.textContent?.includes('take-01')),
null,
{ timeout: 25000 }
);
await page.waitForTimeout(900);
await page.screenshot({ path: '../preview-p1c-record.png' });
const report = await page.evaluate(() => ({
cores: [...document.querySelectorAll('.core-name')].map((el) => el.textContent),
faces: document.querySelectorAll('.face').length,
selected: document.querySelector('.core-tab.is-on .core-name')?.textContent,
stats: document.querySelector('.face-stats')?.textContent?.replace(/\s+/g, ' ').trim(),
wire: [...document.querySelectorAll('.wire-label')].map((el) => el.textContent).slice(-6),
recordText: document.querySelector('.quarry-actions .record')?.textContent?.trim()
}));
console.log(JSON.stringify({ during, report, errors }, null, 2));
const failures = [];
if (!during || !during.includes('stop')) failures.push('record button did not switch to stop');
if (!report.cores.includes('take-01')) failures.push('take-01 was not imported');
if (report.faces !== 16) failures.push('take did not produce 16 faces');
if (!report.stats || !report.stats.includes('recorded')) failures.push('seal is not recorded');
if (errors.length) failures.push('console errors');
await browser.close();
if (failures.length) {
console.error('RECORD SMOKE FAILED: ' + failures.join('; '));
process.exit(1);
}
console.log('record smoke passed');
+3 -2
View File
@@ -15,6 +15,7 @@
Event, Event,
HelloMessage, HelloMessage,
MeterFrame, MeterFrame,
Seal,
State, State,
Status, Status,
VeinId, VeinId,
@@ -143,13 +144,13 @@
void run({ type: 'set_vein_gain', vein, gain }); void run({ type: 'set_vein_gain', vein, gain });
} }
async function importCore(file: File) { async function importCore(file: File, seal: Seal = 'found') {
if (!bridge || importing) return; if (!bridge || importing) return;
importing = true; importing = true;
addEntry('send', `import ${file.name}`); addEntry('send', `import ${file.name}`);
try { try {
const data = await file.arrayBuffer(); const data = await file.arrayBuffer();
const core = await bridge.importCore(file.name, 'found', data); const core = await bridge.importCore(file.name, seal, data);
selectedCoreId = core.id; selectedCoreId = core.id;
selectedFace = core.faces.find((face) => face.rank === 0)?.index ?? 0; selectedFace = core.faces.find((face) => face.rank === 0)?.index ?? 0;
addEntry('recv', 'core', `${core.name} · ${core.faces.length} faces · ${core.duration_sec.toFixed(1)} s`); addEntry('recv', 'core', `${core.name} · ${core.faces.length} faces · ${core.duration_sec.toFixed(1)} s`);
+74
View File
@@ -1167,6 +1167,71 @@ button.room {
cursor: wait; cursor: wait;
} }
.record {
display: inline-flex;
align-items: center;
gap: 6px;
padding: 7px 10px;
border: 1px solid rgba(155, 133, 121, 0.28);
background: transparent;
color: var(--bone);
font-size: 9.5px;
letter-spacing: 0.03em;
transition: border-color 0.2s ease, color 0.2s ease, background 0.2s ease;
}
.record:hover:not(:disabled) {
border-color: var(--ember);
}
.record.is-on {
border-color: var(--ember);
background: rgba(232, 64, 42, 0.06);
color: var(--ember);
}
.record:disabled {
opacity: 0.4;
cursor: not-allowed;
}
.record-dot {
width: 5px;
height: 5px;
border-radius: 50%;
background: var(--ember);
}
.record.is-on .record-dot {
animation: pulse-dot 1.1s ease-in-out infinite;
}
@keyframes pulse-dot {
0%,
100% {
opacity: 1;
box-shadow: 0 0 0 0 rgba(232, 64, 42, 0.5);
}
50% {
opacity: 0.4;
box-shadow: 0 0 0 4px rgba(232, 64, 42, 0);
}
}
.record-error {
margin: 8px 0 0;
font-size: 9px;
line-height: 1.6;
color: var(--ember);
}
.empty-actions {
display: flex;
flex-wrap: wrap;
justify-content: center;
gap: 10px;
}
.file { .file {
position: absolute; position: absolute;
width: 1px; width: 1px;
@@ -1795,12 +1860,21 @@ button.room {
.quarry-head h1 { .quarry-head h1 {
font-size: 26px; font-size: 26px;
} }
.quarry-head {
flex-wrap: wrap;
}
.quarry-actions { .quarry-actions {
width: 100%;
justify-content: flex-end;
gap: 6px; gap: 6px;
} }
.import-state { .import-state {
display: none; display: none;
} }
.record {
padding: 6px 8px;
font-size: 8.5px;
}
.face-wall { .face-wall {
grid-template-columns: repeat(4, minmax(0, 1fr)); grid-template-columns: repeat(4, minmax(0, 1fr));
} }
+86 -8
View File
@@ -2,7 +2,8 @@
<script lang="ts"> <script lang="ts">
import { onMount } from 'svelte'; import { onMount } from 'svelte';
import type { CoreSummary, Pattern } from '../lib/types'; import type { CoreSummary, Pattern, Seal } from '../lib/types';
import { MAX_RECORD_SECONDS, startRecording, stopRecording } from '../lib/record';
let { let {
cores, cores,
@@ -28,7 +29,7 @@
onselectcore: (id: string) => void; onselectcore: (id: string) => void;
onselectface: (face: number) => void; onselectface: (face: number) => void;
onselectrow: (row: number) => void; onselectrow: (row: number) => void;
onimport: (file: File) => void; onimport: (file: File, seal?: Seal) => void;
onassign: () => void; onassign: () => void;
onstrikeface: (core: string, face: number) => void; onstrikeface: (core: string, face: number) => void;
onstep: (row: number, step: number, on: boolean) => void; onstep: (row: number, step: number, on: boolean) => void;
@@ -36,6 +37,11 @@
let fileInput: HTMLInputElement | null = null; let fileInput: HTMLInputElement | null = null;
let wave: HTMLCanvasElement | null = $state(null); let wave: HTMLCanvasElement | null = $state(null);
let recording = $state(false);
let recordSeconds = $state(0);
let recordError = $state<string | null>(null);
let finishing = $state(false);
let takeIndex = $state(1);
let observer: ResizeObserver | null = null; let observer: ResizeObserver | null = null;
const core = $derived(cores.find((item) => item.id === selectedCore) ?? cores[0] ?? null); const core = $derived(cores.find((item) => item.id === selectedCore) ?? cores[0] ?? null);
@@ -117,7 +123,53 @@
} }
function pickFile() { function pickFile() {
fileInput?.click(); if (!importing && !recording) fileInput?.click();
}
function formatTime(seconds: number): string {
const whole = Math.floor(seconds);
return `${Math.floor(whole / 60)}:${String(whole % 60).padStart(2, '0')}`;
}
async function toggleRecording() {
if (finishing) return;
if (!recording) {
recordError = null;
recordSeconds = 0;
try {
await startRecording((seconds) => {
recordSeconds = seconds;
if (seconds >= MAX_RECORD_SECONDS) void finishRecording();
});
recording = true;
} catch (error) {
recordError =
error instanceof Error ? error.message : 'the microphone could not be opened';
}
return;
}
await finishRecording();
}
async function finishRecording() {
if (finishing || !recording) return;
finishing = true;
try {
const take = await stopRecording();
recording = false;
if (!take) {
recordError = 'nothing was captured';
return;
}
const name = `take-${String(takeIndex).padStart(2, '0')}.wav`;
takeIndex += 1;
onimport(new File([take.blob], name, { type: 'audio/wav' }), 'recorded');
} catch (error) {
recording = false;
recordError = error instanceof Error ? error.message : 'the take could not be closed';
} finally {
finishing = false;
}
} }
function fileChanged() { function fileChanged() {
@@ -143,7 +195,20 @@
<span class="import-state"> <span class="import-state">
{importing ? 'cutting…' : `${cores.length} core${cores.length === 1 ? '' : 's'}`} {importing ? 'cutting…' : `${cores.length} core${cores.length === 1 ? '' : 's'}`}
</span> </span>
<button class="import" onclick={pickFile} disabled={importing}>take a core</button> <button
class="record"
class:is-on={recording}
onclick={toggleRecording}
disabled={importing || finishing}
aria-label={recording ? 'stop recording' : 'record a take'}
title={recording ? 'stop and take the core' : 'record from the microphone'}
>
<span class="record-dot" aria-hidden="true"></span>
{recording ? `stop ${formatTime(recordSeconds)}` : 'record'}
</button>
<button class="import" onclick={pickFile} disabled={importing || recording}>
take a core
</button>
<input <input
class="file" class="file"
bind:this={fileInput} bind:this={fileInput}
@@ -160,11 +225,24 @@
<div class="quarry-empty"> <div class="quarry-empty">
<p class="empty-title">the archive is empty.</p> <p class="empty-title">the archive is empty.</p>
<p class="empty-copy"> <p class="empty-copy">
Import a field recording. The quarry finds the seams and cuts sixteen faces. They can be Import a field recording or record a take. The quarry finds the seams and cuts sixteen
struck on their own or laid on a vein. faces. They can be struck on their own or laid on a vein.
</p> </p>
<button class="empty-action" onclick={pickFile} disabled={importing}>take a core</button> <div class="empty-actions">
<p class="empty-formats">WAV · FLAC · MP3 · OGG</p> <button class="empty-action" onclick={pickFile} disabled={importing || recording}>
take a core
</button>
<button
class="record empty-record"
class:is-on={recording}
onclick={toggleRecording}
disabled={importing || finishing}
>
<span class="record-dot" aria-hidden="true"></span>
{recording ? `stop ${formatTime(recordSeconds)}` : 'record a take'}
</button>
</div>
<p class="empty-formats">WAV · FLAC · MP3 · OGG · or the microphone</p>
</div> </div>
{:else if core} {:else if core}
<div class="core-strip" aria-label="cores"> <div class="core-strip" aria-label="cores">
+123
View File
@@ -0,0 +1,123 @@
import workletUrl from './recorder-worklet.js?url';
export const MAX_RECORD_SECONDS = 30;
interface ActiveRecorder {
context: AudioContext;
stream: MediaStream;
node: AudioWorkletNode;
mute: GainNode;
chunks: Float32Array[];
started: number;
timer: number;
}
let active: ActiveRecorder | null = null;
export async function startRecording(tick: (seconds: number) => void): Promise<void> {
if (active) return;
const stream = await navigator.mediaDevices.getUserMedia({
audio: {
channelCount: 1,
echoCancellation: false,
noiseSuppression: false,
autoGainControl: false
}
});
const context = new AudioContext({ sampleRate: 48000, latencyHint: 'interactive' });
try {
await context.audioWorklet.addModule(workletUrl);
} catch (error) {
stream.getTracks().forEach((track) => track.stop());
await context.close();
throw error;
}
const source = context.createMediaStreamSource(stream);
const node = new AudioWorkletNode(context, 'zhila-recorder', {
numberOfInputs: 1,
numberOfOutputs: 1,
outputChannelCount: [1]
});
const chunks: Float32Array[] = [];
node.port.onmessage = (event: MessageEvent<Float32Array>) => {
if (event.data instanceof Float32Array) chunks.push(event.data);
};
// The processor needs a path to the destination to run; the mute keeps the
// microphone out of the speakers.
const mute = context.createGain();
mute.gain.value = 0;
source.connect(node);
node.connect(mute);
mute.connect(context.destination);
if (context.state === 'suspended') await context.resume();
const started = performance.now();
const timer = window.setInterval(() => {
tick(Math.min(MAX_RECORD_SECONDS, (performance.now() - started) / 1000));
}, 100);
active = { context, stream, node, mute, chunks, started, timer };
}
export async function stopRecording(): Promise<{
blob: Blob;
seconds: number;
sampleRate: number;
} | null> {
if (!active) return null;
const recorder = active;
active = null;
window.clearInterval(recorder.timer);
recorder.node.port.onmessage = null;
recorder.node.disconnect();
recorder.mute.disconnect();
recorder.stream.getTracks().forEach((track) => track.stop());
const seconds = (performance.now() - recorder.started) / 1000;
const sampleRate = recorder.context.sampleRate;
await recorder.context.close();
const length = recorder.chunks.reduce((total, chunk) => total + chunk.length, 0);
if (length === 0) return null;
const samples = new Float32Array(length);
let offset = 0;
for (const chunk of recorder.chunks) {
samples.set(chunk, offset);
offset += chunk.length;
}
return { blob: encodeWav(samples, sampleRate), seconds, sampleRate };
}
export function isRecording(): boolean {
return active !== null;
}
function encodeWav(samples: Float32Array, sampleRate: number): Blob {
const channels = 1;
const bytesPerSample = 2;
const dataBytes = samples.length * bytesPerSample;
const buffer = new ArrayBuffer(44 + dataBytes);
const view = new DataView(buffer);
const text = (offset: number, value: string) => {
for (let index = 0; index < value.length; index += 1) {
view.setUint8(offset + index, value.charCodeAt(index));
}
};
text(0, 'RIFF');
view.setUint32(4, 36 + dataBytes, true);
text(8, 'WAVE');
text(12, 'fmt ');
view.setUint32(16, 16, true);
view.setUint16(20, 1, true);
view.setUint16(22, channels, true);
view.setUint32(24, sampleRate, true);
view.setUint32(28, sampleRate * channels * bytesPerSample, true);
view.setUint16(32, channels * bytesPerSample, true);
view.setUint16(34, bytesPerSample * 8, true);
text(36, 'data');
view.setUint32(40, dataBytes, true);
let offset = 44;
for (let index = 0; index < samples.length; index += 1) {
const sample = Math.max(-1, Math.min(1, samples[index]));
view.setInt16(offset, sample < 0 ? sample * 0x8000 : sample * 0x7fff, true);
offset += 2;
}
return new Blob([buffer], { type: 'audio/wav' });
}
+11
View File
@@ -0,0 +1,11 @@
// Runs on the audio thread. Copies each render quantum; the main thread owns
// the take buffer and the WAV encoding.
class ZhilaRecorderProcessor extends AudioWorkletProcessor {
process(inputs) {
const channel = inputs[0] && inputs[0][0];
if (channel && channel.length) this.port.postMessage(channel.slice(0));
return true;
}
}
registerProcessor('zhila-recorder', ZhilaRecorderProcessor);
+1 -1
View File
@@ -1,7 +1,7 @@
// Wire types for protocol v1. Kept in step with crates/zhila-protocol. // Wire types for protocol v1. Kept in step with crates/zhila-protocol.
export type VeinId = 'bass' | 'crack' | 'voice' | 'noise' | 'silt'; export type VeinId = 'bass' | 'crack' | 'voice' | 'noise' | 'silt';
export type Status = 'connecting' | 'live' | 'offline'; export type Status = 'connecting' | 'live' | 'offline';
export type Seal = 'found' | 'taken' | 'gift' | 'loan' | 'unknown'; export type Seal = 'found' | 'recorded' | 'taken' | 'gift' | 'loan' | 'unknown';
export interface FaceSummary { export interface FaceSummary {
index: number; index: number;