P1c: microphone recording through the material pipeline
This commit is contained in:
@@ -2,7 +2,7 @@
|
||||
|
||||
Sound is terrain. A sampling instrument where material is cut in a quarry, laid as stones in a nave, and played by a hand or an agent through the same verbs.
|
||||
|
||||
This repository is **P1b**: the engine, the wire, the descent, the material room, and a project that survives a restart. A found sound can be imported, cut into sixteen faces, laid on a vein and heard in the loop; tempo, gains and pattern edits are autosaved and reversible. The visual demo in [`reference/demo`](reference/demo) remains a design reference, not the codebase. The roadmap is in [`docs/PLAN.md`](docs/PLAN.md).
|
||||
This repository is **P1c**: the engine, the wire, the descent, the material room, a project that survives a restart, and a microphone. A found or recorded sound is cut into sixteen faces, laid on a vein and heard in the loop; tempo, gains and pattern edits are autosaved and reversible. The visual demo in [`reference/demo`](reference/demo) remains a design reference, not the codebase. The roadmap is in [`docs/PLAN.md`](docs/PLAN.md).
|
||||
|
||||

|
||||
|
||||
@@ -10,12 +10,15 @@ This repository is **P1b**: the engine, the wire, the descent, the material room
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
## What is real
|
||||
|
||||
- **Rust engine.** `cpal` output at 48 kHz, five veins, a sixteen-step phrase, a fixed voice pool, per-vein levels and gains, a bar clock. Procedural voices for empty pattern rows; real sample voices for rows with a face. No Web Audio.
|
||||
- **Material.** Import through the local wire; `symphonia` decodes WAV/FLAC/MP3/OGG; onset analysis finds seams and cuts sixteen faces with RMS, rank and waveform peaks; the archive keeps the mono f32 samples. The sample rate is preserved and the engine handles the playback ratio.
|
||||
- **Project.** A portable folder: `project.db` (SQLite) plus `assets/`. On daemon start it loads the cores, pattern, tempo, gains and journal; every project edit autosaves. The default is `~/.local/share/zhila/descent-04`; use `--project PATH` for another. The project name is shown in the top rail.
|
||||
- **Undo/redo.** Tempo, gains and pattern steps/faces push a small history; `Ctrl/⌘+Z` and `Shift+Ctrl/⌘+Z`, or the rail buttons. The journal records every edit. History is per session: after a reload the project is restored, the undo stack starts empty.
|
||||
- **Recording.** `record` in the quarry opens the microphone, captures mono PCM through an AudioWorklet, encodes a 16-bit WAV in the surface and sends it through the same import path. The take becomes a core with sixteen faces, seal `recorded`, and is autosaved in the project. Stops at 30 s; no monitoring.
|
||||
- **Quarry.** Import, core strip, zoomed waveform with face boundaries, sixteen-face wall, face audition, seal and stats; assign a face to a pattern row; edit the sixteen steps; strike a face; all against the live session.
|
||||
- **Protocol v1.** JSON control frames, a 31-byte binary meter frame, and a binary import path (an import header followed by the audio bytes). Snapshot, results, events, token and origin guard.
|
||||
- **Session + journal.** Validation, revisions, a journal for project edits (tempo, gains, pattern faces and steps); play, stop and strikes are performance, not history.
|
||||
@@ -45,7 +48,7 @@ npm --prefix ui run dev
|
||||
|
||||
1. Open **quarry** in the top rail.
|
||||
2. `take a core` and choose a WAV (a short field recording works; the fixture `fixtures/clicks-01.wav` is one).
|
||||
3. The file is decoded, the seams are found and sixteen faces are cut. Click a face, then `lay on bass`.
|
||||
3. The file is decoded, the seams are found and sixteen faces are cut. Or press `record`, speak/play into the microphone, press `stop`; the take arrives the same way. Click a face, then `lay on bass`.
|
||||
4. `space` plays the loop; the bass row now triggers the sample while the other veins keep their procedural voices. Edit steps in the pattern; each edit goes to the engine and the journal.
|
||||
5. Go back to **nave** to see the pattern on the road and the levels in the room.
|
||||
|
||||
@@ -58,7 +61,7 @@ cd app
|
||||
./src-tauri/target/release/zhila-app
|
||||
```
|
||||
|
||||
The built window embeds the surface and connects to the daemon at `ws://127.0.0.1:4317`; start the daemon first. `tauri dev` uses the Vite server at `127.0.0.1:5173`. The engine is still the separate daemon in this phase. Rebuild the native app after any UI change; the binary embeds `ui/dist` at compile time.
|
||||
The built window embeds the surface and connects to the daemon at `ws://127.0.0.1:4317`; start the daemon first. `tauri dev` uses the Vite server at `127.0.0.1:5173`. The engine is still the separate daemon in this phase. Rebuild the native app after any UI change; the binary embeds `ui/dist` at compile time. The recording UI is in the shared surface; WebKitGTK microphone permission still needs a manual check on this machine.
|
||||
|
||||
Controls: `space` play/stop · `s` strike the selected face in the quarry or the selected vein in the nave · `1`–`5` select a vein (nave) or a pattern row (quarry) · click a face to select, double-click to strike · tempo `−` / `+`.
|
||||
|
||||
@@ -90,19 +93,21 @@ node scripts/persistence-smoke.mjs # spawned daemon: import/edit/undo/restart
|
||||
npm --prefix ui run smoke # daemon + Vite: P0 regression, screenshots
|
||||
npm --prefix ui run quarry-smoke # daemon + Vite: import, assign, play, screenshots
|
||||
npm --prefix ui run project-smoke # daemon + Vite: project, undo, import, screenshots
|
||||
npm --prefix ui run record-smoke # daemon + Vite: fake microphone take, faces, seal
|
||||
npm --prefix ui run restored-smoke # after restarting the daemon on the saved project
|
||||
```
|
||||
|
||||
`npm --prefix ui run smoke` writes `preview-p0*.png`; `quarry-smoke` writes `preview-p1-*.png`.
|
||||
|
||||
## P1b limits (next, not hidden)
|
||||
## P1c limits (next, not hidden)
|
||||
|
||||
- The project persists, but undo history does not: after a reload the undo stack starts empty. The journal is stored for audit, not reconstructed into undo.
|
||||
- No recording, no waveform trimming, no manual cut/loop points; the sixteen faces are automatic seams.
|
||||
- Recording is mono, 16-bit, requested 48 kHz, up to 30 s, no monitoring and no input device choice. WebKitGTK/Tauri microphone permission is untested; the browser path is verified with Chromium's fake capture.
|
||||
- No waveform trimming, no manual cut/loop points; the sixteen faces are automatic seams.
|
||||
- One fixed two-bar phrase and five veins; pattern rows are veins, one face per row, no per-step face or pitch.
|
||||
- The Tauri binary currently embeds an older surface build; rebuild it after UI changes (`npm --prefix ui run build`, then `tauri build`). The engine remains the separate daemon.
|
||||
- Formats depend on symphonia; WAV is the tested path. AIFF, multi-channel files and very long files are not specifically handled.
|
||||
- No agent/MCP in this phase.
|
||||
- The Tauri shell now builds and runs; the engine is still the separate daemon (embedding is next).
|
||||
|
||||
The next slice is P1c: recording, waveform trimming, reconstructed undo across reloads, and embedding the engine inside the Tauri backend. Then the arrangement and the agent. The plan is in [`docs/PLAN.md`](docs/PLAN.md); the phase notes are in [`docs/P1.md`](docs/P1.md).
|
||||
The next slice is P1d: waveform trimming and manual faces, reconstructed undo across reloads, and embedding the engine inside the Tauri backend. Then the arrangement and the agent. The plan is in [`docs/PLAN.md`](docs/PLAN.md); the phase notes are in [`docs/P1.md`](docs/P1.md).
|
||||
|
||||
@@ -83,6 +83,7 @@ pub enum Actor {
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum Seal {
|
||||
Found,
|
||||
Recorded,
|
||||
Taken,
|
||||
Gift,
|
||||
Loan,
|
||||
|
||||
+45
@@ -98,3 +98,48 @@ Screenshots: `preview-p1b-quarry.png`, `preview-p1b-restored.png`.
|
||||
- Only the source bytes are archived; trimming/recording will add derived assets.
|
||||
- The engine is still the standalone daemon; the Tauri backend does not host it yet.
|
||||
- A failed autosave logs a warning and leaves the in-memory session running; the surface does not yet show a save error.
|
||||
|
||||
---
|
||||
|
||||
# P1c · recording
|
||||
|
||||
The quarry can now take a microphone take and turn it into material.
|
||||
|
||||
## Path
|
||||
|
||||
```
|
||||
record click
|
||||
→ getUserMedia({ mono, no AGC/NS/echo })
|
||||
→ AudioContext (requested 48 kHz) → AudioWorklet 'zhila-recorder'
|
||||
→ Float32 chunks on the main thread
|
||||
→ 16-bit PCM WAV encoder (in the surface)
|
||||
→ the existing binary import path with seal 'recorded'
|
||||
→ material decode + onset analysis + sixteen faces
|
||||
→ project asset + autosave
|
||||
```
|
||||
|
||||
The worklet copies each 128-frame render quantum; the main thread owns the growing chunk list so the audio thread never allocates the take. A mute GainNode keeps the input out of the speakers. The take name is `take-01.wav`, `take-02.wav`, …; the core name drops the extension. The recorder stops at 30 s; the user can stop earlier. Errors (permission denied, no device, worklet failure) surface under the quarry header.
|
||||
|
||||
The WAV is mono 16-bit PCM at the AudioContext rate (requested 48 kHz). The material crate already handles WAV; no daemon or protocol change was needed beyond `Seal::Recorded`.
|
||||
|
||||
## UI
|
||||
|
||||
- `record` in the quarry header and in the empty state: dot + label; `stop 0:12` while active with a pulsing dot.
|
||||
- `take a core` is disabled while recording; import is disabled while recording.
|
||||
- The take is imported with the recorded seal; the core strip and side facts show `recorded`.
|
||||
- The timer is in the button; no modal, no separate recorder panel.
|
||||
|
||||
## Verification
|
||||
|
||||
- `npm --prefix ui run record-smoke` launches Chromium with `--use-fake-ui-for-media-stream --use-fake-device-for-media-stream --use-file-for-fake-audio-capture=fixtures/clicks-01.wav`, records 2.3 s through the real UI/worklet/encoder, stops, waits for `take-01`, and checks 16 faces, the recorded seal, and the wire import.
|
||||
- The take is persisted in `~/.local/share/zhila/descent-04` (`core-4525165cd5.wav`, 16 faces, seal recorded) and shown by the state.
|
||||
- `./scripts/check.sh` remains green.
|
||||
|
||||
Screenshot: `preview-p1c-record.png`.
|
||||
|
||||
## Limits
|
||||
|
||||
- Mono, 16-bit, 30 s max, no monitor, no device picker, no gain control.
|
||||
- WebKitGTK/Tauri microphone support is untested; the Tauri binary also embeds an older surface build and must be rebuilt to see the record button.
|
||||
- No trim, fade or manual loop points yet; the take is cut by the automatic seams.
|
||||
- The take is stored as the encoded WAV, not the float PCM; trimming will later add derived assets.
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
# ZHILA — план проекта
|
||||
|
||||
Черновик v1.1 · 1 ноября 2026
|
||||
Статус: **P0, P1a и P1b выполнены** (см. `P0.md`, `P1.md`). Демо `reference/demo/` — референс, не кодовая база.
|
||||
Статус: **P0, P1a, P1b и P1c выполнены** (см. `P0.md`, `P1.md`). Демо `reference/demo/` — референс, не кодовая база.
|
||||
|
||||
**Зафиксировано 1 ноября:** descent — основной вид; камень = паттерн; климат — на жилу (концепт погоды остаётся); платформа v1 — Linux; UI — Svelte 5. Журнал — §9.
|
||||
|
||||
|
||||
@@ -152,3 +152,7 @@ Events: `pattern_step`, `pattern_face`, `face_struck`.
|
||||
`State` adds `project_name` (string), `can_undo` and `can_redo` (bools). Commands `undo` and `redo` take no arguments; event `history { can_undo, can_redo }` is broadcast after them.
|
||||
|
||||
The project is loaded by the daemon from its `--project` folder and saved after every project command or import. The protocol only exposes the project name; the path, SQLite file and assets stay server-side. There is no explicit save command: autosave is the contract. The journal is persisted, but undo history is not reconstructed on load.
|
||||
|
||||
## P1c addition
|
||||
|
||||
`Seal` gains `recorded`. Recording introduces no new wire messages: the surface encodes a WAV and uses the existing `import_core` header + binary payload. The recorded seal is persisted with the core row and is visible in the snapshot.
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 321 KiB |
@@ -11,6 +11,7 @@
|
||||
"smoke": "node scripts/smoke.mjs",
|
||||
"quarry-smoke": "node scripts/quarry-smoke.mjs",
|
||||
"project-smoke": "node scripts/project-smoke.mjs",
|
||||
"record-smoke": "node scripts/record-smoke.mjs",
|
||||
"restored-smoke": "node scripts/restored-smoke.mjs"
|
||||
},
|
||||
"devDependencies": {
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
import { chromium } from 'playwright-core';
|
||||
import path from 'node:path';
|
||||
|
||||
const chrome = process.env.CHROME_PATH ?? '/home/emil/.cache/ms-playwright/chromium-1248/chrome-linux64/chrome';
|
||||
const fixture = path.resolve('../fixtures/clicks-01.wav');
|
||||
const browser = await chromium.launch({
|
||||
executablePath: chrome,
|
||||
args: [
|
||||
'--no-sandbox',
|
||||
'--autoplay-policy=no-user-gesture-required',
|
||||
'--use-fake-ui-for-media-stream',
|
||||
'--use-fake-device-for-media-stream',
|
||||
`--use-file-for-fake-audio-capture=${fixture}`
|
||||
]
|
||||
});
|
||||
const page = await browser.newPage({ viewport: { width: 1600, height: 1000 }, deviceScaleFactor: 1 });
|
||||
const errors = [];
|
||||
page.on('pageerror', (error) => errors.push(`pageerror: ${error.message}`));
|
||||
page.on('console', (message) => { if (message.type() === 'error') errors.push(`console: ${message.text()}`); });
|
||||
|
||||
await page.goto('http://127.0.0.1:5173/', { waitUntil: 'load' });
|
||||
await page.waitForTimeout(2200);
|
||||
await page.locator('.room', { hasText: 'quarry' }).click();
|
||||
await page.waitForTimeout(500);
|
||||
const record = page.locator('.quarry-actions .record');
|
||||
await record.click();
|
||||
await page.waitForSelector('.quarry-actions .record.is-on', { timeout: 5000 });
|
||||
await page.waitForTimeout(2300);
|
||||
const during = await record.textContent();
|
||||
await record.click();
|
||||
await page.waitForFunction(
|
||||
() => [...document.querySelectorAll('.core-name')].some((el) => el.textContent?.includes('take-01')),
|
||||
null,
|
||||
{ timeout: 25000 }
|
||||
);
|
||||
await page.waitForTimeout(900);
|
||||
await page.screenshot({ path: '../preview-p1c-record.png' });
|
||||
|
||||
const report = await page.evaluate(() => ({
|
||||
cores: [...document.querySelectorAll('.core-name')].map((el) => el.textContent),
|
||||
faces: document.querySelectorAll('.face').length,
|
||||
selected: document.querySelector('.core-tab.is-on .core-name')?.textContent,
|
||||
stats: document.querySelector('.face-stats')?.textContent?.replace(/\s+/g, ' ').trim(),
|
||||
wire: [...document.querySelectorAll('.wire-label')].map((el) => el.textContent).slice(-6),
|
||||
recordText: document.querySelector('.quarry-actions .record')?.textContent?.trim()
|
||||
}));
|
||||
console.log(JSON.stringify({ during, report, errors }, null, 2));
|
||||
|
||||
const failures = [];
|
||||
if (!during || !during.includes('stop')) failures.push('record button did not switch to stop');
|
||||
if (!report.cores.includes('take-01')) failures.push('take-01 was not imported');
|
||||
if (report.faces !== 16) failures.push('take did not produce 16 faces');
|
||||
if (!report.stats || !report.stats.includes('recorded')) failures.push('seal is not recorded');
|
||||
if (errors.length) failures.push('console errors');
|
||||
await browser.close();
|
||||
if (failures.length) {
|
||||
console.error('RECORD SMOKE FAILED: ' + failures.join('; '));
|
||||
process.exit(1);
|
||||
}
|
||||
console.log('record smoke passed');
|
||||
+3
-2
@@ -15,6 +15,7 @@
|
||||
Event,
|
||||
HelloMessage,
|
||||
MeterFrame,
|
||||
Seal,
|
||||
State,
|
||||
Status,
|
||||
VeinId,
|
||||
@@ -143,13 +144,13 @@
|
||||
void run({ type: 'set_vein_gain', vein, gain });
|
||||
}
|
||||
|
||||
async function importCore(file: File) {
|
||||
async function importCore(file: File, seal: Seal = 'found') {
|
||||
if (!bridge || importing) return;
|
||||
importing = true;
|
||||
addEntry('send', `import ${file.name}`);
|
||||
try {
|
||||
const data = await file.arrayBuffer();
|
||||
const core = await bridge.importCore(file.name, 'found', data);
|
||||
const core = await bridge.importCore(file.name, seal, data);
|
||||
selectedCoreId = core.id;
|
||||
selectedFace = core.faces.find((face) => face.rank === 0)?.index ?? 0;
|
||||
addEntry('recv', 'core', `${core.name} · ${core.faces.length} faces · ${core.duration_sec.toFixed(1)} s`);
|
||||
|
||||
@@ -1167,6 +1167,71 @@ button.room {
|
||||
cursor: wait;
|
||||
}
|
||||
|
||||
.record {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
padding: 7px 10px;
|
||||
border: 1px solid rgba(155, 133, 121, 0.28);
|
||||
background: transparent;
|
||||
color: var(--bone);
|
||||
font-size: 9.5px;
|
||||
letter-spacing: 0.03em;
|
||||
transition: border-color 0.2s ease, color 0.2s ease, background 0.2s ease;
|
||||
}
|
||||
|
||||
.record:hover:not(:disabled) {
|
||||
border-color: var(--ember);
|
||||
}
|
||||
|
||||
.record.is-on {
|
||||
border-color: var(--ember);
|
||||
background: rgba(232, 64, 42, 0.06);
|
||||
color: var(--ember);
|
||||
}
|
||||
|
||||
.record:disabled {
|
||||
opacity: 0.4;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.record-dot {
|
||||
width: 5px;
|
||||
height: 5px;
|
||||
border-radius: 50%;
|
||||
background: var(--ember);
|
||||
}
|
||||
|
||||
.record.is-on .record-dot {
|
||||
animation: pulse-dot 1.1s ease-in-out infinite;
|
||||
}
|
||||
|
||||
@keyframes pulse-dot {
|
||||
0%,
|
||||
100% {
|
||||
opacity: 1;
|
||||
box-shadow: 0 0 0 0 rgba(232, 64, 42, 0.5);
|
||||
}
|
||||
50% {
|
||||
opacity: 0.4;
|
||||
box-shadow: 0 0 0 4px rgba(232, 64, 42, 0);
|
||||
}
|
||||
}
|
||||
|
||||
.record-error {
|
||||
margin: 8px 0 0;
|
||||
font-size: 9px;
|
||||
line-height: 1.6;
|
||||
color: var(--ember);
|
||||
}
|
||||
|
||||
.empty-actions {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
justify-content: center;
|
||||
gap: 10px;
|
||||
}
|
||||
|
||||
.file {
|
||||
position: absolute;
|
||||
width: 1px;
|
||||
@@ -1795,12 +1860,21 @@ button.room {
|
||||
.quarry-head h1 {
|
||||
font-size: 26px;
|
||||
}
|
||||
.quarry-head {
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
.quarry-actions {
|
||||
width: 100%;
|
||||
justify-content: flex-end;
|
||||
gap: 6px;
|
||||
}
|
||||
.import-state {
|
||||
display: none;
|
||||
}
|
||||
.record {
|
||||
padding: 6px 8px;
|
||||
font-size: 8.5px;
|
||||
}
|
||||
.face-wall {
|
||||
grid-template-columns: repeat(4, minmax(0, 1fr));
|
||||
}
|
||||
|
||||
@@ -2,7 +2,8 @@
|
||||
|
||||
<script lang="ts">
|
||||
import { onMount } from 'svelte';
|
||||
import type { CoreSummary, Pattern } from '../lib/types';
|
||||
import type { CoreSummary, Pattern, Seal } from '../lib/types';
|
||||
import { MAX_RECORD_SECONDS, startRecording, stopRecording } from '../lib/record';
|
||||
|
||||
let {
|
||||
cores,
|
||||
@@ -28,7 +29,7 @@
|
||||
onselectcore: (id: string) => void;
|
||||
onselectface: (face: number) => void;
|
||||
onselectrow: (row: number) => void;
|
||||
onimport: (file: File) => void;
|
||||
onimport: (file: File, seal?: Seal) => void;
|
||||
onassign: () => void;
|
||||
onstrikeface: (core: string, face: number) => void;
|
||||
onstep: (row: number, step: number, on: boolean) => void;
|
||||
@@ -36,6 +37,11 @@
|
||||
|
||||
let fileInput: HTMLInputElement | null = null;
|
||||
let wave: HTMLCanvasElement | null = $state(null);
|
||||
let recording = $state(false);
|
||||
let recordSeconds = $state(0);
|
||||
let recordError = $state<string | null>(null);
|
||||
let finishing = $state(false);
|
||||
let takeIndex = $state(1);
|
||||
let observer: ResizeObserver | null = null;
|
||||
|
||||
const core = $derived(cores.find((item) => item.id === selectedCore) ?? cores[0] ?? null);
|
||||
@@ -117,7 +123,53 @@
|
||||
}
|
||||
|
||||
function pickFile() {
|
||||
fileInput?.click();
|
||||
if (!importing && !recording) fileInput?.click();
|
||||
}
|
||||
|
||||
function formatTime(seconds: number): string {
|
||||
const whole = Math.floor(seconds);
|
||||
return `${Math.floor(whole / 60)}:${String(whole % 60).padStart(2, '0')}`;
|
||||
}
|
||||
|
||||
async function toggleRecording() {
|
||||
if (finishing) return;
|
||||
if (!recording) {
|
||||
recordError = null;
|
||||
recordSeconds = 0;
|
||||
try {
|
||||
await startRecording((seconds) => {
|
||||
recordSeconds = seconds;
|
||||
if (seconds >= MAX_RECORD_SECONDS) void finishRecording();
|
||||
});
|
||||
recording = true;
|
||||
} catch (error) {
|
||||
recordError =
|
||||
error instanceof Error ? error.message : 'the microphone could not be opened';
|
||||
}
|
||||
return;
|
||||
}
|
||||
await finishRecording();
|
||||
}
|
||||
|
||||
async function finishRecording() {
|
||||
if (finishing || !recording) return;
|
||||
finishing = true;
|
||||
try {
|
||||
const take = await stopRecording();
|
||||
recording = false;
|
||||
if (!take) {
|
||||
recordError = 'nothing was captured';
|
||||
return;
|
||||
}
|
||||
const name = `take-${String(takeIndex).padStart(2, '0')}.wav`;
|
||||
takeIndex += 1;
|
||||
onimport(new File([take.blob], name, { type: 'audio/wav' }), 'recorded');
|
||||
} catch (error) {
|
||||
recording = false;
|
||||
recordError = error instanceof Error ? error.message : 'the take could not be closed';
|
||||
} finally {
|
||||
finishing = false;
|
||||
}
|
||||
}
|
||||
|
||||
function fileChanged() {
|
||||
@@ -143,7 +195,20 @@
|
||||
<span class="import-state">
|
||||
{importing ? 'cutting…' : `${cores.length} core${cores.length === 1 ? '' : 's'}`}
|
||||
</span>
|
||||
<button class="import" onclick={pickFile} disabled={importing}>take a core</button>
|
||||
<button
|
||||
class="record"
|
||||
class:is-on={recording}
|
||||
onclick={toggleRecording}
|
||||
disabled={importing || finishing}
|
||||
aria-label={recording ? 'stop recording' : 'record a take'}
|
||||
title={recording ? 'stop and take the core' : 'record from the microphone'}
|
||||
>
|
||||
<span class="record-dot" aria-hidden="true"></span>
|
||||
{recording ? `stop ${formatTime(recordSeconds)}` : 'record'}
|
||||
</button>
|
||||
<button class="import" onclick={pickFile} disabled={importing || recording}>
|
||||
take a core
|
||||
</button>
|
||||
<input
|
||||
class="file"
|
||||
bind:this={fileInput}
|
||||
@@ -160,11 +225,24 @@
|
||||
<div class="quarry-empty">
|
||||
<p class="empty-title">the archive is empty.</p>
|
||||
<p class="empty-copy">
|
||||
Import a field recording. The quarry finds the seams and cuts sixteen faces. They can be
|
||||
struck on their own or laid on a vein.
|
||||
Import a field recording or record a take. The quarry finds the seams and cuts sixteen
|
||||
faces. They can be struck on their own or laid on a vein.
|
||||
</p>
|
||||
<button class="empty-action" onclick={pickFile} disabled={importing}>take a core</button>
|
||||
<p class="empty-formats">WAV · FLAC · MP3 · OGG</p>
|
||||
<div class="empty-actions">
|
||||
<button class="empty-action" onclick={pickFile} disabled={importing || recording}>
|
||||
take a core
|
||||
</button>
|
||||
<button
|
||||
class="record empty-record"
|
||||
class:is-on={recording}
|
||||
onclick={toggleRecording}
|
||||
disabled={importing || finishing}
|
||||
>
|
||||
<span class="record-dot" aria-hidden="true"></span>
|
||||
{recording ? `stop ${formatTime(recordSeconds)}` : 'record a take'}
|
||||
</button>
|
||||
</div>
|
||||
<p class="empty-formats">WAV · FLAC · MP3 · OGG · or the microphone</p>
|
||||
</div>
|
||||
{:else if core}
|
||||
<div class="core-strip" aria-label="cores">
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
import workletUrl from './recorder-worklet.js?url';
|
||||
|
||||
export const MAX_RECORD_SECONDS = 30;
|
||||
|
||||
interface ActiveRecorder {
|
||||
context: AudioContext;
|
||||
stream: MediaStream;
|
||||
node: AudioWorkletNode;
|
||||
mute: GainNode;
|
||||
chunks: Float32Array[];
|
||||
started: number;
|
||||
timer: number;
|
||||
}
|
||||
|
||||
let active: ActiveRecorder | null = null;
|
||||
|
||||
export async function startRecording(tick: (seconds: number) => void): Promise<void> {
|
||||
if (active) return;
|
||||
const stream = await navigator.mediaDevices.getUserMedia({
|
||||
audio: {
|
||||
channelCount: 1,
|
||||
echoCancellation: false,
|
||||
noiseSuppression: false,
|
||||
autoGainControl: false
|
||||
}
|
||||
});
|
||||
const context = new AudioContext({ sampleRate: 48000, latencyHint: 'interactive' });
|
||||
try {
|
||||
await context.audioWorklet.addModule(workletUrl);
|
||||
} catch (error) {
|
||||
stream.getTracks().forEach((track) => track.stop());
|
||||
await context.close();
|
||||
throw error;
|
||||
}
|
||||
const source = context.createMediaStreamSource(stream);
|
||||
const node = new AudioWorkletNode(context, 'zhila-recorder', {
|
||||
numberOfInputs: 1,
|
||||
numberOfOutputs: 1,
|
||||
outputChannelCount: [1]
|
||||
});
|
||||
const chunks: Float32Array[] = [];
|
||||
node.port.onmessage = (event: MessageEvent<Float32Array>) => {
|
||||
if (event.data instanceof Float32Array) chunks.push(event.data);
|
||||
};
|
||||
// The processor needs a path to the destination to run; the mute keeps the
|
||||
// microphone out of the speakers.
|
||||
const mute = context.createGain();
|
||||
mute.gain.value = 0;
|
||||
source.connect(node);
|
||||
node.connect(mute);
|
||||
mute.connect(context.destination);
|
||||
if (context.state === 'suspended') await context.resume();
|
||||
const started = performance.now();
|
||||
const timer = window.setInterval(() => {
|
||||
tick(Math.min(MAX_RECORD_SECONDS, (performance.now() - started) / 1000));
|
||||
}, 100);
|
||||
active = { context, stream, node, mute, chunks, started, timer };
|
||||
}
|
||||
|
||||
export async function stopRecording(): Promise<{
|
||||
blob: Blob;
|
||||
seconds: number;
|
||||
sampleRate: number;
|
||||
} | null> {
|
||||
if (!active) return null;
|
||||
const recorder = active;
|
||||
active = null;
|
||||
window.clearInterval(recorder.timer);
|
||||
recorder.node.port.onmessage = null;
|
||||
recorder.node.disconnect();
|
||||
recorder.mute.disconnect();
|
||||
recorder.stream.getTracks().forEach((track) => track.stop());
|
||||
const seconds = (performance.now() - recorder.started) / 1000;
|
||||
const sampleRate = recorder.context.sampleRate;
|
||||
await recorder.context.close();
|
||||
|
||||
const length = recorder.chunks.reduce((total, chunk) => total + chunk.length, 0);
|
||||
if (length === 0) return null;
|
||||
const samples = new Float32Array(length);
|
||||
let offset = 0;
|
||||
for (const chunk of recorder.chunks) {
|
||||
samples.set(chunk, offset);
|
||||
offset += chunk.length;
|
||||
}
|
||||
return { blob: encodeWav(samples, sampleRate), seconds, sampleRate };
|
||||
}
|
||||
|
||||
export function isRecording(): boolean {
|
||||
return active !== null;
|
||||
}
|
||||
|
||||
function encodeWav(samples: Float32Array, sampleRate: number): Blob {
|
||||
const channels = 1;
|
||||
const bytesPerSample = 2;
|
||||
const dataBytes = samples.length * bytesPerSample;
|
||||
const buffer = new ArrayBuffer(44 + dataBytes);
|
||||
const view = new DataView(buffer);
|
||||
const text = (offset: number, value: string) => {
|
||||
for (let index = 0; index < value.length; index += 1) {
|
||||
view.setUint8(offset + index, value.charCodeAt(index));
|
||||
}
|
||||
};
|
||||
text(0, 'RIFF');
|
||||
view.setUint32(4, 36 + dataBytes, true);
|
||||
text(8, 'WAVE');
|
||||
text(12, 'fmt ');
|
||||
view.setUint32(16, 16, true);
|
||||
view.setUint16(20, 1, true);
|
||||
view.setUint16(22, channels, true);
|
||||
view.setUint32(24, sampleRate, true);
|
||||
view.setUint32(28, sampleRate * channels * bytesPerSample, true);
|
||||
view.setUint16(32, channels * bytesPerSample, true);
|
||||
view.setUint16(34, bytesPerSample * 8, true);
|
||||
text(36, 'data');
|
||||
view.setUint32(40, dataBytes, true);
|
||||
let offset = 44;
|
||||
for (let index = 0; index < samples.length; index += 1) {
|
||||
const sample = Math.max(-1, Math.min(1, samples[index]));
|
||||
view.setInt16(offset, sample < 0 ? sample * 0x8000 : sample * 0x7fff, true);
|
||||
offset += 2;
|
||||
}
|
||||
return new Blob([buffer], { type: 'audio/wav' });
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
// Runs on the audio thread. Copies each render quantum; the main thread owns
|
||||
// the take buffer and the WAV encoding.
|
||||
class ZhilaRecorderProcessor extends AudioWorkletProcessor {
|
||||
process(inputs) {
|
||||
const channel = inputs[0] && inputs[0][0];
|
||||
if (channel && channel.length) this.port.postMessage(channel.slice(0));
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
registerProcessor('zhila-recorder', ZhilaRecorderProcessor);
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
// Wire types for protocol v1. Kept in step with crates/zhila-protocol.
|
||||
export type VeinId = 'bass' | 'crack' | 'voice' | 'noise' | 'silt';
|
||||
export type Status = 'connecting' | 'live' | 'offline';
|
||||
export type Seal = 'found' | 'taken' | 'gift' | 'loan' | 'unknown';
|
||||
export type Seal = 'found' | 'recorded' | 'taken' | 'gift' | 'loan' | 'unknown';
|
||||
|
||||
export interface FaceSummary {
|
||||
index: number;
|
||||
|
||||
Reference in New Issue
Block a user