Soundgen/README.md

# Soundgen

8-bit sound synthesizer in Rust with AI-powered sound generation via MCP + human feedback loop.

## Why Soundgen?

Most free 8-bit sound tools are either closed-source, hard to automate, or don't integrate with AI workflows. Soundgen fixes this:

- **LLM generates sounds** by calling MCP tools — no manual parameter tweaking
- **You rate the results** in the GUI — the AI learns from your feedback
- **The AI improves** — next time it generates a similar sound, it references your top-rated examples
- **Export to WAV** — drop the files straight into your game

```
┌──────┐  generate_batch   ┌─────────┐  rate 1-5★  ┌──────────┐
│  AI  │ ────────────────→ │ Sounds  │ ──────────→ │  You     │
└──────┘                   └─────────┘             └──────────┘
  ▲  uses top-rated examples as reference                 │
  │                                                       │
  └──────────────── ┌──────────────┐ ◄───────────────────┘
                    │ Feedback DB  │
                    └──────────────┘
```

## Features

- **Training pipeline**: AI generates → you rate → AI learns (SQLite feedback database)
- **6 voice types**: pulse (NES duty cycles), triangle, noise (LFSR), DPCM, wavetable, FM
- **Effects**: ADSR envelope, frequency sweep, biquad filter, vibrato (LFO)
- **JSON-first**: every sound is a `SoundSpec` JSON — LLMs generate JSON naturally
- **MCP server**: 5 tools for LLM integration (`list_presets`, `generate_sfx`, `render_sound`, `generate_batch`, `get_reference_sounds`)
- **GUI**: egui editor with virtual keyboard, preset browser, channel controls, training tab
- **13 built-in presets**: SFX, UI, ambient — extend with JSON files, no recompilation
- **Sequencer**: pattern-based song playback (JSON)
- **Runtime library**: NES-authentic nonlinear DAC + SoundBank for game embedding
- **No allocations in audio hot path**: `tick() -> f32`

## Quick Start

```bash
# CLI — list presets
cargo run --bin soundgen -- list-presets

# Generate from preset
cargo run --bin soundgen -- gen jump --out jump.wav

# Render custom spec
cargo run --bin soundgen -- render presets/sfx/laser.json --out laser.wav

# Render a song
cargo run --bin soundgen -- render-song song.json --out music.wav

# Launch the GUI editor
cargo run -p soundgen-gui
```

## Training Pipeline (AI + Human Feedback)

### 1. Connect the MCP server

```bash
cargo run -p soundgen-mcp -- --presets-dir presets --db feedback.db
```

Add to your MCP client config (Claude Desktop, OpenCode, etc.):

```json
{
  "mcpServers": {
    "soundgen": {
      "command": "/path/to/soundgen-mcp",
      "args": ["--presets-dir", "/path/to/presets", "--db", "/path/to/feedback.db"]
    }
  }
}
```

### 2. AI generates sounds

The AI calls MCP tools to create sounds:

| Tool | What it does |
|---|---|
| `list_presets` | List available built-in presets |
| `generate_sfx` | Generate WAV from a named preset |
| `render_sound` | Generate WAV from a custom SoundSpec JSON — returns reference examples from your top-rated sounds |
| `generate_batch` | Generate multiple sounds, store in feedback DB (unrated) |
| `get_reference_sounds` | Search DB for highly-rated similar sounds |

### 3. You rate them

Open the GUI → **Training** tab → play each sound, rate 1-5 stars, type feedback:
- "too loud" → AI lowers volume next time
- "pitch зачем-то возрастает" → AI fixes the sweep
- "Идеально!" → AI uses this as a reference for future sounds

### 4. AI improves

When the AI calls `render_sound` with `name: "explosion"`, the MCP server searches the feedback DB for similar highly-rated sounds and returns their specs as reference examples. The AI sees what worked and adjusts its approach.

### 5. Export for fine-tuning (optional)

Once you have 100+ rated sounds, export them as a JSONL dataset for LoRA fine-tuning:

```bash
# In GUI: Training tab → "Export Dataset"
# Creates feedback_dataset.jsonl with all 4★+ sounds
```

## SoundSpec JSON Format

```json
{
  "name": "jump",
  "duration": 0.3,
  "sample_rate": 44100,
  "channels": [
    {
      "type": "pulse",
      "duty": 50,
      "frequency": { "start": 200, "end": 800, "curve": "exponential" },
      "envelope": { "attack": 0.01, "decay": 0.15, "sustain": 0.0, "release": 0.14 },
      "vibrato": { "rate": 8, "depth": 200 },
      "volume": 0.7
    }
  ]
}
```

Channel types: `pulse`, `triangle`, `noise`
Optional fields: `filter` (lowpass/highpass + cutoff sweep), `vibrato` (LFO frequency modulation)

## Architecture

| Crate | Purpose |
|---|---|
| `soundgen-core` | Synthesis engine (generators, effects, mixer). No I/O. |
| `soundgen-fmt` | `SoundSpec` JSON schema + `PresetRegistry` |
| `soundgen-io` | WAV writer (`hound`) + playback |
| `soundgen-seq` | Sequencer: patterns, songs |
| `soundgen-cli` | `gen`, `render`, `render-song`, `list-presets` |
| `soundgen-mcp` | MCP server: 5 tools for LLM integration |
| `soundgen-gui` | egui GUI: editor + training tab |
| `soundgen-feedback` | SQLite feedback database + similarity search |
| `soundgen-runtime` | NES-authentic DAC + `SoundBank` for game embedding |

## Runtime Library (for game integration)

```rust
use soundgen_runtime::SoundBank;

// Load all presets at init time
let bank = SoundBank::load_dir(std::path::Path::new("presets"))?;

// Play by name (zero-allocation, returns pre-rendered buffer)
let (samples, sample_rate) = bank.get("jump").unwrap();

// Pitch-shifted variant
let (pitched, sr) = bank.get_pitched("jump", 1.5)?;
```

## Built-in Presets

**SFX**: jump, explosion, coin, laser, hit, powerup
**UI**: click, hover, confirm, error
**Ambient**: wind, rain, drone

## License

MIT