Soundgen/README.md

Soundgen

8-bit sound synthesizer in Rust with AI-powered sound generation via MCP + human feedback loop.

Why Soundgen?

Most free 8-bit sound tools are either closed-source, hard to automate, or don't integrate with AI workflows. Soundgen fixes this:

• LLM generates sounds by calling MCP tools — no manual parameter tweaking
• You rate the results in the GUI — the AI learns from your feedback
• The AI improves — next time it generates a similar sound, it references your top-rated examples
• Export to WAV — drop the files straight into your game

┌──────┐  generate_batch   ┌─────────┐  rate 1-5★  ┌──────────┐
│  AI  │ ────────────────→ │ Sounds  │ ──────────→ │  You     │
└──────┘                   └─────────┘             └──────────┘
  ▲  uses top-rated examples as reference                 │
  │                                                       │
  └──────────────── ┌──────────────┐ ◄───────────────────┘
                    │ Feedback DB  │
                    └──────────────┘

Features

• Training pipeline: AI generates → you rate → AI learns (SQLite feedback database)
• 6 voice types: pulse (NES duty cycles), triangle, noise (LFSR), DPCM, wavetable, FM
• Effects: ADSR envelope, frequency sweep, biquad filter, vibrato (LFO)
• JSON-first: every sound is a SoundSpec JSON — LLMs generate JSON naturally
• MCP server: 5 tools for LLM integration (list_presets, generate_sfx, render_sound, generate_batch, get_reference_sounds)
• GUI: egui editor with virtual keyboard, preset browser, channel controls, training tab
• 13 built-in presets: SFX, UI, ambient — extend with JSON files, no recompilation
• Sequencer: pattern-based song playback (JSON)
• Runtime library: NES-authentic nonlinear DAC + SoundBank for game embedding
• No allocations in audio hot path: tick() -> f32

Quick Start

# CLI — list presets
cargo run --bin soundgen -- list-presets

# Generate from preset
cargo run --bin soundgen -- gen jump --out jump.wav

# Render custom spec
cargo run --bin soundgen -- render presets/sfx/laser.json --out laser.wav

# Render a song
cargo run --bin soundgen -- render-song song.json --out music.wav

# Launch the GUI editor
cargo run -p soundgen-gui

Training Pipeline (AI + Human Feedback)

1. Connect the MCP server

cargo run -p soundgen-mcp -- --presets-dir presets --db feedback.db

Add to your MCP client config (Claude Desktop, OpenCode, etc.):

{
  "mcpServers": {
    "soundgen": {
      "command": "/path/to/soundgen-mcp",
      "args": ["--presets-dir", "/path/to/presets", "--db", "/path/to/feedback.db"]
    }
  }
}

2. AI generates sounds

The AI calls MCP tools to create sounds:

Tool	What it does
list_presets	List available built-in presets
generate_sfx	Generate WAV from a named preset
render_sound	Generate WAV from a custom SoundSpec JSON — returns reference examples from your top-rated sounds
generate_batch	Generate multiple sounds, store in feedback DB (unrated)
get_reference_sounds	Search DB for highly-rated similar sounds

3. You rate them

Open the GUI → Training tab → play each sound, rate 1-5 stars, type feedback:

• "too loud" → AI lowers volume next time
• "pitch зачем-то возрастает" → AI fixes the sweep
• "Идеально!" → AI uses this as a reference for future sounds

4. AI improves

When the AI calls render_sound with name: "explosion", the MCP server searches the feedback DB for similar highly-rated sounds and returns their specs as reference examples. The AI sees what worked and adjusts its approach.

5. Export for fine-tuning (optional)

Once you have 100+ rated sounds, export them as a JSONL dataset for LoRA fine-tuning:

# In GUI: Training tab → "Export Dataset"
# Creates feedback_dataset.jsonl with all 4★+ sounds

SoundSpec JSON Format

{
  "name": "jump",
  "duration": 0.3,
  "sample_rate": 44100,
  "channels": [
    {
      "type": "pulse",
      "duty": 50,
      "frequency": { "start": 200, "end": 800, "curve": "exponential" },
      "envelope": { "attack": 0.01, "decay": 0.15, "sustain": 0.0, "release": 0.14 },
      "vibrato": { "rate": 8, "depth": 200 },
      "volume": 0.7
    }
  ]
}

Channel types: pulse, triangle, noise Optional fields: filter (lowpass/highpass + cutoff sweep), vibrato (LFO frequency modulation)

Architecture

Crate	Purpose
soundgen-core	Synthesis engine (generators, effects, mixer). No I/O.
soundgen-fmt	SoundSpec JSON schema + PresetRegistry
soundgen-io	WAV writer (hound) + playback
soundgen-seq	Sequencer: patterns, songs
soundgen-cli	gen, render, render-song, list-presets
soundgen-mcp	MCP server: 5 tools for LLM integration
soundgen-gui	egui GUI: editor + training tab
soundgen-feedback	SQLite feedback database + similarity search
soundgen-runtime	NES-authentic DAC + SoundBank for game embedding

Runtime Library (for game integration)

use soundgen_runtime::SoundBank;

// Load all presets at init time
let bank = SoundBank::load_dir(std::path::Path::new("presets"))?;

// Play by name (zero-allocation, returns pre-rendered buffer)
let (samples, sample_rate) = bank.get("jump").unwrap();

// Pitch-shifted variant
let (pitched, sr) = bank.get_pitched("jump", 1.5)?;

Built-in Presets

SFX: jump, explosion, coin, laser, hit, powerup UI: click, hover, confirm, error Ambient: wind, rain, drone

License

MIT