emilandClaude Opus 4.7
75c9ee4576
C++ core (libmemba.so): - include/memba/state.h — C API (state_new/free/save/load/get_size) - src/state.cpp — MEMB file format: magic, version, SHA-256 model_id, CRC-32, opaque llama_state_*_data() blob - src/cli.cpp — minimal demo binary with greedy sampler - CMakeLists.txt + build.sh with llama.cpp submodule, CUDA auto-detect Python SDK (memba): - core.py — file I/O via llama-cpp-python's exposed C functions, unwraps _LlamaContext to access raw context pointer (≥0.3.x) - session.py — high-level Session with auto-save/load, ChatML wrapper for instruct models, raw mode for base models - cli.py — typer-based: chat (REPL), run (one-shot), list, rm, info Examples: - 01_basic_save_load.py, 02_chat_session.py Experiments (throwaway POCs documenting product-direction findings): - recall_poc.py — git log → state → cross-process query - mood_poc.py — batch sentiment trajectory, Mamba vs Transformer - mood_stream_poc.py, mood_batch_poc.py — variants - diag_saveload.py — minimal save/load isolation test - README.md documents the headline finding: save/load is byte-identical, but Falcon-Mamba-7B-Instruct does not retain facts across conversation turns even in-process — limits viable products to single-prompt analysis and persona priming. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
33 lines
1.6 KiB
Markdown
33 lines
1.6 KiB
Markdown
# experiments/
|
|
|
|
Throwaway scripts used to probe capabilities of SSM models with memba.
|
|
Not part of the library API — kept in the repo as reference and reproducible
|
|
evidence for product decisions.
|
|
|
|
Each script is self-contained and prints what it finds; read the source for
|
|
the test's claim and run it yourself if you want to verify on different
|
|
models or hardware.
|
|
|
|
## Scripts
|
|
|
|
| File | What it measures |
|
|
|------|------------------|
|
|
| `recall_poc.py` | Can a memba state, built from N days of git activity, answer "what did I work on last month" in a fresh process? |
|
|
| `mood_poc.py` | Batch sentiment-trajectory test (single prompt with full chat log). Compares Falcon-Mamba vs a Transformer. |
|
|
| `mood_stream_poc.py` | The same trajectory but fed turn-by-turn through `Session.chat()`, then queried cross-process. |
|
|
| `mood_batch_poc.py` | Batch ingest in build process, save, then query in a fresh process. |
|
|
| `diag_saveload.py` | Minimal diagnostic: tell the model one fact, ask it back before save, after save, after cross-process load. |
|
|
|
|
## Headline finding (2026-05-16, Falcon-Mamba-7B-Instruct Q4_K_M)
|
|
|
|
- **Batch single-prompt analysis** (all input + question in one call): works
|
|
for both sentiment and recall.
|
|
- **Multi-turn fact recall** (ingest in turn 1, ask in turn 2): fails even
|
|
in the *same process*. The model does not preserve specific facts in its
|
|
hidden state across conversation turns.
|
|
- **Save/load roundtrip**: byte-identical, no information loss attributable
|
|
to memba's file format. The persistence layer works correctly; the
|
|
trained model just doesn't use the state for cross-turn recall.
|
|
|
|
See the script outputs (or rerun) for the raw evidence.
|