Two bugs were blocking memba's main promise (load .memb in a fresh
process → model continues with full recalled context):
1. llama_state_set_data() restores the C-level KV-cache + SSM hidden
state, but llama-cpp-python's Python wrapper still reports
n_tokens=0. The next eval() then decodes new tokens at offset 0
and overwrites the loaded state.
Fix: extend MEMB format with an optional 12-byte trailer appended
after the CRC32. It carries the wrapper's n_tokens. The C library
reads up to CRC and ignores anything past it, so files stay
backward-compatible with libmemba; only the Python loader uses it.
2. Llama.__call__ / create_chat_completion / generate all re-tokenise
the prompt on every call and clear the KV-cache when the new
tokens don't prefix-match input_ids. That destroys any state we
just loaded.
Fix: rewrite Session.chat() to use raw tokenize → eval → sample.
eval() appends tokens to the live state without resetting, and we
handle stop-token detection ourselves.
Verified end-to-end on Nemotron-3-Nano-4B (hybrid 21x Mamba-2 + 4x
attention) — see experiments/README.md for the full findings log.
diag_session_nemotron.py, mood_batch_poc.py, mood_stream_poc.py and
recall_poc.py now all pass their cross-process tests; Falcon-Mamba
still fails because the trained model itself can't do cross-turn
recall — that was the original misdiagnosis.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
