Memba/experiments

experiments/

Throwaway scripts used to probe SSM/hybrid model capabilities with memba. Not part of the library API — kept in the repo as reference and reproducible evidence for product decisions.

Each script is self-contained. Read the source for what it claims to test and run it yourself if you want to verify on different models or hardware.

Scripts

File	What it measures
recall_poc.py	git log → state → cross-process "what did I work on last month"
mood_poc.py	Batch sentiment trajectory in one prompt — Falcon-Mamba vs Gemma
mood_batch_poc.py	Batch ingest, save, query in fresh process
mood_stream_poc.py	15 separate observation turns, save, query in fresh process
diag_saveload.py	Minimal hamster recall test on Falcon-Mamba
diag_nemotron.py	Same hamster test on Nemotron 4B hybrid
diag_nemotron2.py	Cross-process recall via raw generate() (bypass create_chat_completion)
diag_nemotron3.py	Same with llama-cpp-python's native save_state()/load_state()
diag_session_nemotron.py	Full hamster recall through the rewritten Session.chat()

Findings log

2026-05-16 (initial, Falcon-Mamba-7B-Instruct Q4_K_M)

• Batch single-prompt analysis works for both sentiment and recall.
• Multi-turn fact recall failed even in-process. Misdiagnosed at first as a model capability issue.

2026-05-16 (revised, after Nemotron-3-Nano-4B Q4_K_M test)

• Real root cause: Llama.create_chat_completion() and Llama.generate() retokenise the entire prompt each call and reset KV-cache when the new tokens don't prefix-match input_ids. Loaded memba state was being wiped.
• Secondary issue: llama_state_set_data() restores the C-level cache but llama-cpp-python's wrapper still reports n_tokens=0, so the next eval() writes new tokens at offset 0 and overwrites the loaded state.
• Two fixes in memba:
  1. MEMB file now carries a Python-only trailer with n_tokens so the wrapper position is restored after load_state (12 extra bytes, backward-compatible — the C library ignores anything after the CRC).
  2. Session.chat() rewritten to use raw tokenize → eval → sample instead of Llama.__call__, avoiding the prefix-matching reset.
• Results on Nemotron-3-Nano-4B (hybrid: 21 Mamba-2 + 4 attention layers) through the new Session.chat():
  • diag_session_nemotron.py — cross-process hamster recall: ✅
  • mood_batch_poc.py — cross-process mood trajectory: ✅
  • mood_stream_poc.py — 15 streaming turns + cross-process: ✅
  • recall_poc.py — per-project digest from 30-day git log: ✅ accurate, no hallucinated projects.

Falcon-Mamba was rerun through the new Session.chat() but still fails on the same tests — the trained model genuinely doesn't have the cross-turn recall capability that the hybrid Nemotron does. The 4 attention layers make the difference.