Add memba MVP: C++ core, Python SDK, CLI, examples, experiments
C++ core (libmemba.so): - include/memba/state.h — C API (state_new/free/save/load/get_size) - src/state.cpp — MEMB file format: magic, version, SHA-256 model_id, CRC-32, opaque llama_state_*_data() blob - src/cli.cpp — minimal demo binary with greedy sampler - CMakeLists.txt + build.sh with llama.cpp submodule, CUDA auto-detect Python SDK (memba): - core.py — file I/O via llama-cpp-python's exposed C functions, unwraps _LlamaContext to access raw context pointer (≥0.3.x) - session.py — high-level Session with auto-save/load, ChatML wrapper for instruct models, raw mode for base models - cli.py — typer-based: chat (REPL), run (one-shot), list, rm, info Examples: - 01_basic_save_load.py, 02_chat_session.py Experiments (throwaway POCs documenting product-direction findings): - recall_poc.py — git log → state → cross-process query - mood_poc.py — batch sentiment trajectory, Mamba vs Transformer - mood_stream_poc.py, mood_batch_poc.py — variants - diag_saveload.py — minimal save/load isolation test - README.md documents the headline finding: save/load is byte-identical, but Falcon-Mamba-7B-Instruct does not retain facts across conversation turns even in-process — limits viable products to single-prompt analysis and persona priming. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,58 @@
|
||||
"""
|
||||
02_chat_session.py — multi-turn chat that persists across Python processes.
|
||||
|
||||
First run : model has no prior context.
|
||||
Second run : model continues from the saved SSM state.
|
||||
|
||||
Run twice:
|
||||
python examples/02_chat_session.py --model path/to/falcon-mamba-7b-Q4_K_M.gguf
|
||||
python examples/02_chat_session.py --model path/to/falcon-mamba-7b-Q4_K_M.gguf
|
||||
"""
|
||||
|
||||
import argparse
|
||||
from memba import Session
|
||||
|
||||
TURNS = [
|
||||
"My name is Alex. I am a researcher studying ancient Roman aqueducts.",
|
||||
"What is the most famous aqueduct I should know about?",
|
||||
"How long did it take to build?",
|
||||
]
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--model", required=True, help="Path to GGUF file")
|
||||
parser.add_argument("--gpu-layers", type=int, default=0, help="GPU layers (0=CPU)")
|
||||
parser.add_argument("--session", default="aqueduct_research")
|
||||
parser.add_argument("--state-dir", default="~/.memba/states")
|
||||
args = parser.parse_args()
|
||||
|
||||
print(f"Session: {args.session!r}")
|
||||
print(f"Model : {args.model}")
|
||||
print("-" * 60)
|
||||
|
||||
sess = Session(
|
||||
model_path=args.model,
|
||||
session_id=args.session,
|
||||
state_dir=args.state_dir,
|
||||
n_gpu_layers=args.gpu_layers,
|
||||
verbose=False,
|
||||
)
|
||||
print(f"State size on load: {sess.state_size:,} bytes\n")
|
||||
|
||||
# Only feed the turns that haven't been answered yet.
|
||||
# A real app would track which turns were already fed; here we keep it simple
|
||||
# and feed all TURNS every run — the SSM state update is idempotent in terms
|
||||
# of demonstrating cross-process continuity.
|
||||
for i, turn in enumerate(TURNS, 1):
|
||||
print(f"[Turn {i}] User: {turn}")
|
||||
reply = sess.chat(turn, max_tokens=200)
|
||||
print(f"[Turn {i}] Model: {reply}")
|
||||
print()
|
||||
|
||||
saved_path = sess.save()
|
||||
print(f"State saved → {saved_path}")
|
||||
print(f"State size : {sess.state_size:,} bytes")
|
||||
print("\nRun this script again — the model will continue from this checkpoint.")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user