chat: interactive mode (natural language -> tool calls), shared LLM driver refactor, run_chat.sh; verified with gemma4:e2b and gemma4:12b

This commit is contained in:
opencode
2026-08-08 12:27:05 +03:00
parent 9f877339f8
commit 346d918118
70 changed files with 950 additions and 297 deletions
+29
View File
@@ -22,6 +22,8 @@ over the protocol, no engine hooks.
| `server.py` entry | `python -m testbed.bridge` — WebSocket server. |
| `conformance.py` | Runs the 9 core conformance scenarios against this bridge. |
| `live.py` | Real-time browser viewer (itself an AICC client). |
| `chat.py` | Interactive chat: natural language -> tool calls. |
| `llm_agent.py` | Shared LLM driver (controller + autonomous loop). |
| `demo.py` | Agent demo: LLM driver (OpenAI-compatible) or scripted. |
| `tests/` | pytest suite (world, renderer, bridge, protocol). |
@@ -62,6 +64,33 @@ taken); stop the old one with `fuser -k 8765/tcp` or Ctrl-C in its terminal.
The demo prints a full transcript of tool calls/results to stdout and saves the
capsule's final first-person frame to `demo_final_frame.png`.
## Interactive chat mode
Talk to the capsule's brain in natural language (any language):
```bash
scripts/run_chat.sh --model gemma4:e2b # small + fast
scripts/run_chat.sh --model gemma4:12b # bigger gemma, slower (~20 s/turn)
```
It starts the bridge, the live viewer and a chat REPL — open
[http://127.0.0.1:8000](http://127.0.0.1:8000) to watch the capsule while you
type. The model translates your words into tool calls:
```
you> иди к маяку → look_at + move step by step (auto-continue)
you> повернись налево → turn(-90)
you> осмотрись → vision + description
you> активируй маяк → interact (when close)
you> /state /look /map /models /model gemma4:12b /steps N /help /exit
```
Each turn's transcript is printed; the current frame lands in `chat_frame.png`
and the sensor-built map in `chat_map.png`. Any OpenAI-compatible endpoint
works: `python -m testbed.chat --base-url https://api.openai.com/v1
--model gpt-4o-mini --api-key $OPENAI_API_KEY`. `--auto-steps N` controls how
many tool steps the model may chain per request (0 = one action per turn).
## Real-time mode
Watch the capsule drive live in your browser: