chat: interactive mode (natural language -> tool calls), shared LLM driver refactor, run_chat.sh; verified with gemma4:e2b and gemma4:12b
This commit is contained in:
@@ -22,6 +22,8 @@ over the protocol, no engine hooks.
|
||||
| `server.py` entry | `python -m testbed.bridge` — WebSocket server. |
|
||||
| `conformance.py` | Runs the 9 core conformance scenarios against this bridge. |
|
||||
| `live.py` | Real-time browser viewer (itself an AICC client). |
|
||||
| `chat.py` | Interactive chat: natural language -> tool calls. |
|
||||
| `llm_agent.py` | Shared LLM driver (controller + autonomous loop). |
|
||||
| `demo.py` | Agent demo: LLM driver (OpenAI-compatible) or scripted. |
|
||||
| `tests/` | pytest suite (world, renderer, bridge, protocol). |
|
||||
|
||||
@@ -62,6 +64,33 @@ taken); stop the old one with `fuser -k 8765/tcp` or Ctrl-C in its terminal.
|
||||
The demo prints a full transcript of tool calls/results to stdout and saves the
|
||||
capsule's final first-person frame to `demo_final_frame.png`.
|
||||
|
||||
## Interactive chat mode
|
||||
|
||||
Talk to the capsule's brain in natural language (any language):
|
||||
|
||||
```bash
|
||||
scripts/run_chat.sh --model gemma4:e2b # small + fast
|
||||
scripts/run_chat.sh --model gemma4:12b # bigger gemma, slower (~20 s/turn)
|
||||
```
|
||||
|
||||
It starts the bridge, the live viewer and a chat REPL — open
|
||||
[http://127.0.0.1:8000](http://127.0.0.1:8000) to watch the capsule while you
|
||||
type. The model translates your words into tool calls:
|
||||
|
||||
```
|
||||
you> иди к маяку → look_at + move step by step (auto-continue)
|
||||
you> повернись налево → turn(-90)
|
||||
you> осмотрись → vision + description
|
||||
you> активируй маяк → interact (when close)
|
||||
you> /state /look /map /models /model gemma4:12b /steps N /help /exit
|
||||
```
|
||||
|
||||
Each turn's transcript is printed; the current frame lands in `chat_frame.png`
|
||||
and the sensor-built map in `chat_map.png`. Any OpenAI-compatible endpoint
|
||||
works: `python -m testbed.chat --base-url https://api.openai.com/v1
|
||||
--model gpt-4o-mini --api-key $OPENAI_API_KEY`. `--auto-steps N` controls how
|
||||
many tool steps the model may chain per request (0 = one action per turn).
|
||||
|
||||
## Real-time mode
|
||||
|
||||
Watch the capsule drive live in your browser:
|
||||
|
||||
Reference in New Issue
Block a user