docs: testbed README (run/agent/conformance), setup.sh deps, build log, project status

2026-08-08 04:29:08 +03:00
parent 8e467f22af
commit 132eb25c62
4 changed files with 134 additions and 10 deletions
@@ -0,0 +1,116 @@
+# aicc-capsule testbed
+
+Reference implementation of the [AICC Protocol](https://github.com/emil28092005/AICC-Protocol)
+as a 3D room with an AI-controlled capsule. An agent (any LLM with tool
+calling, or the bundled scripted agent) connects over WebSocket, perceives the
+room through sensors, walks to a glowing beacon, and activates it — every step
+over the protocol, no engine hooks.
+
+```
+        agent (LLM)  <--AICC over WebSocket-->  RoomBridge  <-->  Room (world + renderer)
+       (testbed/demo.py)                        (testbed/bridge.py)   (testbed/room/)
+```
+
+## Layout
+
+| Path                    | Purpose                                                      |
+|-------------------------|--------------------------------------------------------------|
+| `room/world.py`         | World state: 16x16 room, capsule physics, crates, beacon, audio. Single source of truth. |
+| `room/render.py`        | Headless first-person raycaster (Pillow): honest frames from world state. |
+| `bridge.py`             | `RoomBridge(Bridge)`: registers all tools, emits events.     |
+| `server.py` entry       | `python -m testbed.bridge` — WebSocket server.               |
+| `conformance.py`        | Runs the 9 core conformance scenarios against this bridge.   |
+| `demo.py`               | Agent demo: LLM driver (OpenAI-compatible) or scripted.      |
+| `tests/`                | pytest suite (world, renderer, bridge, protocol).            |
+
+## Setup
+
+```bash
+scripts/setup.sh                 # venv + aicc-py + pillow/websockets/openai
+source testbed/.venv/bin/activate
+```
+
+The testbed needs the `aicc` SDK installed from `~/Desktop/aicc-py` (the setup
+script does `pip install -e`).
+
+## Run
+
+```bash
+# 1. Start the bridge (headless)
+scripts/run_bridge.sh            # ws://127.0.0.1:8765
+
+# 2. Run the demo — default `auto` tries the LLM, then hands off to the
+#    scripted agent so the run always completes
+python -m testbed.demo --agent auto
+
+#   scripted only (deterministic, no LLM needed)
+python -m testbed.demo --agent scripted
+
+#   LLM only (any OpenAI-compatible endpoint; ollama by default)
+python -m testbed.demo --agent llm --model gemma4:e2b
+python -m testbed.demo --agent llm \
+    --base-url https://api.openai.com/v1 --model gpt-4o-mini --api-key $OPENAI_API_KEY
+```
+
+The demo prints a full transcript of tool calls/results to stdout and saves the
+capsule's final first-person frame to `demo_final_frame.png`.
+
+## Conformance
+
+```bash
+python -m testbed.conformance ~/Desktop/aicc-spec/conformance/scenarios
+# [PASS] core-01..core-09  ->  9/9 scenarios passed
+```
+
+The suite runs against the *same* bridge class used by the server and demo.
+
+## Tools
+
+Registered in the manifest (sensors first, then actuators):
+
+| id              | class     | purpose                                            |
+|-----------------|-----------|----------------------------------------------------|
+| `proprioception`| sensor    | position, rotation, velocity, health, tick         |
+| `vision`        | sensor    | first-person RGB frame as base64 PNG (160x120)     |
+| `depth`         | sensor    | aligned depth map (40x30, meters)                  |
+| `hear`          | sensor    | audio since last call: beacon hum, collision thuds |
+| `world_query`   | sensor    | room bounds, obstacle layout, beacon position      |
+| `move`          | actuator  | move forward N meters, collision-aware             |
+| `turn`          | actuator  | rotate yaw/pitch                                   |
+| `look_at`       | actuator  | aim camera at a named target (`beacon`)            |
+| `interact`      | actuator  | activate the beacon within reach                   |
+| `echo`/`boom`/`bump` | —    | conformance tools (design doc requirement)         |
+
+World data flows only through sensors: the manifest carries session metadata
+and tool schemas, never world state (protocol §7, single source of truth).
+
+## Tick model
+
+`tick_mode` is `event`: the world advances one tick per tool call, so every
+observation and event shares a monotonic `tick`. Collision events
+(`topic: collision`, payload `{other, normal, impulse}`) are pushed
+asynchronously when `move` hits a wall or crate.
+
+## Demo agents
+
+- **LLM agent** (`--agent llm`): generic tool-use loop — manifest tools are
+  converted to OpenAI function schemas; every response is executed via
+  `AICCClient.call_tool` and fed back as a `tool` message. Vision frames are
+  decoded into a coarse color grid so text-only models can navigate. The loop
+  keeps a compact `CURRENT STATE` note (agent-side working memory, protocol
+  §9) and gently corrects a model that drifts: nudge after text-only replies,
+  re-aim corrections when the capsule moves away or faces the wrong way,
+  collision guidance. Works with any OpenAI-compatible endpoint (ollama,
+  vLLM, OpenAI, ...). Model quality varies — a capable model completes on its
+  own; a weak local model may hand off (see `auto`).
+- **Scripted agent** (`--agent scripted`): deterministic bug-algorithm robot —
+  sensor-driven steering toward the beacon with detour-on-collision. No LLM,
+  always completes. Used as the reference/fallback.
+- **Auto** (`--agent auto`): tries the LLM (bounded steps), then hands off to
+  the scripted agent so a demo run always ends with an activated beacon.
+
+## Tests
+
+```bash
+python -m pytest -q        # 33 tests: physics, renderer, tools, protocol
+```