Reference implementation of the AICC Protocol as a 3D room with an AI-controlled capsule. An agent (any LLM with tool calling, or the bundled scripted agent) connects over WebSocket, perceives the room through sensors, walks to a glowing beacon, and activates it — every step over the protocol, no engine hooks.

        agent (LLM)  <--AICC over WebSocket-->  RoomBridge  <-->  Room (world + renderer)
       (testbed/demo.py)                        (testbed/bridge.py)   (testbed/room/)

Layout

Path	Purpose
`room/world.py`	World state: 16x16 room, capsule physics, crates, beacon, audio. Single source of truth.
`room/render.py`	Headless first-person raycaster (Pillow): honest frames from world state.
`room/mapview.py`	Top-down map drawn from sensor data (shared by recorder + live viewer).
`bridge.py`	`RoomBridge(Bridge)`: registers all tools, emits events.
`server.py` entry	`python -m testbed.bridge` — WebSocket server.
`conformance.py`	Runs the 9 core conformance scenarios against this bridge.
`live.py`	Real-time browser viewer (itself an AICC client).
`chat.py`	Interactive chat: natural language -> tool calls.
`llm_agent.py`	Shared LLM driver (controller + autonomous loop).
`demo.py`	Agent demo: LLM driver (OpenAI-compatible) or scripted.
`tests/`	pytest suite (world, renderer, bridge, protocol).

Setup

scripts/setup.sh                 # venv + aicc-py + pillow/websockets/openai

The testbed needs the aicc SDK installed from ~/Desktop/aicc-py (the setup script does pip install -e). The launcher scripts below use the venv interpreter directly, so python does not need to be on your PATH. To run commands by hand instead, activate the venv first: source testbed/.venv/bin/activate.

Run

# 1. Start the bridge (headless; keep it running in its own terminal)
scripts/run_bridge.sh            # ws://127.0.0.1:8765

# 2. Run the demo — default `auto` tries the LLM, then hands off to the
#    scripted agent so the run always completes
scripts/run_demo.sh --agent auto

#   scripted only (deterministic, no LLM needed)
scripts/run_demo.sh --agent scripted

#   LLM only (any OpenAI-compatible endpoint; ollama by default)
scripts/run_demo.sh --agent llm --model gemma4:e2b
scripts/run_demo.sh --agent llm \
    --base-url https://api.openai.com/v1 --model gpt-4o-mini --api-key $OPENAI_API_KEY

If the bridge is already running, run_bridge.sh will say so (the port is taken); stop the old one with fuser -k 8765/tcp or Ctrl-C in its terminal.

The demo prints a full transcript of tool calls/results to stdout and saves the capsule's final first-person frame to demo_final_frame.png.

Interactive chat mode

Talk to the capsule's brain in natural language (any language):

scripts/run_chat.sh --model gemma4:e2b    # small + fast
scripts/run_chat.sh --model gemma4:12b    # bigger gemma, slower (~20 s/turn)

It starts the bridge, the live viewer and a chat REPL — open http://127.0.0.1:8000 to watch the capsule while you type. The model translates your words into tool calls:

you> иди к маяку            → look_at + move step by step (auto-continue)
you> повернись налево       → turn(-90)
you> осмотрись              → vision + description
you> активируй маяк         → interact (when close)
you> /mission               → autonomous goal: reach & activate the beacon,
                             keeps trying until done (retries + corrections)
you> /mission дойди до маяка
you> /status  /stop         → mission progress / cancel
you> /state  /look  /map  /models  /model gemma4:12b  /steps N  /help  /exit

Missions run in the background while the REPL stays usable — watch the capsule on http://127.0.0.1:8000 as it works. Start one directly: python -m testbed.chat --mission [--mission-steps 50] [--mission-retries 3]. The mission keeps trying (corrections when it drifts, nudges when it stalls, fresh attempts on failure) until the goal is achieved; /stop cancels it.

Each turn's transcript is printed; the current frame lands in chat_frame.png and the sensor-built map in chat_map.png. Any OpenAI-compatible endpoint works: python -m testbed.chat --base-url https://api.openai.com/v1 --model gpt-4o-mini --api-key $OPENAI_API_KEY. --auto-steps N controls how many tool steps the model may chain per request (0 = one action per turn).

Real vision

By default the model gets frames as a color-grid digest (works for any text-only model). If the model can actually see images (gemma3/gemma4, qwen2.5-vl, gpt-4o-mini, ...), pass the frames as real images:

auto-detected for local ollama (/api/show capabilities) — nothing to do
otherwise force it: --vision (or --no-vision to disable)

With vision ON, vision tool results attach the actual camera frame as an image to the conversation — the model sees the crate, the wall, the glowing beacon, and orients itself. Verified locally: gemma4:12b completed a beacon mission on its first attempt using look_at/move navigation.

Real-time mode

Watch the capsule drive live in your browser:

scripts/run_live.sh --agent scripted

This starts the bridge, a viewer server, and the demo; open http://127.0.0.1:8000 while the agent acts. The page shows the first-person frame (vision) and a top-down map (world_query + proprioception) updating a few times per second, with the capsule's path, heading, and distance to the beacon.

The viewer (testbed/live.py) is itself a plain AICC client — it sees the world only through the protocol sensors, so it works against any bridge, not just this one. You can also run it standalone:

testbed/.venv/bin/python -m testbed.live            # then run the demo in another terminal

Visual mode

scripts/run_demo.sh --agent scripted --frames-dir frames

Saves, for every step, the first-person frame (step_NNN_view.png) and a top-down map of the room with the capsule's path (step_NNN_map.png), then writes a demo.gif animation and a demo_summary.png (final map + last view). The map is rebuilt purely from sensor data (world_query, proprioception, vision) — the same view the agent itself has. Generated examples are committed at the repo root (demo.gif, demo_summary.png).

Conformance

testbed/.venv/bin/python -m testbed.conformance ~/Desktop/aicc-spec/conformance/scenarios
# [PASS] core-01..core-09  ->  9/9 scenarios passed

The suite runs against the same bridge class used by the server and demo.

Tools

Registered in the manifest (sensors first, then actuators):

id	class	purpose
`proprioception`	sensor	position, rotation, velocity, health, tick
`vision`	sensor	first-person RGB frame as base64 PNG (160x120)
`depth`	sensor	aligned depth map (40x30, meters)
`hear`	sensor	audio since last call: beacon hum, collision thuds
`world_query`	sensor	room bounds, obstacle layout, beacon position
`move`	actuator	move forward N meters, collision-aware
`turn`	actuator	rotate yaw/pitch
`look_at`	actuator	aim camera at a named target (`beacon`)
`interact`	actuator	activate the beacon within reach
`echo`/`boom`/`bump`	—	conformance tools (design doc requirement)

World data flows only through sensors: the manifest carries session metadata and tool schemas, never world state (protocol §7, single source of truth).

Tick model

tick_mode is event: the world advances one tick per tool call, so every observation and event shares a monotonic tick. Collision events (topic: collision, payload {other, normal, impulse}) are pushed asynchronously when move hits a wall or crate.

Demo agents

LLM agent (--agent llm): generic tool-use loop — manifest tools are converted to OpenAI function schemas; every response is executed via AICCClient.call_tool and fed back as a tool message. Vision frames are decoded into a coarse color grid so text-only models can navigate. The loop keeps a compact CURRENT STATE note (agent-side working memory, protocol §9) and gently corrects a model that drifts: nudge after text-only replies, re-aim corrections when the capsule moves away or faces the wrong way, collision guidance. Works with any OpenAI-compatible endpoint (ollama, vLLM, OpenAI, ...). Model quality varies — a capable model completes on its own; a weak local model may hand off (see auto).
Scripted agent (--agent scripted): deterministic bug-algorithm robot — sensor-driven steering toward the beacon with detour-on-collision. No LLM, always completes. Used as the reference/fallback.
Auto (--agent auto): tries the LLM (bounded steps), then hands off to the scripted agent so a demo run always ends with an activated beacon.

Tests

testbed/.venv/bin/python -m pytest -q   # 33 tests: physics, renderer, tools, protocol