aicc-capsule/docs/testbed-design.md

Testbed design notes

How the aicc-capsule testbed maps onto the AICC Protocol. Read aicc-core.md for the protocol itself; this file is testbed-specific.

World

• One room: a floor, four walls, 2–3 obstacles (boxes), one interactable (a glowing beacon).
• Coordinate system: right-handed, Y-up. Room ~16×16 units.
• The capsule starts at a fixed corner; the beacon sits in the opposite area.

The capsule

• A cylinder/capsule body with a heading (yaw) and a camera (pitch).
• Physics: simple — position, velocity, collision against walls/obstacles. AABB or capsule-vs-box is enough. No gravity needed (or trivial gravity).
• move(forward) pushes along heading; turn(yaw) rotates; collisions stop movement.

Tools (all must exist in the bridge)

id	class	purpose	returns
proprioception	sensor	agent's own state	position, rotation, velocity, health
vision	sensor	first-person RGB frame	base64 PNG + width/height/tick
hear	sensor	audio events since last call	list of {kind, direction, intensity}
move	actuator	translate along heading	new position
turn	actuator	rotate yaw/pitch	new rotation
look_at	actuator	orient camera at a target	new rotation
interact	actuator	use the beacon	result message

Optional: depth (depth map), world_query (room bounds). If you add tools beyond the list, document them in the bridge manifest via descriptions.

Vision

The most important sensor. Render the capsule's view to an image and return it as base64 PNG. Resolution small (e.g. 160×120) to keep latency and tokens down. If the engine can't render, fall back to a canvas-drawn approximation (raycast floor + box silhouettes) — but it must reflect actual world state, not a placeholder.

Events

• collision event with payload {other, normal, impulse} when the capsule hits something.
• Use bridge.emit_event(...) (available in aicc-py) from tool handlers.
• The agent can subscribe to tick for a periodic heartbeat if useful.

Agent loop (demo)

1. AICCClient(WebSocketClientTransport("ws://localhost:8765"))
2. handshake() → manifest
3. Loop: call vision + proprioception, feed to LLM with tool schemas, execute returned tool calls, repeat until interact succeeds.
4. Print every tool call and result to stdout (transcript).

The demo should work with any tool-calling LLM. Provide a generic loop that takes a model function; include one example wired to a local/cheap model (ollama or similar) and note in README how to swap providers.

Conformance

The bridge must pass all 9 core scenarios. Note: the conformance reference bridge registers tools echo, boom, bump in addition to the world tools — register those three on the testbed bridge too (trivial: echo returns input; boom raises; bump emits a collision event) so the scenario suite runs green against the same bridge instance used in the demo.

Non-goals

• No networking beyond WebSocket. No multi-agent. No persistence. No rendering window (headless preferred; a window is optional debug aid).
• No engine-specific protocol extensions. If the engine needs something extra, it goes in the manifest as an extra tool, not a protocol change.