Emil Shanaty
fb3b4935d9
- AGENTS.md: build instructions for AI coding agents - docs/: AICC core spec copy, JSON schema, testbed design notes - scripts/: setup.sh, BUILDLOG.md - README.md: overview and reading order
58 lines
3.2 KiB
Markdown
58 lines
3.2 KiB
Markdown
# Testbed design notes
|
||
|
||
How the aicc-capsule testbed maps onto the AICC Protocol. Read `aicc-core.md` for the protocol itself; this file is testbed-specific.
|
||
|
||
## World
|
||
|
||
- One room: a floor, four walls, 2–3 obstacles (boxes), one interactable (a glowing beacon).
|
||
- Coordinate system: right-handed, Y-up. Room ~16×16 units.
|
||
- The capsule starts at a fixed corner; the beacon sits in the opposite area.
|
||
|
||
## The capsule
|
||
|
||
- A cylinder/capsule body with a heading (yaw) and a camera (pitch).
|
||
- Physics: simple — position, velocity, collision against walls/obstacles. AABB or capsule-vs-box is enough. No gravity needed (or trivial gravity).
|
||
- `move(forward)` pushes along heading; `turn(yaw)` rotates; collisions stop movement.
|
||
|
||
## Tools (all must exist in the bridge)
|
||
|
||
| id | class | purpose | returns |
|
||
|----|-------|---------|---------|
|
||
| `proprioception` | sensor | agent's own state | position, rotation, velocity, health |
|
||
| `vision` | sensor | first-person RGB frame | base64 PNG + width/height/tick |
|
||
| `hear` | sensor | audio events since last call | list of {kind, direction, intensity} |
|
||
| `move` | actuator | translate along heading | new position |
|
||
| `turn` | actuator | rotate yaw/pitch | new rotation |
|
||
| `look_at` | actuator | orient camera at a target | new rotation |
|
||
| `interact` | actuator | use the beacon | result message |
|
||
|
||
Optional: `depth` (depth map), `world_query` (room bounds). If you add tools beyond the list, document them in the bridge manifest via descriptions.
|
||
|
||
## Vision
|
||
|
||
The most important sensor. Render the capsule's view to an image and return it as base64 PNG. Resolution small (e.g. 160×120) to keep latency and tokens down. If the engine can't render, fall back to a canvas-drawn approximation (raycast floor + box silhouettes) — but it must reflect actual world state, not a placeholder.
|
||
|
||
## Events
|
||
|
||
- `collision` event with payload `{other, normal, impulse}` when the capsule hits something.
|
||
- Use `bridge.emit_event(...)` (available in aicc-py) from tool handlers.
|
||
- The agent can subscribe to `tick` for a periodic heartbeat if useful.
|
||
|
||
## Agent loop (demo)
|
||
|
||
1. `AICCClient(WebSocketClientTransport("ws://localhost:8765"))`
|
||
2. `handshake()` → manifest
|
||
3. Loop: call `vision` + `proprioception`, feed to LLM with tool schemas, execute returned tool calls, repeat until `interact` succeeds.
|
||
4. Print every tool call and result to stdout (transcript).
|
||
|
||
The demo should work with any tool-calling LLM. Provide a generic loop that takes a model function; include one example wired to a local/cheap model (ollama or similar) and note in README how to swap providers.
|
||
|
||
## Conformance
|
||
|
||
The bridge must pass all 9 core scenarios. Note: the conformance reference bridge registers tools `echo`, `boom`, `bump` in addition to the world tools — register those three on the testbed bridge too (trivial: echo returns input; boom raises; bump emits a collision event) so the scenario suite runs green against the same bridge instance used in the demo.
|
||
|
||
## Non-goals
|
||
|
||
- No networking beyond WebSocket. No multi-agent. No persistence. No rendering window (headless preferred; a window is optional debug aid).
|
||
- No engine-specific protocol extensions. If the engine needs something extra, it goes in the manifest as an extra tool, not a protocol change.
|