Emil_Shanaty/aicc-capsule
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: context pack for aicc-capsule reference testbed

Browse Source 
- AGENTS.md: build instructions for AI coding agents
- docs/: AICC core spec copy, JSON schema, testbed design notes
- scripts/: setup.sh, BUILDLOG.md
- README.md: overview and reading order
This commit is contained in:
Emil Shanaty
2026-08-08 03:31:42 +03:00
commit fb3b4935d9
7 changed files with 1001 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/aicc-core.md
+477
View File
@@ -0,0 +1,477 @@
# AICC Protocol — Core Specification (Draft 0.1)
## Overview
AICC (AI-Controlled Character) is a transport-agnostic protocol
for connecting language models to interactive virtual environments. It defines
how an agent perceives a world, reasons about it, and acts within it through
a structured set of sensor, actuator, and generator tools.
The protocol makes no assumptions about:
- The host engine (Godot, Unity, Unreal, custom simulators, robotics stacks)
- The reasoning model (any LLM with tool-use capability)
- The transport layer (HTTP, WebSocket, stdio, gRPC, in-process)
- The visual style (2D, 3D, text-based, abstract)
It only specifies message formats, tool taxonomy, capability negotiation, and
causality semantics.
## Design Goals
1. **Engine-agnostic.** The same protocol speaks to a game character, a robot,
or a sandboxed LLM playroom.
2. **Model-agnostic.** Any LLM that supports tool-use / function-calling can
act as the reasoning layer. No prompt engineering baked into the wire
format.
3. **Transport-agnostic.** Wire format is JSON. Transport is pluggable.
4. **Capability-aware.** An environment advertises what an agent may do. The
agent never has to guess.
5. **Deterministic when needed.** Every tool call returns a structured
outcome with timestamps, source, and confidence. Replay is a first-class
concept.
6. **Extensible.** New sensors, actuators, and generators can be added
without breaking older agents, provided they follow the extension rules
in §11.
7. **Single source of truth.** All world data flows through sensor tools.
The manifest carries session metadata only — never world state. This
makes the contract honest and uniform across any environment.
## 1. Model of the World
An AICC interaction involves three actors:
- **Agent** — the reasoning layer (typically an LLM). Holds goals, plans,
and short-term working memory.
- **Environment** — the host system that owns world state. Renders, ticks
physics, runs NPCs, persists state.
- **Bridge** — the thin adapter that translates between the protocol and
the environment's native APIs.
```
+----------------+ +-------------+ +------------------+
| Agent | <--> | Bridge | <--> | Environment |
| (LLM + tools) | AICC | (adapter) | native| (engine/world) |
+----------------+ +-------------+ +------------------+
```
The bridge is implementation-specific and not part of this spec. The agent
sees only the AICC contract.
### Tick
A **tick** is one discrete update of the environment. The cadence is defined
by the environment (e.g. 60 Hz physics, 10 Hz AI updates, event-driven).
Agents do not assume a fixed tick rate — they negotiate it via the session
manifest (§3).
### Session
A **session** is the lifetime of one agent connected to one environment.
Sessions are identified by a UUID assigned at handshake. Sessions are
stateful: tool calls within a session may accumulate history that influences
later calls (e.g. episodic memory).
## 2. Message Format
All messages are UTF-8 JSON. Every message carries a top-level `protocol`
field with the version string `"aicc/0.1"`.
### Common envelope
```json
{
"protocol": "aicc/0.1",
"type": "...",
"session_id": "...",
"message_id": "..."
}
```
| Field | Type | Notes |
|---------------|--------|---------------------------------------------------|
| `protocol` | string | Must be `"aicc/0.1"` for this version. |
| `type` | string | One of the message types below. |
| `session_id` | string | UUID assigned by the bridge at handshake. |
| `message_id` | string | UUID for this message. Used for correlation. |
### Message types
| `type` | Direction | Purpose |
|---------------------|----------------------|--------------------------------------------|
| `session_init` | bridge -> agent | Open a new session, advertise manifest. |
| `session_resume` | agent -> bridge | Resume a session by `session_id`. |
| `session_close` | both | Graceful close. |
| `manifest_request` | agent -> bridge | Re-fetch the manifest. |
| `tool_call` | agent -> bridge | Invoke a tool. |
| `tool_result` | bridge -> agent | Return outcome of a tool call. |
| `event` | bridge -> agent | Asynchronous notification from environment.|
| `error` | bridge -> agent | Protocol-level or environment error. |
| `heartbeat` | both | Keep-alive ping with optional payload. |
### Versioning
The protocol string uses semantic versioning (`aicc/MAJOR.MINOR`).
- MAJOR bump: any breaking change to message format or tool schema.
- MINOR bump: additive changes only. Agents must accept messages from a
bridge with a higher MINOR (forward compatibility), ignoring unknown
optional fields.
## 3. Session Manifest
Sent in `session_init` and on `manifest_request`. Describes the environment
and what the agent may do.
```json
{
"type": "session_init",
"session_id": "8f3a...",
"tick_rate_hz": 10,
"tick_mode": "fixed",
"world": {
"name": "testbed_room_01",
"kind": "3d"
},
"capabilities": {
"sensors": ["vision", "depth", "proprioception", "memory_query"],
"actuators": ["move", "turn", "look_at", "interact", "say"],
"generators": []
},
"tools": [
{
"id": "vision",
"class": "sensor",
"description": "Return the current first-person RGB frame as base64 PNG.",
"input_schema": { "type": "object", "properties": {}, "additionalProperties": false },
"output_schema": { "type": "object", "properties": { "png_b64": { "type": "string" }, "width": { "type": "integer" }, "height": { "type": "integer" }, "tick": { "type": "integer" } }, "required": ["png_b64", "width", "height", "tick"] }
}
]
}
```
> **Note.** World state — including the agent's own position, rotation,
> inventory, and the world's bounds — is **never** included in the manifest.
> It is accessible only through the appropriate sensor tools. The manifest
> is a contract on the session, not a snapshot of the world.
### Tick modes
- `fixed` — environment advances in regular intervals. Tools see a coherent
snapshot.
- `event` — environment advances only on tool calls or external triggers.
- `hybrid` — fixed physics tick, but agents may request immediate evaluation
via a `tick_now` tool.
## 4. Tool Taxonomy
Three top-level classes. Every tool belongs to exactly one.
| Class | Side effect | Returns |
|--------------|-------------|----------------------------------|
| `sensor` | none | Read-only observation of world. |
| `actuator` | yes | Action performed on the world. |
| `generator` | yes | New world content created. |
A tool is identified by its `id`. IDs are scoped to the session.
### Standard sensors (initial set)
| ID | Returns |
|-----------------|---------------------------------------------------|
| `vision` | RGB frame from agent's camera. |
| `depth` | Depth map aligned to vision. |
| `hear` | Audio events since last call. |
| `smell` | Active scent zones near the agent. |
| `touch` | Surface contact info from last physics tick. |
| `proprioception`| Position, rotation, velocity, health, inventory. |
| `memory_query` | Episodic memory hits. |
| `inspect` | Detailed view of a target entity or location. |
### Standard actuators (initial set)
| ID | Effect |
|-------------|-----------------------------------------------------|
| `move` | Translate along agent-relative or world axes. |
| `turn` | Rotate yaw / pitch. |
| `look_at` | Orient camera toward a target. |
| `interact` | Use / activate / pick up / talk to an entity. |
| `say` | Emit speech (or text for dialog-only sessions). |
| `wait` | Skip ticks without action. |
### Standard generators (initial set)
| ID | Effect |
|----------------|---------------------------------------------------|
| `place_object` | Spawn a registered prefab at a location. |
| `modify_terrain`| Edit terrain heightmap / material. |
| `spawn_entity` | Create a registered entity at a location. |
Generators require the `can_modify_world` capability and a per-tool rate
limit. They are optional in the manifest.
## 5. Tool Invocation
### Request (`tool_call`)
```json
{
"type": "tool_call",
"message_id": "...",
"call_id": "tc_001",
"tool": "vision",
"input": {}
}
```
| Field | Type | Notes |
|--------------|--------|------------------------------------------------|
| `call_id` | string | Agent-assigned. Echoed in `tool_result`. |
| `tool` | string | Tool id from the manifest. |
| `input` | object | Must validate against the tool's input schema. |
### Response (`tool_result`)
```json
{
"type": "tool_result",
"message_id": "...",
"call_id": "tc_001",
"ok": true,
"output": { "...": "..." },
"meta": {
"tick": 142,
"latency_ms": 12,
"source": "render_thread"
}
}
```
| Field | Type | Notes |
|---------------|---------|---------------------------------------------|
| `call_id` | string | Echoed from the request. |
| `ok` | boolean | False on tool-level failure. |
| `output` | object | Conforms to the tool's output schema. |
| `error` | object | Present only when `ok` is false. |
| `meta` | object | Optional environment metadata. |
### Failure shape
```json
{
"ok": false,
"error": {
"code": "tool_unavailable",
"message": "Generator 'spawn_entity' requires capability 'can_spawn_entities'.",
"retryable": false
}
}
```
Standard error codes are listed in §10.
## 6. Events
Asynchronous notifications from the environment. The agent may subscribe to a
subset at session init.
```json
{
"type": "event",
"topic": "collision",
"payload": {
"other": "wall_segment_03",
"normal": { "x": -1, "y": 0, "z": 0 },
"impulse": 4.2
},
"meta": { "tick": 142 }
}
```
Reserved topics:
- `tick` — fired on every environment tick (only if subscribed).
- `collision` — physical contact.
- `audio` — sound event outside the agent's request cycle.
- `state_change` — entity added, removed, or substantially changed.
- `agent_message` — incoming inter-agent communication in multi-agent sessions.
## 7. Capabilities
Capabilities are advertised in the manifest. They are coarse-grained
permissions, not per-entity ACLs. Fine-grained permissions are an extension
(see §11).
### Model class declaration
The bridge declares which model class the agent runs on. This lets
environments tune tick rates, sensor cadence, and event batching to match
the agent's expected response time.
```json
{
"agent_model": {
"class": "edge_small | edge_medium | cloud_medium | cloud_large",
"expected_first_token_ms": 800,
"expected_full_response_ms": 2500
}
}
```
| Class | Typical target | First-token budget |
|----------------|---------------------------------------------|--------------------|
| `edge_small` | E2B / phone-class, on-device | 100-400 ms |
| `edge_medium` | E4B / 12B on consumer GPU | 300-800 ms |
| `cloud_medium` | 12B-26B MoE via hosted inference | 500-1500 ms |
| `cloud_large` | 31B+ via hosted inference, with reasoning | 1500-3000 ms |
These are advisory. The bridge uses them to inform tick-rate choice and
sensor buffering strategy. They do not constrain the agent — the agent
may exceed or beat the budget — but environments should design their
default cadence so that a conformant agent of the declared class can
participate in real-time interaction.
### Reserved capabilities
- `can_observe_world` — read any sensor.
- `can_move_self` — invoke movement actuators.
- `can_modify_world` — invoke generators.
- `can_spawn_entities` — generators that create NPCs / objects.
- `can_modify_terrain` — generators that change terrain.
- `can_communicate` — invoke `say` and `agent_message` events.
- `can_persist_state` — write to long-term storage.
An agent must not call a tool that requires an absent capability. Bridges
should reject such calls with `tool_unavailable`.
## 8. Time and Causality
### Tick clock
Every message that carries a `meta.tick` field references the same monotonic
counter maintained by the environment. The agent uses this to reason about
ordering of observations.
### Eventual consistency for events
`event` messages may arrive between `tool_result` messages. The agent MUST
process events in `meta.tick` order; out-of-order arrival is the bridge's
responsibility to prevent or flag.
### Heartbeat
For long sessions, the bridge may send `heartbeat` every N seconds. Agents
may respond with `heartbeat` of their own. This is also the vehicle for
pushing manifest updates (e.g. capability changes).
## 9. Memory and State
Memory is split:
- **World state** — owned by the environment, accessed **only** via sensors.
The manifest carries no world state.
- **Agent memory** — owned by the agent. Not transmitted over AICC unless
via the optional `memory_query` sensor, which is environment-side episodic
memory tagged with session events.
The protocol does not prescribe how an LLM maintains its context window.
That's the agent's concern. The protocol only guarantees that any state
observable via a sensor is reproducible from the session.
### Single source of truth
Every piece of world data has exactly one way to reach the agent: through
a sensor tool. The agent's own position, the world's bounds, the contents
of nearby containers, the time of day — all of these are observable only
by calling the appropriate sensor. The manifest never duplicates or
caches this data. This eliminates a class of bugs where handshake data
drifts from real-time sensors, and keeps the contract uniform across
single-agent and multi-agent sessions.
## 10. Errors
Standard error codes returned in `tool_result.error` or top-level `error`
messages:
| Code | When |
|----------------------|-----------------------------------------------------|
| `protocol_mismatch` | Wire version incompatible. |
| `session_expired` | Session id unknown or closed. |
| `tool_unknown` | Tool id not in manifest. |
| `tool_unavailable` | Capability missing or rate limit hit. |
| `invalid_input` | Input fails schema validation. |
| `execution_failed` | Tool ran but failed at the environment level. |
| `timeout` | Tool exceeded environment-defined timeout. |
| `internal_error` | Unspecified bridge failure. |
All errors include `retryable: boolean` to indicate whether the agent may
safely retry the same call.
## 11. Extension Rules
New tools and capabilities MAY be added in a MINOR version. Additions MUST:
1. Use a new `id` not previously registered.
2. Be advertised in the manifest.
3. Define input and output schemas as JSON Schema.
4. Use only documented error codes.
Breaking changes (renaming, removing, changing semantics of existing tools,
changing required fields) MUST trigger a MAJOR bump.
Agents SHOULD ignore unknown fields in `meta` and unknown optional fields in
`output`. Bridges SHOULD accept tool calls with extra fields by ignoring them
(unless `strict_input: true` is set on the tool in the manifest).
## 12. Security and Trust
- The bridge authenticates the agent (out of band of this spec).
- The agent authenticates the bridge (out of band of this spec).
- Tools that may be expensive or destructive MUST advertise rate limits in
the manifest under `tool.limits`, e.g. `{"calls_per_minute": 60}`.
- All transport MUST be encrypted when crossing untrusted networks. AICC
itself does not mandate TLS — that is a transport concern.
## 13. Conformance
A bridge is AICC-conformant for version `aicc/0.1` if it:
1. Emits and accepts the message types in §2.
2. Validates all `tool_call` inputs against the manifest.
3. Returns errors using the codes in §10.
4. Preserves event ordering per §8.
5. Passes the conformance scenarios in `conformance/scenarios/`.
An agent is conformant if it:
1. Never invokes tools outside its advertised capabilities.
2. Echoes `call_id` correctly on result correlation.
3. Handles all standard error codes.
4. Respects tick ordering for events.
## Appendix A: Example — Walking Around a Room
1. Bridge -> Agent: `session_init` with vision, depth, proprioception,
move, turn, look_at.
2. Agent -> Bridge: `tool_call` `vision`.
3. Bridge -> Agent: `tool_result` with PNG.
4. Agent reasons, decides to walk forward.
5. Agent -> Bridge: `tool_call` `move` `{ "forward": 2.0 }`.
6. Bridge -> Agent: `tool_result` with new proprioception snapshot.
7. Bridge -> Agent: `event` `collision` when wall hit.
8. Agent -> Bridge: `tool_call` `turn` `{ "yaw": -45 }` to redirect.
9. Agent -> Bridge: `tool_call` `move` `{ "forward": 1.5 }`.
10. Loop continues.
## Appendix B: Open Questions (to resolve in 0.2)
- How are multi-agent sessions coordinated? Sub-sessions or shared session?
- How are assets (prefabs, models, audio) referenced — by id, URL, or
inline binary?
- Should `vision` support streaming chunks, or is single-frame enough?
- Standard for episodic memory schema (events vs summaries vs both)?
- Should `inspect` allow arbitrary queries, or a fixed set of detail
modes (`color`, `bbox`, `semantic`, `full`)?
- Resolved in 0.1.1: model class declaration (see §7).
---
Status: **draft**. Not yet published. Internal review only.
docs/aicc.schema.json
+369
View File
@@ -0,0 +1,369 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://aicc.dev/schemas/0.1/aicc.schema.json",
"title": "AICC Protocol",
"description": "Schema for the AI-Controlled Character Protocol (AICC) version 0.1. Defines the message envelope, session manifest, tool definitions, and all message types.",
"type": "object",
"oneOf": [
{ "$ref": "#/$defs/sessionInit" },
{ "$ref": "#/$defs/sessionResume" },
{ "$ref": "#/$defs/sessionClose" },
{ "$ref": "#/$defs/manifestRequest" },
{ "$ref": "#/$defs/toolCall" },
{ "$ref": "#/$defs/toolResult" },
{ "$ref": "#/$defs/event" },
{ "$ref": "#/$defs/errorMessage" },
{ "$ref": "#/$defs/heartbeat" }
],
"$defs": {
"protocolString": {
"type": "string",
"const": "aicc/0.1",
"description": "Protocol version string. Must be exactly 'aicc/0.1' for this version."
},
"uuid": {
"type": "string",
"format": "uuid",
"pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$"
},
"sessionId": {
"type": "string",
"format": "uuid",
"description": "UUID assigned by the bridge at handshake."
},
"messageId": {
"type": "string",
"format": "uuid",
"description": "UUID for this message. Used for correlation."
},
"callId": {
"type": "string",
"minLength": 1,
"description": "Agent-assigned id. Echoed in tool_result for correlation."
},
"envelope": {
"type": "object",
"required": ["protocol", "type", "session_id", "message_id"],
"properties": {
"protocol": { "$ref": "#/$defs/protocolString" },
"type": {
"type": "string",
"enum": [
"session_init",
"session_resume",
"session_close",
"manifest_request",
"tool_call",
"tool_result",
"event",
"error",
"heartbeat"
]
},
"session_id": { "$ref": "#/$defs/sessionId" },
"message_id": { "$ref": "#/$defs/messageId" }
},
"additionalProperties": false
},
"tickMode": {
"type": "string",
"enum": ["fixed", "event", "hybrid"],
"description": "How the environment advances over time. fixed: regular intervals. event: only on tool calls or triggers. hybrid: fixed physics but agents may request immediate evaluation."
},
"worldKind": {
"type": "string",
"enum": ["2d", "3d", "text", "abstract"],
"description": "Visual / spatial style of the world. Does not constrain tool implementations."
},
"modelClass": {
"type": "string",
"enum": ["edge_small", "edge_medium", "cloud_medium", "cloud_large"],
"description": "Declared class of the agent's reasoning model. Advisory only — used by the bridge to tune tick rate and sensor cadence."
},
"toolClass": {
"type": "string",
"enum": ["sensor", "actuator", "generator"]
},
"errorCode": {
"type": "string",
"enum": [
"protocol_mismatch",
"session_expired",
"tool_unknown",
"tool_unavailable",
"invalid_input",
"execution_failed",
"timeout",
"internal_error"
]
},
"error": {
"type": "object",
"required": ["code", "message", "retryable"],
"properties": {
"code": { "$ref": "#/$defs/errorCode" },
"message": { "type": "string", "minLength": 1 },
"retryable": { "type": "boolean" }
},
"additionalProperties": false
},
"meta": {
"type": "object",
"description": "Optional environment-supplied metadata for any message that reports state.",
"properties": {
"tick": { "type": "integer", "minimum": 0 },
"latency_ms": { "type": "integer", "minimum": 0 },
"source": { "type": "string" }
},
"additionalProperties": true
},
"tool": {
"type": "object",
"required": ["id", "class", "description", "input_schema", "output_schema"],
"properties": {
"id": {
"type": "string",
"pattern": "^[a-z][a-z0-9_]*$",
"description": "Lowercase snake_case identifier, unique within the session."
},
"class": { "$ref": "#/$defs/toolClass" },
"description": {
"type": "string",
"minLength": 1,
"description": "Human-readable description surfaced to the agent. Should make the tool's purpose and any non-obvious side effects clear."
},
"input_schema": {
"type": "object",
"description": "JSON Schema describing the tool's input object."
},
"output_schema": {
"type": "object",
"description": "JSON Schema describing the tool's output object."
},
"limits": {
"type": "object",
"description": "Rate limits and quotas. Enforced by the bridge.",
"properties": {
"calls_per_minute": { "type": "integer", "minimum": 1 },
"calls_per_session": { "type": "integer", "minimum": 1 }
},
"additionalProperties": false
},
"requires_capability": {
"type": "string",
"description": "Capability the agent must hold to invoke this tool."
},
"strict_input": {
"type": "boolean",
"default": false,
"description": "If true, the bridge rejects tool calls with input fields not declared in input_schema. Default false (extra fields ignored)."
}
},
"additionalProperties": false
},
"capabilities": {
"type": "object",
"required": ["sensors", "actuators", "generators"],
"properties": {
"sensors": {
"type": "array",
"items": { "type": "string" },
"description": "Reserved capability names enabled for sensors."
},
"actuators": {
"type": "array",
"items": { "type": "string" },
"description": "Reserved capability names enabled for actuators."
},
"generators": {
"type": "array",
"items": { "type": "string" },
"description": "Reserved capability names enabled for generators."
}
},
"additionalProperties": false
},
"modelDecl": {
"type": "object",
"required": ["class"],
"properties": {
"class": { "$ref": "#/$defs/modelClass" },
"expected_first_token_ms": {
"type": "integer",
"minimum": 0,
"description": "Advisory budget for time-to-first-token."
},
"expected_full_response_ms": {
"type": "integer",
"minimum": 0,
"description": "Advisory budget for a full tool-call decision cycle."
}
},
"additionalProperties": false
},
"sessionInit": {
"type": "object",
"required": ["protocol", "type", "session_id", "message_id", "tick_rate_hz", "tick_mode", "world", "capabilities", "tools", "agent_model"],
"properties": {
"protocol": { "$ref": "#/$defs/protocolString" },
"type": { "const": "session_init" },
"session_id": { "$ref": "#/$defs/sessionId" },
"message_id": { "$ref": "#/$defs/messageId" },
"tick_rate_hz": {
"type": "number",
"exclusiveMinimum": 0,
"description": "Environment's nominal tick rate in Hertz. Informational; bridges may exceed or undershoot."
},
"tick_mode": { "$ref": "#/$defs/tickMode" },
"world": {
"type": "object",
"required": ["name", "kind"],
"properties": {
"name": { "type": "string", "minLength": 1 },
"kind": { "$ref": "#/$defs/worldKind" }
},
"additionalProperties": false
},
"agent_model": { "$ref": "#/$defs/modelDecl" },
"capabilities": { "$ref": "#/$defs/capabilities" },
"tools": {
"type": "array",
"items": { "$ref": "#/$defs/tool" },
"minItems": 0,
"description": "Full catalog of tools available in this session."
},
"event_subscriptions": {
"type": "array",
"items": { "type": "string" },
"description": "Optional. Topics the agent is subscribed to. Empty or omitted means only events emitted as direct responses are sent."
}
},
"additionalProperties": false
},
"sessionResume": {
"type": "object",
"required": ["protocol", "type", "session_id", "message_id"],
"properties": {
"protocol": { "$ref": "#/$defs/protocolString" },
"type": { "const": "session_resume" },
"session_id": { "$ref": "#/$defs/sessionId" },
"message_id": { "$ref": "#/$defs/messageId" }
},
"additionalProperties": false
},
"sessionClose": {
"type": "object",
"required": ["protocol", "type", "session_id", "message_id"],
"properties": {
"protocol": { "$ref": "#/$defs/protocolString" },
"type": { "const": "session_close" },
"session_id": { "$ref": "#/$defs/sessionId" },
"message_id": { "$ref": "#/$defs/messageId" },
"reason": { "type": "string" }
},
"additionalProperties": false
},
"manifestRequest": {
"type": "object",
"required": ["protocol", "type", "session_id", "message_id"],
"properties": {
"protocol": { "$ref": "#/$defs/protocolString" },
"type": { "const": "manifest_request" },
"session_id": { "$ref": "#/$defs/sessionId" },
"message_id": { "$ref": "#/$defs/messageId" }
},
"additionalProperties": false
},
"toolCall": {
"type": "object",
"required": ["protocol", "type", "session_id", "message_id", "call_id", "tool", "input"],
"properties": {
"protocol": { "$ref": "#/$defs/protocolString" },
"type": { "const": "tool_call" },
"session_id": { "$ref": "#/$defs/sessionId" },
"message_id": { "$ref": "#/$defs/messageId" },
"call_id": { "$ref": "#/$defs/callId" },
"tool": {
"type": "string",
"pattern": "^[a-z][a-z0-9_]*$",
"description": "Tool id from the manifest."
},
"input": {
"type": "object",
"description": "Must validate against the tool's input_schema."
}
},
"additionalProperties": false
},
"toolResult": {
"type": "object",
"required": ["protocol", "type", "session_id", "message_id", "call_id", "ok"],
"properties": {
"protocol": { "$ref": "#/$defs/protocolString" },
"type": { "const": "tool_result" },
"session_id": { "$ref": "#/$defs/sessionId" },
"message_id": { "$ref": "#/$defs/messageId" },
"call_id": { "$ref": "#/$defs/callId" },
"ok": { "type": "boolean" },
"output": {
"type": "object",
"description": "Tool output. Must conform to the tool's output_schema when ok is true. Absent or null when ok is false."
},
"error": {
"$ref": "#/$defs/error",
"description": "Present only when ok is false."
},
"meta": { "$ref": "#/$defs/meta" }
},
"additionalProperties": false
},
"event": {
"type": "object",
"required": ["protocol", "type", "session_id", "message_id", "topic", "payload"],
"properties": {
"protocol": { "$ref": "#/$defs/protocolString" },
"type": { "const": "event" },
"session_id": { "$ref": "#/$defs/sessionId" },
"message_id": { "$ref": "#/$defs/messageId" },
"topic": {
"type": "string",
"enum": ["tick", "collision", "audio", "state_change", "agent_message"],
"description": "Event topic. Reserved topics are listed; bridges may define additional topics but agents may not rely on them."
},
"payload": {
"type": "object",
"description": "Topic-specific payload. Bridges MAY extend with additional fields; agents SHOULD ignore unknown fields."
},
"meta": { "$ref": "#/$defs/meta" }
},
"additionalProperties": false
},
"errorMessage": {
"type": "object",
"required": ["protocol", "type", "session_id", "message_id", "error"],
"properties": {
"protocol": { "$ref": "#/$defs/protocolString" },
"type": { "const": "error" },
"session_id": { "$ref": "#/$defs/sessionId" },
"message_id": { "$ref": "#/$defs/messageId" },
"error": { "$ref": "#/$defs/error" }
},
"additionalProperties": false
},
"heartbeat": {
"type": "object",
"required": ["protocol", "type", "session_id", "message_id"],
"properties": {
"protocol": { "$ref": "#/$defs/protocolString" },
"type": { "const": "heartbeat" },
"session_id": { "$ref": "#/$defs/sessionId" },
"message_id": { "$ref": "#/$defs/messageId" },
"manifest_update": {
"type": "object",
"description": "If present, replaces the current manifest (e.g. capability changes). Same shape as session_init minus session_id and message_id."
}
},
"additionalProperties": false
}
}
}
docs/testbed-design.md
+57
View File
@@ -0,0 +1,57 @@
# Testbed design notes
How the aicc-capsule testbed maps onto the AICC Protocol. Read `aicc-core.md` for the protocol itself; this file is testbed-specific.
## World
- One room: a floor, four walls, 2–3 obstacles (boxes), one interactable (a glowing beacon).
- Coordinate system: right-handed, Y-up. Room ~16×16 units.
- The capsule starts at a fixed corner; the beacon sits in the opposite area.
## The capsule
- A cylinder/capsule body with a heading (yaw) and a camera (pitch).
- Physics: simple — position, velocity, collision against walls/obstacles. AABB or capsule-vs-box is enough. No gravity needed (or trivial gravity).
- `move(forward)` pushes along heading; `turn(yaw)` rotates; collisions stop movement.
## Tools (all must exist in the bridge)
| id | class | purpose | returns |
|----|-------|---------|---------|
| `proprioception` | sensor | agent's own state | position, rotation, velocity, health |
| `vision` | sensor | first-person RGB frame | base64 PNG + width/height/tick |
| `hear` | sensor | audio events since last call | list of {kind, direction, intensity} |
| `move` | actuator | translate along heading | new position |
| `turn` | actuator | rotate yaw/pitch | new rotation |
| `look_at` | actuator | orient camera at a target | new rotation |
| `interact` | actuator | use the beacon | result message |
Optional: `depth` (depth map), `world_query` (room bounds). If you add tools beyond the list, document them in the bridge manifest via descriptions.
## Vision
The most important sensor. Render the capsule's view to an image and return it as base64 PNG. Resolution small (e.g. 160×120) to keep latency and tokens down. If the engine can't render, fall back to a canvas-drawn approximation (raycast floor + box silhouettes) — but it must reflect actual world state, not a placeholder.
## Events
- `collision` event with payload `{other, normal, impulse}` when the capsule hits something.
- Use `bridge.emit_event(...)` (available in aicc-py) from tool handlers.
- The agent can subscribe to `tick` for a periodic heartbeat if useful.
## Agent loop (demo)
1. `AICCClient(WebSocketClientTransport("ws://localhost:8765"))`
2. `handshake()` → manifest
3. Loop: call `vision` + `proprioception`, feed to LLM with tool schemas, execute returned tool calls, repeat until `interact` succeeds.
4. Print every tool call and result to stdout (transcript).
The demo should work with any tool-calling LLM. Provide a generic loop that takes a model function; include one example wired to a local/cheap model (ollama or similar) and note in README how to swap providers.
## Conformance
The bridge must pass all 9 core scenarios. Note: the conformance reference bridge registers tools `echo`, `boom`, `bump` in addition to the world tools — register those three on the testbed bridge too (trivial: echo returns input; boom raises; bump emits a collision event) so the scenario suite runs green against the same bridge instance used in the demo.
## Non-goals
- No networking beyond WebSocket. No multi-agent. No persistence. No rendering window (headless preferred; a window is optional debug aid).
- No engine-specific protocol extensions. If the engine needs something extra, it goes in the manifest as an extra tool, not a protocol change.