vision: real-time perception — auto frame attach every N turns (--look-every) and on collision, frame context cap (6), digest fallback for text models

This commit is contained in:
opencode
2026-08-08 20:06:22 +03:00
parent d8373a50fc
commit 1d7dbb7b79
5 changed files with 449 additions and 18 deletions
+12
View File
@@ -101,6 +101,18 @@ works: `python -m testbed.chat --base-url https://api.openai.com/v1
--model gpt-4o-mini --api-key $OPENAI_API_KEY`. `--auto-steps N` controls how
many tool steps the model may chain per request (0 = one action per turn).
### Real-time perception
The model sees continuously, not only when it asks:
- `--look-every N` attaches a fresh camera frame every N steps/turns (default 3,
`0` disables). Frames go in as images for multimodal models, as digests for
text-only ones, and old frames are trimmed (max 6) to keep the context
bounded.
- On a collision the current frame is attached immediately, so the model sees
what it bumped into.
- The live viewer (http://127.0.0.1:8000) shows the same frames in the browser.
### Real vision
Vision is the **primary channel**: when the model is multimodal, `vision`