vision: real-time perception — auto frame attach every N turns (--look-every) and on collision, frame context cap (6), digest fallback for text models
This commit is contained in:
@@ -101,6 +101,18 @@ works: `python -m testbed.chat --base-url https://api.openai.com/v1
|
||||
--model gpt-4o-mini --api-key $OPENAI_API_KEY`. `--auto-steps N` controls how
|
||||
many tool steps the model may chain per request (0 = one action per turn).
|
||||
|
||||
### Real-time perception
|
||||
|
||||
The model sees continuously, not only when it asks:
|
||||
|
||||
- `--look-every N` attaches a fresh camera frame every N steps/turns (default 3,
|
||||
`0` disables). Frames go in as images for multimodal models, as digests for
|
||||
text-only ones, and old frames are trimmed (max 6) to keep the context
|
||||
bounded.
|
||||
- On a collision the current frame is attached immediately, so the model sees
|
||||
what it bumped into.
|
||||
- The live viewer (http://127.0.0.1:8000) shows the same frames in the browser.
|
||||
|
||||
### Real vision
|
||||
|
||||
Vision is the **primary channel**: when the model is multimodal, `vision`
|
||||
|
||||
Reference in New Issue
Block a user