vision: images are the primary channel for multimodal models (digest off by default); digest is the text-only fallback; --digest/--no-digest to override

This commit is contained in:
opencode
2026-08-08 19:44:18 +03:00
parent ef02e5f9ea
commit d8373a50fc
5 changed files with 164 additions and 19 deletions
+11 -9
View File
@@ -103,17 +103,19 @@ many tool steps the model may chain per request (0 = one action per turn).
### Real vision
By default the model gets frames as a color-grid digest (works for any
text-only model). If the model can actually see images (gemma3/gemma4,
qwen2.5-vl, gpt-4o-mini, ...), pass the frames as real images:
Vision is the **primary channel**: when the model is multimodal, `vision`
tool results attach the actual camera frame as an image to the conversation —
the model sees the crate, the wall, the glowing beacon. The color-grid
digest is only the **fallback** for text-only models:
- auto-detected for local ollama (`/api/show` capabilities) — nothing to do
- otherwise force it: `--vision` (or `--no-vision` to disable)
- multimodal model (auto-detected for local ollama via `/api/show`;
otherwise `--vision`): frame as image, digest off
- text-only model: digest on automatically; `--digest` forces the digest to
be included even alongside images, `--no-digest` disables it everywhere
With vision ON, `vision` tool results attach the actual camera frame as an
image to the conversation — the model sees the crate, the wall, the glowing
beacon, and orients itself. Verified locally: gemma4:12b completed a beacon
mission on its first attempt using look_at/move navigation.
Verified locally: gemma4:12b sees frames and completed a beacon mission on
its first attempt; gemma4:e2b describes what it sees (crates, beacon) and
orients itself from the image.
## Real-time mode