These four endpoints checked model existence but never verified the user has read access via AccessGrants, allowing any authenticated user to use restricted models.
Uses the canonical check_model_access helper from utils.access_control.
The /responses proxy endpoint only required authentication via
get_verified_user but did not check per-model access grants. This
allowed any authenticated user to access any model through this
endpoint, bypassing the access control system.
Extract a shared check_model_access helper into utils/access_control
and replace all inline access control blocks across openai.py and
ollama.py (7 locations) with calls to this helper. This eliminates
code duplication and prevents future policy drift between endpoints.
CWE-862: Missing Authorization
CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:N/A:H (6.5 Medium)
Ollama recently added Responses API support via its OpenAI-compatible
endpoint (/v1/responses). This adds a proxy endpoint to the Ollama
router that forwards requests to Ollama's /v1/responses, applying
the same model resolution, access control, and prefix_id handling
used by the existing /v1/chat/completions and /v1/messages proxies.
Uses a typed ResponsesForm Pydantic model with required model field
and extra='allow' for forward compatibility, consistent with other
endpoint schemas in the file.
This allows API consumers (Codex, Claude Code, etc.) to use the
Responses API directly with Ollama-hosted models without requiring
a separate OpenAI-compatible connection.
`raise "string"` in Python raises TypeError instead of the intended
error, making error messages confusing and debugging difficult.
Co-authored-by: gambletan <ethanchang32@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
fix: reduce TTFT by caching model lookups in chat completion
Skip expensive get_all_models() calls when models are already cached
in app.state. This significantly reduces Time To First Token (TTFT)
for chat completions and embeddings requests.
Previously, every request called get_all_models() which fetches model
lists from all configured backends. Now we check the cache first and
only call get_all_models() on cache miss.
Affected endpoints:
- openai: generate_chat_completion, embeddings
- ollama: embed, embeddings
Fixes#20069
Co-authored-by: Michael <42099345+mickeytheseal@users.noreply.github.com>
Remove Depends(get_session) from the /v1/completions endpoint to prevent database connections from being held during the entire duration of LLM calls.
Previously, the database session was acquired at request start and held until the response completed. Under concurrent load, this exhausted the connection pool, causing QueuePool timeout errors.
The fix allows Models.get_model_by_id() and has_access() to manage their own short-lived sessions internally, releasing the connection immediately after authorization checks complete.
Remove Depends(get_session) from the /v1/chat/completions endpoint to prevent database connections from being held during the entire duration of LLM calls.
Previously, the database session was acquired at request start and held until the streaming response completed. Under concurrent load, this exhausted the connection pool, causing QueuePool timeout errors.
The fix allows Models.get_model_by_id() and has_access() to manage their own short-lived sessions internally, releasing the connection immediately after authorization checks complete.
Remove Depends(get_session) from the /api/chat endpoint to prevent database connections from being held during the entire duration of LLM calls (30-60+ seconds for streaming responses).
Previously, the database session was acquired at request start and held until the streaming response completed. Under concurrent load, this exhausted the connection pool, causing QueuePool timeout errors for other database operations.
The fix allows Models.get_model_by_id() and has_access() to manage their own short-lived sessions internally, releasing the connection immediately after the quick authorization checks complete - before the slow external LLM API call begins.
- Fix file handle memory leak in download_file_stream by properly closing and reopening files
- Add requests.Session context manager for proper HTTP connection cleanup
- Remove unnecessary file.seek(0) after file reopening
- Add timeout to prevent hanging connections
This prevents memory accumulation during large file downloads and ensures
proper resource cleanup in all scenarios.
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
- Replace Request object with user.id in cache key for get_all_models
- Request objects are new instances per HTTP request, preventing cache hits
- Cache keys now use user.id ensuring proper cache functionality
- Affects both Ollama and OpenAI model list endpoints
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Timothy Jaeryang Baek