Add Chat.id as a secondary sort key to all paginated chat queries
that use offset/limit pagination. When multiple chats share the same
updated_at timestamp, the database does not guarantee a stable order
across page boundaries, causing chats to appear on multiple pages.
This produces duplicate keys in the Svelte sidebar each-block
(each_key_duplicate error). Adding Chat.id as a tiebreaker ensures
fully deterministic ordering.
Extends the fix from #22383 (which addressed get_chat_ids_by_model_id)
to all remaining paginated chat queries.
* fix: normalize usage tokens + migration streaming/batching
- Migration: replace .fetchall() with yield_per streaming, replace per-message INSERT+SAVEPOINT with batched inserts (5k/batch) with fallback to row-by-row on error, add progress logging
- Write path: call normalize_usage() in upsert_message() before saving to ensure input_tokens/output_tokens always present
- Read path: analytics queries now COALESCE across input_tokens/prompt_tokens and output_tokens/completion_tokens so historical data with OpenAI-format keys is visible
* fix: restore defensive timestamp conversion in migration
Re-add try/except around int(float(timestamp)) that was accidentally dropped. Without this, a non-numeric timestamp string would cause a TypeError on the subsequent comparison, breaking the entire upgrade.
* revert: remove changes to chat_messages.py
The non-streaming response handler was saving assistant messages without
their usage/token data. While the streaming handler correctly extracted
and saved usage information, the non-streaming path discarded it entirely.
This caused assistant messages from non-streaming completions to have
NULL usage in the chat_message table, making them invisible to the
analytics token aggregation queries and contributing to the '0 tokens'
display in Admin Panel Analytics.
Extract and normalize the usage data from the API response and include
it in the database upsert, matching the pattern already used by the
streaming handler.
* perf(models): batch-fetch function valves to eliminate N+1 queries
get_action_priority() called Functions.get_function_valves_by_id()
individually for every action on every model — an N+1 query pattern
that issued one DB round-trip per (action x model) pair.
Add Functions.get_function_valves_by_ids() that fetches all valves in
a single WHERE IN query, then look up each action's valves from the
pre-fetched dict inside get_action_priority().
No functional change — same priority resolution, same sort order.
* Update models.py
* Update models.py
Replace the recursive spread-based implementation with an iterative
push+reverse approach. The recursive version created a new array at
each level of recursion via spread, resulting in O(d^2) array copies
where d is the conversation depth. The iterative version walks from
the target message to the root, pushes each message, and reverses
once at the end for O(d) total work.
No behavioral change - same input produces the same output array.
removeAllDetails() uses replaceOutsideCode() which splits content on
triple-backtick code blocks before applying the details-removal regex.
When thinking/reasoning content inside a <details> block contained
code blocks (backticks survive html.escape), the <details> opening
and </details> closing tags ended up in different split segments,
making the regex unable to match either. This caused thinking content
to leak through to TTS playback.
Fix: add a direct <details> strip (without code-block splitting) as
the first step of getMessageContentParts(), which is the TTS-specific
entry point. This catches the edge case while keeping removeAllDetails
safe for copy-to-clipboard (where legitimate <details> inside code
blocks should be preserved).
Fixes#22197
The chatCompletionEventHandler runs getMessageContentParts() and
removeAllDetails() on every streaming token to extract sentences
for real-time TTS dispatch via CustomEvent('chat'). These functions
perform multiple O(n) regex passes over the full accumulated message
content, resulting in O(n^2) total work over a streaming response.
The only consumer of these events is CallOverlay.svelte, which is
only mounted when showCallOverlay is true. Without the overlay open,
the parsing runs but the dispatched events have no listeners.
Wrap all three TTS parsing blocks in an if () guard
so the expensive regex work is skipped entirely for the vast majority
of users who are not using the voice call feature.
The SQLCipher engine used a dummy sqlite:// URL with a creator function,
which caused SQLAlchemy to auto-select SingletonThreadPool. This pool
non-deterministically closes in-use connections when thread count exceeds
pool_size (default 5), leading to use-after-free segfaults (exit code 139)
in the native sqlcipher3 C library during multi-threaded operations like
user signup.
Now defaults to NullPool (each operation creates/closes its own connection)
for maximum safety with the native C extension. Also respects the
DATABASE_POOL_SIZE setting: if explicitly set >0, QueuePool is used with
the configured pool parameters, matching the behavior of other DB paths.
Fixes#22258
Array.unshift() is O(n) per call because it shifts all existing
elements. In a loop building an n-element array, this makes the
total cost O(n²). Replace with push() + reverse() which is O(n)
total. Produces the identical message ordering.
codeHighlight.ts had a top-level static import of shiki that pulled
the entire highlighter engine (~5-10MB of JavaScript including all
language grammars) into any page that imported the module - even if
only the lightweight isCodeFile() function was used.
Replace the static shiki import with:
- A static set of ~85 common language IDs for synchronous extension
checks (isCodeFile, extToLang) - no shiki dependency needed
- A dynamic import('shiki') inside highlightCode(), which is already
async so callers are completely unaffected
The static language set covers all commonly-used file extensions.
Obscure extensions not in the set simply won't be detected by
isCodeFile() (the file still opens fine, just won't show the code
file indicator). Highlighting itself still works for all shiki
languages since the full bundle loads on demand.
* perf: pre-compile KaTeX Unicode regex at module load time
The katexStart() function was creating a new RegExp with Unicode
property escapes (\p{Script=Han}, \p{Script=Hiragana}, etc.) on
every invocation. Unicode property escapes are extremely expensive
to compile as the regex engine must build character class tables
covering tens of thousands of code points.
Since marked calls the start() function at every character position
while scanning source text, this meant hundreds of regex compilations
per marked.lexer() call, and lexer runs ~60 times/sec during streaming.
Profiling showed KaTeX regex consuming 87% (320ms/365ms) of total
markdown rendering time.
Changes:
- Pre-compile SURROUNDING_CHARS_REGEX once at module load time
- Use .test() instead of .match() to avoid array allocations
- Fix delimiter search to find earliest match, not last match
* perf: replace katexStart with single-pass character scan
The katexStart() function was the dominant cost in marked's lexer,
consuming 55-58% of total markdown rendering time per profiling.
It was called at every character position by marked and each call:
- Looped through 3-5 delimiters, each doing indexOf() on the full
remaining source (3-5 x O(n) string scans per call)
- Ran the complex ruleReg regex with Unicode lookaheads for validation
- On failed validation, created substrings and looped again
Replace with a single linear character scan using charCodeAt that:
- Checks only for $ (charCode 36) or backslash (charCode 92)
- Filters backslash hits by next character to avoid false positives
- Preserves the surrounding-character validation
- Returns immediately on first valid candidate
- Lets the tokenizer handle full validation (it already does this)
This reduces start() from O(n * delimiters * retries) to O(n) with
a very small constant factor per call.
* Update katex-extension.ts
* fix: enforce ownership check on user-memory collection queries
fix: enforce ownership check on user-memory collection queries
Prevent authenticated users from querying other users' memory
collections via the /query/doc and /query/collection endpoints.
A new _validate_collection_access helper rejects requests for
user-memory-{UUID} collections where the UUID does not match
the requesting user. Admins bypass the check.
* Update retrieval.py
* Update retrieval.py
Same optimization as the merged ResponseMessage PR: replace JSON.parse(JSON.stringify()) with structuredClone and add an O(1) fast-path check on content before falling back to full JSON.stringify comparison.
- Use os.path.basename() for filename sanitization instead of fragile blocklist
- Replace ERROR_MESSAGES.DEFAULT(e) with generic error message in both except blocks to prevent CWE-209 information disclosure
- Server-side logging via log.exception(e) is preserved for debugging
Same optimization as ResponseMessage: add O(1) fast-path check on content and done fields before falling back to full JSON.stringify comparison. Avoids expensive serialization when only content changes during streaming.
Three improvements to the socket event emitter hot path (when realtime chat save is enabled):
1. Wrap all synchronous Chats.* DB calls in asyncio.to_thread() to avoid blocking the event loop during streaming. With N concurrent users, sync DB calls serialize all writes and block socket event delivery.
2. Only persist final (done=True) status events to DB. Intermediate statuses (tool calling progress, web search progress, etc.) are ephemeral UI-only data already delivered via socket — writing every one to DB is unnecessary I/O.
3. Convert if/if/if chain to if/elif since event types are mutually exclusive, avoiding unnecessary string comparisons after a match.
Replace 7 instances of JSON.parse(JSON.stringify()) deep cloning with the native structuredClone API. All are on cold paths (model selection, file preparation, history saving) but structuredClone is ~2x faster and more readable.
During streaming, every token change triggers a full JSON.stringify comparison on the code block token object. Add an O(1) fast-path check on token.text and token.raw — the fields that actually change during streaming — before falling back to the expensive JSON.stringify comparison for infrequent structural changes.
Add O(1) array length check before expensive JSON.stringify comparison. During streaming, status history typically only grows via appends, so a length mismatch catches most updates without serialization.
Replace JSON roundtrip with native structuredClone for tool execution result cloning. Also remove unnecessary JSON roundtrip on a static error object literal that is already a fresh value.
Messages.svelte rebuilds the message list by walking the parent chain and creating spread copies on every history.messages change. During streaming, this runs on every token — hundreds of times per second — even though each ResponseMessage already has its own reactive binding for content updates. Throttle the rebuild to once per animation frame (~60Hz) during content-only updates, while keeping immediate rebuilds for structural changes (currentId changes like chat switches, navigation, or new messages). Adds onDestroy cleanup for the pending rAF.
* perf: cache KaTeX module import as singleton across all renderer instances
KatexRenderer.svelte dynamically imports katex, mhchem, and the CSS on every component mount. When a message contains multiple math expressions, this triggers redundant module resolution for each one. Move the import promise to a module-level singleton using Svelte's context='module' script block so it loads once and is shared across all KatexRenderer instances.
* Update KatexRenderer.svelte
ResponseMessage compared the entire message object via JSON.stringify on every reactive tick to detect changes. During streaming, content changes on every token, making the two O(content_length) JSON.stringify calls always return different results — pure wasted work. Add a fast O(1) comparison on content and done fields first. When either differs (the common streaming case), skip straight to cloning. Only fall through to the expensive JSON.stringify comparison for infrequent changes like sources, annotations, or status updates.
fix: resolve valve priority for actions and filters via class instantiation
The priority sorting for action buttons and filter execution order
read valve data directly from the database JSON column using
Functions.get_function_valves_by_id(). This returns only explicitly
saved values — when a developer defines priority as a class default
in their Valves definition (e.g. priority: int = 5) without ever
opening the Valves UI to persist it, the database column remains
empty. Every function then resolves to priority 0, and the preceding
set() deduplication produces non-deterministic iteration order that
the stable sort preserves — resulting in random button placement on
every page load.
The fix instantiates the Valves class with database values as keyword
overrides: Valves(**(db_valves or {})). This merges any persisted
overrides onto the code-defined defaults, matching the pattern already
established in the action execution handler, filter processing
pipeline, and tool module initialization. A secondary sort key (the
function ID) ensures fully deterministic ordering even when multiple
functions share the same priority value.
Affected locations:
- get_action_priority in utils/models.py (action button ordering)
- get_priority in utils/filter.py (filter execution ordering)
feat: sort action buttons by valve priority
Action buttons under assistant messages were rendered in
non-deterministic order due to set() deduplication. They now
respect the priority field from function Valves, sorted ascending
(lower value = appears first, default 0), matching the existing
filter priority mechanism.
fix: model fallback routing for all model types and default model selection
Backend: When ENABLE_CUSTOM_MODEL_FALLBACK is active and a custom model's
base model is unavailable, the fallback now swaps the model and form data
to the configured default model directly. This ensures routing uses the
fallback model's type (pipe, Ollama, or OpenAI) instead of the original
model's type, which previously caused "Model not found" errors when the
fallback was a different backend type.
Frontend: Fixed default model selection in new chat initialization where
the admin-configured default models were always overwritten by the first
available model. The first-available fallback now only triggers when the
configured defaults don't resolve to valid available models.
This commit adds aria-labels to the text inputs and textareas that previously lacked them, applies role=switch to inputs, and adds accessible titles to floating quick actions.
This commit adds aria-labels to the search inputs, select fields, action buttons, and close buttons on modals across the admin users layout and the site changelog modal for improved screen reader support.
fix: reset taskIds and messageQueue on new chat
Fixes a bug where clicking "New Chat" after sending a message would
silently drop subsequent messages. The initNewChat function reset most
chat state but did not clear taskIds or messageQueue, causing
submitPrompt to queue messages indefinitely instead of sending them.
Adds critical accessibility fixes across various app components:
- auth/+page: provide alt text for logo, turn on screenReader support for password input, add aria-required, hide decorative SVGs from AT
- AppSidebar: wrap navigation icons in a <nav> structure, provide ARIA labels for Home and Chat icons
- s/[id]/+page: convert structural divs into semantically accurate h1 heading and time element, wrap message display in main region
- OnBoarding: replace flawed aria-labelledby with direct aria-label on start button
- NotificationToast: provide role='status' and aria-live='polite' for proper screen reader broadcasting
- ChangelogModal: add required heading semantics for structure
- AddFilesPlaceholder: provide heading element role for standalone text content
- ImportModal: provide aria-label for close button
Addresses WCAG 4.1.3, 1.1.1, 3.3.2, and 1.3.1.
Add aria-hidden='true' to 112 SVG icon components in src/lib/components/icons/ that were missing this attribute. Decorative icons that convey no semantic meaning should be hidden from the accessibility tree to prevent screen readers from attempting to read meaningless SVG markup (WCAG 1.1.1 Non-text Content, WCAG 4.1.2 Name, Role, Value).
The remaining 60 icon files already had aria-hidden='true' set. All 172 icon components now consistently declare aria-hidden='true' on their root svg element.
- Replace incorrect aria-roledescription='model-item' with role='option' and aria-selected on ModelItem.svelte. The previous attribute was not a valid ARIA role description and provided no useful information to screen readers.
- Add contextual aria-label to each model item button (e.g. 'Select GPT-4 model') instead of just the raw model name, making the action clear to screen reader users.
- Add role='listbox' and aria-label='Available models' to the scrollable model list container in Selector.svelte so screen readers announce the container's purpose and navigate items correctly.
- Make the model selector trigger button's aria-label dynamic: it now announces 'Selected model: GPT-4' when a model is selected, falling back to 'Select a model' when nothing is selected.
- Add aria-label to the eject (unload) button in ModelItem.svelte so screen readers announce its purpose.
- Add aria-label to the cancel download button in Selector.svelte with the specific model name being canceled.
- Improve model profile image alt text from generic 'Model' to contextual '{{modelName}} profile image'.
* perf: eliminate 2 redundant full chat deserialization on every message send (#162)
Problem:
Every message send triggered get_chat_by_id_and_user_id which loads the
entire Chat row — including the potentially massive JSON blob containing
the full conversation history — even when the caller only needed a
simple yes/no ownership check or a single column value.
Two call sites in the message-send hot path were doing this:
1. main.py ownership verification: loaded the entire chat object including
all message history JSON, then checked `if chat is None`. The JSON blob
was immediately discarded — only the existence of the row mattered.
2. middleware.py folder check: loaded the entire chat object including all
message history JSON, then read only `chat.folder_id` — a plain column
on the chat table that requires zero JSON parsing.
Fix:
- Added `chat_exists_by_id_and_user_id()`: uses SQL EXISTS subquery which
returns a boolean without loading any row data. The database can satisfy
this from the primary key index alone.
- Added `get_chat_folder_id()`: queries only the `folder_id` column via
`db.query(Chat.folder_id)`, which tells SQLAlchemy to SELECT only that
single column instead of the entire row.
Both new methods preserve the same error handling semantics (return
False/None on exception) and user_id filtering (ownership check) as
the original get_chat_by_id_and_user_id.
Impact:
- Best case (typical): eliminates deserializing 2 full chat JSON blobs per
message send. For long conversations (hundreds of messages with tool
calls, images, file attachments), this blob can be multiple megabytes.
- Worst case: no regression — the new queries are strictly cheaper than
the old ones (less data transferred, less Python object construction,
no Pydantic model_validate overhead).
- The 3 remaining full chat loads in process_chat_payload (load_messages_from_db,
add_file_context, chat_image_generation_handler) are left untouched as
they genuinely need the full history and require separate analysis.
* Address maintainer feedback: rename method and inline call (#166)
- Rename chat_exists_by_id_and_user_id -> is_chat_owner
- Remove intermediate chat_owned variable; call is_chat_owner directly in if condition
feat: add citation sources for fetch_url tool results
URL fetches now produce clickable citation sources in the UI, matching
the existing behavior of search_web and knowledge file tools. When a
model calls fetch_url during native tool calling, the fetched URL
appears as a citable source with a content preview, giving users full
transparency into what pages the model referenced.
* feat: add sortable columns to groups admin panel
Make the Group and Users column headers in the admin groups list clickable to sort groups alphabetically by name or numerically by member count. Clicking a column toggles ascending/descending order, indicated by a chevron icon. When no sort is active, the default API order (by updated_at) is preserved.
* Update Groups.svelte
* Update Groups.svelte
fix: gate model default features on global config and user permissions
If you disabled code interpreter globally and in user permissions but
enabled it as a default feature on a model, the code interpreter pill
still appeared in the chat input. Same issue for web search and image
generation.
The setDefaults function in Chat.svelte activated model default features
based solely on the model's capability flag, ignoring whether the feature
was globally enabled or allowed by user permissions. Added the same
global config and user permission checks already used by the integrations
menu visibility and the features object sent to the backend.
Previously loaded the entire ChatModel (including the full conversation JSON
blob) just to extract the title string. Now queries only the Chat.title
column directly, which is already a top-level DB column.
The GET /chats/shared endpoint was loading full Chat rows including
the entire conversation history JSON blob, only to discard it and
return SharedChatResponse (id, title, share_id, timestamps). Now
uses with_entities() to select only the 5 needed columns, avoiding
deserialization of potentially large chat JSON for every shared chat.
When models reference functions (via filterIds/actionIds) that no longer
exist in the database, the /api/models endpoint crashes with a 500 error,
preventing the UI from loading chats entirely. This can happen after
upgrades when built-in functions are removed or when user-created
functions are deleted while still referenced by models.
Instead of raising an exception, log at INFO level and skip the missing
function so the rest of the models load successfully.
Fixes#21464https://claude.ai/code/session_015JRM7m2bNeZPBBmV2Gv4Mj
Co-authored-by: Claude <noreply@anthropic.com>
The sharePublic prop in editor components (Knowledge, Tools, Skills,
Prompts, Models) incorrectly included an "|| edit" / "|| write_access"
condition, allowing users with write access to see and use the "Public"
sharing option regardless of their actual public sharing permission.
Additionally, all backend access/update endpoints only verified write
authorization but did not check the corresponding sharing.public_*
permission, allowing direct API calls to bypass frontend restrictions
entirely.
Frontend: removed the edit/write_access bypass from sharePublic in all
five editor components so visibility is gated solely by the user's
sharing.public_* permission or admin role.
Backend: added has_public_read_access_grant checks to the access/update
endpoints in knowledge.py, tools.py, prompts.py, skills.py, models.py,
and notes.py. Public grants are silently stripped when the user lacks
the corresponding permission.
Fixes#21356
Ensure chat_id is reliably passed to function pipelines/manifolds during internal task invocations (web search query generation, RAG query generation, image prompt generation).
This allows stateful functions to maintain per-chat state without fragmentation, as they will now receive a consistent chat_id for all chat-scoped invocations including internal tasks.
Backend changes:
- Pass chat_id in generate_queries call for web search
- Pass chat_id in generate_queries call for RAG/retrieval
- Pass chat_id in generate_image_prompt call
Frontend changes:
- Add optional chat_id parameter to generateQueries API function
- Add optional chat_id parameter to generateAutoCompletion API function
Fixes#20563
* fix: prevent worker death during document upload by using run_coroutine_threadsafe
Replace asyncio.run() with asyncio.run_coroutine_threadsafe() in
save_docs_to_vector_db() to prevent uvicorn worker health check failures.
The issue: asyncio.run() creates a new event loop and blocks the thread
completely, preventing the worker from responding to health checks during
long-running embedding operations (>5 seconds default timeout).
The fix: Schedule the async embedding work on the main event loop using
run_coroutine_threadsafe(). This keeps the main loop responsive to health
check pings while the sync caller waits for the result.
Changes:
- main.py: Store main event loop reference in app.state.main_loop at startup
- retrieval.py: Use run_coroutine_threadsafe() instead of asyncio.run()
https://claude.ai/code/session_01UQSYvSTkXb57sFb7M85Kcw
* add env var
---------
Co-authored-by: Claude <noreply@anthropic.com>
fix: reduce TTFT by caching model lookups in chat completion
Skip expensive get_all_models() calls when models are already cached
in app.state. This significantly reduces Time To First Token (TTFT)
for chat completions and embeddings requests.
Previously, every request called get_all_models() which fetches model
lists from all configured backends. Now we check the cache first and
only call get_all_models() on cache miss.
Affected endpoints:
- openai: generate_chat_completion, embeddings
- ollama: embed, embeddings
Fixes#20069
Co-authored-by: Michael <42099345+mickeytheseal@users.noreply.github.com>
* fix: add ScopedSession.remove() to prevent idle transaction leaks
The HTTP middleware was calling ScopedSession.commit() but not
ScopedSession.remove(), causing database connections to remain
"checked out" from the pool indefinitely. This resulted in
"idle in transaction" connections in PostgreSQL that could persist
for 30-50+ minutes.
With SQLAlchemy's scoped_session:
- commit() commits but keeps the session active
- remove() is required to return the connection to the pool
This fix adds the missing remove() call, ensuring connections are
properly returned after each HTTP request.
Also includes IDLE_TRANSACTION_ANALYSIS.md documenting the full
root cause analysis and additional recommendations.
* Delete IDLE_TRANSACTION_ANALYSIS.md
---------
Co-authored-by: Claude <noreply@anthropic.com>
Pre-download NLTK punkt_tab during Docker build instead of at runtime.
This fixes document extraction failures in offline/airgapped environments
where the container cannot download the tokenizer data after restarts.
Fixes#21150
Classic298