feat: add RAG_RERANKING_BATCH_SIZE configuration option

Add configurable reranker batch size (env var RAG_RERANKING_BATCH_SIZE,
default 32) following the same pattern as RAG_EMBEDDING_BATCH_SIZE.

- config.py: PersistentConfig for RAG_RERANKING_BATCH_SIZE
- main.py: import, state init, pass to get_reranking_function
- colbert.py: accept batch_size param in predict() (was hardcoded 32)
- utils.py: get_reranking_function passes batch_size at call time
- retrieval.py: expose in config GET/POST endpoints and ConfigForm
- Documents.svelte: add Reranking Batch Size input in admin settings

Closes #23730
This commit is contained in:
Timothy Jaeryang Baek
2026-04-17 08:35:45 +09:00
parent 70a6a24f14
commit 4d2f189810
6 changed files with 40 additions and 5 deletions
@@ -1185,6 +1185,23 @@
</div>
{/if}
<div class=" mb-2.5 flex w-full justify-between">
<div class=" self-center text-xs font-medium">
{$i18n.t('Reranking Batch Size')}
</div>
<div class="">
<input
bind:value={RAGConfig.RAG_RERANKING_BATCH_SIZE}
type="number"
class=" bg-transparent text-center w-14 outline-none"
min="1"
max="16000"
step="1"
/>
</div>
</div>
<div class=" mb-2.5 flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Top K')}</div>
<div class="flex items-center relative">