feat: add RAG_RERANKING_BATCH_SIZE configuration option
Add configurable reranker batch size (env var RAG_RERANKING_BATCH_SIZE, default 32) following the same pattern as RAG_EMBEDDING_BATCH_SIZE. - config.py: PersistentConfig for RAG_RERANKING_BATCH_SIZE - main.py: import, state init, pass to get_reranking_function - colbert.py: accept batch_size param in predict() (was hardcoded 32) - utils.py: get_reranking_function passes batch_size at call time - retrieval.py: expose in config GET/POST endpoints and ConfigForm - Documents.svelte: add Reranking Batch Size input in admin settings Closes #23730
This commit is contained in:
@@ -2965,6 +2965,12 @@ RAG_RERANKING_MODEL_TRUST_REMOTE_CODE = (
|
||||
os.environ.get('RAG_RERANKING_MODEL_TRUST_REMOTE_CODE', 'True').lower() == 'true'
|
||||
)
|
||||
|
||||
RAG_RERANKING_BATCH_SIZE = PersistentConfig(
|
||||
'RAG_RERANKING_BATCH_SIZE',
|
||||
'rag.reranking_batch_size',
|
||||
int(os.environ.get('RAG_RERANKING_BATCH_SIZE', '32')),
|
||||
)
|
||||
|
||||
RAG_EXTERNAL_RERANKER_URL = PersistentConfig(
|
||||
'RAG_EXTERNAL_RERANKER_URL',
|
||||
'rag.external_reranker_url',
|
||||
|
||||
Reference in New Issue
Block a user