feat: add RAG_RERANKING_BATCH_SIZE configuration option
Add configurable reranker batch size (env var RAG_RERANKING_BATCH_SIZE, default 32) following the same pattern as RAG_EMBEDDING_BATCH_SIZE. - config.py: PersistentConfig for RAG_RERANKING_BATCH_SIZE - main.py: import, state init, pass to get_reranking_function - colbert.py: accept batch_size param in predict() (was hardcoded 32) - utils.py: get_reranking_function passes batch_size at call time - retrieval.py: expose in config GET/POST endpoints and ConfigForm - Documents.svelte: add Reranking Batch Size input in admin settings Closes #23730
This commit is contained in:
@@ -919,7 +919,7 @@ async def generate_embeddings(
|
||||
return embeddings[0] if isinstance(text, str) else embeddings
|
||||
|
||||
|
||||
def get_reranking_function(reranking_engine, reranking_model, reranking_function):
|
||||
def get_reranking_function(reranking_engine, reranking_model, reranking_function, reranking_batch_size=32):
|
||||
if reranking_function is None:
|
||||
return None
|
||||
if reranking_engine == 'external':
|
||||
@@ -928,7 +928,7 @@ def get_reranking_function(reranking_engine, reranking_model, reranking_function
|
||||
)
|
||||
else:
|
||||
return lambda query, documents, user=None: reranking_function.predict(
|
||||
[(query, doc.page_content) for doc in documents]
|
||||
[(query, doc.page_content) for doc in documents], batch_size=int(reranking_batch_size)
|
||||
)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user