Document team and scaling roadmap

This commit is contained in:
Emil
2026-07-26 20:40:25 +03:00
parent 16db1e41f7
commit f5ead0a450
2 changed files with 46 additions and 2 deletions
+44 -1
View File
@@ -820,6 +820,47 @@ CTX-09 enables final result downloads.
- failure/retry scenarios have automated coverage;
- README contains architecture diagram, security caveat, and troubleshooting.
### CTX-13 — In-worker CPU parallelism
**Goal:** Allow a worker to use a bounded, configured number of CPU threads or
processes while preserving the existing one-task-per-lease coordinator model.
**Depends on:** CTX-12.
**Acceptance criteria:**
- worker concurrency is an explicit configuration value with a safe default of
one;
- a task's internal parallel execution has bounded memory and does not build a
dense N×N similarity matrix;
- CPU-parallel `similarity-search` and `similarity-graph` outputs match the
single-threaded local reference byte-for-byte where ordering is observable;
- result ordering is deterministic across worker counts and block sizes;
- cancellation, lease loss, and worker failure stop child work safely and do
not report a successful result;
- benchmarks and tests cover one-worker and multi-worker configurations.
### CTX-14 — GPU-accelerated workload execution
**Goal:** Add an optional GPU execution backend for supported molecular
workloads, while retaining the validated CPU implementation as the reference
and fallback.
**Depends on:** CTX-13.
**Acceptance criteria:**
- GPU capability and backend version are advertised explicitly by a worker;
- the coordinator schedules GPU work only to compatible workers and CPU-only
workers continue to claim CPU tasks;
- unsupported hardware, unavailable drivers, and GPU execution errors produce
sanitized failures or a documented CPU fallback;
- GPU results match the CPU reference within a documented, tested numerical
tolerance and preserve deterministic output ordering;
- GPU memory use is bounded and no dense N×N similarity matrix is created;
- CPU-only CI verifies backend selection and contract behavior, with GPU
integration tests documented for compatible runners.
---
## 10. Suggested assignment bundles
@@ -948,7 +989,9 @@ Before merging a task, reviewer checks:
Do not start these before CTX-12 is accepted.
- Replace local artifact storage with S3/MinIO behind an `ArtifactStore` API.
- Add worker labels/capacity-aware scheduling and concurrency > 1.
- Add worker labels and capacity-aware scheduling.
- Implement CTX-13 for bounded in-worker CPU parallelism.
- Implement CTX-14 for optional GPU-accelerated workload execution.
- Add cancellation propagation to workers.
- Add image outputs and final PDF reporting to job artifacts.
- Add CV/video workloads using the same planner/runner/reducer contract.
+2 -1
View File
@@ -137,5 +137,6 @@ The package separates common dataset parsing and fingerprints from independent w
- [Emil](https://github.com/emil28092005) — Project Lead
- [Kristina](https://github.com/kristtma) — Tech Lead
- [Veniamin](https://t.me/Veniamin_Kt) — Scientific Lead
- [Arkhip](https://github.com/hIpa-ussr) — Programmer
- [Makar](https://github.com/RERAN4K) — Programmer
- [Reranchik](https://github.com/RERAN4K) — Programmer