SciMesh/STATUS.md

SciMesh Status

Updated: 2026-07-23
Branch baseline: main at b4a89dd (coordinator merge)

Current state

The local Python molecular workloads are implemented and tested. They provide the reference behaviour for future distributed execution:

• similarity-search: streaming ChEMBL TSV search, Morgan fingerprints, Tanimoto scoring, heap-based top-k, CSV and image output;
• similarity-graph: exact sparse graph, block-based pair comparisons, deterministic CSV output;
• Python Worker skeleton: claim, heartbeat, input checksum validation, artifact upload, completion and failure reporting.

The Go coordinator and its PostgreSQL-backed task lifecycle are implemented: registration, atomic claiming, lease renewal, artifact storage, dataset chunking, result/failure reporting, and job progress. The Python worker now uses the live coordinator contract; its HTTP path was exercised against a real Docker PostgreSQL stack on 2026-07-23.

Milestone tracker

CTX	Status	Notes
CTX-00 API and error contract	Implemented	Contract, OpenAPI, and request examples are in docs/.
CTX-01 Go coordinator bootstrap	Implemented	Go service and Docker runtime in coordinator/.
CTX-02 PostgreSQL migrations	Implemented	Applied by the Compose migration service.
CTX-03 Transactional queue	Implemented	Real-PostgreSQL integration tests cover atomic claims and concurrency.
CTX-04 Worker registry and HTTP API	Implemented	Registration, claim, heartbeat, result, failure, and status endpoints.
CTX-05 Artifact storage	Implemented	Coordinator-owned inputs/results, checksum verification, and upload flow.
CTX-06 Python Worker live-contract alignment	Implemented	Worker completed a real uploaded shard via HTTP on 2026-07-23.
CTX-07 Distributed workload protocol	Not started	Depends on artifact and Worker contracts.
CTX-08 Distributed similarity-search	Not started	Local reference exists.
CTX-09 Reducer and final-result API	Not started	Depends on CTX-07 and CTX-08.
CTX-10 Distributed similarity-graph	Not started	Local reference exists.
CTX-11 Dashboard/operator view	Not started	Deferred until API and reducer work.
CTX-12 Reliability, security, CI	In progress	Unit, race, PostgreSQL integration, and smoke checks exist; CI hardening remains.

Next recommended assignment

Assign CTX-07 to the workload role: define distributed job planning and reduction boundaries before implementing distributed search or graph execution.

Known constraints

• Planner/reducer semantics are not implemented; use the local scimesh CLI for complete workload results.
• The worker/coordinator flow currently accepts both underscore API workload names and hyphenated CLI names while the contract is consolidated.
• A real-stack worker test uses a small query_smiles shard. Resolving a query_id once and sharing it across shards belongs to CTX-07.

Update rule

The integration role updates this file only after collecting command output, test evidence, and accepted changes. State facts, revision hashes, blockers, and the next unblocked CTX task; do not mark work complete based on plans alone.