Merge branch 'fix/worker-contract-hardening'
# Conflicts: # docs/api-contract.md # scimesh/worker/artifacts.py # scimesh/worker/coordinator.py # scimesh/worker/daemon.py # scimesh/worker/models.py # tests/test_worker_daemon.py
This commit is contained in:
@@ -1,6 +1,11 @@
|
||||
# SciMesh
|
||||
|
||||
SciMesh is a small local framework for scientific workloads on molecular datasets. It currently provides exact molecular similarity search and exact sparse similarity-graph construction. It runs in one local Python process: there is no network service, multiprocessing, coordinator, database, or dense similarity matrix.
|
||||
SciMesh is a scientific-workload framework for molecular datasets. Its public CLI
|
||||
currently runs exact similarity search and sparse similarity-graph construction
|
||||
locally in one Python process; it creates no dense similarity matrix. A Python
|
||||
Worker client and the planned Go/PostgreSQL coordinator contract are tracked in
|
||||
the repository, but distributed execution is not available yet; see
|
||||
[`STATUS.md`](STATUS.md).
|
||||
|
||||
The ChEMBL TSV database is intentionally not included in this repository. Download it separately and pass its path to the commands below. The expected columns are `chembl_id` and `canonical_smiles`.
|
||||
|
||||
|
||||
@@ -27,7 +27,9 @@ inside the daemon.
|
||||
`scimesh-worker`.
|
||||
2. Configuration via environment variables and CLI overrides:
|
||||
- `SCIMESH_COORDINATOR_URL` (required);
|
||||
- `SCIMESH_WORKER_ID` (required, stable UUID or hostname-derived value);
|
||||
- `SCIMESH_WORKER_NAME` (optional; defaults to the hostname);
|
||||
- `SCIMESH_WORKER_ID` (optional legacy/test override; production identity is
|
||||
returned by registration);
|
||||
- working directory for downloaded inputs and generated outputs;
|
||||
- poll interval and request timeout;
|
||||
- optional bearer token.
|
||||
@@ -43,6 +45,28 @@ inside the daemon.
|
||||
Use JSON over HTTPS. Claiming a task changes its state, so use `POST`, even if
|
||||
the initial diagram labels the endpoint as `GET /get_task`.
|
||||
|
||||
`docs/api-contract.md` is the authoritative API schema. This document explains
|
||||
the daemon workflow and must not introduce a different request or response
|
||||
shape.
|
||||
|
||||
### Register worker
|
||||
|
||||
At daemon startup, register the worker capabilities before claiming tasks:
|
||||
|
||||
```http
|
||||
POST /workers/register
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"name": "lab-worker-01",
|
||||
"capabilities": ["similarity-search", "similarity-graph"],
|
||||
"cpu_count": 8,
|
||||
"memory_mb": 16384
|
||||
}
|
||||
```
|
||||
|
||||
The `worker_id` returned by this endpoint is used for the daemon lifetime.
|
||||
|
||||
### Claim a task
|
||||
|
||||
```http
|
||||
@@ -90,8 +114,8 @@ Content-Type: application/json
|
||||
{
|
||||
"worker_id": "worker-01",
|
||||
"attempt": 1,
|
||||
"status": "completed",
|
||||
"result": {
|
||||
"artifact_id": "0d2d5a53-4c7e-467e-93d2-45ed2dc18e46",
|
||||
"uri": "https://coordinator.example/tasks/0d2d/result.csv",
|
||||
"sha256": "...",
|
||||
"content_type": "text/csv"
|
||||
@@ -121,7 +145,10 @@ The coordinator streams the artifact to its configured storage and responds:
|
||||
|
||||
```json
|
||||
{
|
||||
"uri": "https://coordinator.example/tasks/0d2d/artifacts/result.csv"
|
||||
"artifact_id": "0d2d5a53-4c7e-467e-93d2-45ed2dc18e46",
|
||||
"uri": "https://coordinator.example/tasks/0d2d/artifacts/result.csv",
|
||||
"sha256": "...",
|
||||
"size_bytes": 1234
|
||||
}
|
||||
```
|
||||
|
||||
@@ -142,7 +169,9 @@ idle -> claiming -> downloading -> running -> uploading -> submitting -> idle
|
||||
- Verify the input checksum before running.
|
||||
- Create one isolated task directory: `<work-dir>/<task-id>/<attempt>/`.
|
||||
- Invoke the runner with an explicit argument list, never `shell=True`.
|
||||
- Upload/submit exactly the produced result files listed by the runner.
|
||||
- Upload the produced result artifact before submitting its manifest.
|
||||
- Version 1 produces exactly one CSV partial result. Multi-artifact manifests
|
||||
require an explicit future API-contract change.
|
||||
- Do not mark a task completed until every submitted artifact has a durable
|
||||
coordinator-provided URI.
|
||||
- A timeout, network error, or rejected submission must leave the local task
|
||||
|
||||
Reference in New Issue
Block a user