Files
SciMesh/docs/api-contract.md
T
Efremenko Arhip dc92121acc feat(coordinator): worker registry, API contract, logging & DB retry
Align the coordinator with the master PLAN.md (CTX-00, CTX-04) and harden
process startup.

- CTX-00: freeze docs/api-contract.md as the v1 source of truth for the
  Go coordinator and Python worker.
- CTX-04: worker registry — workers table (migration 0002), domain.Worker,
  RegisterWorker use case, WorkerRepository, and POST /workers/register.
- Contract alignment: claim uses `capabilities` (was `workloads`),
  COORDINATOR_TOKEN env (WORKER_AUTH_TOKEN kept as fallback), and
  GET /health now reports database readiness (503 when the DB is down).
- Logging: logs are teed to stdout and an optional rotated file (LOG_FILE)
  via lumberjack, so they survive a container rebuild.
- Startup resilience: the initial DB connection is retried with backoff,
  so the coordinator waits for Postgres to boot instead of crash-looping.
2026-07-23 13:45:31 +03:00

4.6 KiB

SciMesh coordinator ↔ worker API contract (v1)

Status marker: v1. This document is the single source of truth for the Go coordinator and the Python Worker Daemon. It is derived from PLAN.md §5 and must be updated in the same change as any behaviour it describes.

  • Auth: every endpoint except readiness requires Authorization: Bearer <token>.
  • Identity: every mutating worker request carries worker_id and attempt; they are checked against the current task lease in PostgreSQL. A stale attempt gets 409.
  • Timestamps: UTC, RFC 3339 (e.g. 2026-07-22T12:05:00Z).
  • Unknown JSON fields are rejected with 400.

Implementation status

Endpoint Contract Coordinator
GET /health readiness incl. DB done
POST /workers/register register + capabilities done
POST /tasks/claim atomic lease done
POST /tasks/{id}/heartbeat renew lease done
POST /tasks/{id}/result complete 🟡 done, but result is a URI today; moves to artifact_id in CTX-05
POST /tasks/{id}/failure fail done
GET /jobs/{id} progress done
GET /tasks/{id}/input download shard CTX-05
PUT /tasks/{id}/artifacts/{name} upload partial CTX-05

Readiness

GET /health

200 {"status":"ok"} when the database is reachable; 503 {"status":"unavailable"} otherwise. Unauthenticated.

Register worker

POST /workers/register
Authorization: Bearer <token>
Content-Type: application/json

{
  "name": "lab-worker-01",
  "capabilities": ["similarity-search", "similarity-graph"],
  "cpu_count": 8,
  "memory_mb": 16384
}

201:

{ "worker_id": "uuid", "heartbeat_interval_seconds": 15 }

cpu_count/memory_mb are accepted for forward compatibility and not yet persisted. capabilities must be non-empty (an allowlisted workload set).

Claim task

POST /tasks/claim
Authorization: Bearer <token>
Content-Type: application/json

{ "worker_id": "uuid", "capabilities": ["similarity-search"], "max_concurrency": 1 }
  • 204 No Content: no compatible task.
  • 200 OK: a task is leased atomically.
{
  "task_id": "uuid",
  "attempt": 1,
  "lease_expires_at": "2026-07-22T12:05:00Z",
  "workload": "similarity-search",
  "input": { "uri": "https://coordinator/tasks/uuid/input", "sha256": "hex" },
  "parameters": { "query_id": "CHEMBL939", "top_k": 20 }
}

max_concurrency is accepted; the coordinator leases one task per call for now.

Renew lease (heartbeat)

POST /tasks/{task_id}/heartbeat
Authorization: Bearer <token>
Content-Type: application/json

{ "worker_id": "uuid", "attempt": 1 }

Response must contain a renewed deadline:

{ "lease_expires_at": "2026-07-22T12:10:00Z" }

The worker schedules the next heartbeat before half of the returned TTL, never on a fixed interval alone.

Download input or shard (CTX-05)

GET /tasks/{task_id}/input returns the artifact owned by the current task. The worker verifies its SHA-256 before execution. If the URI redirects to another origin, the worker removes the coordinator bearer token.

Upload a partial artifact (CTX-05)

PUT /tasks/{task_id}/artifacts/{filename}
Authorization: Bearer <token>
Content-Type: text/csv
X-Worker-ID: uuid
X-Task-Attempt: 1

<streamed bytes>

200:

{ "artifact_id": "uuid", "uri": "https://coordinator/artifacts/uuid/download",
  "sha256": "hex", "size_bytes": 1234 }

Complete or fail task

POST /tasks/{task_id}/result
Authorization: Bearer <token>
Content-Type: application/json

{
  "worker_id": "uuid",
  "attempt": 1,
  "result": { "artifact_id": "uuid", "sha256": "hex", "content_type": "text/csv" },
  "metrics": { "elapsed_seconds": 12.4, "processed_rows": 10000 }
}

Transitional: the coordinator currently accepts result_uri + result_sha256 instead of result.artifact_id. This switches to the artifact form in CTX-05, once upload exists. Until then no worker-supplied file:///worker:// URI is valid in persisted metadata.

POST /tasks/{task_id}/failure

Same identity fields, plus sanitized error_code, error_message, retryable. Never a traceback, token, or absolute worker path.

Idempotency and errors

Situation Response
No compatible task 204
Worker/attempt does not own lease 409
Artifact does not belong to task/attempt 409
Same completion, same manifest 200 idempotent
Same attempt, different manifest 409
Invalid parameters/input 400
Auth failure 401
Unknown job/task 404