Files
SciMesh/docs/api-contract.md
T
Efremenko Arhip 6d45406ee0 feat(coordinator): artifact upload/download endpoints (CTX-05, part 2)
Wire the artifact storage foundation to HTTP.

- PUT /tasks/{id}/artifacts/{filename}: a worker streams a partial result;
  the coordinator verifies lease ownership (foreign worker → 409), streams
  the bytes to blob storage while hashing, and records the metadata. An
  orphaned blob from a failed metadata insert is cleaned up.
- GET /artifacts/{id}/download: streams an artifact back with its content
  type, length, and checksum.
- Ownership is read with a new non-locking TaskRepository.Get, so no row lock
  is held across a long upload. Identity travels in X-Worker-ID / X-Task-Attempt
  headers per the contract; upload/download bypass the short request timeout.
- docker-compose mounts ./data for durable artifact storage; smoke and
  requests.http exercise an upload → foreign-409 → download round-trip.
2026-07-23 14:12:18 +03:00

4.7 KiB

SciMesh coordinator ↔ worker API contract (v1)

Status marker: v1. This document is the single source of truth for the Go coordinator and the Python Worker Daemon. It is derived from PLAN.md §5 and must be updated in the same change as any behaviour it describes.

  • Auth: every endpoint except readiness requires Authorization: Bearer <token>.
  • Identity: every mutating worker request carries worker_id and attempt; they are checked against the current task lease in PostgreSQL. A stale attempt gets 409.
  • Timestamps: UTC, RFC 3339 (e.g. 2026-07-22T12:05:00Z).
  • Unknown JSON fields are rejected with 400.

Implementation status

Endpoint Contract Coordinator
GET /health readiness incl. DB done
POST /workers/register register + capabilities done
POST /tasks/claim atomic lease done
POST /tasks/{id}/heartbeat renew lease done
POST /tasks/{id}/result complete 🟡 done, but result is a URI today; moves to artifact_id next
POST /tasks/{id}/failure fail done
GET /jobs/{id} progress done
PUT /tasks/{id}/artifacts/{name} upload partial done
GET /artifacts/{id}/download download by id done
GET /tasks/{id}/input download shard needs input artifacts (upload+chunking)

Readiness

GET /health

200 {"status":"ok"} when the database is reachable; 503 {"status":"unavailable"} otherwise. Unauthenticated.

Register worker

POST /workers/register
Authorization: Bearer <token>
Content-Type: application/json

{
  "name": "lab-worker-01",
  "capabilities": ["similarity-search", "similarity-graph"],
  "cpu_count": 8,
  "memory_mb": 16384
}

201:

{ "worker_id": "uuid", "heartbeat_interval_seconds": 15 }

cpu_count/memory_mb are accepted for forward compatibility and not yet persisted. capabilities must be non-empty (an allowlisted workload set).

Claim task

POST /tasks/claim
Authorization: Bearer <token>
Content-Type: application/json

{ "worker_id": "uuid", "capabilities": ["similarity-search"], "max_concurrency": 1 }
  • 204 No Content: no compatible task.
  • 200 OK: a task is leased atomically.
{
  "task_id": "uuid",
  "attempt": 1,
  "lease_expires_at": "2026-07-22T12:05:00Z",
  "workload": "similarity-search",
  "input": { "uri": "https://coordinator/tasks/uuid/input", "sha256": "hex" },
  "parameters": { "query_id": "CHEMBL939", "top_k": 20 }
}

max_concurrency is accepted; the coordinator leases one task per call for now.

Renew lease (heartbeat)

POST /tasks/{task_id}/heartbeat
Authorization: Bearer <token>
Content-Type: application/json

{ "worker_id": "uuid", "attempt": 1 }

Response must contain a renewed deadline:

{ "lease_expires_at": "2026-07-22T12:10:00Z" }

The worker schedules the next heartbeat before half of the returned TTL, never on a fixed interval alone.

Download input or shard (CTX-05)

GET /tasks/{task_id}/input returns the artifact owned by the current task. The worker verifies its SHA-256 before execution. If the URI redirects to another origin, the worker removes the coordinator bearer token.

Upload a partial artifact (CTX-05)

PUT /tasks/{task_id}/artifacts/{filename}
Authorization: Bearer <token>
Content-Type: text/csv
X-Worker-ID: uuid
X-Task-Attempt: 1

<streamed bytes>

200:

{ "artifact_id": "uuid", "uri": "https://coordinator/artifacts/uuid/download",
  "sha256": "hex", "size_bytes": 1234 }

Complete or fail task

POST /tasks/{task_id}/result
Authorization: Bearer <token>
Content-Type: application/json

{
  "worker_id": "uuid",
  "attempt": 1,
  "result": { "artifact_id": "uuid", "sha256": "hex", "content_type": "text/csv" },
  "metrics": { "elapsed_seconds": 12.4, "processed_rows": 10000 }
}

Transitional: the coordinator currently accepts result_uri + result_sha256 instead of result.artifact_id. This switches to the artifact form in CTX-05, once upload exists. Until then no worker-supplied file:///worker:// URI is valid in persisted metadata.

POST /tasks/{task_id}/failure

Same identity fields, plus sanitized error_code, error_message, retryable. Never a traceback, token, or absolute worker path.

Idempotency and errors

Situation Response
No compatible task 204
Worker/attempt does not own lease 409
Artifact does not belong to task/attempt 409
Same completion, same manifest 200 idempotent
Same attempt, different manifest 409
Invalid parameters/input 400
Auth failure 401
Unknown job/task 404