Worker integration

The Go worker agent (coordinator/cmd/worker-agent, built with make agent) is the worker: a coordinator client, never a database client. It polls the coordinator over HTTP, executes SDK workloads, and uploads partial results through the coordinator — results never carry file:// or worker:// URIs, and failures go to /failure.

The same scientific handlers run in three places: the local CLI cores, the LocalCoreBatchExecutor conformance harness, and the agent — because the agent spawns the workload's own SDK runner in a Python subprocess per task.

Claim lifecycle

register -> claim (one task) -> download input + verify sha256
         -> spawn python -m scimesh.worker.task -> upload partial -> submit

Register: the agent advertises its capabilities (CAPABILITIES, default similarity-search,similarity_search).
Claim: atomic lease of one task; 204 means idle.
Download: the input is streamed and its SHA-256 verified; the bearer token is stripped on cross-origin redirects.
Heartbeat: a background goroutine renews the lease from the returned deadline at less than half the remaining TTL.
Task execution: a Python subprocess (TASK_RUNNER, default python -m scimesh.worker.task) runs the SDK workload; exit 0 writes the result manifest, exit 3 means permanent failure, exit 1 retryable.
Upload: the partial CSV is streamed with X-Worker-ID / X-Task-Attempt headers, then completion references the coordinator-owned artifact id.
Failure: sanitized error_code + message (≤300 chars, no local paths); transient errors are retried with backoff; lost leases stop quietly.

Authentication

WORKER_AUTH_TOKEN — a static bearer token (the shared service token).
WORKER_KEY + USERSERVICE_URL — a long-lived worker key exchanged at the userservice for short-lived JWTs; the agent refreshes them before expiry and retries once after a 401.

Configuration

Variable	Meaning
`COORDINATOR_URL`	Coordinator base URL (required)
`WORKER_AUTH_TOKEN`	Static bearer token (when no worker key)
`WORKER_KEY` / `USERSERVICE_URL`	Worker-key authentication
`WORK_DIR`	Attempt directory root (default `./scimesh-agent-data`)
`WORKER_NAME`	Registered name (default: hostname)
`WORKER_ID`	Fixed identity override (tests)
`CPU_COUNT` / `MEMORY_MB`	Advertised capacity
`POLL_INTERVAL` / `REQUEST_TIMEOUT` / `HEARTBEAT_INTERVAL`	Timings
`CLEANUP_AFTER_SECONDS`	Delete attempt dirs older than this
`CAPABILITIES`	JSON array of advertised capabilities
`TASK_RUNNER`	JSON command array for the task subprocess
`MAX_TASKS` / `EXIT_WHEN_IDLE`	Lifecycle limits

Loading workloads

The task subprocess loads workloads from SCIMESH_WORKLOAD_ALLOWLIST (a JSON array of {distribution, name, version, digest} entries matched against installed scimesh.workloads entry points) or falls back to the built-in similarity-search. Discovery measures the installed package before and after importing and fails transactionally on any mismatch.

make agent
COORDINATOR_URL=https://coordinator.example \
WORKER_AUTH_TOKEN=... \
TASK_RUNNER='["/opt/scimesh/.venv/bin/python","-m","scimesh.worker.task"]' \
./coordinator/bin/worker-agent

v1 contract limits

The coordinator protocol v1 persists flat one-input/one-result tasks. Until a versioned protocol rollout:

map stages with more than one input port (for example similarity-graph's block pairs) are rejected by the task runner — the coordinator does not create such tasks anyway;
max_rows is a plan-time option and is rejected per task;
workloads beyond the allowlisted set are rejected as unsupported.

3.7 KiB Raw Blame History