# Worker integration

The **Go worker agent** (`coordinator/cmd/worker-agent`, built with
`make agent`) is the worker: a coordinator client, never a database client.
It polls the coordinator over HTTP, executes SDK workloads, and uploads
partial results through the coordinator — results never carry `file://` or
`worker://` URIs, and failures go to `/failure`.

The same scientific handlers run in three places: the local CLI cores, the
`LocalCoreBatchExecutor` conformance harness, and the agent — because the
agent spawns the workload's own SDK runner in a Python subprocess per task.

## Claim lifecycle

```text
register -> claim (one task) -> download input + verify sha256
         -> spawn python -m scimesh.worker.task -> upload partial -> submit
```

- **Register**: the agent advertises its capabilities
  (`CAPABILITIES`, default `similarity-search,similarity_search`).
- **Claim**: atomic lease of one task; `204` means idle.
- **Download**: the input is streamed and its SHA-256 verified; the bearer
  token is stripped on cross-origin redirects.
- **Heartbeat**: a background goroutine renews the lease from the returned
  deadline at less than half the remaining TTL.
- **Task execution**: a Python subprocess (`TASK_RUNNER`, default
  `python -m scimesh.worker.task`) runs the SDK workload; exit 0 writes the
  result manifest, exit 3 means permanent failure, exit 1 retryable.
- **Upload**: the partial CSV is streamed with `X-Worker-ID` /
  `X-Task-Attempt` headers, then completion references the
  coordinator-owned artifact id.
- **Failure**: sanitized `error_code` + message (≤300 chars, no local
  paths); transient errors are retried with backoff; lost leases stop
  quietly.

## Authentication

- `WORKER_AUTH_TOKEN` — a static bearer token (the shared service token).
- `WORKER_KEY` + `USERSERVICE_URL` — a long-lived worker key exchanged at
  the userservice for short-lived JWTs; the agent refreshes them before
  expiry and retries once after a 401.

## Configuration

| Variable | Meaning |
| --- | --- |
| `COORDINATOR_URL` | Coordinator base URL (required) |
| `WORKER_AUTH_TOKEN` | Static bearer token (when no worker key) |
| `WORKER_KEY` / `USERSERVICE_URL` | Worker-key authentication |
| `WORK_DIR` | Attempt directory root (default `./scimesh-agent-data`) |
| `WORKER_NAME` | Registered name (default: hostname) |
| `WORKER_ID` | Fixed identity override (tests) |
| `CPU_COUNT` / `MEMORY_MB` | Advertised capacity |
| `POLL_INTERVAL` / `REQUEST_TIMEOUT` / `HEARTBEAT_INTERVAL` | Timings |
| `CLEANUP_AFTER_SECONDS` | Delete attempt dirs older than this |
| `CAPABILITIES` | JSON array of advertised capabilities |
| `TASK_RUNNER` | JSON command array for the task subprocess |
| `MAX_TASKS` / `EXIT_WHEN_IDLE` | Lifecycle limits |

## Loading workloads

The task subprocess loads workloads from `SCIMESH_WORKLOAD_ALLOWLIST` (a
JSON array of `{distribution, name, version, digest}` entries matched
against installed `scimesh.workloads` entry points) or falls back to the
built-in `similarity-search`. Discovery measures the installed package
before and after importing and fails transactionally on any mismatch.

```bash
make agent
COORDINATOR_URL=https://coordinator.example \
WORKER_AUTH_TOKEN=... \
TASK_RUNNER='["/opt/scimesh/.venv/bin/python","-m","scimesh.worker.task"]' \
./coordinator/bin/worker-agent
```

## v1 contract limits

The coordinator protocol v1 persists flat one-input/one-result tasks. Until
a versioned protocol rollout:

- map stages with more than one input port (for example
  `similarity-graph`'s block pairs) are **rejected by the task runner** —
  the coordinator does not create such tasks anyway;
- `max_rows` is a plan-time option and is rejected per task;
- workloads beyond the allowlisted set are rejected as unsupported.