The container runs as uid 10001, but the ./data and ./logs bind mounts were
root-owned, so blob storage failed with "mkdir .staging: permission denied"
and the coordinator crash-looped.
Pre-create the storage and log directories in the image owned by the
coordinator user, and switch the bind mounts to named volumes, which inherit
that ownership. The process can now write to them without running as root.
Wire the artifact storage foundation to HTTP.
- PUT /tasks/{id}/artifacts/{filename}: a worker streams a partial result;
the coordinator verifies lease ownership (foreign worker → 409), streams
the bytes to blob storage while hashing, and records the metadata. An
orphaned blob from a failed metadata insert is cleaned up.
- GET /artifacts/{id}/download: streams an artifact back with its content
type, length, and checksum.
- Ownership is read with a new non-locking TaskRepository.Get, so no row lock
is held across a long upload. Identity travels in X-Worker-ID / X-Task-Attempt
headers per the contract; upload/download bypass the short request timeout.
- docker-compose mounts ./data for durable artifact storage; smoke and
requests.http exercise an upload → foreign-409 → download round-trip.
Align the coordinator with the master PLAN.md (CTX-00, CTX-04) and harden
process startup.
- CTX-00: freeze docs/api-contract.md as the v1 source of truth for the
Go coordinator and Python worker.
- CTX-04: worker registry — workers table (migration 0002), domain.Worker,
RegisterWorker use case, WorkerRepository, and POST /workers/register.
- Contract alignment: claim uses `capabilities` (was `workloads`),
COORDINATOR_TOKEN env (WORKER_AUTH_TOKEN kept as fallback), and
GET /health now reports database readiness (503 when the DB is down).
- Logging: logs are teed to stdout and an optional rotated file (LOG_FILE)
via lumberjack, so they survive a container rebuild.
- Startup resilience: the initial DB connection is retried with backoff,
so the coordinator waits for Postgres to boot instead of crash-looping.
Adds the SciMesh coordinator: a durable task-queue server on PostgreSQL
that owns all database access, with workers reaching it over HTTP only.
Structured as a modular monolith following Clean Architecture:
domain entities and their invariants, no I/O
usecase business operations + repository/clock ports
transport HTTP handlers, DTOs, auth, error mapping
storage PostgreSQL repositories, transactions carried in context
infra config, pool, clock, server, lease reaper
Dependencies point strictly inward; domain imports nothing from the module.
Working: layer wiring, routing, shared-token auth, access logging, request
IDs, domain-error to status-code mapping, transactional boundaries,
graceful shutdown (HTTP drain -> reaper stop -> pool close), migrations,
and a Compose stack starting Postgres -> migrations -> coordinator.
The domain is complete and covered by unit tests that need no database:
lease ownership, stale attempts, idempotent result replay, retry budgets,
and lease expiry.
Repository methods are stubs returning ErrNotImplemented (HTTP 501). The
SQL for atomic claiming (FOR UPDATE SKIP LOCKED) and for lease expiry is
written and ready to wire up.
See coordinator/ARCHITECTURE.md for the layer map and a request traced
through every layer.
Emil