Let a signed-in user turn their own machine into a worker without the
shared token. The coordinator already binds a JWT-authenticated
registration to owner_id as untrusted; this adds the missing pieces.
userservice: long-lived worker keys (scimesh_wk_live_*, hash-at-rest)
with create/list/revoke and a public /worker-tokens/exchange that trades
a key for a short-lived JWT carrying the owner current role/verified.
python worker: SCIMESH_WORKER_KEY + SCIMESH_USERSERVICE_URL; a token
provider exchanges the key and refreshes the JWT proactively and on 401,
so a long-running worker survives token expiry. Static bearer token path
is unchanged.
coordinator UI: an "add your machine" page that mints a key and shows a
ready-to-run command, proxying key management to the userservice; the
dashboard gains an owner-scoped "my machines" section.
docs: how to run a worker from your account, plus the untrusted/quorum/
verified trust model.
Replaces the C1 quarantine with real verification. Trusted results (lab token,
verified, or admin worker) are accepted directly as before. An untrusted
worker's result is recorded as one vote per (task, owner) in a new task_results
table; the task only completes once QUORUM_SIZE distinct owners submit the same
result hash, otherwise it returns to the queue for another independent compute.
- migration 0013 task_results (one vote per owner, quorum by result_sha256)
- CompleteTask branches on worker trust; unknown worker defaults trusted (safe:
completing needs the lease, whose owner is always a known registered worker)
- claim drops the quarantine and excludes chunks the owner already voted on
- domain Task.ReleaseAfterVote; QUORUM_SIZE config (default 2)
- unit tests: trusted direct-complete, untrusted needs-quorum, can-claim
Reducer and job done/total logic untouched — still one completed task per chunk.
With a UI session, the dashboard and job pages are scoped to the caller:
- Overview lists only the user's own jobs (admin/basic-auth operator: all)
- JobDetail, artifact download, and preview 404 on another user's job
- scoping keys off authctx: no requester (basic auth) still sees everything,
so the fallback operator UI is unchanged
ListJobs gains an owner filter (SQL WHERE) so paging stays correct per user.
Tests cover Overview scoping and cross-user JobDetail rejection.
Polish pass hardening the queue and closing plan gaps.
- Task state machine gains `running`: the first heartbeat moves a task from
leased to running (migrations 0006/0007 add the enum value and extend the
lease-integrity check). verifyLease, ExpireLease, the reaper SQL, and job
progress all treat leased and running alike.
- Worker liveness: a heartbeat from a registered worker (UUID worker_id) bumps
its last_heartbeat_at online; a second background reaper marks workers offline
after WORKER_OFFLINE_AFTER of silence (RunReaper generalized to RunPeriodic).
- Request-size limits: JSON bodies capped at 1 MiB; dataset/artifact uploads
capped at MAX_UPLOAD_BYTES (default 1 GiB) via http.MaxBytesReader.
- Tests cover the running transition, liveness + offline reaper (unit over
memstore and integration over Postgres).
- domain: NewJobWithTasks, DeriveStatus, NewUploadedJob, NewShardTask,
NewWorker, NewArtifact/SetContent (domain 47% -> 88%).
- internal/memstore: in-memory implementations of every usecase port, so
orchestration can be tested without Postgres or a filesystem.
- usecase: claim/renew/complete/fail/create/register/upload/submit-dataset
flows over memstore, including rule-10 cross-task rejection, idempotent
replay, lease sweep-on-claim, and dataset chunking (usecase 0% -> 73%).
- transport: httptest end-to-end over real use cases + memstore — auth,
readiness, full lifecycle, multipart upload + shard input, error mappings
(0% -> 70%).
- postgres integration: fix the tests broken by the artifact_id switch and add
worker-repo, artifact-repo, and shard-task (nullable input_uri) round-trips.
go test -race ./... is clean; golangci-lint (incl. integration tag) reports 0.
Emil