demo-down / stop intentionally keep the Postgres and artifact volumes, so a
fresh start still carried old workers/jobs/tasks. Add a reset action (compose
down -v) and a make demo-reset target for a truly pristine restart.
The Add-your-machine page sent the typed name only to the worker key, while
the copyable command hard-coded --worker-name my-machine, so every self-service
worker registered as my-machine regardless of what the user entered. Thread the
name into the command (shell-quoted; falls back to my-machine when blank).
Let a signed-in user turn their own machine into a worker without the
shared token. The coordinator already binds a JWT-authenticated
registration to owner_id as untrusted; this adds the missing pieces.
userservice: long-lived worker keys (scimesh_wk_live_*, hash-at-rest)
with create/list/revoke and a public /worker-tokens/exchange that trades
a key for a short-lived JWT carrying the owner current role/verified.
python worker: SCIMESH_WORKER_KEY + SCIMESH_USERSERVICE_URL; a token
provider exchanges the key and refreshes the JWT proactively and on 401,
so a long-running worker survives token expiry. Static bearer token path
is unchanged.
coordinator UI: an "add your machine" page that mints a key and shows a
ready-to-run command, proxying key management to the userservice; the
dashboard gains an owner-scoped "my machines" section.
docs: how to run a worker from your account, plus the untrusted/quorum/
verified trust model.
Replaces the C1 quarantine with real verification. Trusted results (lab token,
verified, or admin worker) are accepted directly as before. An untrusted
worker's result is recorded as one vote per (task, owner) in a new task_results
table; the task only completes once QUORUM_SIZE distinct owners submit the same
result hash, otherwise it returns to the queue for another independent compute.
- migration 0013 task_results (one vote per owner, quorum by result_sha256)
- CompleteTask branches on worker trust; unknown worker defaults trusted (safe:
completing needs the lease, whose owner is always a known registered worker)
- claim drops the quarantine and excludes chunks the owner already voted on
- domain Task.ReleaseAfterVote; QUORUM_SIZE config (default 2)
- unit tests: trusted direct-complete, untrusted needs-quorum, can-claim
Reducer and job done/total logic untouched — still one completed task per chunk.
A scrape-time collector reports scimesh_tasks/jobs/workers gauges keyed by
status, sourced from cheap GROUP BY queries (StatsRepo), zero-filled across all
known statuses so the dashboard shows flat zeros instead of gaps. A failed
query yields no samples for that scrape rather than crashing it.
Metrics is now built in main so the DB-backed collector can be registered
(NewServer takes *metrics.Metrics; nil self-provisions for tests). Grafana
dashboard gains a Domain state row: tasks/jobs/workers by status and a queue-
depth stat.
The metrics deps pull in modules whose go directive is 1.25, so go mod tidy
raised the module to go 1.25.0. The build image is bumped golang:1.24 ->
golang:1.25-alpine to match (the image runs GOTOOLCHAIN=local and can't
auto-fetch a newer toolchain). CI reads go-version-file, so it follows along.
client_golang pinned to v1.19.1 (stable, same API).
make demo-ui now also starts Prometheus (scrapes coordinator:8080/metrics) and
Grafana with a provisioned datasource and a SciMesh Coordinator dashboard
(request rate & p95 by route, status mix, goroutines, RSS). Grafana allows
anonymous viewing so the dashboard opens without a login; admin/admin to edit.
- monitoring/prometheus.yml + grafana provisioning + dashboard JSON
- docker-compose.monitoring.yml overlay (third -f in demo-ui.sh)
- demo prints the Grafana and Prometheus URLs
Adds internal/metrics: a private registry with the Go runtime + process
collectors and HTTP RED instrumentation (scimesh_http_requests_total and
request_duration_seconds), labelled by method/status and a normalized route so
per-id paths collapse to {id} and never blow up label cardinality.
/metrics is unauthenticated (like /health) for a Prometheus scraper; the
middleware wraps the whole router so every request is measured once.
Shows the signed-in user's id, email, role, verified status, and created-at by
proxying the session token to the userservice /me endpoint (email/created_at
are not in the JWT). Profile link added to the dashboard, job, and admin headers.
Tests: /me proxy forwards the bearer and renders the account; redirect without a session.
Admin-only page to promote/demote/verify/unverify a user by id. Actions proxy
to the userservice forwarding the admin's session JWT, which the userservice
re-checks (defense in depth). Non-admins are redirected off the panel; the
Admin link shows only for admins. Admins already see all jobs on the dashboard.
Tests: requireAdmin gate, bearer forwarding, action/id validation, admin link.
POST /users/{id}/promote and /demote set a user's role (admin/user), admin-only
(403 otherwise). Mirrors the verify endpoints: SetRole use case + repo method,
validated role. Unit, admin-flow, and integration tests included.
Emil