Compare commits
11
Commits
v1.0.0
...
v1.1.0-alpha.1
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0494b8bc2a | ||
|
|
0f9cbcec48 | ||
|
|
a4abdb970a | ||
|
|
b4fe7aaf72 | ||
|
|
e308085fe2 | ||
|
|
988101afab | ||
|
|
215325a9e0 | ||
|
|
c06d8673ce | ||
|
|
1473bbe2a8 | ||
|
|
9883def0c2 | ||
|
|
079eca071e |
@@ -58,6 +58,8 @@ jobs:
|
||||
needs: binaries
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
path: artifacts
|
||||
@@ -72,7 +74,12 @@ jobs:
|
||||
|
||||
- uses: softprops/action-gh-release@v2
|
||||
with:
|
||||
files: artifacts/*
|
||||
files: |
|
||||
artifacts/*
|
||||
install.sh
|
||||
install.ps1
|
||||
# Pre-release tags (e.g. v1.1.0-alpha.1) publish as pre-releases.
|
||||
prerelease: ${{ contains(github.ref_name, '-alpha') }}
|
||||
generate_release_notes: true
|
||||
|
||||
image:
|
||||
|
||||
+16
-15
@@ -1,21 +1,22 @@
|
||||
COMPLETED
|
||||
# Session Goal
|
||||
|
||||
давай теперь почистим проект от линего кода
|
||||
COMPLETED
|
||||
|
||||
выполни полностью намеченный сейчас план. автономно
|
||||
|
||||
## Plan
|
||||
|
||||
1. Аудит: ruff/pyflakes — неиспользуемые импорты по scimesh/ и tests/; grep — неиспользуемые функции/модули (после рефакторингов могли остаться мёртвые экспорты, например в descriptors/search/graph core и sdk/_validation).
|
||||
2. Удаление мёртвого кода: неиспользуемые импорты, функции, дубли (например write_descriptor_shards/concatenate_descriptor_shards, если вытеснены дефолтами batch), устаревшие файлы-обёртки.
|
||||
3. Проверка, что ничего публичного/API не сломано: pyright 0 ошибок, pytest зелёный, go test/vet, mkdocs build.
|
||||
4. Финал: полный прогон, COMPLETED.
|
||||
1. Фаза 1 — SQLite-хранилище: `coordinator/internal/storage/sqlite` (все порты, TxManager, миграции, `SCIMESH_DB=sqlite|postgres`, тесты).
|
||||
2. Фаза 2 — Встроенный userservice: перенос `users/internal/*` в `coordinator/internal/userservice/` (sqlite-хранилище), запуск на 127.0.0.1, BootstrapAdmin.
|
||||
3. Фаза 3 — `coordinator serve` (data-dir, всё-в-одном, --workers N, --open) + subcommand `coordinator agent`.
|
||||
4. Фаза 4 — Управляемый venv + install.sh/install.ps1 + ассеты релиза.
|
||||
5. Документация: mkdocs, README, PLAN.md (CTX-17 done, CTX-18), STATUS.md.
|
||||
6. Проверка: полный E2E без внешних сервисов + все тесты/lint/vet.
|
||||
|
||||
## Progress
|
||||
|
||||
Эта сессия (доп. задача): Go-агент доведён до паритета, Python-демон удалён.
|
||||
- [x] Go-агент: token provider (static + worker-key exchange + 401 refresh на API/download/upload), CLEANUP_AFTER_SECONDS (очистка attempt-директорий), тесты auth (exchange/cache/reject/select/401-retry).
|
||||
- [x] Python: удалены daemon.py, cli.py, config.py, coordinator.py, artifacts.py, auth.py, transport.py; `scimesh/worker/` = только task.py + runners.py (SDK-мост, allowlist из env) + models.py (ClaimedTask/RunResult); консольный скрипт scimesh-worker убран из pyproject.
|
||||
- [x] Тесты: удалены test_worker_daemon.py, test_worker_auth.py; 208 pytest зелёные.
|
||||
- [x] Демо/смок переведены на Go-агент (demo-ui.sh: build_agent + env; two-worker-smoke.sh: AGENT_BIN + TASK_RUNNER_JSON). `make smoke-two-worker` PASS: 4/4 шардов, worker-a=2, worker-b=2.
|
||||
- [x] Документация: README, AGENTS.md, STATUS, handoff, mkdocs worker-integration/cli.
|
||||
- [x] Верификация: ruff clean; 208 pytest; pyright 0 (scimesh+tests); go test 11 пакетов + vet; mkdocs 0 warnings.
|
||||
- ВСЕ ФАЗЫ ВЫПОЛНЕНЫ И ЗАПУШЕНЫ:
|
||||
- 9883def — SQLite-бэкенд (SCIMESH_DB=sqlite|postgres, миграции, тесты).
|
||||
- 1473bbe — встроенный userservice + serve/agent subcommands.
|
||||
- c06d867 — install.sh/install.ps1 + make serve.
|
||||
- 215325a — документация (README, mkdocs, PLAN CTX-17/CTX-18, STATUS).
|
||||
- E2E «чистая машина»: `coordinator serve` → health/login (embedded userservice) → molwt-filter джоб через локального агента → результат byte-точный.
|
||||
- Верификация: 208 pytest, pyright 0, 18 Go-пакетов ok, gofmt чист, vet чист, golangci 0 issues, postgres integration ok, все CI-раны success.
|
||||
|
||||
@@ -1003,11 +1003,12 @@ with as little manual configuration as possible: it provisions its own
|
||||
schema, and a `setup` command walks the operator through the remaining
|
||||
environment (database creation, secrets, admin account).
|
||||
|
||||
**Depends on:** the Go coordinator and the release build pipeline. Step 1
|
||||
(embedded migrations, `AUTO_MIGRATE`) and step 2 (the `coordinator setup`
|
||||
wizard: database reachability and creation, schema migration, `.env` with a
|
||||
generated `JWT_SECRET`, readiness summary) are implemented; step 3 — a fully
|
||||
embedded userservice — is the remaining work.
|
||||
**Depends on:** the Go coordinator and the release build pipeline. All steps
|
||||
are implemented: embedded migrations with `AUTO_MIGRATE`, the `setup` wizard,
|
||||
the embedded SQLite storage backend (`SCIMESH_DB=sqlite`), the embedded
|
||||
userservice, and the `coordinator serve` single-binary mode with local worker
|
||||
agents. The standalone `users/` service and the PostgreSQL engine remain for
|
||||
cluster deployments.
|
||||
|
||||
**Acceptance criteria:**
|
||||
|
||||
@@ -1027,6 +1028,34 @@ embedded userservice — is the remaining work.
|
||||
|
||||
---
|
||||
|
||||
### CTX-18 — Single-binary platform (`coordinator serve`)
|
||||
|
||||
**Goal:** A scientist installs one file, runs one command, and gets the whole
|
||||
platform: coordinator, both databases, the userservice, and local workers —
|
||||
no PostgreSQL, no Docker, no Python setup.
|
||||
|
||||
**Depends on:** CTX-17 (embedded migrations, SQLite, embedded userservice).
|
||||
|
||||
**Acceptance criteria:**
|
||||
|
||||
- `coordinator serve` provisions `~/.scimesh` (databases, secrets chmod 0600,
|
||||
generated admin password printed once, artifacts dir) and serves the UI on
|
||||
127.0.0.1:8080 by default; `--open` opens the browser;
|
||||
- `--workers N` spawns N `coordinator agent` subprocesses that claim and
|
||||
execute tasks locally; agents are stopped on shutdown;
|
||||
- the embedded userservice listens on the loopback interface and shares the
|
||||
JWT secret with the coordinator, so UI login/registration work unchanged;
|
||||
- a managed scientific runtime venv (`~/.scimesh/venv`) is bootstrapped on
|
||||
first start; `SCIMESH_PIP_PACKAGE` controls what gets installed (the PyPI
|
||||
name is not ours), and `TASK_RUNNER` points at the venv python;
|
||||
- `install.sh` / `install.ps1` detect the platform, download the release
|
||||
binary, and print the start command; both are release assets;
|
||||
- the PostgreSQL engine and the standalone userservice stay fully supported;
|
||||
- `coordinator serve` passes the full local E2E without any external service:
|
||||
health, login, upload, claim, compute, reduction, byte-exact result.
|
||||
|
||||
---
|
||||
|
||||
## 10. Suggested assignment bundles
|
||||
|
||||
These bundles minimize overlap. Do not run tasks from the same bundle in
|
||||
@@ -1166,12 +1195,11 @@ Do not start these before CTX-12 is accepted.
|
||||
- Add shard caching and content-addressed input deduplication.
|
||||
- Add job priority and fair scheduling.
|
||||
- Add a CLI for submitting and monitoring remote jobs.
|
||||
- Implement CTX-17 step 3: a fully embedded userservice
|
||||
(`coordinator userservice` subcommand) so one binary can serve the whole
|
||||
platform without containers.
|
||||
- Replace PostgreSQL with an embedded SQLite backend for fully self-contained
|
||||
single-binary deployments (large storage-layer change; postgres row locks,
|
||||
transactions, and integration tests must be re-derived).
|
||||
- Publish the scimesh Python package to PyPI so the managed venv bootstrap
|
||||
(`SCIMESH_PIP_PACKAGE`) works out of the box on a scientist's machine.
|
||||
- Bundle a Python runtime (python-build-standalone) into the release so local
|
||||
workers need no system Python at all.
|
||||
- Native installers (.msi/.dmg/.deb) built by the release workflow.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,15 +1,26 @@
|
||||
# SciMesh
|
||||
|
||||
SciMesh is a scientific-workload framework for molecular datasets. Its public CLI
|
||||
runs exact similarity search and sparse similarity-graph construction locally in
|
||||
one Python process; it creates no dense similarity matrix. The Go/PostgreSQL
|
||||
coordinator and Go worker agents (which execute SDK workloads in a Python
|
||||
subprocess) can run a shard-based `similarity-search`
|
||||
pipeline locally. After every shard succeeds, the coordinator deterministically
|
||||
merges its candidates into one final global top-k CSV. See
|
||||
[`STATUS.md`](STATUS.md).
|
||||
SciMesh is a local-first platform for scientific computation on molecular
|
||||
datasets. It turns a scientific run into independent tasks, dispatches them
|
||||
to worker agents, and deterministically combines the partial results into a
|
||||
checksum-protected final artifact.
|
||||
|
||||
The ChEMBL TSV database is intentionally not included in this repository. Download it separately and pass its path to the commands below. The expected columns are `chembl_id` and `canonical_smiles`.
|
||||
- **The Workload SDK (`scimesh.sdk`)** — a strict Python framework for
|
||||
authoring scientific workloads: `similarity-search` (exact top-k Tanimoto),
|
||||
`similarity-graph` (exact sparse graph), `descriptor-batch`, and
|
||||
`molwt-filter`. Workloads are ordinary user scripts built on the SDK; they
|
||||
run locally, in the conformance harness, and on claimed coordinator tasks
|
||||
without touching any other part of the program.
|
||||
- **The coordinator and worker agents** — a Go/PostgreSQL coordinator with an
|
||||
operator UI and Go worker agents that execute SDK workloads in a Python
|
||||
subprocess. The UI is workload-agnostic: the "New computation" form offers
|
||||
every workload from the embedded SDK library, and each workload declares its
|
||||
own form controls (`UIElement`) through the SDK.
|
||||
|
||||
The ChEMBL TSV database is intentionally not included in this repository.
|
||||
Download it separately and pass its path to the commands below. The expected
|
||||
columns are `chembl_id` and `canonical_smiles`. See
|
||||
[`STATUS.md`](STATUS.md) and [`PLAN.md`](PLAN.md).
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -28,6 +39,50 @@ conda install -c conda-forge rdkit
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
## Releases
|
||||
|
||||
Every `v*` tag pushes a GitHub Release with static binaries for `coordinator`
|
||||
and `worker-agent` on linux/darwin/windows × amd64/arm64 (plus SHA-256
|
||||
checksums), the installer scripts, and the `coordinator` image on GHCR:
|
||||
|
||||
```bash
|
||||
docker pull ghcr.io/emil28092005/SciMesh/coordinator:latest
|
||||
```
|
||||
|
||||
For scientists: one command downloads the right binary and prints the start
|
||||
instructions:
|
||||
|
||||
```bash
|
||||
# Linux / macOS
|
||||
curl -fsSL https://raw.githubusercontent.com/emil28092005/SciMesh/main/install.sh | bash
|
||||
coordinator serve --open
|
||||
|
||||
# Windows (PowerShell)
|
||||
powershell -ExecutionPolicy Bypass -c "irm https://raw.githubusercontent.com/emil28092005/SciMesh/main/install.ps1 | iex"
|
||||
coordinator serve --open
|
||||
```
|
||||
|
||||
`coordinator serve` is the single-binary mode: it embeds SQLite (coordinator +
|
||||
userservice databases), the userservice itself, and local worker agents
|
||||
(`--workers N`, default 1). On first start it generates secrets and the admin
|
||||
password under `~/.scimesh`, prints the login, and opens the UI. No
|
||||
PostgreSQL, no Docker, no environment variables. The scientific runtime is a
|
||||
managed venv (`~/.scimesh/venv`); point `SCIMESH_PIP_PACKAGE` at your scimesh
|
||||
wheel to install it automatically.
|
||||
|
||||
Manual download and run of a release binary:
|
||||
|
||||
```bash
|
||||
curl -L -o coordinator https://github.com/emil28092005/SciMesh/releases/latest/download/coordinator-linux-amd64
|
||||
chmod +x coordinator
|
||||
./coordinator --version
|
||||
```
|
||||
|
||||
Cluster deployments keep the PostgreSQL engine (`SCIMESH_DB=postgres` with
|
||||
`DATABASE_URL`, or `coordinator setup` to provision it) and the standalone
|
||||
userservice (`users/`). `coordinator agent` runs a worker agent from the same
|
||||
binary.
|
||||
|
||||
## Quick start
|
||||
|
||||
Run the built-in help command for copy-paste examples of both workloads:
|
||||
@@ -47,9 +102,9 @@ scimesh similarity-graph --help
|
||||
|
||||
## Manual pipeline demo
|
||||
|
||||
To inspect the coordinator, Web UI, and distributed `similarity-search`
|
||||
pipeline by hand, install development dependencies once and start the isolated
|
||||
demo from the repository root:
|
||||
To inspect the coordinator, Web UI, and distributed pipeline by hand, install
|
||||
development dependencies once and start the isolated demo from the repository
|
||||
root:
|
||||
|
||||
```bash
|
||||
python3 -m venv .venv
|
||||
@@ -64,17 +119,19 @@ covers the complete Workload SDK: guides (`mkdocs/sdk/`), the full
|
||||
auto-generated API reference for `scimesh.sdk` (`mkdocs/api/`), and the
|
||||
documentation rules the site is written by (`mkdocs/approach.md`).
|
||||
|
||||
Open `http://localhost:18080/ui` and sign in with username `operator` and
|
||||
password `demo-ui-secret`. The command starts PostgreSQL, the coordinator, and
|
||||
two Go worker agents (built by `make agent`; each executes the SDK workload
|
||||
in a Python subprocess). Upload a small ChEMBL TSV, then use the job page
|
||||
to follow shard progress, inspect bounded **Preview CSV** results, and see a
|
||||
live processing-speed chart in shards per minute. The **Workloads** page shows
|
||||
the installed SDK workload library (descriptions, parameters, and artifact
|
||||
schemas) from the embedded catalog; regenerate it with
|
||||
`make workloads-export` (or `scimesh workload export`) whenever workloads
|
||||
change. To change the worker count, run `make demo-ui WORKERS=3`; stop
|
||||
everything with `make demo-down`.
|
||||
Open `http://localhost:18080/ui` and sign in with username
|
||||
`root@scimesh.local` and password `rootpassword`. The command starts
|
||||
PostgreSQL, the coordinator, and two Go worker agents (built by `make agent`;
|
||||
each executes the SDK workload in a Python subprocess). The **New computation**
|
||||
form offers every upload-ready workload from the installed library — the
|
||||
controls come from each workload's own SDK declarations. Upload a small ChEMBL
|
||||
TSV, then use the job page to follow shard progress, inspect bounded
|
||||
**Preview CSV** results, and see a live processing-speed chart in shards per
|
||||
minute. The **Workloads** page shows the installed SDK workload library
|
||||
(descriptions, parameters, and artifact schemas) from the embedded catalog;
|
||||
regenerate it with `make workloads-export` (or `scimesh workload export`)
|
||||
whenever workloads change. To change the worker count, run
|
||||
`make demo-ui WORKERS=3`; stop everything with `make demo-down`.
|
||||
|
||||
Run `make help` to display these commands in the terminal.
|
||||
|
||||
@@ -146,6 +203,15 @@ pytest
|
||||
|
||||
The package separates common dataset parsing and fingerprints from independent workloads. Add future workloads through the workload registry without changing the main CLI.
|
||||
|
||||
The coordinator and worker agent are Go modules under `coordinator/` and `users/`:
|
||||
|
||||
```bash
|
||||
cd coordinator && make coordinator agent && go test ./...
|
||||
```
|
||||
|
||||
`make check` runs the full gate: vet, lint, race tests, the PostgreSQL
|
||||
integration suite, and the two-worker end-to-end smoke script.
|
||||
|
||||
## Workload SDK
|
||||
|
||||
`scimesh.sdk` is the framework only: strict and immutable workload manifests,
|
||||
@@ -153,14 +219,25 @@ typed artifact ports, static map/reduce plans, resource eligibility and local
|
||||
reservations, exact/canonical/numeric verifier primitives, installed-package
|
||||
allowlisting, and a local conformance executor. It contains no scientific
|
||||
workload code. Workloads are user scripts built on the SDK: the built-in
|
||||
`similarity-search`, `similarity-graph`, and `descriptor-batch` live in
|
||||
`scimesh/workloads/` (each a small package with `core.py` + `definition.py`),
|
||||
composed by `scimesh/workloads/library.py` and registered through
|
||||
`scimesh.workloads` entry points. The Worker Agent executes those SDK-built
|
||||
workloads directly (see `scimesh/worker/runners.py`), so the same scientific
|
||||
handlers run locally, in conformance, and on claimed coordinator tasks.
|
||||
`scimesh workload list` and `scimesh workload run` run any SDK workload from
|
||||
the command line. See the
|
||||
`similarity-search`, `similarity-graph`, `descriptor-batch`, and
|
||||
`molwt-filter` live in `scimesh/workloads/` (each a small package with
|
||||
`core.py` + `definition.py`), composed by `scimesh/workloads/library.py` and
|
||||
registered through `scimesh.workloads` entry points. The Worker Agent executes
|
||||
those SDK-built workloads directly (see `scimesh/worker/runners.py`), so the
|
||||
same scientific handlers run locally, in conformance, and on claimed
|
||||
coordinator tasks. `scimesh workload list` and `scimesh workload run` run any
|
||||
SDK workload from the command line; `scimesh workload export` writes the
|
||||
coordinator's embedded workload catalog, and `scimesh workload allowlist`
|
||||
prints the JSON for `SCIMESH_WORKLOAD_ALLOWLIST`.
|
||||
|
||||
Workloads can also declare how they should appear in the coordinator UI:
|
||||
a tuple of `UIElement`s (`scimesh.sdk.UIElement`) shapes the "New computation"
|
||||
form — widget, label, help, defaults, and ordering — plus the coordinator-side
|
||||
reduction mode (`reduction`: `top-k` or `ordered-concat`) and whether a single
|
||||
uploaded dataset can drive the workload (`upload_ready`). The strict parameter
|
||||
schema stays the authoritative validation contract.
|
||||
|
||||
See the
|
||||
[SDK author guide](docs/workload-sdk.md), [contract](docs/scimesh-sdk-contract.md),
|
||||
and [delivery roadmap](docs/scimesh-sdk-roadmap.md).
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# SciMesh Status
|
||||
|
||||
**Updated:** 2026-08-01
|
||||
**Branch baseline:** `main`; this revision adds the Workload SDK foundation.
|
||||
**Updated:** 2026-08-02
|
||||
**Branch baseline:** `main`; this revision adds the single-binary platform.
|
||||
|
||||
## Current state
|
||||
|
||||
@@ -17,11 +17,22 @@ the reference behaviour for future distributed execution:
|
||||
|
||||
The Go coordinator and its PostgreSQL-backed task lifecycle are implemented:
|
||||
registration, atomic claiming, lease renewal, artifact storage, dataset
|
||||
chunking, result/failure reporting, and job progress. The Python worker now
|
||||
uses the live coordinator contract. Completed similarity-search shard results
|
||||
are reduced once into a checksum-protected final CSV, which is downloadable
|
||||
through the coordinator. The full Go checks (including a fresh migration and
|
||||
real PostgreSQL smoke test) passed on 2026-07-24.
|
||||
chunking, result/failure reporting, and job progress. The Go worker agent now
|
||||
uses the live coordinator contract. Completed shard results are reduced once
|
||||
into a checksum-protected final CSV, downloadable through the coordinator.
|
||||
|
||||
**Single-binary platform (`coordinator serve`)**: the coordinator now ships
|
||||
an embedded SQLite storage backend (`SCIMESH_DB=sqlite`), an embedded
|
||||
userservice, and `serve`/`agent` subcommands, so one downloaded binary runs
|
||||
the whole platform — coordinator, both databases, UI logins, and local
|
||||
workers — with no PostgreSQL, no Docker, and no environment variables. The
|
||||
first start provisions `~/.scimesh` (secrets, admin password printed once,
|
||||
managed scientific-runtime venv) and opens the UI. `install.sh` / `install.ps1`
|
||||
download the right release binary in one command and are release assets. The
|
||||
PostgreSQL engine, the `setup` wizard, and the standalone `users/` service
|
||||
remain fully supported for cluster deployments. The full E2E passes with zero
|
||||
external services: health, UI login, job upload, local agent compute,
|
||||
reduction, and a byte-exact final CSV.
|
||||
|
||||
The User Service is merged into `main`. It owns user accounts, authentication,
|
||||
roles, and verified-contributor status; the coordinator scopes user jobs and
|
||||
@@ -37,24 +48,25 @@ the complete result-artifact SHA-256 before a task is accepted.
|
||||
| --- | --- | --- |
|
||||
| CTX-00 API and error contract | Implemented | Contract, OpenAPI, and request examples are in `docs/`. |
|
||||
| CTX-01 Go coordinator bootstrap | Implemented | Go service and Docker runtime in `coordinator/`. |
|
||||
| CTX-02 PostgreSQL migrations | Implemented | Applied by the Compose migration service. |
|
||||
| CTX-03 Transactional queue | Implemented | Real-PostgreSQL integration tests cover atomic claims and concurrency. |
|
||||
| CTX-02 PostgreSQL migrations | Implemented | Embedded into the binary (`AUTO_MIGRATE`); the CLI path is still available for managed databases. |
|
||||
| CTX-03 Transactional queue | Implemented | Real-PostgreSQL integration tests cover atomic claims and concurrency; the SQLite backend mirrors the semantics. |
|
||||
| CTX-04 Worker registry and HTTP API | Implemented | Registration, claim, heartbeat, result, failure, and status endpoints. |
|
||||
| CTX-05 Artifact storage | Implemented | Coordinator-owned inputs/results, checksum verification, and upload flow. |
|
||||
| CTX-06 Python Worker live-contract alignment | Superseded | The Python worker daemon was removed; the Go worker agent (`coordinator/internal/agent/` + `cmd/worker-agent`) now implements the lifecycle (register/claim/heartbeat/download/upload/submit/fail, token refresh, cleanup) and executes SDK workloads via the Python task entry `scimesh/worker/task.py`. E2E: `make smoke-two-worker` passes 4/4 shards with two agents. |
|
||||
| CTX-07 Distributed workload protocol | Implemented | Versioned Python contract models, registry, strict plan validation, and deterministic reduction ordering are in `scimesh/distributed/`. |
|
||||
| CTX-08 Distributed similarity-search | Implemented | Python planner resolves `query_id` once, creates deterministic shard plans, worker adapter emits exact partial top-k CSVs/metrics, and reducer matches the local reference. |
|
||||
| CTX-09 Reducer and final-result API | Implemented | Atomic `reducing` claim, deterministic coordinator-side top-k reducer, sanitized reducer failure, final artifact persistence, `result_uri`, and final CSV download. |
|
||||
| CTX-10 Distributed similarity-graph | Not started | Local reference exists; the SDK-built local graph workload already enforces the pair-coverage invariant. |
|
||||
| CTX-11 Dashboard/operator view | Implemented | Protected live control room: MkDocs documentation served at `/ui/docs/` (SCIMESH_DOCS_DIR; the demo mounts `site/` automatically), recent-run/worker overview, real pipeline-stage visualization, shard attempts and safe failures, validated similarity-search upload, coordinator artifacts, final-result download, bounded polling, a Workload library page rendering the embedded catalog from `scimesh workload export` (`/ui/workloads`, regenerated via `make workloads-export`). |
|
||||
| CTX-12 Reliability, security, CI | In progress | Unit, race, PostgreSQL integration, and smoke checks exist; CI hardening remains. |
|
||||
| CTX-15 User Service and access control | Implemented | User/owner scoping, verified contributors, worker keys, self-service enrollment, and quorum-backed untrusted workers are merged; local Go/Python and Docker/PostgreSQL checks passed. |
|
||||
| MkDocs documentation site | Implemented | A standalone documentation site (`mkdocs/`, `docs_dir: mkdocs`) covering the complete Workload SDK: guides (overview, authoring workloads, CLI, worker integration), the full auto-generated API reference for all 15 `scimesh.sdk` modules (mkdocstrings), and the writing rules (`mkdocs/approach.md`). Built with `make docs`, served inside the UI at `/ui/docs/`; the project's internal `docs/` directory is not part of the site. |
|
||||
| CTX-16 Workload SDK foundation | Implemented | `scimesh.sdk` provides strict immutable manifests/plans/artifacts, digest/trust-pinned tasks, typed DAGs, compatibility negotiation, verifier primitives with owner/binding-safe quorum inputs, resource eligibility/local allocation, measured package discovery, a trusted local core-batch conformance harness, and strict package discovery. Enforcing coordinator/Worker profiles remain fail-closed. |
|
||||
| SDK roadmap step 3: `descriptor-batch` | Implemented | The first SDK-built reference workload (`scimesh/workloads/descriptors/`): pinned 81-name RDKit 2D descriptor set, canonical one-row-per-input CSV, deterministic row-bounded shards, shard-index concatenation with one header, byte-identical local/distributed output, and a two-worker `untrusted_quorum` verifier test. |
|
||||
| SDK-built `similarity-search` and `similarity-graph` | Implemented | Both workloads are SDK-built packages (`scimesh/workloads/search/`, `scimesh/workloads/graph/`) built on the `MapReduceWorkload` authoring scaffold (`scimesh/sdk/batch.py`); they reuse the local scientific cores and are byte-identical to the single-process references (search; graph for both threshold directions and any block size). The graph reducer enforces the CTX-10 pair-coverage invariant. `scimesh/workloads/library.py` composes the built-in registry/runtime. |
|
||||
| SDK-built `molwt-filter` | Implemented | The minimal authoring example (`scimesh/workloads/molwt_filter/`): filters molecules by exact RDKit molecular weight with only one scientific hook, using the scaffold's new default sharding and concatenation hooks. Registered in the built-in library and as a `scimesh.workloads` entry point. |
|
||||
| SDK authoring scaffold | Implemented | `MapReduceWorkload` (exported from `scimesh.sdk`) assembles manifest, map/reduce stages, workflow, and digest-pinned handlers from three scientific hooks (partition/compute/merge), with overridable hooks for domain validation, plan-time resolution, custom task planning, and partial-key policy. The generic `scimesh workload list|run` CLI and the worker's allowlist-driven loading (`SCIMESH_WORKLOAD_ALLOWLIST`, `SCIMESH_CAPABILITIES`) let new workloads run without touching other code. |
|
||||
| CTX-06 Python Worker live-contract alignment | Superseded | The Python worker daemon was removed; the Go worker agent (`coordinator/internal/agent/` + `cmd/worker-agent`, or `coordinator agent`) implements the lifecycle and executes SDK workloads via `scimesh/worker/task.py`. |
|
||||
| CTX-07 Distributed workload protocol | Implemented | Versioned Python contract models, registry, strict plan validation, and deterministic reduction ordering. |
|
||||
| CTX-08 Distributed similarity-search | Implemented | Planner/worker/reducer match the local reference byte-exactly. |
|
||||
| CTX-09 Reducer and final-result API | Implemented | Atomic `reducing` claim, deterministic coordinator-side reducers (`top-k` and `ordered-concat`), sanitized failure, final artifact, `result_uri`. |
|
||||
| CTX-10 Distributed similarity-graph | Not started | Local reference exists; the SDK-built local graph workload enforces the pair-coverage invariant. |
|
||||
| CTX-11 Dashboard/operator view | Implemented | Protected live control room, workload library, workload-agnostic "New computation" form (SDK-declared `UIElement`s), MkDocs at `/ui/docs/`, final-result download. |
|
||||
| CTX-12 Reliability, security, CI | In progress | vet, gofmt, race tests, golangci-lint (0 issues), PostgreSQL integration, and smoke checks exist. |
|
||||
| CTX-15 User Service and access control | Implemented | User/owner scoping, verified contributors, worker keys, quorum; also embeddable (`coordinator serve`). |
|
||||
| CTX-16 Workload SDK foundation | Implemented | Strict immutable manifests/plans/artifacts, digest/trust-pinned tasks, negotiation, verifier primitives, conformance harness. |
|
||||
| CTX-17 Self-provisioning + setup wizard | Implemented | Embedded migrations, `coordinator setup`, SQLite backend, embedded userservice, `serve` mode. |
|
||||
| CTX-18 Single-binary platform | Implemented | `coordinator serve` (data dir, secrets, admin bootstrap, local agents, managed venv) + `install.sh`/`install.ps1`; full no-external-service E2E green. |
|
||||
| SDK roadmap step 3: `descriptor-batch` | Implemented | Byte-identical local/distributed output, quorum verifier test. |
|
||||
| SDK-built `similarity-search` and `similarity-graph` | Implemented | SDK-built packages, byte-identical to single-process references. |
|
||||
| SDK-built `molwt-filter` | Implemented | Minimal authoring example; also the single-binary E2E workload. |
|
||||
| SDK authoring scaffold | Implemented | `MapReduceWorkload` with `UIElement` declarations, `reduction`, `upload_ready`; generic `scimesh workload list|run|export|allowlist` CLI. |
|
||||
|
||||
## Next recommended assignment
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
.DEFAULT_GOAL := help
|
||||
|
||||
.PHONY: help build run test test-integration vet lint tidy check migrate-up migrate-down up down down-clean logs ps rebuild psql smoke agent coordinator setup workloads-export demo-ui demo-down demo-reset demo-logs
|
||||
.PHONY: help build run test test-integration vet lint tidy check migrate-up migrate-down up down down-clean logs ps rebuild psql smoke agent coordinator setup serve workloads-export demo-ui demo-down demo-reset demo-logs
|
||||
|
||||
# `check` deliberately uses its own Compose project and host ports. This keeps
|
||||
# it from connecting to or replacing a developer's local PostgreSQL instance.
|
||||
@@ -57,6 +57,11 @@ coordinator:
|
||||
setup: coordinator
|
||||
./bin/coordinator setup $(SETUP_ARGS)
|
||||
|
||||
# The single-binary mode: everything embedded (sqlite + userservice + local
|
||||
# workers), no PostgreSQL or Docker. SETUP_ARGS=--workers 2 --open.
|
||||
serve: coordinator
|
||||
./bin/coordinator serve $(SETUP_ARGS)
|
||||
|
||||
workloads-export:
|
||||
cd .. && .venv/bin/scimesh workload export -o coordinator/$(WORKLOADS_JSON)
|
||||
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/agent"
|
||||
)
|
||||
|
||||
// runAgent implements `coordinator agent`: the worker agent as a subcommand of
|
||||
// the same binary, so one file can serve the whole platform. `serve` spawns
|
||||
// these for its local workers.
|
||||
func runAgent(args []string) error {
|
||||
flags := flag.NewFlagSet("agent", flag.ContinueOnError)
|
||||
flags.Usage = func() {
|
||||
_, _ = fmt.Fprintf(flags.Output(), "usage: coordinator agent [options]\n")
|
||||
_, _ = fmt.Fprintf(flags.Output(), "Runs as a worker agent: claims tasks, executes SDK workloads in a\n")
|
||||
_, _ = fmt.Fprintf(flags.Output(), "Python subprocess, uploads results.\n\n")
|
||||
flags.PrintDefaults()
|
||||
}
|
||||
var (
|
||||
coordinatorURL = flags.String("coordinator-url", os.Getenv("COORDINATOR_URL"), "coordinator base URL")
|
||||
token = flags.String("token", os.Getenv("WORKER_AUTH_TOKEN"), "worker bearer token")
|
||||
workDir = flags.String("work-dir", os.Getenv("WORK_DIR"), "worker work directory")
|
||||
name = flags.String("name", os.Getenv("WORKER_NAME"), "worker name (default: hostname)")
|
||||
workerID = flags.String("worker-id", os.Getenv("WORKER_ID"), "persistent worker id (optional)")
|
||||
cpuCount = flags.Int("cpu", envInt("CPU_COUNT", 1), "advertised CPU cores")
|
||||
memoryMB = flags.Int("memory-mb", envInt("MEMORY_MB", 1024), "advertised memory in MiB")
|
||||
poll = flags.Duration("poll-interval", 2*time.Second, "claim poll interval")
|
||||
taskRunner = flags.String("task-runner", os.Getenv("TASK_RUNNER"), "python command + args that run scimesh.worker.task")
|
||||
maxTasks = flags.Int("max-tasks", envInt("MAX_TASKS", 0), "stop after N completed tasks (0 = unlimited)")
|
||||
exitWhenIdle = flags.Bool("exit-when-idle", false, "exit when the queue is empty")
|
||||
)
|
||||
if err := flags.Parse(args); err != nil {
|
||||
return err
|
||||
}
|
||||
if flags.NArg() > 0 {
|
||||
return fmt.Errorf("agent takes no positional arguments")
|
||||
}
|
||||
if *coordinatorURL == "" || *token == "" || *workDir == "" {
|
||||
return fmt.Errorf("--coordinator-url, --token, and --work-dir are required")
|
||||
}
|
||||
if *taskRunner == "" {
|
||||
*taskRunner = "python -m scimesh.worker.task"
|
||||
}
|
||||
|
||||
logger := slog.New(slog.NewTextHandler(os.Stderr, nil))
|
||||
config := agent.Config{
|
||||
CoordinatorURL: strings.TrimRight(*coordinatorURL, "/"),
|
||||
Capabilities: agent.DefaultCapabilities(),
|
||||
Token: *token,
|
||||
WorkerName: *name,
|
||||
WorkerID: *workerID,
|
||||
WorkDir: *workDir,
|
||||
CPUCount: *cpuCount,
|
||||
MemoryMB: *memoryMB,
|
||||
PollInterval: *poll,
|
||||
RequestTimeout: 30 * time.Second,
|
||||
Heartbeat: 15 * time.Second,
|
||||
TaskRunner: strings.Fields(*taskRunner),
|
||||
MaxTasks: *maxTasks,
|
||||
ExitWhenIdle: *exitWhenIdle,
|
||||
}
|
||||
tokens := agent.NewTokenProvider("", "", config.Token, config.RequestTimeout)
|
||||
client := agent.NewClient(config.CoordinatorURL, tokens, config.RequestTimeout)
|
||||
runner := agent.NewTaskRunner(config.TaskRunner)
|
||||
daemon := agent.NewDaemon(&config, client, runner, logger)
|
||||
return daemon.RunForever()
|
||||
}
|
||||
|
||||
func envInt(name string, fallback int) int {
|
||||
raw := os.Getenv(name)
|
||||
if raw == "" {
|
||||
return fallback
|
||||
}
|
||||
var n int
|
||||
if _, err := fmt.Sscanf(raw, "%d", &n); err != nil {
|
||||
return fallback
|
||||
}
|
||||
return n
|
||||
}
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/metrics"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/storage/blob"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/storage/postgres"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/storage/sqlite"
|
||||
httptransport "github.com/emil28092005/SciMesh/coordinator/internal/transport/http"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/usecase"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/workloads"
|
||||
@@ -25,11 +26,24 @@ var version = "dev"
|
||||
|
||||
func main() {
|
||||
args := os.Args[1:]
|
||||
if len(args) > 0 && args[0] == "setup" {
|
||||
if err := runSetup(args[1:]); err != nil {
|
||||
os.Exit(1)
|
||||
if len(args) > 0 {
|
||||
switch args[0] {
|
||||
case "setup":
|
||||
if err := runSetup(args[1:]); err != nil {
|
||||
os.Exit(1)
|
||||
}
|
||||
return
|
||||
case "serve":
|
||||
if err := runServe(args[1:]); err != nil {
|
||||
os.Exit(1)
|
||||
}
|
||||
return
|
||||
case "agent":
|
||||
if err := runAgent(args[1:]); err != nil {
|
||||
os.Exit(1)
|
||||
}
|
||||
return
|
||||
}
|
||||
return
|
||||
}
|
||||
showVersion := flag.Bool("version", false, "print the build version and exit")
|
||||
flag.Parse()
|
||||
@@ -44,9 +58,25 @@ func main() {
|
||||
}
|
||||
}
|
||||
|
||||
// storageDeps carries the engine-specific database handles and the repository
|
||||
// implementations. The usecases below only ever see the ports.
|
||||
type storageDeps struct {
|
||||
tx usecase.TxManager
|
||||
taskRepo usecase.TaskRepository
|
||||
jobRepo usecase.JobRepository
|
||||
workerRepo usecase.WorkerRepository
|
||||
artifactRepo usecase.ArtifactRepository
|
||||
uiReadRepo usecase.UIReadRepository
|
||||
taskResultRepo usecase.TaskResultRepository
|
||||
statsRepo interface {
|
||||
Counts(ctx context.Context) (tasks, jobs, workers map[string]int, err error)
|
||||
}
|
||||
ready func(ctx context.Context) error
|
||||
migrate func(ctx context.Context, log *slog.Logger) error
|
||||
close func()
|
||||
}
|
||||
|
||||
func run() error {
|
||||
// Bootstrap logger, used only until config says where logs should go. It
|
||||
// writes to stderr so it never contaminates the configured stdout stream.
|
||||
boot := slog.New(slog.NewJSONHandler(os.Stderr, nil))
|
||||
|
||||
cfg, err := infra.LoadConfig()
|
||||
@@ -54,6 +84,16 @@ func run() error {
|
||||
boot.Error("load config", "err", err)
|
||||
return err
|
||||
}
|
||||
return runWithConfig(cfg)
|
||||
}
|
||||
|
||||
// runWithConfig boots the coordinator server with an explicit config. The
|
||||
// `serve` subcommand builds such a config for the single-binary mode; the
|
||||
// plain `coordinator` binary loads it from the environment.
|
||||
func runWithConfig(cfg infra.Config) error {
|
||||
// Bootstrap logger, used only until config says where logs should go. It
|
||||
// writes to stderr so it never contaminates the configured stdout stream.
|
||||
boot := slog.New(slog.NewJSONHandler(os.Stderr, nil))
|
||||
|
||||
// The real logger: stdout plus an optional rotated file (LOG_FILE).
|
||||
log, logCloser, err := infra.NewLogger(cfg)
|
||||
@@ -66,17 +106,25 @@ func run() error {
|
||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||
defer stop()
|
||||
|
||||
pool, err := infra.NewPool(ctx, cfg, log)
|
||||
var deps *storageDeps
|
||||
switch cfg.DatabaseEngine {
|
||||
case "sqlite":
|
||||
deps, err = openSQLite(ctx, cfg, log)
|
||||
case "postgres":
|
||||
deps, err = openPostgres(ctx, cfg, log)
|
||||
default:
|
||||
err = fmt.Errorf("SCIMESH_DB must be sqlite or postgres")
|
||||
}
|
||||
if err != nil {
|
||||
log.Error("connect database", "err", err)
|
||||
log.Error("init storage", "err", err)
|
||||
return err
|
||||
}
|
||||
defer pool.Close()
|
||||
defer deps.close()
|
||||
|
||||
// A downloaded binary provisions its own schema; AUTO_MIGRATE=false keeps
|
||||
// out-of-band migration workflows (the migrate CLI, CI, managed databases).
|
||||
if cfg.AutoMigrate {
|
||||
if err := postgres.Migrate(ctx, cfg.DatabaseURL, log); err != nil {
|
||||
if err := deps.migrate(ctx, log); err != nil {
|
||||
log.Error("apply migrations", "err", err)
|
||||
return err
|
||||
}
|
||||
@@ -88,16 +136,9 @@ func run() error {
|
||||
return err
|
||||
}
|
||||
|
||||
var (
|
||||
clk = infra.NewClock()
|
||||
tx = postgres.NewTxManager(pool)
|
||||
taskRepo = postgres.NewTaskRepo(pool)
|
||||
jobRepo = postgres.NewJobRepo(pool)
|
||||
workerRepo = postgres.NewWorkerRepo(pool)
|
||||
artifactRepo = postgres.NewArtifactRepo(pool)
|
||||
uiReadRepo = postgres.NewUIReadRepo(pool)
|
||||
taskResultRepo = postgres.NewTaskResultRepo(pool)
|
||||
)
|
||||
clk := infra.NewClock()
|
||||
tx, taskRepo, jobRepo, workerRepo, artifactRepo, uiReadRepo, taskResultRepo :=
|
||||
deps.tx, deps.taskRepo, deps.jobRepo, deps.workerRepo, deps.artifactRepo, deps.uiReadRepo, deps.taskResultRepo
|
||||
|
||||
catalog, err := workloads.Load()
|
||||
if err != nil {
|
||||
@@ -125,7 +166,7 @@ func run() error {
|
||||
}
|
||||
|
||||
// Background reapers are tracked so shutdown can wait for them. Without this
|
||||
// the process would exit mid-UPDATE, and the deferred pool.Close() would pull
|
||||
// the process would exit mid-UPDATE, and the deferred close() would pull
|
||||
// connections out from under them.
|
||||
expireLeases := usecase.NewExpireLeases(taskRepo, jobRepo, tx, clk, catalog)
|
||||
markOffline := usecase.NewMarkWorkersOffline(workerRepo, clk, cfg.WorkerOfflineAfter)
|
||||
@@ -147,16 +188,15 @@ func run() error {
|
||||
|
||||
// Business metrics: gauges of tasks/jobs/workers by status, sampled from the
|
||||
// database on every Prometheus scrape.
|
||||
statsRepo := postgres.NewStatsRepo(pool)
|
||||
m := metrics.New()
|
||||
m.RegisterBusiness(func(ctx context.Context) (metrics.Stats, error) {
|
||||
tasks, jobs, workers, err := statsRepo.Counts(ctx)
|
||||
tasks, jobs, workers, err := deps.statsRepo.Counts(ctx)
|
||||
return metrics.Stats{Tasks: tasks, Jobs: jobs, Workers: workers}, err
|
||||
})
|
||||
|
||||
// pool.Ping backs /health: readiness means the database answers, not just
|
||||
// deps.ready backs /health: readiness means the database answers, not just
|
||||
// that the process is alive.
|
||||
api := httptransport.NewServer(useCases, log, cfg.RequestTimeout, cfg.HeartbeatInterval, cfg.MaxUploadBytes, cfg.JWTSecret, cfg.UserserviceURL, m, pool.Ping, cfg.PublicCoordinatorURL, cfg.PublicUserserviceURL, cfg.DocsDir)
|
||||
api := httptransport.NewServer(useCases, log, cfg.RequestTimeout, cfg.HeartbeatInterval, cfg.MaxUploadBytes, cfg.JWTSecret, cfg.UserserviceURL, m, deps.ready, cfg.PublicCoordinatorURL, cfg.PublicUserserviceURL, cfg.DocsDir)
|
||||
err = infra.RunServer(ctx, log, cfg.Addr, api.Handler(cfg.Token, cfg.UIToken))
|
||||
|
||||
// Shutdown order matters, and defers alone cannot express it (they run
|
||||
@@ -164,7 +204,7 @@ func run() error {
|
||||
//
|
||||
// 1. stop() cancel the context, telling the reaper to finish
|
||||
// 2. wg.Wait() let it return from its current tick
|
||||
// 3. deferred pool.Close() closes an idle pool, not a busy one
|
||||
// 3. deferred close() closes an idle pool, not a busy one
|
||||
//
|
||||
// Calling stop() here also covers the path where RunServer failed on its
|
||||
// own: the context would never be cancelled otherwise and wg.Wait()
|
||||
@@ -175,3 +215,50 @@ func run() error {
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
// openSQLite opens the embedded database and builds the sqlite repositories.
|
||||
func openSQLite(ctx context.Context, cfg infra.Config, log *slog.Logger) (*storageDeps, error) {
|
||||
if err := os.MkdirAll(cfg.StorageDir, 0o750); err != nil {
|
||||
return nil, fmt.Errorf("create storage dir: %w", err)
|
||||
}
|
||||
db, err := sqlite.Open(cfg.DBPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
closeOnce := &sync.Once{}
|
||||
return &storageDeps{
|
||||
tx: sqlite.NewTxManager(db),
|
||||
taskRepo: sqlite.NewTaskRepo(db),
|
||||
jobRepo: sqlite.NewJobRepo(db),
|
||||
workerRepo: sqlite.NewWorkerRepo(db),
|
||||
artifactRepo: sqlite.NewArtifactRepo(db),
|
||||
uiReadRepo: sqlite.NewUIReadRepo(db),
|
||||
taskResultRepo: sqlite.NewTaskResultRepo(db),
|
||||
statsRepo: sqlite.NewStatsRepo(db),
|
||||
ready: func(ctx context.Context) error { return db.PingContext(ctx) },
|
||||
migrate: func(ctx context.Context, log *slog.Logger) error { return sqlite.Migrate(ctx, db, log) },
|
||||
close: func() { closeOnce.Do(func() { _ = db.Close() }) },
|
||||
}, nil
|
||||
}
|
||||
|
||||
// openPostgres connects to PostgreSQL and builds the postgres repositories.
|
||||
func openPostgres(ctx context.Context, cfg infra.Config, log *slog.Logger) (*storageDeps, error) {
|
||||
pool, err := infra.NewPool(ctx, cfg, log)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
closeOnce := &sync.Once{}
|
||||
return &storageDeps{
|
||||
tx: postgres.NewTxManager(pool),
|
||||
taskRepo: postgres.NewTaskRepo(pool),
|
||||
jobRepo: postgres.NewJobRepo(pool),
|
||||
workerRepo: postgres.NewWorkerRepo(pool),
|
||||
artifactRepo: postgres.NewArtifactRepo(pool),
|
||||
uiReadRepo: postgres.NewUIReadRepo(pool),
|
||||
taskResultRepo: postgres.NewTaskResultRepo(pool),
|
||||
statsRepo: postgres.NewStatsRepo(pool),
|
||||
ready: func(ctx context.Context) error { return pool.Ping(ctx) },
|
||||
migrate: func(ctx context.Context, log *slog.Logger) error { return postgres.Migrate(ctx, cfg.DatabaseURL, log) },
|
||||
close: func() { closeOnce.Do(pool.Close) },
|
||||
}, nil
|
||||
}
|
||||
|
||||
@@ -0,0 +1,318 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"flag"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/infra"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice"
|
||||
)
|
||||
|
||||
// runServe implements `coordinator serve`: the single-binary mode for a
|
||||
// scientist. It provisions a data directory (default ~/.scimesh) with the
|
||||
// coordinator and userservice sqlite databases, secrets, the admin account,
|
||||
// and optionally local worker agents — then runs the same server run() does.
|
||||
func runServe(args []string) error {
|
||||
flags := flag.NewFlagSet("serve", flag.ContinueOnError)
|
||||
flags.Usage = func() {
|
||||
_, _ = fmt.Fprintf(flags.Output(), "usage: coordinator serve [options]\n")
|
||||
_, _ = fmt.Fprintf(flags.Output(), "Runs the whole SciMesh platform from one binary: embedded databases, the\n")
|
||||
_, _ = fmt.Fprintf(flags.Output(), "userservice, and optional local workers. No PostgreSQL or Docker needed.\n\n")
|
||||
flags.PrintDefaults()
|
||||
}
|
||||
var (
|
||||
dataDir = flags.String("data-dir", defaultDataDir(), "data directory (default: ~/.scimesh)")
|
||||
addr = flags.String("addr", "127.0.0.1:8080", "listen address")
|
||||
workers = flags.Int("workers", 1, "number of local worker agents to spawn")
|
||||
open = flags.Bool("open", false, "open the UI in the browser")
|
||||
docsDir = flags.String("docs-dir", "", "built MkDocs site directory to serve at /ui/docs/")
|
||||
email = flags.String("admin-email", "admin@scimesh.local", "admin account email")
|
||||
password = flags.String("admin-password", "", "admin password (generated on first run when empty)")
|
||||
)
|
||||
if err := flags.Parse(args); err != nil {
|
||||
return err
|
||||
}
|
||||
if flags.NArg() > 0 {
|
||||
return fmt.Errorf("serve takes no positional arguments")
|
||||
}
|
||||
if *workers < 0 {
|
||||
return fmt.Errorf("--workers must be >= 0")
|
||||
}
|
||||
|
||||
log := slog.New(slog.NewTextHandler(os.Stderr, nil))
|
||||
if err := os.MkdirAll(*dataDir, 0o750); err != nil {
|
||||
return fmt.Errorf("create data dir: %w", err)
|
||||
}
|
||||
|
||||
// 1. Secrets, persisted in the data dir so restarts keep working.
|
||||
jwtSecret, err := loadOrGenerate(filepath.Join(*dataDir, "jwt.secret"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
workerToken, err := loadOrGenerate(filepath.Join(*dataDir, "worker.token"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 2. Admin account: generated once and printed, remembered for later boots.
|
||||
if *password == "" {
|
||||
*password, err = loadOrGenerate(filepath.Join(*dataDir, "admin.password"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Scientific runtime: ensure the managed venv (best effort).
|
||||
venvPython := filepath.Join(*dataDir, "venv", binName("bin/python"))
|
||||
ensureRuntime(log, *dataDir, venvPython)
|
||||
|
||||
// 4. Embedded userservice on the loopback interface.
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
usersAddr, closeUsers, err := userservice.Serve(ctx, userservice.Config{
|
||||
DBPath: filepath.Join(*dataDir, "users.db"),
|
||||
JWTSecret: jwtSecret,
|
||||
AdminEmail: *email,
|
||||
AdminPassword: *password,
|
||||
Log: log,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("embedded userservice: %w", err)
|
||||
}
|
||||
defer func() { _ = closeUsers() }()
|
||||
|
||||
// 5. Local worker agents before the server, so they can claim immediately.
|
||||
coordinatorURL := "http://" + *addr
|
||||
agents, err := spawnAgents(ctx, log, *dataDir, *workers, coordinatorURL, workerToken, venvPython)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer stopAgents(agents)
|
||||
|
||||
// 6. The coordinator server itself.
|
||||
cfg := infra.Config{
|
||||
Addr: *addr,
|
||||
DatabaseEngine: "sqlite",
|
||||
DBPath: filepath.Join(*dataDir, "scimesh.db"),
|
||||
Token: workerToken,
|
||||
JWTSecret: jwtSecret,
|
||||
UserserviceURL: "http://" + usersAddr,
|
||||
PublicUserserviceURL: "http://" + usersAddr,
|
||||
LogLevel: "info",
|
||||
StorageDir: filepath.Join(*dataDir, "artifacts"),
|
||||
DocsDir: *docsDir,
|
||||
MaxUploadBytes: 1 << 30,
|
||||
DBMaxConns: 4,
|
||||
DBConnectTimeout: 10 * time.Second,
|
||||
RequestTimeout: 15 * time.Second,
|
||||
HeartbeatInterval: 15 * time.Second,
|
||||
LeaseDuration: 2 * time.Minute,
|
||||
DefaultMaxAttempts: 3,
|
||||
QuorumSize: 2,
|
||||
ReaperInterval: 30 * time.Second,
|
||||
WorkerOfflineAfter: 1 * time.Minute,
|
||||
AutoMigrate: true,
|
||||
}
|
||||
if *open {
|
||||
openBrowser("http://" + *addr + "/ui")
|
||||
}
|
||||
|
||||
// Print the login once the server is about to start.
|
||||
fmt.Printf("\nSciMesh is starting at http://%s/ui\n", *addr)
|
||||
fmt.Printf(" admin login: %s / %s\n", *email, *password)
|
||||
if runtimeStatus(venvPython) {
|
||||
fmt.Printf(" scientific runtime: ready (%s)\n", venvPython)
|
||||
} else {
|
||||
fmt.Printf(" scientific runtime: NOT ready — install Python 3, then restart serve\n")
|
||||
}
|
||||
fmt.Printf(" data directory: %s\n\n", *dataDir)
|
||||
|
||||
err = runWithConfig(cfg)
|
||||
cancel()
|
||||
stopAgents(agents)
|
||||
return err
|
||||
}
|
||||
|
||||
// defaultDataDir returns the platform-appropriate data directory.
|
||||
func defaultDataDir() string {
|
||||
if dir := os.Getenv("SCIMESH_DATA_DIR"); dir != "" {
|
||||
return dir
|
||||
}
|
||||
home, err := os.UserHomeDir()
|
||||
if err != nil || home == "" {
|
||||
return ".scimesh"
|
||||
}
|
||||
return filepath.Join(home, ".scimesh")
|
||||
}
|
||||
|
||||
// loadOrGenerate reads a secret file, creating it with fresh random content
|
||||
// (chmod 0600) when missing.
|
||||
// #nosec G304 -- the path is an operator-supplied secret file inside the data dir.
|
||||
func loadOrGenerate(path string) (string, error) {
|
||||
if raw, err := os.ReadFile(path); err == nil {
|
||||
return strings.TrimSpace(string(raw)), nil
|
||||
}
|
||||
buffer := make([]byte, 32)
|
||||
if _, err := rand.Read(buffer); err != nil {
|
||||
return "", err
|
||||
}
|
||||
secret := hex.EncodeToString(buffer)
|
||||
if err := os.WriteFile(path, []byte(secret+"\n"), 0o600); err != nil {
|
||||
return "", fmt.Errorf("write %s: %w", path, err)
|
||||
}
|
||||
return secret, nil
|
||||
}
|
||||
|
||||
// spawnAgents starts `coordinator agent` subprocesses that claim tasks from
|
||||
// the coordinator. Each gets its own work directory under the data dir.
|
||||
func spawnAgents(ctx context.Context, log *slog.Logger, dataDir string, count int,
|
||||
coordinatorURL, token, venvPython string) ([]*exec.Cmd, error) {
|
||||
|
||||
var agents []*exec.Cmd
|
||||
for i := 0; i < count; i++ {
|
||||
workDir := filepath.Join(dataDir, "workers", fmt.Sprintf("%d", i))
|
||||
if err := os.MkdirAll(workDir, 0o750); err != nil {
|
||||
return agents, err
|
||||
}
|
||||
taskRunner := defaultTaskRunner(venvPython)
|
||||
// #nosec G204,G702 -- the command is this binary itself with operator flags.
|
||||
cmd := exec.CommandContext(ctx, os.Args[0], "agent",
|
||||
"--coordinator-url", coordinatorURL,
|
||||
"--token", token,
|
||||
"--work-dir", workDir,
|
||||
"--task-runner", taskRunner,
|
||||
)
|
||||
cmd.Stdout = os.Stdout
|
||||
cmd.Stderr = os.Stderr
|
||||
if err := cmd.Start(); err != nil {
|
||||
return agents, fmt.Errorf("start local agent %d: %w", i, err)
|
||||
}
|
||||
agents = append(agents, cmd)
|
||||
log.Info("local worker agent started", "index", i)
|
||||
}
|
||||
return agents, nil
|
||||
}
|
||||
|
||||
// stopAgents terminates the spawned agents and waits briefly for them.
|
||||
func stopAgents(agents []*exec.Cmd) {
|
||||
for _, agent := range agents {
|
||||
if agent.Process != nil {
|
||||
_ = agent.Process.Kill()
|
||||
}
|
||||
}
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
for _, agent := range agents {
|
||||
_, _ = agent.Process.Wait()
|
||||
}
|
||||
close(done)
|
||||
}()
|
||||
select {
|
||||
case <-done:
|
||||
case <-time.After(5 * time.Second):
|
||||
}
|
||||
}
|
||||
|
||||
// defaultTaskRunner picks the managed venv python when present, else the
|
||||
// system `python`.
|
||||
func defaultTaskRunner(venvPython string) string {
|
||||
if runtimeStatus(venvPython) {
|
||||
return venvPython + " -m scimesh.worker.task"
|
||||
}
|
||||
return "python -m scimesh.worker.task"
|
||||
}
|
||||
|
||||
// ensureRuntime creates the managed venv and installs scimesh into it, unless
|
||||
// it already exists. Best effort: a missing Python only logs a hint.
|
||||
func ensureRuntime(log *slog.Logger, dataDir, venvPython string) {
|
||||
if runtimeStatus(venvPython) {
|
||||
return
|
||||
}
|
||||
python := findPython()
|
||||
if python == "" {
|
||||
log.Warn("python3 not found; local workers need it to run scientific workloads")
|
||||
return
|
||||
}
|
||||
log.Info("creating the scientific runtime venv", "python", python)
|
||||
venvDir := filepath.Dir(filepath.Dir(venvPython))
|
||||
// #nosec G204 -- python comes from PATH and venvDir from the data dir.
|
||||
create := exec.CommandContext(context.Background(), python, "-m", "venv", venvDir)
|
||||
if out, err := create.CombinedOutput(); err != nil {
|
||||
log.Warn("venv creation failed; local workers need a manual Python install", "err", err, "output", string(out))
|
||||
return
|
||||
}
|
||||
pip := filepath.Join(venvDir, binName("bin/pip"))
|
||||
// The scimesh package is installed from an explicit source only: the PyPI
|
||||
// name is not ours yet, so `pip install scimesh` would fetch a stranger's
|
||||
// package. Operators publish a wheel or index via SCIMESH_PIP_PACKAGE.
|
||||
source := os.Getenv("SCIMESH_PIP_PACKAGE")
|
||||
if source == "" {
|
||||
log.Warn("scientific runtime venv created, but scimesh is not installed",
|
||||
"hint", pip+" install <your scimesh wheel or index> (or set SCIMESH_PIP_PACKAGE)")
|
||||
return
|
||||
}
|
||||
// #nosec G204,G702 -- pip and source are operator-configured paths.
|
||||
install := exec.CommandContext(context.Background(), pip, "install", source)
|
||||
if out, err := install.CombinedOutput(); err != nil {
|
||||
log.Warn("pip install failed", "err", err, "output", string(out))
|
||||
return
|
||||
}
|
||||
log.Info("scientific runtime installed", "venv", venvDir)
|
||||
}
|
||||
|
||||
// findPython locates a usable python3.
|
||||
func findPython() string {
|
||||
for _, candidate := range []string{"python3", "python"} {
|
||||
path, err := exec.LookPath(candidate)
|
||||
if err == nil {
|
||||
return path
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// runtimeStatus reports whether the managed venv python exists.
|
||||
func runtimeStatus(venvPython string) bool {
|
||||
info, err := os.Stat(venvPython)
|
||||
return err == nil && !info.IsDir()
|
||||
}
|
||||
|
||||
// binName adapts a relative path to the platform layout.
|
||||
func binName(relative string) string {
|
||||
if runtime.GOOS == "windows" {
|
||||
parts := strings.Split(relative, "/")
|
||||
parts[len(parts)-1] += ".exe"
|
||||
return strings.Join(parts, string(filepath.Separator))
|
||||
}
|
||||
return relative
|
||||
}
|
||||
|
||||
// openBrowser opens the UI in the platform's default browser.
|
||||
func openBrowser(target string) {
|
||||
command := ""
|
||||
switch runtime.GOOS {
|
||||
case "darwin":
|
||||
command = "open"
|
||||
case "windows":
|
||||
command = "rundll32"
|
||||
default:
|
||||
command = "xdg-open"
|
||||
}
|
||||
if command == "rundll32" {
|
||||
// #nosec G204 -- target is the local UI URL the operator asked to open.
|
||||
_ = exec.CommandContext(context.Background(), "rundll32", "url.dll,FileProtocolHandler", target).Start()
|
||||
return
|
||||
}
|
||||
// #nosec G204 -- target is the local UI URL the operator asked to open.
|
||||
_ = exec.CommandContext(context.Background(), command, target).Start()
|
||||
}
|
||||
@@ -16,18 +16,26 @@ require (
|
||||
require (
|
||||
github.com/beorn7/perks v1.0.1 // indirect
|
||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||
github.com/dustin/go-humanize v1.0.1 // indirect
|
||||
github.com/jackc/pgpassfile v1.0.0 // indirect
|
||||
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect
|
||||
github.com/jackc/puddle/v2 v2.2.1 // indirect
|
||||
github.com/lann/builder v0.0.0-20180802200727-47ae307949d0 // indirect
|
||||
github.com/lann/ps v0.0.0-20150810152359-62de8c46ede0 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
|
||||
github.com/ncruces/go-strftime v1.0.0 // indirect
|
||||
github.com/prometheus/client_model v0.6.2 // indirect
|
||||
github.com/prometheus/common v0.55.0 // indirect
|
||||
github.com/prometheus/procfs v0.21.1 // indirect
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||
golang.org/x/crypto v0.17.0 // indirect
|
||||
golang.org/x/sync v0.22.0 // indirect
|
||||
golang.org/x/sys v0.47.0 // indirect
|
||||
golang.org/x/text v0.40.0 // indirect
|
||||
google.golang.org/protobuf v1.36.11 // indirect
|
||||
modernc.org/libc v1.74.1 // indirect
|
||||
modernc.org/mathutil v1.7.1 // indirect
|
||||
modernc.org/memory v1.11.0 // indirect
|
||||
modernc.org/sqlite v1.55.0 // indirect
|
||||
)
|
||||
|
||||
@@ -9,6 +9,8 @@ github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XL
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
|
||||
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
|
||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||
@@ -29,8 +31,12 @@ github.com/lann/builder v0.0.0-20180802200727-47ae307949d0 h1:SOEGU9fKiNWd/HOJuq
|
||||
github.com/lann/builder v0.0.0-20180802200727-47ae307949d0/go.mod h1:dXGbAdH5GtBTC4WfIxhKZfyBF/HBFgRZSWwZ9g/He9o=
|
||||
github.com/lann/ps v0.0.0-20150810152359-62de8c46ede0 h1:P6pPBnrTSX3DEVR4fDembhRWSsG5rVo6hYhAB/ADZrk=
|
||||
github.com/lann/ps v0.0.0-20150810152359-62de8c46ede0/go.mod h1:vmVJ0l/dxyfGW6FmdpVm2joNMFikkuWg0EoCKLGUMNw=
|
||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
|
||||
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
|
||||
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
|
||||
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/prometheus/client_golang v1.19.1 h1:wZWJDwK+NameRJuPGDhlnFgx8e8HN3XHQeLaYJFJBOE=
|
||||
@@ -41,6 +47,8 @@ github.com/prometheus/common v0.55.0 h1:KEi6DK7lXW/m7Ig5i47x0vRzuBsHuvJdi5ee6Y3G
|
||||
github.com/prometheus/common v0.55.0/go.mod h1:2SECS4xJG1kd8XF9IcM1gMX6510RAEL65zxzNImwdc8=
|
||||
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
|
||||
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
|
||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||
@@ -51,6 +59,7 @@ golang.org/x/crypto v0.17.0 h1:r8bRNjWL3GshPW3gkd+RpvzWrZAwPS49OmTGZ/uhM4k=
|
||||
golang.org/x/crypto v0.17.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4=
|
||||
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
|
||||
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
|
||||
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
|
||||
@@ -63,3 +72,11 @@ gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYs
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
modernc.org/libc v1.74.1 h1:bdR4VTKFMC4966QSNZ05XLGI/VwzVa2kTUX51Dm0riQ=
|
||||
modernc.org/libc v1.74.1/go.mod h1:uH4t5bOx3G3g9Xcmj10YKlTcVISlRDwv8VoQJG9n8Os=
|
||||
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
|
||||
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
|
||||
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
|
||||
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
|
||||
modernc.org/sqlite v1.55.0 h1:hIFh0MCH0rGinQ/4KYb5/UbCkRkb+UP+OkLCVWa5MTM=
|
||||
modernc.org/sqlite v1.55.0/go.mod h1:4ntCLuNmnH8+GNqjka1wNg7KJd5/Hi5FYp8K+XQ7GZw=
|
||||
|
||||
@@ -101,7 +101,7 @@ func LoadConfig() (*Config, error) {
|
||||
return nil, err
|
||||
}
|
||||
if len(capabilities) == 0 {
|
||||
capabilities = defaultCapabilities()
|
||||
capabilities = DefaultCapabilities()
|
||||
}
|
||||
runner, err := envList("TASK_RUNNER")
|
||||
if err != nil {
|
||||
@@ -160,11 +160,11 @@ func durationEnv(name string, fallback time.Duration) (time.Duration, error) {
|
||||
return parsed, nil
|
||||
}
|
||||
|
||||
// defaultCapabilities derives the worker's advertised capabilities from the
|
||||
// DefaultCapabilities derives the worker's advertised capabilities from the
|
||||
// embedded workload catalog, so an agent is workload-agnostic out of the box:
|
||||
// it claims whatever enabled workloads the coordinator library declares.
|
||||
// Explicit CAPABILITIES still overrides this for operators who want a subset.
|
||||
func defaultCapabilities() []string {
|
||||
func DefaultCapabilities() []string {
|
||||
catalog, err := workloads.Load()
|
||||
if err != nil {
|
||||
return []string{"similarity-search"}
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"io/fs"
|
||||
"math"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
@@ -82,6 +83,12 @@ type Config struct {
|
||||
// On by default so a downloaded binary provisions its own database; set
|
||||
// AUTO_MIGRATE=false when an operator manages migrations out of band.
|
||||
AutoMigrate bool
|
||||
// DatabaseEngine selects the storage backend: "sqlite" (embedded, the
|
||||
// single-binary default) or "postgres" (cluster deployments). The
|
||||
// postgres engine requires DATABASE_URL.
|
||||
DatabaseEngine string
|
||||
// DBPath is the sqlite database file (engine=sqlite only).
|
||||
DBPath string
|
||||
}
|
||||
|
||||
// Load reads the environment and fails fast on anything required-but-missing
|
||||
@@ -128,8 +135,16 @@ func LoadConfig() (Config, error) {
|
||||
WorkerOfflineAfter: 1 * time.Minute,
|
||||
}
|
||||
|
||||
if cfg.DatabaseURL == "" {
|
||||
return Config{}, fmt.Errorf("DATABASE_URL is required")
|
||||
cfg.DatabaseEngine = getEnv("SCIMESH_DB", "sqlite")
|
||||
switch cfg.DatabaseEngine {
|
||||
case "sqlite", "postgres":
|
||||
default:
|
||||
return Config{}, fmt.Errorf("SCIMESH_DB must be sqlite or postgres")
|
||||
}
|
||||
cfg.DBPath = getEnv("SCIMESH_DB_PATH", filepath.Join(cfg.StorageDir, "scimesh.db"))
|
||||
|
||||
if cfg.DatabaseEngine == "postgres" && cfg.DatabaseURL == "" {
|
||||
return Config{}, fmt.Errorf("DATABASE_URL is required for the postgres engine")
|
||||
}
|
||||
if cfg.UIToken != "" && cfg.Token != "" && cfg.UIToken == cfg.Token {
|
||||
return Config{}, fmt.Errorf("UI_AUTH_TOKEN must differ from the worker auth token")
|
||||
@@ -185,7 +200,6 @@ func LoadConfig() (Config, error) {
|
||||
}
|
||||
cfg.AutoMigrate = parsed
|
||||
}
|
||||
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/domain"
|
||||
)
|
||||
|
||||
// ArtifactRepo implements usecase.ArtifactRepository on SQLite.
|
||||
type ArtifactRepo struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewArtifactRepo(db *sql.DB) *ArtifactRepo {
|
||||
return &ArtifactRepo{db: db}
|
||||
}
|
||||
|
||||
const artifactColumns = `id, job_id, task_id, attempt, kind, filename, storage_key,
|
||||
content_type, size_bytes, sha256, created_at`
|
||||
|
||||
// scanArtifact maps one row onto a domain.Artifact.
|
||||
func scanArtifact(row interface{ Scan(dest ...any) error }) (*domain.Artifact, error) {
|
||||
var (
|
||||
a domain.Artifact
|
||||
kind string
|
||||
)
|
||||
var (
|
||||
taskID sql.NullString
|
||||
attempt sql.NullInt64
|
||||
createdAt sql.NullInt64
|
||||
)
|
||||
if err := row.Scan(
|
||||
&a.ID, &a.JobID, &taskID, &attempt, &kind, &a.Filename, &a.StorageKey,
|
||||
&a.ContentType, &a.SizeBytes, &a.SHA256, &createdAt,
|
||||
); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
a.CreatedAt = decodeTime(createdAt.Int64)
|
||||
a.Kind = domain.ArtifactKind(kind)
|
||||
if taskID.Valid {
|
||||
if id, err := uuid.Parse(taskID.String); err == nil {
|
||||
a.TaskID = &id
|
||||
}
|
||||
}
|
||||
if attempt.Valid {
|
||||
value := int(attempt.Int64)
|
||||
a.Attempt = &value
|
||||
}
|
||||
return &a, nil
|
||||
}
|
||||
|
||||
func (r *ArtifactRepo) Insert(ctx context.Context, a *domain.Artifact) error {
|
||||
_, err := conn(ctx, r.db).ExecContext(ctx, `
|
||||
INSERT INTO artifacts (id, job_id, task_id, attempt, kind, filename, storage_key,
|
||||
content_type, size_bytes, sha256, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
a.ID.String(), a.JobID.String(), nullableUUID(a.TaskID), nullableInt(a.Attempt),
|
||||
string(a.Kind), a.Filename, a.StorageKey, a.ContentType, a.SizeBytes, a.SHA256,
|
||||
encodeTime(a.CreatedAt))
|
||||
return err
|
||||
}
|
||||
|
||||
func (r *ArtifactRepo) Get(ctx context.Context, id uuid.UUID) (*domain.Artifact, error) {
|
||||
row := conn(ctx, r.db).QueryRowContext(ctx,
|
||||
"SELECT "+artifactColumns+" FROM artifacts WHERE id = ?", id.String())
|
||||
artifact, err := scanArtifact(row)
|
||||
return artifact, mapErrNoRows(err, domain.ErrArtifactNotFound)
|
||||
}
|
||||
|
||||
func (r *ArtifactRepo) FindPartialResult(ctx context.Context, taskID uuid.UUID, attempt int) (*domain.Artifact, error) {
|
||||
row := conn(ctx, r.db).QueryRowContext(ctx,
|
||||
"SELECT "+artifactColumns+" FROM artifacts WHERE task_id = ? AND attempt = ? AND kind = ?",
|
||||
taskID.String(), attempt, string(domain.ArtifactPartialResult))
|
||||
artifact, err := scanArtifact(row)
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, nil
|
||||
}
|
||||
return artifact, err
|
||||
}
|
||||
|
||||
// nullableInt renders a nilable int as its value, or NULL.
|
||||
func nullableInt(n *int) any {
|
||||
if n == nil {
|
||||
return nil
|
||||
}
|
||||
return *n
|
||||
}
|
||||
@@ -0,0 +1,168 @@
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/domain"
|
||||
)
|
||||
|
||||
// JobRepo implements usecase.JobRepository on SQLite.
|
||||
type JobRepo struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewJobRepo(db *sql.DB) *JobRepo {
|
||||
return &JobRepo{db: db}
|
||||
}
|
||||
|
||||
const jobColumns = `id, workload, input_uri, parameters, status, created_at, completed_at,
|
||||
input_artifact_id, result_artifact_id, error_code, error_message, reducer_started_at, owner_id`
|
||||
|
||||
// scanJob maps one row onto a domain.Job. Scanned values follow the sqlite
|
||||
// column order exactly: ids are TEXT, parameters JSON TEXT, timestamps unix
|
||||
// nanoseconds (nullable), statuses plain strings.
|
||||
func scanJob(row interface{ Scan(dest ...any) error }) (*domain.Job, error) {
|
||||
var (
|
||||
j domain.Job
|
||||
status string
|
||||
params string
|
||||
)
|
||||
var (
|
||||
createdAt sql.NullInt64
|
||||
completedAt, reducerStartedAt sql.NullInt64
|
||||
inputArtifact, resultArtifact, ownerID sql.NullString
|
||||
errorCode, errorMessage sql.NullString
|
||||
)
|
||||
if err := row.Scan(
|
||||
&j.ID, &j.Workload, &j.InputURI, ¶ms, &status, &createdAt,
|
||||
&completedAt, &inputArtifact, &resultArtifact, &errorCode, &errorMessage,
|
||||
&reducerStartedAt, &ownerID,
|
||||
); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := decodeJSON(params, &j.Parameters); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
j.Status = domain.JobStatus(status)
|
||||
j.CreatedAt = decodeTime(createdAt.Int64)
|
||||
if completedAt.Valid {
|
||||
value := decodeTime(completedAt.Int64)
|
||||
j.CompletedAt = &value
|
||||
}
|
||||
if reducerStartedAt.Valid {
|
||||
value := decodeTime(reducerStartedAt.Int64)
|
||||
j.ReducerStartedAt = &value
|
||||
}
|
||||
if inputArtifact.Valid {
|
||||
if id, err := uuid.Parse(inputArtifact.String); err == nil {
|
||||
j.InputArtifactID = &id
|
||||
}
|
||||
}
|
||||
if resultArtifact.Valid {
|
||||
if id, err := uuid.Parse(resultArtifact.String); err == nil {
|
||||
j.ResultArtifactID = &id
|
||||
}
|
||||
}
|
||||
if ownerID.Valid {
|
||||
if id, err := uuid.Parse(ownerID.String); err == nil {
|
||||
j.OwnerID = &id
|
||||
}
|
||||
}
|
||||
if errorCode.Valid {
|
||||
j.ErrorCode = &errorCode.String
|
||||
}
|
||||
if errorMessage.Valid {
|
||||
j.ErrorMessage = &errorMessage.String
|
||||
}
|
||||
return &j, nil
|
||||
}
|
||||
|
||||
// Insert runs inside the caller's transaction alongside the job's tasks.
|
||||
func (r *JobRepo) Insert(ctx context.Context, j *domain.Job) error {
|
||||
_, err := conn(ctx, r.db).ExecContext(ctx, `
|
||||
INSERT INTO jobs (id, workload, input_uri, parameters, status, created_at, owner_id)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)`,
|
||||
j.ID.String(), j.Workload, j.InputURI, encodeJSON(j.Parameters), string(j.Status),
|
||||
encodeTime(j.CreatedAt), nullableUUID(j.OwnerID))
|
||||
return err
|
||||
}
|
||||
|
||||
func (r *JobRepo) Get(ctx context.Context, id uuid.UUID) (*domain.Job, error) {
|
||||
row := conn(ctx, r.db).QueryRowContext(ctx,
|
||||
"SELECT "+jobColumns+" FROM jobs WHERE id = ?", id.String())
|
||||
job, err := scanJob(row)
|
||||
return job, mapErrNoRows(err, domain.ErrJobNotFound)
|
||||
}
|
||||
|
||||
func (r *JobRepo) ClaimReduction(ctx context.Context, id uuid.UUID, startedAt time.Time) (bool, error) {
|
||||
res, err := conn(ctx, r.db).ExecContext(ctx, `
|
||||
UPDATE jobs SET reducer_started_at = ?
|
||||
WHERE id = ? AND status = ? AND reducer_started_at IS NULL`,
|
||||
encodeTime(startedAt), id.String(), string(domain.JobReducing))
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
affected, err := res.RowsAffected()
|
||||
return affected == 1, err
|
||||
}
|
||||
|
||||
func (r *JobRepo) CompleteWithResult(ctx context.Context, id, resultArtifactID uuid.UUID, completedAt time.Time) error {
|
||||
res, err := conn(ctx, r.db).ExecContext(ctx, `
|
||||
UPDATE jobs SET status = ?, result_artifact_id = ?, completed_at = ?,
|
||||
reducer_started_at = NULL, error_code = NULL, error_message = NULL
|
||||
WHERE id = ? AND status = ?`,
|
||||
string(domain.JobCompleted), resultArtifactID.String(), encodeTime(completedAt),
|
||||
id.String(), string(domain.JobReducing))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
affected, err := res.RowsAffected()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if affected == 0 {
|
||||
return domain.ErrJobNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *JobRepo) FailReduction(ctx context.Context, id uuid.UUID, code, message string, completedAt time.Time) error {
|
||||
_, err := conn(ctx, r.db).ExecContext(ctx, `
|
||||
UPDATE jobs SET status = ?, completed_at = ?, error_code = ?, error_message = ?,
|
||||
reducer_started_at = NULL
|
||||
WHERE id = ? AND status = ?`,
|
||||
string(domain.JobFailed), encodeTime(completedAt), code, message,
|
||||
id.String(), string(domain.JobReducing))
|
||||
return err
|
||||
}
|
||||
|
||||
func (r *JobRepo) UpdateStatus(ctx context.Context, id uuid.UUID,
|
||||
status domain.JobStatus, completedAt *time.Time) error {
|
||||
|
||||
res, err := conn(ctx, r.db).ExecContext(ctx,
|
||||
"UPDATE jobs SET status = ?, completed_at = ? WHERE id = ?",
|
||||
string(status), encodeTimePtr(completedAt), id.String())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
affected, err := res.RowsAffected()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if affected == 0 {
|
||||
return domain.ErrJobNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// nullableUUID renders a nilable UUID as its text, or NULL.
|
||||
func nullableUUID(id *uuid.UUID) any {
|
||||
if id == nil {
|
||||
return nil
|
||||
}
|
||||
return id.String()
|
||||
}
|
||||
@@ -0,0 +1,85 @@
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"embed"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
//go:embed migrations/*.sql
|
||||
var migrationFiles embed.FS
|
||||
|
||||
var migrationNamePattern = regexp.MustCompile(`^([0-9]+)_[a-z0-9_]+\.sql$`)
|
||||
|
||||
// Migrate applies every embedded migration above the current PRAGMA
|
||||
// user_version watermark, each inside its own transaction. It is idempotent:
|
||||
// the watermark only advances after a migration commits.
|
||||
func Migrate(ctx context.Context, db *sql.DB, log *slog.Logger) error {
|
||||
entries, err := migrationFiles.ReadDir("migrations")
|
||||
if err != nil {
|
||||
return fmt.Errorf("read embedded migrations: %w", err)
|
||||
}
|
||||
type file struct {
|
||||
version int
|
||||
name string
|
||||
}
|
||||
var files []file
|
||||
byVersion := map[int]string{}
|
||||
for _, entry := range entries {
|
||||
match := migrationNamePattern.FindStringSubmatch(entry.Name())
|
||||
if match == nil {
|
||||
continue
|
||||
}
|
||||
version, err := strconv.Atoi(match[1])
|
||||
if err != nil {
|
||||
return fmt.Errorf("migration %q has an invalid version: %w", entry.Name(), err)
|
||||
}
|
||||
if _, duplicate := byVersion[version]; duplicate {
|
||||
return fmt.Errorf("migration version %d is duplicated", version)
|
||||
}
|
||||
body, err := migrationFiles.ReadFile("migrations/" + entry.Name())
|
||||
if err != nil {
|
||||
return fmt.Errorf("read migration %q: %w", entry.Name(), err)
|
||||
}
|
||||
byVersion[version] = string(body)
|
||||
files = append(files, file{version: version, name: entry.Name()})
|
||||
}
|
||||
if len(files) == 0 {
|
||||
return fmt.Errorf("no sqlite migrations are embedded")
|
||||
}
|
||||
sort.Slice(files, func(i, j int) bool { return files[i].version < files[j].version })
|
||||
|
||||
var applied int
|
||||
if err := db.QueryRowContext(ctx, "PRAGMA user_version").Scan(&applied); err != nil {
|
||||
return fmt.Errorf("read schema version: %w", err)
|
||||
}
|
||||
for _, item := range files {
|
||||
if item.version <= applied {
|
||||
continue
|
||||
}
|
||||
if log != nil {
|
||||
log.Info("applying sqlite migration", "version", item.version, "file", item.name)
|
||||
}
|
||||
tx, err := db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx, byVersion[item.version]); err != nil {
|
||||
_ = tx.Rollback()
|
||||
return fmt.Errorf("apply migration %s: %w", item.name, err)
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx, fmt.Sprintf("PRAGMA user_version = %d", item.version)); err != nil {
|
||||
_ = tx.Rollback()
|
||||
return fmt.Errorf("advance schema version after %s: %w", item.name, err)
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return fmt.Errorf("commit migration %s: %w", item.name, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
-- 0001: core schema. SQLite stores enums as TEXT with CHECK constraints and
|
||||
-- JSON documents as TEXT; timestamps are unix nanoseconds (INTEGER).
|
||||
CREATE TABLE IF NOT EXISTS jobs (
|
||||
id TEXT PRIMARY KEY,
|
||||
workload TEXT NOT NULL,
|
||||
input_uri TEXT NOT NULL DEFAULT '',
|
||||
parameters TEXT NOT NULL DEFAULT '{}',
|
||||
status TEXT NOT NULL DEFAULT 'pending'
|
||||
CHECK (status IN ('pending','running','reducing','completed','failed','cancelled')),
|
||||
created_at INTEGER NOT NULL,
|
||||
completed_at INTEGER,
|
||||
input_artifact_id TEXT,
|
||||
result_artifact_id TEXT,
|
||||
error_code TEXT,
|
||||
error_message TEXT,
|
||||
reducer_started_at INTEGER,
|
||||
owner_id TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS tasks (
|
||||
id TEXT PRIMARY KEY,
|
||||
job_id TEXT NOT NULL REFERENCES jobs(id) ON DELETE CASCADE,
|
||||
chunk_index INTEGER NOT NULL,
|
||||
workload TEXT NOT NULL,
|
||||
input_uri TEXT,
|
||||
input_artifact_id TEXT,
|
||||
input_sha256 TEXT NOT NULL,
|
||||
parameters TEXT NOT NULL DEFAULT '{}',
|
||||
status TEXT NOT NULL DEFAULT 'pending'
|
||||
CHECK (status IN ('pending','leased','running','completed','failed','cancelled')),
|
||||
attempt INTEGER NOT NULL DEFAULT 0 CHECK (attempt >= 0),
|
||||
max_attempts INTEGER NOT NULL DEFAULT 3 CHECK (max_attempts > 0),
|
||||
lease_owner TEXT,
|
||||
lease_expires_at INTEGER,
|
||||
result_artifact_id TEXT,
|
||||
metrics TEXT,
|
||||
error_code TEXT,
|
||||
error_message TEXT,
|
||||
created_at INTEGER NOT NULL,
|
||||
started_at INTEGER,
|
||||
completed_at INTEGER,
|
||||
version INTEGER NOT NULL DEFAULT 0,
|
||||
CONSTRAINT uq_tasks_job_chunk UNIQUE (job_id, chunk_index),
|
||||
CONSTRAINT ck_tasks_completed_result CHECK (
|
||||
status <> 'completed' OR (result_artifact_id IS NOT NULL)
|
||||
),
|
||||
CONSTRAINT ck_tasks_leased_owner CHECK (
|
||||
status <> 'leased' OR (lease_owner IS NOT NULL AND lease_expires_at IS NOT NULL)
|
||||
)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_tasks_claim ON tasks (status, lease_expires_at, created_at);
|
||||
CREATE INDEX IF NOT EXISTS ix_tasks_job ON tasks (job_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS artifacts (
|
||||
id TEXT PRIMARY KEY,
|
||||
job_id TEXT NOT NULL REFERENCES jobs(id) ON DELETE CASCADE,
|
||||
task_id TEXT,
|
||||
attempt INTEGER,
|
||||
kind TEXT NOT NULL
|
||||
CHECK (kind IN ('input','shard','partial_result','final_result','log')),
|
||||
filename TEXT NOT NULL,
|
||||
storage_key TEXT NOT NULL,
|
||||
content_type TEXT NOT NULL,
|
||||
size_bytes INTEGER NOT NULL DEFAULT 0,
|
||||
sha256 TEXT NOT NULL DEFAULT '',
|
||||
created_at INTEGER NOT NULL,
|
||||
CONSTRAINT uq_partial_result_task_attempt UNIQUE (task_id, attempt)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_artifacts_job ON artifacts (job_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS workers (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
capabilities TEXT NOT NULL DEFAULT '[]',
|
||||
status TEXT NOT NULL DEFAULT 'online'
|
||||
CHECK (status IN ('online','busy','offline')),
|
||||
owner_id TEXT,
|
||||
trust_level TEXT NOT NULL DEFAULT 'trusted'
|
||||
CHECK (trust_level IN ('trusted','untrusted')),
|
||||
last_heartbeat_at INTEGER NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
updated_at INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS task_results (
|
||||
task_id TEXT NOT NULL REFERENCES tasks(id) ON DELETE CASCADE,
|
||||
owner_id TEXT NOT NULL,
|
||||
result_sha256 TEXT NOT NULL,
|
||||
result_artifact_id TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (task_id, owner_id)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_task_results_task ON task_results (task_id);
|
||||
@@ -0,0 +1,172 @@
|
||||
// Package sqlite implements the usecase repository ports on an embedded
|
||||
// SQLite database. It is the single-binary storage backend: no external
|
||||
// service, one file per database, pure-Go driver (modernc.org/sqlite) so the
|
||||
// static release binaries stay static.
|
||||
//
|
||||
// Concurrency model: SQLite allows exactly one writer. Every repository write
|
||||
// runs inside a TxManager transaction, and the database is opened with a
|
||||
// busy_timeout, so concurrent writers serialize instead of failing. The
|
||||
// postgres claim path uses FOR UPDATE SKIP LOCKED; here the same guarantee
|
||||
// comes from the write lock of the surrounding transaction — ClaimNext is
|
||||
// always called inside WithinTx by the usecase layer, so SELECT + UPDATE
|
||||
// cannot interleave.
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
_ "modernc.org/sqlite"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/usecase"
|
||||
)
|
||||
|
||||
// Open opens (and creates when missing) the database file, applying WAL,
|
||||
// foreign keys, and a busy timeout. Callers own the returned handle.
|
||||
func Open(path string) (*sql.DB, error) {
|
||||
dsn := fmt.Sprintf("file:%s?_pragma=busy_timeout(10000)&_pragma=journal_mode(WAL)&_pragma=foreign_keys(1)&_pragma=synchronous(NORMAL)", path)
|
||||
db, err := sql.Open("sqlite", dsn)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite database: %w", err)
|
||||
}
|
||||
if err := db.PingContext(context.Background()); err != nil {
|
||||
_ = db.Close()
|
||||
return nil, fmt.Errorf("ping sqlite database: %w", err)
|
||||
}
|
||||
return db, nil
|
||||
}
|
||||
|
||||
// querier is satisfied by both *sql.DB and *sql.Tx, letting every repository
|
||||
// method run identically inside or outside a transaction.
|
||||
type querier interface {
|
||||
ExecContext(ctx context.Context, query string, args ...any) (sql.Result, error)
|
||||
QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error)
|
||||
QueryRowContext(ctx context.Context, query string, args ...any) *sql.Row
|
||||
}
|
||||
|
||||
// txKey is an unexported struct type, so no other package can collide with it
|
||||
// or reach the transaction we stash in the context.
|
||||
type txKey struct{}
|
||||
|
||||
// TxManager implements usecase.TxManager.
|
||||
type TxManager struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewTxManager(db *sql.DB) *TxManager {
|
||||
return &TxManager{db: db}
|
||||
}
|
||||
|
||||
var _ usecase.TxManager = (*TxManager)(nil)
|
||||
|
||||
// WithinTx runs fn inside one transaction, committing on success and rolling
|
||||
// back on any error or panic. The transaction travels in the context, the
|
||||
// same pattern as the postgres backend. SQLite write transactions are
|
||||
// serialized by the database's single-writer lock, so a concurrent writer
|
||||
// waits on the busy timeout instead of racing.
|
||||
func (m *TxManager) WithinTx(ctx context.Context, fn func(ctx context.Context) error) error {
|
||||
if _, ok := ctx.Value(txKey{}).(*sql.Tx); ok {
|
||||
return fn(ctx)
|
||||
}
|
||||
tx, err := m.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
|
||||
if err := fn(context.WithValue(ctx, txKey{}, tx)); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Commit()
|
||||
}
|
||||
|
||||
// conn returns the transaction bound to ctx, or the database when there is none.
|
||||
func conn(ctx context.Context, db *sql.DB) querier {
|
||||
if tx, ok := ctx.Value(txKey{}).(*sql.Tx); ok {
|
||||
return tx
|
||||
}
|
||||
return db
|
||||
}
|
||||
|
||||
// --- JSON and null helpers ------------------------------------------------
|
||||
|
||||
// encodeJSON stores a Go value as JSON text, defaulting to "{}".
|
||||
func encodeJSON(value any) string {
|
||||
if value == nil {
|
||||
return "{}"
|
||||
}
|
||||
encoded, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return "{}"
|
||||
}
|
||||
return string(encoded)
|
||||
}
|
||||
|
||||
// decodeJSON reads a JSON text column into the destination.
|
||||
func decodeJSON(raw any, destination any) error {
|
||||
text, ok := raw.(string)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
if text == "" {
|
||||
return nil
|
||||
}
|
||||
return json.Unmarshal([]byte(text), destination)
|
||||
}
|
||||
|
||||
// encodeTime stores a time as unix nanoseconds (NULL for zero time).
|
||||
func encodeTime(t time.Time) any {
|
||||
if t.IsZero() {
|
||||
return nil
|
||||
}
|
||||
return t.UnixNano()
|
||||
}
|
||||
|
||||
// encodeTimePtr stores a nilable time as unix nanoseconds.
|
||||
func encodeTimePtr(t *time.Time) any {
|
||||
if t == nil {
|
||||
return nil
|
||||
}
|
||||
return t.UnixNano()
|
||||
}
|
||||
|
||||
// decodeTime reads a unix-nanosecond column back into a time.Time.
|
||||
func decodeTime(raw any) time.Time {
|
||||
switch v := raw.(type) {
|
||||
case int64:
|
||||
return time.Unix(0, v).UTC()
|
||||
case int:
|
||||
return time.Unix(0, int64(v)).UTC()
|
||||
}
|
||||
return time.Time{}
|
||||
}
|
||||
|
||||
// nullIfEmpty maps "" to SQL NULL.
|
||||
func nullIfEmpty(s string) any {
|
||||
if s == "" {
|
||||
return nil
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// mapErrNoRows translates sql.ErrNoRows into the domain not-found errors.
|
||||
func mapErrNoRows(err error, notFound error) error {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return notFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
var (
|
||||
_ usecase.JobRepository = (*JobRepo)(nil)
|
||||
_ usecase.TaskRepository = (*TaskRepo)(nil)
|
||||
_ usecase.ArtifactRepository = (*ArtifactRepo)(nil)
|
||||
_ usecase.WorkerRepository = (*WorkerRepo)(nil)
|
||||
_ usecase.TaskResultRepository = (*TaskResultRepo)(nil)
|
||||
_ usecase.UIReadRepository = (*UIReadRepo)(nil)
|
||||
_ usecase.TxManager = (*TxManager)(nil)
|
||||
)
|
||||
@@ -0,0 +1,349 @@
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/domain"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/usecase"
|
||||
)
|
||||
|
||||
// newTestDB opens an isolated on-disk database and applies the migrations.
|
||||
func newTestDB(t *testing.T) *sql.DB {
|
||||
t.Helper()
|
||||
db, err := Open(filepath.Join(t.TempDir(), "test.db"))
|
||||
if err != nil {
|
||||
t.Fatalf("open: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = db.Close() })
|
||||
if err := Migrate(context.Background(), db, nil); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
return db
|
||||
}
|
||||
|
||||
func fixedTime() time.Time {
|
||||
return time.Date(2026, 8, 2, 12, 0, 0, 0, time.UTC)
|
||||
}
|
||||
|
||||
func seedJob(t *testing.T, db *sql.DB, n int) *domain.Job {
|
||||
t.Helper()
|
||||
ctx := context.Background()
|
||||
tx := NewTxManager(db)
|
||||
chunks := make([]domain.ChunkSpec, 0, n)
|
||||
for i := 0; i < n; i++ {
|
||||
chunks = append(chunks, domain.ChunkSpec{
|
||||
ChunkIndex: i,
|
||||
InputURI: "s3://chunk-" + string(rune('a'+i)),
|
||||
InputSHA256: "sha-" + string(rune('a'+i)),
|
||||
})
|
||||
}
|
||||
job, tasks, err := domain.NewJobWithTasks("similarity_search", "s3://ds", nil, chunks, fixedTime())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := tx.WithinTx(ctx, func(ctx context.Context) error {
|
||||
if err := NewJobRepo(db).Insert(ctx, job); err != nil {
|
||||
return err
|
||||
}
|
||||
return NewTaskRepo(db).InsertBatch(ctx, tasks)
|
||||
}); err != nil {
|
||||
t.Fatalf("seed: %v", err)
|
||||
}
|
||||
return job
|
||||
}
|
||||
|
||||
func TestMigrateIsIdempotent(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
if err := Migrate(ctx, db, nil); err != nil {
|
||||
t.Fatalf("second migrate: %v", err)
|
||||
}
|
||||
var version int
|
||||
if err := db.QueryRowContext(ctx, "PRAGMA user_version").Scan(&version); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if version != 1 {
|
||||
t.Errorf("user_version = %d, want 1", version)
|
||||
}
|
||||
}
|
||||
|
||||
func TestJobRepoRoundTrip(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
job := seedJob(t, db, 1)
|
||||
|
||||
got, err := NewJobRepo(db).Get(ctx, job.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Workload != job.Workload || got.Status != domain.JobPending {
|
||||
t.Errorf("job = %+v", got)
|
||||
}
|
||||
if err := NewJobRepo(db).UpdateStatus(ctx, job.ID, domain.JobRunning, nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err = NewJobRepo(db).Get(ctx, job.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Status != domain.JobRunning {
|
||||
t.Errorf("status = %q, want running", got.Status)
|
||||
}
|
||||
if _, err := NewJobRepo(db).Get(ctx, uuid.New()); !errors.Is(err, domain.ErrJobNotFound) {
|
||||
t.Errorf("missing job err = %v, want ErrJobNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClaimGivesEachTaskToExactlyOneWorker(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
seedJob(t, db, 3)
|
||||
repo := NewTaskRepo(db)
|
||||
|
||||
claimed := map[uuid.UUID]bool{}
|
||||
for i := 0; i < 3; i++ {
|
||||
task, err := repo.ClaimNext(ctx, usecase.ClaimFilter{
|
||||
Workloads: []string{"similarity_search"},
|
||||
Owner: "w1",
|
||||
Now: fixedTime(),
|
||||
LeaseUntil: fixedTime().Add(time.Minute),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if task == nil {
|
||||
t.Fatal("claim returned nil on a non-empty queue")
|
||||
}
|
||||
if claimed[task.ID] {
|
||||
t.Fatalf("task %s claimed twice", task.ID)
|
||||
}
|
||||
claimed[task.ID] = true
|
||||
if task.Status != domain.TaskLeased || task.Attempt != 1 || task.LeaseOwner == nil || *task.LeaseOwner != "w1" {
|
||||
t.Errorf("task = %+v", task)
|
||||
}
|
||||
}
|
||||
task, err := repo.ClaimNext(ctx, usecase.ClaimFilter{Workloads: []string{"similarity_search"}, Owner: "w1", Now: fixedTime(), LeaseUntil: fixedTime().Add(time.Minute)})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if task != nil {
|
||||
t.Fatal("claim must return nil on an empty queue")
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpdateRejectsStaleVersion(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
seedJob(t, db, 1)
|
||||
repo := NewTaskRepo(db)
|
||||
task, err := repo.ClaimNext(ctx, usecase.ClaimFilter{Workloads: nil, Owner: "w1", Now: fixedTime(), LeaseUntil: fixedTime().Add(time.Minute)})
|
||||
if err != nil || task == nil {
|
||||
t.Fatalf("claim: %v", err)
|
||||
}
|
||||
stale := *task
|
||||
task.Status = domain.TaskRunning
|
||||
task.Version++ // as a domain method would have done
|
||||
if err := repo.Update(ctx, task); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
stale.Status = domain.TaskCompleted
|
||||
stale.Version++
|
||||
if err := repo.Update(ctx, &stale); !errors.Is(err, domain.ErrLeaseConflict) {
|
||||
t.Errorf("stale update err = %v, want ErrLeaseConflict", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExpireLeasesRequeuesElapsedTasks(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
seedJob(t, db, 1)
|
||||
repo := NewTaskRepo(db)
|
||||
task, err := repo.ClaimNext(ctx, usecase.ClaimFilter{Workloads: nil, Owner: "w1", Now: fixedTime(), LeaseUntil: fixedTime().Add(-time.Minute)})
|
||||
if err != nil || task == nil {
|
||||
t.Fatalf("claim: %v", err)
|
||||
}
|
||||
affected, err := repo.ExpireLeases(ctx, fixedTime())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(affected) != 1 {
|
||||
t.Fatalf("affected = %v, want 1 job", affected)
|
||||
}
|
||||
task, err = repo.ClaimNext(ctx, usecase.ClaimFilter{Workloads: nil, Owner: "w2", Now: fixedTime(), LeaseUntil: fixedTime().Add(time.Minute)})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if task == nil || task.Attempt != 2 {
|
||||
t.Errorf("requeued task = %+v, want attempt 2", task)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExpireLeasesFailsAfterFinalAttempt(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
seedJob(t, db, 1)
|
||||
repo := NewTaskRepo(db)
|
||||
for attempt := 1; attempt <= 3; attempt++ {
|
||||
task, err := repo.ClaimNext(ctx, usecase.ClaimFilter{Workloads: nil, Owner: "w1", Now: fixedTime(), LeaseUntil: fixedTime().Add(-time.Minute)})
|
||||
if err != nil || task == nil {
|
||||
t.Fatalf("claim %d: %v", attempt, err)
|
||||
}
|
||||
if _, err := repo.ExpireLeases(ctx, fixedTime()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
task, err := repo.ClaimNext(ctx, usecase.ClaimFilter{Workloads: nil, Owner: "w1", Now: fixedTime(), LeaseUntil: fixedTime().Add(time.Minute)})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if task != nil {
|
||||
t.Fatal("exhausted task must not be claimable")
|
||||
}
|
||||
}
|
||||
|
||||
func TestArtifactRepoRoundTripAndUniqueness(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
job := seedJob(t, db, 1)
|
||||
repo := NewArtifactRepo(db)
|
||||
attempt := 1
|
||||
artifact, err := domain.NewArtifact(job.ID, nil, domain.ArtifactShard, "shard-0.tsv", "text/tab-separated-values", fixedTime())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
artifact.SetContent("abc123", 42)
|
||||
if err := repo.Insert(ctx, artifact); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := repo.Get(ctx, artifact.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.SHA256 != "abc123" || got.SizeBytes != 42 {
|
||||
t.Errorf("artifact = %+v", got)
|
||||
}
|
||||
|
||||
partialTask := uuid.New()
|
||||
partial, err := domain.NewArtifact(job.ID, &partialTask, domain.ArtifactPartialResult, "p.csv", "text/csv", fixedTime())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
partial.Attempt = &attempt
|
||||
if err := repo.Insert(ctx, partial); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
found, err := repo.FindPartialResult(ctx, partialTask, attempt)
|
||||
if err != nil || found == nil {
|
||||
t.Fatalf("find partial: %v", err)
|
||||
}
|
||||
duplicate, err := domain.NewArtifact(job.ID, &partialTask, domain.ArtifactPartialResult, "p2.csv", "text/csv", fixedTime())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
duplicate.Attempt = &attempt
|
||||
if err := repo.Insert(ctx, duplicate); err == nil {
|
||||
t.Fatal("duplicate partial for the same attempt must fail")
|
||||
}
|
||||
}
|
||||
|
||||
func TestWorkerRepoRoundTripAndLiveness(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
repo := NewWorkerRepo(db)
|
||||
worker, err := domain.NewWorker("w1", []string{"similarity-search"}, fixedTime())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := repo.Insert(ctx, worker); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := repo.Get(ctx, worker.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Name != "w1" || len(got.Capabilities) != 1 || got.TrustLevel != domain.WorkerTrusted {
|
||||
t.Errorf("worker = %+v", got)
|
||||
}
|
||||
if err := repo.Touch(ctx, worker.ID, fixedTime().Add(time.Hour)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
changed, err := repo.MarkStaleOffline(ctx, fixedTime().Add(2*time.Hour))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if changed != 1 {
|
||||
t.Errorf("offline changes = %d, want 1", changed)
|
||||
}
|
||||
got, err = repo.Get(ctx, worker.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Status != domain.WorkerOffline {
|
||||
t.Errorf("status = %q, want offline", got.Status)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTaskResultVotes(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
job := seedJob(t, db, 1)
|
||||
repo := NewTaskResultRepo(db)
|
||||
task, err := domain.NewTask(job.ID, 1, "similarity_search", "s3://in", "sha", nil, 3, fixedTime())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := NewTaskRepo(db).InsertBatch(ctx, []*domain.Task{task}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
taskID := task.ID
|
||||
artifactID := uuid.New()
|
||||
ownerA, ownerB := uuid.New(), uuid.New()
|
||||
if err := repo.RecordVote(ctx, taskID, ownerA, "hash", artifactID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := repo.RecordVote(ctx, taskID, ownerB, "hash", artifactID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := repo.RecordVote(ctx, taskID, ownerA, "hash2", artifactID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
n, err := repo.CountAgreeing(ctx, taskID, "hash")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if n != 1 {
|
||||
t.Errorf("agreeing = %d, want 1 (owner A changed its vote)", n)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCancelByJobInvalidatesTasks(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
job := seedJob(t, db, 2)
|
||||
repo := NewTaskRepo(db)
|
||||
task, err := repo.ClaimNext(ctx, usecase.ClaimFilter{Workloads: nil, Owner: "w1", Now: fixedTime(), LeaseUntil: fixedTime().Add(time.Minute)})
|
||||
if err != nil || task == nil {
|
||||
t.Fatalf("claim: %v", err)
|
||||
}
|
||||
cancelled, err := repo.CancelByJob(ctx, job.ID, fixedTime())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cancelled != 2 {
|
||||
t.Errorf("cancelled = %d, want 2", cancelled)
|
||||
}
|
||||
got, err := repo.Get(ctx, task.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Status != domain.TaskCancelled || got.LeaseOwner != nil {
|
||||
t.Errorf("cancelled task = %+v", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/domain"
|
||||
)
|
||||
|
||||
// Known statuses per entity, so counts are zero-filled and every status is
|
||||
// always present in the metrics (a flat 0 line beats a gap on the dashboard).
|
||||
var (
|
||||
taskStatuses = []string{string(domain.TaskPending), string(domain.TaskLeased), string(domain.TaskRunning), string(domain.TaskCompleted), string(domain.TaskFailed), string(domain.TaskCancelled)}
|
||||
jobStatuses = []string{string(domain.JobPending), string(domain.JobRunning), string(domain.JobReducing), string(domain.JobCompleted), string(domain.JobFailed), string(domain.JobCancelled)}
|
||||
workerStatuses = []string{string(domain.WorkerOnline), string(domain.WorkerBusy), string(domain.WorkerOffline)}
|
||||
)
|
||||
|
||||
// StatsRepo answers the aggregate status counts the business metrics report.
|
||||
type StatsRepo struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewStatsRepo(db *sql.DB) *StatsRepo {
|
||||
return &StatsRepo{db: db}
|
||||
}
|
||||
|
||||
// Counts returns status->count maps for tasks, jobs, and workers, each
|
||||
// zero-filled across its known statuses.
|
||||
func (r *StatsRepo) Counts(ctx context.Context) (tasks, jobs, workers map[string]int, err error) {
|
||||
if tasks, err = r.countByStatus(ctx, "tasks", taskStatuses); err != nil {
|
||||
return nil, nil, nil, err
|
||||
}
|
||||
if jobs, err = r.countByStatus(ctx, "jobs", jobStatuses); err != nil {
|
||||
return nil, nil, nil, err
|
||||
}
|
||||
if workers, err = r.countByStatus(ctx, "workers", workerStatuses); err != nil {
|
||||
return nil, nil, nil, err
|
||||
}
|
||||
return tasks, jobs, workers, nil
|
||||
}
|
||||
|
||||
func (r *StatsRepo) countByStatus(ctx context.Context, table string, known []string) (map[string]int, error) {
|
||||
out := make(map[string]int, len(known))
|
||||
for _, s := range known {
|
||||
out[s] = 0 // zero-fill
|
||||
}
|
||||
// table is a fixed internal constant, never user input — safe to format.
|
||||
rows, err := r.db.QueryContext(ctx, fmt.Sprintf("SELECT status, count(*) FROM %s GROUP BY status", table))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("count %s by status: %w", table, err)
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
for rows.Next() {
|
||||
var status string
|
||||
var n int
|
||||
if err := rows.Scan(&status, &n); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out[status] = n
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
@@ -0,0 +1,314 @@
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/domain"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/usecase"
|
||||
)
|
||||
|
||||
// TaskRepo implements usecase.TaskRepository on SQLite.
|
||||
type TaskRepo struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewTaskRepo(db *sql.DB) *TaskRepo {
|
||||
return &TaskRepo{db: db}
|
||||
}
|
||||
|
||||
const taskColumns = `id, job_id, chunk_index, workload, input_uri, input_artifact_id, input_sha256,
|
||||
parameters, status, attempt, max_attempts, lease_owner, lease_expires_at,
|
||||
result_artifact_id, metrics, error_code, error_message,
|
||||
created_at, started_at, completed_at, version`
|
||||
|
||||
// scanTask maps one row onto a domain.Task.
|
||||
func scanTask(row interface{ Scan(dest ...any) error }) (*domain.Task, error) {
|
||||
var (
|
||||
t domain.Task
|
||||
status string
|
||||
params string
|
||||
metrics sql.NullString
|
||||
)
|
||||
var (
|
||||
inputURI, leaseOwner, errorCode, errorMessage sql.NullString
|
||||
inputArtifact, resultArtifact sql.NullString
|
||||
leaseExpiresAt, startedAt, completedAt sql.NullInt64
|
||||
createdAt sql.NullInt64
|
||||
)
|
||||
if err := row.Scan(
|
||||
&t.ID, &t.JobID, &t.ChunkIndex, &t.Workload, &inputURI, &inputArtifact, &t.InputSHA256,
|
||||
¶ms, &status, &t.Attempt, &t.MaxAttempts, &leaseOwner, &leaseExpiresAt,
|
||||
&resultArtifact, &metrics, &errorCode, &errorMessage,
|
||||
&createdAt, &startedAt, &completedAt, &t.Version,
|
||||
); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := decodeJSON(params, &t.Parameters); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if metrics.Valid && metrics.String != "" {
|
||||
if err := decodeJSON(metrics.String, &t.Metrics); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
t.Status = domain.TaskStatus(status)
|
||||
t.CreatedAt = decodeTime(createdAt.Int64)
|
||||
if inputURI.Valid {
|
||||
t.InputURI = inputURI.String
|
||||
}
|
||||
if leaseOwner.Valid {
|
||||
t.LeaseOwner = &leaseOwner.String
|
||||
}
|
||||
if inputArtifact.Valid {
|
||||
if id, err := uuid.Parse(inputArtifact.String); err == nil {
|
||||
t.InputArtifactID = &id
|
||||
}
|
||||
}
|
||||
if resultArtifact.Valid {
|
||||
if id, err := uuid.Parse(resultArtifact.String); err == nil {
|
||||
t.ResultArtifactID = &id
|
||||
}
|
||||
}
|
||||
if leaseExpiresAt.Valid {
|
||||
value := decodeTime(leaseExpiresAt.Int64)
|
||||
t.LeaseExpiresAt = &value
|
||||
}
|
||||
if startedAt.Valid {
|
||||
value := decodeTime(startedAt.Int64)
|
||||
t.StartedAt = &value
|
||||
}
|
||||
if completedAt.Valid {
|
||||
value := decodeTime(completedAt.Int64)
|
||||
t.CompletedAt = &value
|
||||
}
|
||||
if errorCode.Valid {
|
||||
t.ErrorCode = &errorCode.String
|
||||
}
|
||||
if errorMessage.Valid {
|
||||
t.ErrorMessage = &errorMessage.String
|
||||
}
|
||||
return &t, nil
|
||||
}
|
||||
|
||||
// ClaimNext atomically leases the next eligible task. SQLite has no SKIP
|
||||
// LOCKED: the guarantee comes from the surrounding transaction's write lock —
|
||||
// the usecase layer always calls ClaimNext inside WithinTx, so SELECT + UPDATE
|
||||
// cannot interleave with another claimant.
|
||||
func (r *TaskRepo) ClaimNext(ctx context.Context, f usecase.ClaimFilter) (*domain.Task, error) {
|
||||
workloadClause := ""
|
||||
workloadArgs := []any{}
|
||||
if len(f.Workloads) > 0 {
|
||||
placeholders := make([]string, 0, len(f.Workloads))
|
||||
for _, w := range f.Workloads {
|
||||
placeholders = append(placeholders, "?")
|
||||
workloadArgs = append(workloadArgs, w)
|
||||
}
|
||||
workloadClause = " AND workload IN (" + strings.Join(placeholders, ", ") + ")"
|
||||
}
|
||||
voterClause := ""
|
||||
var voterArg any
|
||||
if f.VoterOwner != nil {
|
||||
voterClause = " AND NOT EXISTS (SELECT 1 FROM task_results tr WHERE tr.task_id = tasks.id AND tr.owner_id = ?)"
|
||||
voterArg = f.VoterOwner.String()
|
||||
}
|
||||
args := append([]any{f.Owner, f.LeaseUntil.UnixNano(), f.Now.UnixNano()}, workloadArgs...)
|
||||
if f.VoterOwner != nil {
|
||||
args = append(args, voterArg)
|
||||
}
|
||||
query := `
|
||||
UPDATE tasks SET
|
||||
status = 'leased',
|
||||
attempt = attempt + 1,
|
||||
lease_owner = ?,
|
||||
lease_expires_at = ?,
|
||||
started_at = COALESCE(started_at, ?),
|
||||
version = version + 1
|
||||
WHERE id IN (
|
||||
SELECT id FROM tasks
|
||||
WHERE status = 'pending' AND attempt < max_attempts` + workloadClause + voterClause + `
|
||||
ORDER BY created_at, chunk_index
|
||||
LIMIT 1
|
||||
)
|
||||
RETURNING ` + taskColumns
|
||||
|
||||
row := conn(ctx, r.db).QueryRowContext(ctx, query, args...)
|
||||
task, err := scanTask(row)
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return task, nil
|
||||
}
|
||||
|
||||
func (r *TaskRepo) Get(ctx context.Context, id uuid.UUID) (*domain.Task, error) {
|
||||
row := conn(ctx, r.db).QueryRowContext(ctx,
|
||||
"SELECT "+taskColumns+" FROM tasks WHERE id = ?", id.String())
|
||||
task, err := scanTask(row)
|
||||
return task, mapErrNoRows(err, domain.ErrTaskNotFound)
|
||||
}
|
||||
|
||||
// GetForUpdate reads a task. SQLite serializes writers inside a transaction,
|
||||
// so no row lock is needed: the surrounding write transaction already isolates
|
||||
// the read-modify-write sequence.
|
||||
func (r *TaskRepo) GetForUpdate(ctx context.Context, id uuid.UUID) (*domain.Task, error) {
|
||||
return r.Get(ctx, id)
|
||||
}
|
||||
|
||||
// Update writes the mutated entity back under optimistic concurrency.
|
||||
func (r *TaskRepo) Update(ctx context.Context, t *domain.Task) error {
|
||||
res, err := conn(ctx, r.db).ExecContext(ctx, `
|
||||
UPDATE tasks SET
|
||||
status = ?, attempt = ?, lease_owner = ?, lease_expires_at = ?,
|
||||
result_artifact_id = ?, metrics = ?, error_code = ?, error_message = ?,
|
||||
started_at = ?, completed_at = ?, version = ?
|
||||
WHERE id = ? AND version = ?`,
|
||||
string(t.Status), t.Attempt, nullableString(t.LeaseOwner), encodeTimePtr(t.LeaseExpiresAt),
|
||||
nullableUUID(t.ResultArtifactID), nullableMetrics(t.Metrics),
|
||||
nullableString(t.ErrorCode), nullableString(t.ErrorMessage),
|
||||
encodeTimePtr(t.StartedAt), encodeTimePtr(t.CompletedAt), t.Version,
|
||||
t.ID.String(), t.Version-1)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
affected, err := res.RowsAffected()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if affected == 0 {
|
||||
return domain.ErrLeaseConflict
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *TaskRepo) InsertBatch(ctx context.Context, tasks []*domain.Task) error {
|
||||
for _, t := range tasks {
|
||||
_, err := conn(ctx, r.db).ExecContext(ctx, `
|
||||
INSERT INTO tasks (id, job_id, chunk_index, workload, input_uri, input_artifact_id,
|
||||
input_sha256, parameters, status, attempt, max_attempts, created_at, version)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
t.ID.String(), t.JobID.String(), t.ChunkIndex, t.Workload,
|
||||
nullIfEmpty(t.InputURI), nullableUUID(t.InputArtifactID),
|
||||
t.InputSHA256, encodeJSON(t.Parameters), string(t.Status),
|
||||
t.Attempt, t.MaxAttempts, encodeTime(t.CreatedAt), t.Version)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *TaskRepo) ListCompleted(ctx context.Context, jobID uuid.UUID) ([]*domain.Task, error) {
|
||||
rows, err := conn(ctx, r.db).QueryContext(ctx,
|
||||
"SELECT "+taskColumns+" FROM tasks WHERE job_id = ? AND status = ? ORDER BY chunk_index",
|
||||
jobID.String(), string(domain.TaskCompleted))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
var tasks []*domain.Task
|
||||
for rows.Next() {
|
||||
task, err := scanTask(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tasks = append(tasks, task)
|
||||
}
|
||||
return tasks, rows.Err()
|
||||
}
|
||||
|
||||
func (r *TaskRepo) CountByStatus(ctx context.Context, jobID uuid.UUID) (map[domain.TaskStatus]int, error) {
|
||||
rows, err := conn(ctx, r.db).QueryContext(ctx,
|
||||
"SELECT status, count(*) FROM tasks WHERE job_id = ? GROUP BY status", jobID.String())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
counts := make(map[domain.TaskStatus]int)
|
||||
for rows.Next() {
|
||||
var (
|
||||
status string
|
||||
n int
|
||||
)
|
||||
if err := rows.Scan(&status, &n); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
counts[domain.TaskStatus(status)] = n
|
||||
}
|
||||
return counts, rows.Err()
|
||||
}
|
||||
|
||||
func (r *TaskRepo) CancelByJob(ctx context.Context, jobID uuid.UUID, now time.Time) (int64, error) {
|
||||
res, err := conn(ctx, r.db).ExecContext(ctx, `
|
||||
UPDATE tasks SET
|
||||
status = 'cancelled',
|
||||
lease_owner = NULL,
|
||||
lease_expires_at = NULL,
|
||||
error_code = NULL,
|
||||
error_message = NULL,
|
||||
completed_at = ?,
|
||||
version = version + 1
|
||||
WHERE job_id = ? AND status IN ('pending','leased','running')`,
|
||||
now.UnixNano(), jobID.String())
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return res.RowsAffected()
|
||||
}
|
||||
|
||||
// ExpireLeases applies the lease-expiry rule set-based and returns the
|
||||
// distinct jobs whose aggregate status may have changed.
|
||||
func (r *TaskRepo) ExpireLeases(ctx context.Context, now time.Time) ([]uuid.UUID, error) {
|
||||
rows, err := conn(ctx, r.db).QueryContext(ctx, `
|
||||
UPDATE tasks SET
|
||||
status = CASE WHEN attempt < max_attempts THEN 'pending' ELSE 'failed' END,
|
||||
lease_owner = NULL,
|
||||
lease_expires_at = NULL,
|
||||
error_code = CASE WHEN attempt >= max_attempts THEN ? ELSE error_code END,
|
||||
error_message = CASE WHEN attempt >= max_attempts THEN ? ELSE error_message END,
|
||||
completed_at = CASE WHEN attempt >= max_attempts THEN ? ELSE completed_at END,
|
||||
version = version + 1
|
||||
WHERE status IN ('leased','running') AND lease_expires_at < ?
|
||||
RETURNING job_id`,
|
||||
domain.ErrCodeLeaseExpired, "lease expired after the final attempt", now.UnixNano(), now.UnixNano())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
var affected []uuid.UUID
|
||||
seen := map[uuid.UUID]bool{}
|
||||
for rows.Next() {
|
||||
var raw string
|
||||
if err := rows.Scan(&raw); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if id, err := uuid.Parse(raw); err == nil && !seen[id] {
|
||||
seen[id] = true
|
||||
affected = append(affected, id)
|
||||
}
|
||||
}
|
||||
return affected, rows.Err()
|
||||
}
|
||||
|
||||
// nullableString renders a nilable string, or NULL.
|
||||
func nullableString(s *string) any {
|
||||
if s == nil {
|
||||
return nil
|
||||
}
|
||||
return *s
|
||||
}
|
||||
|
||||
// nullableMetrics stores nil metrics as NULL, else JSON text.
|
||||
func nullableMetrics(m map[string]any) any {
|
||||
if m == nil {
|
||||
return nil
|
||||
}
|
||||
return encodeJSON(m)
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// TaskResultRepo records and tallies quorum votes for untrusted task results.
|
||||
type TaskResultRepo struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewTaskResultRepo(db *sql.DB) *TaskResultRepo {
|
||||
return &TaskResultRepo{db: db}
|
||||
}
|
||||
|
||||
// RecordVote stores (or replaces) one owner's vote for a task's result.
|
||||
func (r *TaskResultRepo) RecordVote(ctx context.Context, taskID, ownerID uuid.UUID, sha256 string, artifactID uuid.UUID) error {
|
||||
_, err := conn(ctx, r.db).ExecContext(ctx, `
|
||||
INSERT INTO task_results (task_id, owner_id, result_sha256, result_artifact_id, created_at)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
ON CONFLICT (task_id, owner_id) DO UPDATE SET
|
||||
result_sha256 = excluded.result_sha256,
|
||||
result_artifact_id = excluded.result_artifact_id,
|
||||
created_at = excluded.created_at`,
|
||||
taskID.String(), ownerID.String(), sha256, artifactID.String(), time.Now().UnixNano())
|
||||
return err
|
||||
}
|
||||
|
||||
// CountAgreeing returns how many distinct owners have voted for the given
|
||||
// result hash on this task.
|
||||
func (r *TaskResultRepo) CountAgreeing(ctx context.Context, taskID uuid.UUID, sha256 string) (int, error) {
|
||||
var n int
|
||||
err := conn(ctx, r.db).QueryRowContext(ctx,
|
||||
"SELECT count(DISTINCT owner_id) FROM task_results WHERE task_id = ? AND result_sha256 = ?",
|
||||
taskID.String(), sha256).Scan(&n)
|
||||
return n, err
|
||||
}
|
||||
@@ -0,0 +1,145 @@
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/domain"
|
||||
)
|
||||
|
||||
// UIReadRepo contains bounded, deterministic read queries for the operator UI.
|
||||
type UIReadRepo struct{ db *sql.DB }
|
||||
|
||||
func NewUIReadRepo(db *sql.DB) *UIReadRepo { return &UIReadRepo{db: db} }
|
||||
|
||||
func (r *UIReadRepo) GetJob(ctx context.Context, id uuid.UUID) (*domain.Job, error) {
|
||||
return NewJobRepo(r.db).Get(ctx, id)
|
||||
}
|
||||
|
||||
func (r *UIReadRepo) ListJobs(ctx context.Context, owner *uuid.UUID, limit int) ([]domain.Job, error) {
|
||||
if limit < 1 || limit > 100 {
|
||||
return nil, domain.ErrInvalidInput
|
||||
}
|
||||
query := "SELECT " + jobColumns + " FROM jobs"
|
||||
args := []any{}
|
||||
if owner != nil {
|
||||
query += " WHERE owner_id = ?"
|
||||
args = append(args, owner.String())
|
||||
}
|
||||
query += " ORDER BY created_at DESC, id DESC LIMIT ?"
|
||||
args = append(args, limit)
|
||||
rows, err := conn(ctx, r.db).QueryContext(ctx, query, args...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list jobs: %w", err)
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
jobs := make([]domain.Job, 0)
|
||||
for rows.Next() {
|
||||
job, err := scanJob(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
jobs = append(jobs, *job)
|
||||
}
|
||||
return jobs, rows.Err()
|
||||
}
|
||||
|
||||
func (r *UIReadRepo) ListTasksByJob(ctx context.Context, jobID uuid.UUID) ([]domain.Task, error) {
|
||||
rows, err := conn(ctx, r.db).QueryContext(ctx,
|
||||
"SELECT "+taskColumns+" FROM tasks WHERE job_id = ? ORDER BY chunk_index ASC", jobID.String())
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list tasks: %w", err)
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
tasks := make([]domain.Task, 0)
|
||||
for rows.Next() {
|
||||
task, err := scanTask(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tasks = append(tasks, *task)
|
||||
}
|
||||
return tasks, rows.Err()
|
||||
}
|
||||
|
||||
func (r *UIReadRepo) ListTasksByJobs(ctx context.Context, jobIDs []uuid.UUID) (map[uuid.UUID][]domain.Task, error) {
|
||||
out := make(map[uuid.UUID][]domain.Task, len(jobIDs))
|
||||
if len(jobIDs) == 0 {
|
||||
return out, nil
|
||||
}
|
||||
placeholders := make([]string, 0, len(jobIDs))
|
||||
args := make([]any, 0, len(jobIDs))
|
||||
for _, id := range jobIDs {
|
||||
placeholders = append(placeholders, "?")
|
||||
args = append(args, id.String())
|
||||
}
|
||||
rows, err := conn(ctx, r.db).QueryContext(ctx,
|
||||
"SELECT "+taskColumns+" FROM tasks WHERE job_id IN ("+strings.Join(placeholders, ", ")+") ORDER BY job_id ASC, chunk_index ASC",
|
||||
args...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list tasks by jobs: %w", err)
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
for rows.Next() {
|
||||
task, err := scanTask(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out[task.JobID] = append(out[task.JobID], *task)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func (r *UIReadRepo) ListWorkers(ctx context.Context, limit int) ([]domain.Worker, error) {
|
||||
return r.listWorkers(ctx, "", nil, limit)
|
||||
}
|
||||
|
||||
func (r *UIReadRepo) ListWorkersByOwner(ctx context.Context, owner uuid.UUID, limit int) ([]domain.Worker, error) {
|
||||
return r.listWorkers(ctx, " WHERE owner_id = ?", []any{owner.String()}, limit)
|
||||
}
|
||||
|
||||
func (r *UIReadRepo) listWorkers(ctx context.Context, clause string, args []any, limit int) ([]domain.Worker, error) {
|
||||
if limit < 1 || limit > 100 {
|
||||
return nil, domain.ErrInvalidInput
|
||||
}
|
||||
query := "SELECT " + workerColumns + " FROM workers" + clause +
|
||||
" ORDER BY last_heartbeat_at DESC, id DESC LIMIT ?"
|
||||
fullArgs := append(args, limit)
|
||||
rows, err := conn(ctx, r.db).QueryContext(ctx, query, fullArgs...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list workers: %w", err)
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
workers := make([]domain.Worker, 0)
|
||||
for rows.Next() {
|
||||
worker, err := scanWorker(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
workers = append(workers, *worker)
|
||||
}
|
||||
return workers, rows.Err()
|
||||
}
|
||||
|
||||
func (r *UIReadRepo) ListArtifactsByJob(ctx context.Context, jobID uuid.UUID) ([]domain.Artifact, error) {
|
||||
rows, err := conn(ctx, r.db).QueryContext(ctx,
|
||||
"SELECT "+artifactColumns+" FROM artifacts WHERE job_id = ? ORDER BY created_at ASC, id ASC",
|
||||
jobID.String())
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list artifacts: %w", err)
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
artifacts := make([]domain.Artifact, 0)
|
||||
for rows.Next() {
|
||||
artifact, err := scanArtifact(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
artifacts = append(artifacts, *artifact)
|
||||
}
|
||||
return artifacts, rows.Err()
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/domain"
|
||||
)
|
||||
|
||||
// WorkerRepo implements usecase.WorkerRepository on SQLite.
|
||||
type WorkerRepo struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewWorkerRepo(db *sql.DB) *WorkerRepo {
|
||||
return &WorkerRepo{db: db}
|
||||
}
|
||||
|
||||
const workerColumns = `id, name, capabilities, status, owner_id, trust_level, last_heartbeat_at, created_at, updated_at`
|
||||
|
||||
// scanWorker maps one row onto a domain.Worker.
|
||||
func scanWorker(row interface{ Scan(dest ...any) error }) (*domain.Worker, error) {
|
||||
var (
|
||||
w domain.Worker
|
||||
status string
|
||||
trust string
|
||||
caps string
|
||||
)
|
||||
var (
|
||||
ownerID sql.NullString
|
||||
lastHeartbeat, created, updated sql.NullInt64
|
||||
)
|
||||
if err := row.Scan(
|
||||
&w.ID, &w.Name, &caps, &status, &ownerID, &trust,
|
||||
&lastHeartbeat, &created, &updated,
|
||||
); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
w.LastHeartbeatAt = decodeTime(lastHeartbeat.Int64)
|
||||
w.CreatedAt = decodeTime(created.Int64)
|
||||
w.UpdatedAt = decodeTime(updated.Int64)
|
||||
if err := decodeJSON(caps, &w.Capabilities); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
w.Status = domain.WorkerStatus(status)
|
||||
w.TrustLevel = domain.WorkerTrust(trust)
|
||||
if ownerID.Valid {
|
||||
if id, err := uuid.Parse(ownerID.String); err == nil {
|
||||
w.OwnerID = &id
|
||||
}
|
||||
}
|
||||
return &w, nil
|
||||
}
|
||||
|
||||
func (r *WorkerRepo) Insert(ctx context.Context, w *domain.Worker) error {
|
||||
_, err := conn(ctx, r.db).ExecContext(ctx, `
|
||||
INSERT INTO workers (id, name, capabilities, status, owner_id, trust_level,
|
||||
last_heartbeat_at, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
w.ID.String(), w.Name, encodeJSON(w.Capabilities), string(w.Status),
|
||||
nullableUUID(w.OwnerID), string(w.TrustLevel),
|
||||
encodeTime(w.LastHeartbeatAt), encodeTime(w.CreatedAt), encodeTime(w.UpdatedAt))
|
||||
return err
|
||||
}
|
||||
|
||||
func (r *WorkerRepo) Get(ctx context.Context, id uuid.UUID) (*domain.Worker, error) {
|
||||
row := conn(ctx, r.db).QueryRowContext(ctx,
|
||||
"SELECT "+workerColumns+" FROM workers WHERE id = ?", id.String())
|
||||
worker, err := scanWorker(row)
|
||||
return worker, mapErrNoRows(err, domain.ErrWorkerNotFound)
|
||||
}
|
||||
|
||||
func (r *WorkerRepo) Touch(ctx context.Context, id uuid.UUID, at time.Time) error {
|
||||
_, err := conn(ctx, r.db).ExecContext(ctx,
|
||||
"UPDATE workers SET last_heartbeat_at = ?, status = ?, updated_at = ? WHERE id = ?",
|
||||
encodeTime(at), string(domain.WorkerOnline), encodeTime(at), id.String())
|
||||
return err
|
||||
}
|
||||
|
||||
func (r *WorkerRepo) MarkStaleOffline(ctx context.Context, cutoff time.Time) (int64, error) {
|
||||
res, err := conn(ctx, r.db).ExecContext(ctx,
|
||||
"UPDATE workers SET status = ?, updated_at = ? WHERE last_heartbeat_at < ? AND status <> ?",
|
||||
string(domain.WorkerOffline), encodeTime(cutoff), encodeTime(cutoff),
|
||||
string(domain.WorkerOffline))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return res.RowsAffected()
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/golang-jwt/jwt/v5"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
)
|
||||
|
||||
// Claims is the payload of a signed token. Subject (from RegisteredClaims) is
|
||||
// the user id — it becomes the coordinator's jobs.owner_id; Role drives
|
||||
// authorization; Verified tells the coordinator whether this user's workers are
|
||||
// trusted (results accepted without quorum). Both services verify this token
|
||||
// locally with the shared HS256 secret, so no runtime call back to the
|
||||
// userservice is ever needed.
|
||||
type Claims struct {
|
||||
Role domain.Role `json:"role"`
|
||||
Verified bool `json:"verified"`
|
||||
jwt.RegisteredClaims
|
||||
}
|
||||
|
||||
// Issuer signs and verifies tokens with a shared HS256 secret.
|
||||
type Issuer struct {
|
||||
secret []byte
|
||||
ttl time.Duration
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
// NewIssuer builds an Issuer. now defaults to time.Now when nil; tests inject a
|
||||
// fixed clock to make expiry deterministic.
|
||||
func NewIssuer(secret string, ttl time.Duration, now func() time.Time) Issuer {
|
||||
if now == nil {
|
||||
now = time.Now
|
||||
}
|
||||
return Issuer{secret: []byte(secret), ttl: ttl, now: now}
|
||||
}
|
||||
|
||||
// Issue returns a signed token for the user, valid for the configured TTL. It
|
||||
// takes the whole user so every trust-bearing field (role, verified) travels in
|
||||
// the token, keeping the two services from needing a runtime lookup.
|
||||
func (i Issuer) Issue(u *domain.User) (string, error) {
|
||||
now := i.now()
|
||||
claims := Claims{
|
||||
Role: u.Role,
|
||||
Verified: u.Verified,
|
||||
RegisteredClaims: jwt.RegisteredClaims{
|
||||
Subject: u.ID.String(),
|
||||
IssuedAt: jwt.NewNumericDate(now),
|
||||
ExpiresAt: jwt.NewNumericDate(now.Add(i.ttl)),
|
||||
},
|
||||
}
|
||||
return jwt.NewWithClaims(jwt.SigningMethodHS256, claims).SignedString(i.secret)
|
||||
}
|
||||
|
||||
// Verify checks the signature and expiry and returns the claims. It pins the
|
||||
// algorithm to HMAC, rejecting a token that asks for "none" or an RS256 public
|
||||
// key — the classic algorithm-substitution attack against naive verifiers.
|
||||
func (i Issuer) Verify(token string) (*Claims, error) {
|
||||
var claims Claims
|
||||
_, err := jwt.ParseWithClaims(token, &claims, func(t *jwt.Token) (any, error) {
|
||||
if _, ok := t.Method.(*jwt.SigningMethodHMAC); !ok {
|
||||
return nil, fmt.Errorf("unexpected signing method: %v", t.Header["alg"])
|
||||
}
|
||||
return i.secret, nil
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &claims, nil
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/golang-jwt/jwt/v5"
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
)
|
||||
|
||||
const testSecret = "test-secret-at-least-32-bytes-long!!"
|
||||
|
||||
func TestIssueVerifyRoundTrip(t *testing.T) {
|
||||
iss := NewIssuer(testSecret, time.Hour, nil)
|
||||
id := uuid.New()
|
||||
|
||||
token, err := iss.Issue(&domain.User{ID: id, Role: domain.RoleAdmin, Verified: true})
|
||||
if err != nil {
|
||||
t.Fatalf("issue: %v", err)
|
||||
}
|
||||
|
||||
claims, err := iss.Verify(token)
|
||||
if err != nil {
|
||||
t.Fatalf("verify: %v", err)
|
||||
}
|
||||
if claims.Subject != id.String() {
|
||||
t.Errorf("sub = %q, want %q", claims.Subject, id.String())
|
||||
}
|
||||
if claims.Role != domain.RoleAdmin {
|
||||
t.Errorf("role = %q, want admin", claims.Role)
|
||||
}
|
||||
if !claims.Verified {
|
||||
t.Error("verified claim not carried in token")
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyRejectsExpired(t *testing.T) {
|
||||
// Negative TTL: the token is already expired when issued.
|
||||
iss := NewIssuer(testSecret, -time.Minute, nil)
|
||||
token, _ := iss.Issue(&domain.User{ID: uuid.New(), Role: domain.RoleUser})
|
||||
|
||||
if _, err := iss.Verify(token); err == nil {
|
||||
t.Error("expired token accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyRejectsWrongSecret(t *testing.T) {
|
||||
token, _ := NewIssuer(testSecret, time.Hour, nil).Issue(&domain.User{ID: uuid.New(), Role: domain.RoleUser})
|
||||
|
||||
other := NewIssuer("another-secret-also-32-bytes-long!!!", time.Hour, nil)
|
||||
if _, err := other.Verify(token); err == nil {
|
||||
t.Error("token verified under the wrong secret")
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyRejectsNoneAlgorithm(t *testing.T) {
|
||||
// Forge a token signed with "none" — the classic algorithm-substitution
|
||||
// attack. A verifier that trusts the header's alg would accept it.
|
||||
tok := jwt.NewWithClaims(jwt.SigningMethodNone, Claims{
|
||||
Role: domain.RoleAdmin,
|
||||
RegisteredClaims: jwt.RegisteredClaims{
|
||||
Subject: uuid.New().String(),
|
||||
ExpiresAt: jwt.NewNumericDate(time.Now().Add(time.Hour)),
|
||||
},
|
||||
})
|
||||
raw, err := tok.SignedString(jwt.UnsafeAllowNoneSignatureType)
|
||||
if err != nil {
|
||||
t.Fatalf("sign none: %v", err)
|
||||
}
|
||||
|
||||
iss := NewIssuer(testSecret, time.Hour, nil)
|
||||
if _, err := iss.Verify(raw); err == nil {
|
||||
t.Error("none-signed token accepted")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
// Package auth holds the cryptographic adapters — password hashing and JWT
|
||||
// signing/verification. They implement use-case ports and keep bcrypt and the
|
||||
// JWT library out of the domain and use-case layers.
|
||||
package auth
|
||||
|
||||
import "golang.org/x/crypto/bcrypt"
|
||||
|
||||
// Hasher turns plaintext passwords into storable hashes and checks them back.
|
||||
type Hasher struct {
|
||||
cost int
|
||||
}
|
||||
|
||||
// NewHasher builds a Hasher. A cost of 0 uses bcrypt's default work factor.
|
||||
func NewHasher(cost int) Hasher {
|
||||
if cost == 0 {
|
||||
cost = bcrypt.DefaultCost
|
||||
}
|
||||
return Hasher{cost: cost}
|
||||
}
|
||||
|
||||
// Hash returns the bcrypt hash of password. The salt and the cost are embedded
|
||||
// in the returned string, so nothing else needs to be stored alongside it.
|
||||
//
|
||||
// bcrypt silently ignores input past 72 bytes; the use case rejects longer
|
||||
// passwords before reaching here so a truncated tail never becomes a security
|
||||
// surprise.
|
||||
func (h Hasher) Hash(password string) (string, error) {
|
||||
b, err := bcrypt.GenerateFromPassword([]byte(password), h.cost)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return string(b), nil
|
||||
}
|
||||
|
||||
// Compare reports whether password matches the stored hash. It returns a
|
||||
// non-nil error (bcrypt.ErrMismatchedHashAndPassword) on any mismatch, which
|
||||
// the caller collapses into a generic authentication failure.
|
||||
func (h Hasher) Compare(hash, password string) error {
|
||||
return bcrypt.CompareHashAndPassword([]byte(hash), []byte(password))
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
package auth
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestHashAndCompare(t *testing.T) {
|
||||
h := NewHasher(0) // default cost
|
||||
|
||||
hash, err := h.Hash("correct horse battery staple")
|
||||
if err != nil {
|
||||
t.Fatalf("hash: %v", err)
|
||||
}
|
||||
if hash == "correct horse battery staple" {
|
||||
t.Fatal("hash must not equal the plaintext")
|
||||
}
|
||||
if err := h.Compare(hash, "correct horse battery staple"); err != nil {
|
||||
t.Errorf("correct password rejected: %v", err)
|
||||
}
|
||||
if err := h.Compare(hash, "wrong password"); err == nil {
|
||||
t.Error("wrong password accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestHashSaltsEachTime(t *testing.T) {
|
||||
h := NewHasher(0)
|
||||
a, _ := h.Hash("same")
|
||||
b, _ := h.Hash("same")
|
||||
if a == b {
|
||||
t.Error("two hashes of the same password must differ (random salt)")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
package domain
|
||||
|
||||
import "errors"
|
||||
|
||||
// Domain validation errors. They describe an entity that cannot be constructed,
|
||||
// independent of storage or transport, and the HTTP layer maps them to 400.
|
||||
var (
|
||||
ErrEmptyEmail = errors.New("email is required")
|
||||
ErrInvalidEmail = errors.New("email is not a valid address")
|
||||
ErrEmptyPasswordHash = errors.New("password hash is required")
|
||||
|
||||
ErrWorkerKeyNameTooLong = errors.New("worker key name is too long")
|
||||
)
|
||||
@@ -0,0 +1,84 @@
|
||||
package domain
|
||||
|
||||
import (
|
||||
"net/mail"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
type Role string
|
||||
|
||||
const (
|
||||
RoleAdmin Role = "admin"
|
||||
RoleUser Role = "user"
|
||||
)
|
||||
|
||||
func (r Role) Valid() bool {
|
||||
switch r {
|
||||
case RoleAdmin, RoleUser:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
type User struct {
|
||||
ID uuid.UUID
|
||||
Email string
|
||||
PasswordHash string
|
||||
Role Role
|
||||
// Verified marks a trusted contributor whose workers' results the
|
||||
// coordinator accepts without quorum. Distinct from Role; granted by an
|
||||
// admin, defaults to false.
|
||||
Verified bool
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
// NewUser builds a freshly registered account. It normalises the email and
|
||||
// enforces every invariant a row must satisfy, so an invalid User cannot be
|
||||
// constructed. The caller supplies the already-hashed password — hashing is an
|
||||
// adapter's job, not the domain's.
|
||||
//
|
||||
// Registration always produces a plain user; promotion to admin is a manual,
|
||||
// out-of-band operation, never something a request can trigger.
|
||||
func NewUser(email, passwordHash string, now time.Time) (*User, error) {
|
||||
email = NormalizeEmail(email)
|
||||
if err := validateEmail(email); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if passwordHash == "" {
|
||||
return nil, ErrEmptyPasswordHash
|
||||
}
|
||||
return &User{
|
||||
ID: uuid.New(),
|
||||
Email: email,
|
||||
PasswordHash: passwordHash,
|
||||
Role: RoleUser,
|
||||
CreatedAt: now,
|
||||
UpdatedAt: now,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// NormalizeEmail lower-cases and trims an address so that "Bob@X.com " and
|
||||
// "bob@x.com" resolve to the same account. Every lookup and every insert must
|
||||
// pass through here, matching the ck_users_email_lower database constraint.
|
||||
func NormalizeEmail(email string) string {
|
||||
return strings.ToLower(strings.TrimSpace(email))
|
||||
}
|
||||
|
||||
func validateEmail(email string) error {
|
||||
if email == "" {
|
||||
return ErrEmptyEmail
|
||||
}
|
||||
// A minimal shape check, not full RFC 5322: real deliverability is proven by
|
||||
// sending mail, not by a regex. mail.ParseAddress also accepts the
|
||||
// "Name <addr>" form, so we insist the parsed address equals the input.
|
||||
addr, err := mail.ParseAddress(email)
|
||||
if err != nil || addr.Address != email {
|
||||
return ErrInvalidEmail
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
package domain
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
func TestNewUserNormalisesAndValidates(t *testing.T) {
|
||||
now := time.Date(2026, 7, 26, 12, 0, 0, 0, time.UTC)
|
||||
|
||||
u, err := NewUser(" Bob@Example.COM ", "hashed", now)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if u.Email != "bob@example.com" {
|
||||
t.Errorf("email not normalised: got %q", u.Email)
|
||||
}
|
||||
if u.Role != RoleUser {
|
||||
t.Errorf("new user must default to RoleUser, got %q", u.Role)
|
||||
}
|
||||
if u.ID == uuid.Nil {
|
||||
t.Error("new user must get an id")
|
||||
}
|
||||
if !u.CreatedAt.Equal(now) || !u.UpdatedAt.Equal(now) {
|
||||
t.Error("timestamps not set from clock")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewUserRejectsBadInput(t *testing.T) {
|
||||
now := time.Now()
|
||||
cases := []struct {
|
||||
name string
|
||||
email string
|
||||
hash string
|
||||
wantErr error
|
||||
}{
|
||||
{"empty email", "", "h", ErrEmptyEmail},
|
||||
{"no domain", "bob", "h", ErrInvalidEmail},
|
||||
{"name form", "Bob <bob@x.com>", "h", ErrInvalidEmail},
|
||||
{"empty hash", "bob@x.com", "", ErrEmptyPasswordHash},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
_, err := NewUser(tc.email, tc.hash, now)
|
||||
if !errors.Is(err, tc.wantErr) {
|
||||
t.Errorf("got %v, want %v", err, tc.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRoleValid(t *testing.T) {
|
||||
if !RoleUser.Valid() || !RoleAdmin.Valid() {
|
||||
t.Error("user and admin must be valid")
|
||||
}
|
||||
if Role("root").Valid() {
|
||||
t.Error("unknown role must be invalid")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
package domain
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/hex"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
const (
|
||||
// workerKeyLabel makes a key self-describing when it turns up in a log or an
|
||||
// env var, and lets a client sanity-check the shape before exchanging it.
|
||||
workerKeyLabel = "scimesh_wk_live_"
|
||||
// workerKeyRandomBytes is the entropy behind the secret. 24 bytes (192 bits)
|
||||
// is far beyond guessable, which is why the stored hash needs no salt.
|
||||
workerKeyRandomBytes = 24
|
||||
// workerKeyPrefixChars is how much of the random tail we keep, alongside the
|
||||
// label, as the non-secret identifier shown in the UI.
|
||||
workerKeyPrefixChars = 8
|
||||
// workerKeyNameMax caps the user-supplied label.
|
||||
workerKeyNameMax = 100
|
||||
// workerKeyDefaultName is used when the caller supplies no label.
|
||||
workerKeyDefaultName = "my machine"
|
||||
)
|
||||
|
||||
// WorkerKey is a long-lived, per-user credential for running a worker. The
|
||||
// secret itself is never stored — only TokenHash — so the plaintext returned by
|
||||
// NewWorkerKey is the one and only chance to show it to the user.
|
||||
type WorkerKey struct {
|
||||
ID uuid.UUID
|
||||
UserID uuid.UUID
|
||||
Name string
|
||||
TokenHash string
|
||||
Prefix string
|
||||
CreatedAt time.Time
|
||||
LastUsedAt *time.Time
|
||||
RevokedAt *time.Time
|
||||
}
|
||||
|
||||
// NewWorkerKey mints a key for a user and returns both the entity (carrying only
|
||||
// the hash) and the one-time plaintext to hand back to the caller. The label is
|
||||
// trimmed and defaulted; an over-long one is rejected.
|
||||
func NewWorkerKey(userID uuid.UUID, name string, now time.Time) (*WorkerKey, string, error) {
|
||||
name = strings.TrimSpace(name)
|
||||
if name == "" {
|
||||
name = workerKeyDefaultName
|
||||
}
|
||||
if len(name) > workerKeyNameMax {
|
||||
return nil, "", ErrWorkerKeyNameTooLong
|
||||
}
|
||||
|
||||
b := make([]byte, workerKeyRandomBytes)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
// URL-safe, unpadded: the key rides in env vars and shell commands, so it
|
||||
// must contain no '=', '+', or '/' that a shell might mangle.
|
||||
raw := workerKeyLabel + base64.RawURLEncoding.EncodeToString(b)
|
||||
|
||||
key := &WorkerKey{
|
||||
ID: uuid.New(),
|
||||
UserID: userID,
|
||||
Name: name,
|
||||
TokenHash: HashWorkerKey(raw),
|
||||
Prefix: raw[:len(workerKeyLabel)+workerKeyPrefixChars],
|
||||
CreatedAt: now,
|
||||
}
|
||||
return key, raw, nil
|
||||
}
|
||||
|
||||
// HashWorkerKey returns the hex SHA-256 of a presented key. Exchange hashes the
|
||||
// incoming key the same way and looks the row up by it, so the plaintext never
|
||||
// has to be compared directly.
|
||||
func HashWorkerKey(raw string) string {
|
||||
sum := sha256.Sum256([]byte(raw))
|
||||
return hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
// Revoked reports whether the key has been retired and must no longer exchange.
|
||||
func (k *WorkerKey) Revoked() bool { return k.RevokedAt != nil }
|
||||
@@ -0,0 +1,62 @@
|
||||
package domain_test
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
)
|
||||
|
||||
func TestNewWorkerKeyShape(t *testing.T) {
|
||||
owner := uuid.New()
|
||||
now := time.Date(2026, 7, 26, 0, 0, 0, 0, time.UTC)
|
||||
|
||||
key, raw, err := domain.NewWorkerKey(owner, "home-desktop", now)
|
||||
if err != nil {
|
||||
t.Fatalf("NewWorkerKey: %v", err)
|
||||
}
|
||||
if !strings.HasPrefix(raw, "scimesh_wk_live_") {
|
||||
t.Errorf("raw key has no recognisable label: %q", raw)
|
||||
}
|
||||
if key.TokenHash != domain.HashWorkerKey(raw) {
|
||||
t.Error("stored hash does not match the plaintext")
|
||||
}
|
||||
if key.TokenHash == raw || strings.Contains(key.TokenHash, raw) {
|
||||
t.Error("plaintext leaked into the stored hash")
|
||||
}
|
||||
if !strings.HasPrefix(raw, key.Prefix) {
|
||||
t.Errorf("prefix %q is not a leading slice of the key", key.Prefix)
|
||||
}
|
||||
if key.UserID != owner || key.CreatedAt != now || key.Revoked() {
|
||||
t.Errorf("unexpected key metadata: %+v", key)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewWorkerKeyDefaultsBlankName(t *testing.T) {
|
||||
key, _, err := domain.NewWorkerKey(uuid.New(), " ", time.Now())
|
||||
if err != nil {
|
||||
t.Fatalf("NewWorkerKey: %v", err)
|
||||
}
|
||||
if key.Name == "" {
|
||||
t.Error("blank name was not defaulted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewWorkerKeyRejectsLongName(t *testing.T) {
|
||||
_, _, err := domain.NewWorkerKey(uuid.New(), strings.Repeat("x", 101), time.Now())
|
||||
if !errors.Is(err, domain.ErrWorkerKeyNameTooLong) {
|
||||
t.Errorf("got %v, want ErrWorkerKeyNameTooLong", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewWorkerKeyUniquePerCall(t *testing.T) {
|
||||
a, rawA, _ := domain.NewWorkerKey(uuid.New(), "a", time.Now())
|
||||
b, rawB, _ := domain.NewWorkerKey(uuid.New(), "b", time.Now())
|
||||
if rawA == rawB || a.TokenHash == b.TokenHash || a.ID == b.ID {
|
||||
t.Error("two keys collided; generation is not random")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
// Package userservice is the SciMesh authentication service, embedded into the
|
||||
// coordinator binary for single-binary deployments. The packages here are the
|
||||
// same code the standalone `users/` service runs, with its PostgreSQL storage
|
||||
// replaced by an embedded SQLite backend. The coordinator's HTTP layer talks
|
||||
// to it through the usual USERSERVICE_URL proxy, so no proxy code changes.
|
||||
package userservice
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/auth"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/storage/sqlite"
|
||||
usershttp "github.com/emil28092005/SciMesh/coordinator/internal/userservice/transport/http"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/usecase"
|
||||
)
|
||||
|
||||
// Config wires the embedded userservice.
|
||||
type Config struct {
|
||||
// DBPath is the sqlite database file (for example <data-dir>/users.db).
|
||||
DBPath string
|
||||
// JWTSecret must equal the coordinator's JWT_SECRET so tokens verify.
|
||||
JWTSecret string
|
||||
// AdminEmail/AdminPassword bootstrap the first admin on first run.
|
||||
AdminEmail string
|
||||
AdminPassword string
|
||||
// Log receives the service's log lines.
|
||||
Log *slog.Logger
|
||||
}
|
||||
|
||||
// Serve runs the embedded userservice until ctx is cancelled. It listens only
|
||||
// on the loopback interface; the coordinator proxies to it internally.
|
||||
func Serve(ctx context.Context, cfg Config) (string, func() error, error) {
|
||||
db, err := sqlite.Open(cfg.DBPath)
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
if err := sqlite.Migrate(ctx, db, cfg.Log); err != nil {
|
||||
_ = db.Close()
|
||||
return "", nil, err
|
||||
}
|
||||
|
||||
clock := NewClock()
|
||||
users := sqlite.NewUserRepo(db)
|
||||
workerKeys := sqlite.NewWorkerKeyRepo(db)
|
||||
hasher := auth.NewHasher(0)
|
||||
issuer := auth.NewIssuer(cfg.JWTSecret, 24*time.Hour, clock.Now)
|
||||
|
||||
uc := usershttp.UseCases{
|
||||
Register: usecase.NewRegister(users, hasher, clock),
|
||||
Login: usecase.NewLogin(users, hasher, issuer),
|
||||
SetVerified: usecase.NewSetVerified(users),
|
||||
SetRole: usecase.NewSetRole(users),
|
||||
CreateWorkerKey: usecase.NewCreateWorkerKey(workerKeys, clock),
|
||||
ListWorkerKeys: usecase.NewListWorkerKeys(workerKeys),
|
||||
RevokeWorkerKey: usecase.NewRevokeWorkerKey(workerKeys),
|
||||
ExchangeWorkerKey: usecase.NewExchangeWorkerKey(workerKeys, users, issuer, 24*time.Hour),
|
||||
Users: users,
|
||||
}
|
||||
|
||||
if cfg.AdminEmail != "" && cfg.AdminPassword != "" {
|
||||
created, err := usecase.NewBootstrapAdmin(users, hasher, clock).
|
||||
Execute(ctx, cfg.AdminEmail, cfg.AdminPassword)
|
||||
if err != nil {
|
||||
_ = db.Close()
|
||||
return "", nil, fmt.Errorf("bootstrap admin: %w", err)
|
||||
}
|
||||
if created {
|
||||
cfg.Log.Info("embedded userservice created the admin account", "email", cfg.AdminEmail)
|
||||
}
|
||||
}
|
||||
|
||||
handler := usershttp.NewServer(cfg.Log, uc, issuer)
|
||||
handler = http.TimeoutHandler(handler, 15*time.Second, `{"error":"request timeout"}`)
|
||||
|
||||
// A fixed loopback port lets the coordinator's proxy reuse its config
|
||||
// unchanged; collisions are unlikely (no other process binds 18081 on a
|
||||
// fresh machine) and fail loudly.
|
||||
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", "127.0.0.1:18081")
|
||||
if err != nil {
|
||||
_ = db.Close()
|
||||
return "", nil, fmt.Errorf("listen for embedded userservice: %w", err)
|
||||
}
|
||||
server := &http.Server{
|
||||
Handler: handler,
|
||||
ReadHeaderTimeout: 10 * time.Second,
|
||||
}
|
||||
go func() {
|
||||
<-ctx.Done()
|
||||
shutdownCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 5*time.Second)
|
||||
defer cancel()
|
||||
_ = server.Shutdown(shutdownCtx)
|
||||
}()
|
||||
go func() {
|
||||
if err := server.Serve(listener); err != nil && err != http.ErrServerClosed {
|
||||
cfg.Log.Error("embedded userservice stopped", "err", err)
|
||||
}
|
||||
}()
|
||||
return listener.Addr().String(), func() error { return db.Close() }, nil
|
||||
}
|
||||
|
||||
// NewClock returns the userservice's wall clock.
|
||||
func NewClock() *Clock { return &Clock{} }
|
||||
|
||||
// Clock implements the userservice usecase clock.
|
||||
type Clock struct{}
|
||||
|
||||
// Now returns the current UTC time.
|
||||
func (c *Clock) Now() time.Time { return time.Now().UTC() }
|
||||
@@ -0,0 +1,90 @@
|
||||
// Package memstore provides in-memory implementations of the usecase ports for
|
||||
// fast, deterministic tests that need no database.
|
||||
package memstore
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/usecase"
|
||||
)
|
||||
|
||||
// UserRepo is an in-memory usecase.UserRepository. It stores copies, so callers
|
||||
// mutating a returned user cannot corrupt the store.
|
||||
type UserRepo struct {
|
||||
mu sync.Mutex
|
||||
byID map[uuid.UUID]domain.User
|
||||
byEmail map[string]uuid.UUID
|
||||
}
|
||||
|
||||
func NewUserRepo() *UserRepo {
|
||||
return &UserRepo{
|
||||
byID: make(map[uuid.UUID]domain.User),
|
||||
byEmail: make(map[string]uuid.UUID),
|
||||
}
|
||||
}
|
||||
|
||||
func (r *UserRepo) Insert(_ context.Context, u *domain.User) error {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if _, ok := r.byEmail[u.Email]; ok {
|
||||
return usecase.ErrEmailExists
|
||||
}
|
||||
r.byID[u.ID] = *u
|
||||
r.byEmail[u.Email] = u.ID
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *UserRepo) GetByEmail(_ context.Context, email string) (*domain.User, error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
id, ok := r.byEmail[email]
|
||||
if !ok {
|
||||
return nil, usecase.ErrUserNotFound
|
||||
}
|
||||
u := r.byID[id]
|
||||
return &u, nil
|
||||
}
|
||||
|
||||
func (r *UserRepo) GetByID(_ context.Context, id uuid.UUID) (*domain.User, error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
u, ok := r.byID[id]
|
||||
if !ok {
|
||||
return nil, usecase.ErrUserNotFound
|
||||
}
|
||||
return &u, nil
|
||||
}
|
||||
|
||||
func (r *UserRepo) SetVerified(_ context.Context, id uuid.UUID, verified bool) error {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
u, ok := r.byID[id]
|
||||
if !ok {
|
||||
return usecase.ErrUserNotFound
|
||||
}
|
||||
u.Verified = verified
|
||||
r.byID[id] = u
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *UserRepo) SetRole(_ context.Context, id uuid.UUID, role domain.Role) error {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
u, ok := r.byID[id]
|
||||
if !ok {
|
||||
return usecase.ErrUserNotFound
|
||||
}
|
||||
u.Role = role
|
||||
r.byID[id] = u
|
||||
return nil
|
||||
}
|
||||
|
||||
// Clock is a fixed usecase.Clock for deterministic tests.
|
||||
type Clock struct{ T time.Time }
|
||||
|
||||
func (c Clock) Now() time.Time { return c.T }
|
||||
@@ -0,0 +1,81 @@
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"embed"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
//go:embed migrations/*.sql
|
||||
var migrationFiles embed.FS
|
||||
|
||||
var migrationNamePattern = regexp.MustCompile(`^([0-9]+)_[a-z0-9_]+\.sql$`)
|
||||
|
||||
// Migrate applies every embedded migration above the PRAGMA user_version
|
||||
// watermark, each inside its own transaction.
|
||||
func Migrate(ctx context.Context, db *sql.DB, log *slog.Logger) error {
|
||||
entries, err := migrationFiles.ReadDir("migrations")
|
||||
if err != nil {
|
||||
return fmt.Errorf("read embedded migrations: %w", err)
|
||||
}
|
||||
type file struct {
|
||||
version int
|
||||
name string
|
||||
}
|
||||
var files []file
|
||||
byVersion := map[int]string{}
|
||||
for _, entry := range entries {
|
||||
match := migrationNamePattern.FindStringSubmatch(entry.Name())
|
||||
if match == nil {
|
||||
continue
|
||||
}
|
||||
version, err := strconv.Atoi(match[1])
|
||||
if err != nil {
|
||||
return fmt.Errorf("migration %q has an invalid version: %w", entry.Name(), err)
|
||||
}
|
||||
body, err := migrationFiles.ReadFile("migrations/" + entry.Name())
|
||||
if err != nil {
|
||||
return fmt.Errorf("read migration %q: %w", entry.Name(), err)
|
||||
}
|
||||
byVersion[version] = string(body)
|
||||
files = append(files, file{version: version, name: entry.Name()})
|
||||
}
|
||||
if len(files) == 0 {
|
||||
return fmt.Errorf("no userservice migrations are embedded")
|
||||
}
|
||||
sort.Slice(files, func(i, j int) bool { return files[i].version < files[j].version })
|
||||
|
||||
var applied int
|
||||
if err := db.QueryRowContext(ctx, "PRAGMA user_version").Scan(&applied); err != nil {
|
||||
return fmt.Errorf("read schema version: %w", err)
|
||||
}
|
||||
for _, item := range files {
|
||||
if item.version <= applied {
|
||||
continue
|
||||
}
|
||||
if log != nil {
|
||||
log.Info("applying userservice migration", "version", item.version, "file", item.name)
|
||||
}
|
||||
tx, err := db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx, byVersion[item.version]); err != nil {
|
||||
_ = tx.Rollback()
|
||||
return fmt.Errorf("apply migration %s: %w", item.name, err)
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx, fmt.Sprintf("PRAGMA user_version = %d", item.version)); err != nil {
|
||||
_ = tx.Rollback()
|
||||
return fmt.Errorf("advance schema version after %s: %w", item.name, err)
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return fmt.Errorf("commit migration %s: %w", item.name, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
-- 0001: userservice schema. Users and long-lived worker keys, in the same
|
||||
-- style as the coordinator's sqlite schema: TEXT ids, INTEGER timestamps.
|
||||
CREATE TABLE IF NOT EXISTS users (
|
||||
id TEXT PRIMARY KEY,
|
||||
email TEXT NOT NULL UNIQUE,
|
||||
password_hash TEXT NOT NULL,
|
||||
role TEXT NOT NULL CHECK (role IN ('admin','user')),
|
||||
verified INTEGER NOT NULL DEFAULT 0,
|
||||
created_at INTEGER NOT NULL,
|
||||
updated_at INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS worker_keys (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
name TEXT NOT NULL,
|
||||
token_hash TEXT NOT NULL UNIQUE,
|
||||
prefix TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
last_used_at INTEGER,
|
||||
revoked_at INTEGER
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_worker_keys_user ON worker_keys (user_id);
|
||||
@@ -0,0 +1,256 @@
|
||||
// Package sqlite implements the userservice repository ports on an embedded
|
||||
// SQLite database, mirroring the coordinator's single-binary storage choice.
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
_ "modernc.org/sqlite"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/usecase"
|
||||
)
|
||||
|
||||
const userColumns = `id, email, password_hash, role, verified, created_at, updated_at`
|
||||
|
||||
// scanUser maps one row onto a domain.User.
|
||||
func scanUser(row interface{ Scan(dest ...any) error }) (*domain.User, error) {
|
||||
var (
|
||||
u domain.User
|
||||
role string
|
||||
verified int64
|
||||
)
|
||||
var created, updated sql.NullInt64
|
||||
if err := row.Scan(&u.ID, &u.Email, &u.PasswordHash, &role, &verified, &created, &updated); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
u.Role = domain.Role(role)
|
||||
u.Verified = verified != 0
|
||||
u.CreatedAt = decodeTime(created.Int64)
|
||||
u.UpdatedAt = decodeTime(updated.Int64)
|
||||
return &u, nil
|
||||
}
|
||||
|
||||
// UserRepo implements usecase.UserRepository on SQLite.
|
||||
type UserRepo struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewUserRepo(db *sql.DB) *UserRepo {
|
||||
return &UserRepo{db: db}
|
||||
}
|
||||
|
||||
func (r *UserRepo) Insert(ctx context.Context, u *domain.User) error {
|
||||
_, err := r.db.ExecContext(ctx, `
|
||||
INSERT INTO users (id, email, password_hash, role, verified, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)`,
|
||||
u.ID.String(), u.Email, u.PasswordHash, string(u.Role), boolInt(u.Verified),
|
||||
u.CreatedAt.UnixNano(), u.UpdatedAt.UnixNano())
|
||||
if err != nil && isUnique(err) {
|
||||
return usecase.ErrEmailExists
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (r *UserRepo) GetByEmail(ctx context.Context, email string) (*domain.User, error) {
|
||||
return r.getBy(ctx, "email = ?", email)
|
||||
}
|
||||
|
||||
func (r *UserRepo) GetByID(ctx context.Context, id uuid.UUID) (*domain.User, error) {
|
||||
return r.getBy(ctx, "id = ?", id.String())
|
||||
}
|
||||
|
||||
func (r *UserRepo) getBy(ctx context.Context, clause string, arg any) (*domain.User, error) {
|
||||
// #nosec G202 -- clause is an internal constant, never user input.
|
||||
row := r.db.QueryRowContext(ctx, "SELECT "+userColumns+" FROM users WHERE "+clause, arg)
|
||||
user, err := scanUser(row)
|
||||
return user, mapErrNoRows(err, usecase.ErrUserNotFound)
|
||||
}
|
||||
|
||||
func (r *UserRepo) SetVerified(ctx context.Context, id uuid.UUID, verified bool) error {
|
||||
res, err := r.db.ExecContext(ctx,
|
||||
"UPDATE users SET verified = ?, updated_at = ? WHERE id = ?",
|
||||
boolInt(verified), time.Now().UnixNano(), id.String())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return rowsAffectedOrNotFound(res, usecase.ErrUserNotFound)
|
||||
}
|
||||
|
||||
func (r *UserRepo) SetRole(ctx context.Context, id uuid.UUID, role domain.Role) error {
|
||||
res, err := r.db.ExecContext(ctx,
|
||||
"UPDATE users SET role = ?, updated_at = ? WHERE id = ?",
|
||||
string(role), time.Now().UnixNano(), id.String())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return rowsAffectedOrNotFound(res, usecase.ErrUserNotFound)
|
||||
}
|
||||
|
||||
const workerKeyColumns = `id, user_id, name, token_hash, prefix, created_at, last_used_at, revoked_at`
|
||||
|
||||
func scanWorkerKey(row interface{ Scan(dest ...any) error }) (*domain.WorkerKey, error) {
|
||||
var (
|
||||
k domain.WorkerKey
|
||||
lastUsed sql.NullInt64
|
||||
revoked sql.NullInt64
|
||||
created sql.NullInt64
|
||||
)
|
||||
if err := row.Scan(&k.ID, &k.UserID, &k.Name, &k.TokenHash, &k.Prefix, &created, &lastUsed, &revoked); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
k.CreatedAt = decodeTime(created.Int64)
|
||||
if lastUsed.Valid {
|
||||
value := decodeTime(lastUsed.Int64)
|
||||
k.LastUsedAt = &value
|
||||
}
|
||||
if revoked.Valid {
|
||||
value := decodeTime(revoked.Int64)
|
||||
k.RevokedAt = &value
|
||||
}
|
||||
return &k, nil
|
||||
}
|
||||
|
||||
// WorkerKeyRepo implements usecase.WorkerKeyRepository on SQLite.
|
||||
type WorkerKeyRepo struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewWorkerKeyRepo(db *sql.DB) *WorkerKeyRepo {
|
||||
return &WorkerKeyRepo{db: db}
|
||||
}
|
||||
|
||||
func (r *WorkerKeyRepo) Insert(ctx context.Context, k *domain.WorkerKey) error {
|
||||
_, err := r.db.ExecContext(ctx, `
|
||||
INSERT INTO worker_keys (id, user_id, name, token_hash, prefix, created_at, last_used_at, revoked_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
k.ID.String(), k.UserID.String(), k.Name, k.TokenHash, k.Prefix,
|
||||
k.CreatedAt.UnixNano(), nullableTime(k.LastUsedAt), nullableTime(k.RevokedAt))
|
||||
return err
|
||||
}
|
||||
|
||||
func (r *WorkerKeyRepo) ListByUser(ctx context.Context, userID uuid.UUID) ([]*domain.WorkerKey, error) {
|
||||
rows, err := r.db.QueryContext(ctx,
|
||||
"SELECT "+workerKeyColumns+" FROM worker_keys WHERE user_id = ? AND revoked_at IS NULL ORDER BY created_at DESC",
|
||||
userID.String())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
var keys []*domain.WorkerKey
|
||||
for rows.Next() {
|
||||
key, err := scanWorkerKey(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
keys = append(keys, key)
|
||||
}
|
||||
return keys, rows.Err()
|
||||
}
|
||||
|
||||
func (r *WorkerKeyRepo) GetActiveByHash(ctx context.Context, tokenHash string) (*domain.WorkerKey, error) {
|
||||
row := r.db.QueryRowContext(ctx,
|
||||
"SELECT "+workerKeyColumns+" FROM worker_keys WHERE token_hash = ? AND revoked_at IS NULL",
|
||||
tokenHash)
|
||||
key, err := scanWorkerKey(row)
|
||||
return key, mapErrNoRows(err, usecase.ErrWorkerKeyNotFound)
|
||||
}
|
||||
|
||||
func (r *WorkerKeyRepo) Revoke(ctx context.Context, id, userID uuid.UUID) error {
|
||||
res, err := r.db.ExecContext(ctx,
|
||||
"UPDATE worker_keys SET revoked_at = ? WHERE id = ? AND user_id = ? AND revoked_at IS NULL",
|
||||
time.Now().UnixNano(), id.String(), userID.String())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return rowsAffectedOrNotFound(res, usecase.ErrWorkerKeyNotFound)
|
||||
}
|
||||
|
||||
func (r *WorkerKeyRepo) TouchLastUsed(ctx context.Context, id uuid.UUID) error {
|
||||
_, err := r.db.ExecContext(ctx,
|
||||
"UPDATE worker_keys SET last_used_at = ? WHERE id = ?",
|
||||
time.Now().UnixNano(), id.String())
|
||||
return err
|
||||
}
|
||||
|
||||
// --- helpers ---------------------------------------------------------------
|
||||
|
||||
func boolInt(b bool) int {
|
||||
if b {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func nullableTime(t *time.Time) any {
|
||||
if t == nil {
|
||||
return nil
|
||||
}
|
||||
return t.UnixNano()
|
||||
}
|
||||
|
||||
func decodeTime(raw any) time.Time {
|
||||
switch v := raw.(type) {
|
||||
case int64:
|
||||
return time.Unix(0, v).UTC()
|
||||
}
|
||||
return time.Time{}
|
||||
}
|
||||
|
||||
func mapErrNoRows(err error, notFound error) error {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return notFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func rowsAffectedOrNotFound(res sql.Result, notFound error) error {
|
||||
affected, err := res.RowsAffected()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if affected == 0 {
|
||||
return notFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// isUnique reports whether the error is a SQLite unique-constraint violation.
|
||||
func isUnique(err error) bool {
|
||||
return err != nil && (contains(err.Error(), "UNIQUE constraint failed") ||
|
||||
contains(err.Error(), "constraint failed"))
|
||||
}
|
||||
|
||||
func contains(haystack, needle string) bool {
|
||||
return len(haystack) >= len(needle) && (haystack == needle || len(haystack) > len(needle) &&
|
||||
(indexOf(haystack, needle) >= 0))
|
||||
}
|
||||
|
||||
func indexOf(haystack, needle string) int {
|
||||
for i := 0; i+len(needle) <= len(haystack); i++ {
|
||||
if haystack[i:i+len(needle)] == needle {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
// Open opens (and creates when missing) the userservice database file.
|
||||
func Open(path string) (*sql.DB, error) {
|
||||
dsn := fmt.Sprintf("file:%s?_pragma=busy_timeout(10000)&_pragma=journal_mode(WAL)&_pragma=foreign_keys(1)&_pragma=synchronous(NORMAL)", path)
|
||||
db, err := sql.Open("sqlite", dsn)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open userservice database: %w", err)
|
||||
}
|
||||
if err := db.PingContext(context.Background()); err != nil {
|
||||
_ = db.Close()
|
||||
return nil, fmt.Errorf("ping userservice database: %w", err)
|
||||
}
|
||||
return db, nil
|
||||
}
|
||||
@@ -0,0 +1,137 @@
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/usecase"
|
||||
)
|
||||
|
||||
func newTestDB(t *testing.T) *sql.DB {
|
||||
t.Helper()
|
||||
db, err := Open(filepath.Join(t.TempDir(), "users.db"))
|
||||
if err != nil {
|
||||
t.Fatalf("open: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = db.Close() })
|
||||
if err := Migrate(context.Background(), db, nil); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
return db
|
||||
}
|
||||
|
||||
func TestMigrateIsIdempotent(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
if err := Migrate(context.Background(), db, nil); err != nil {
|
||||
t.Fatalf("second migrate: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUserRepoRoundTripAndSentinels(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
repo := NewUserRepo(db)
|
||||
|
||||
user, err := domain.NewUser("root@scimesh.local", "hash", time.Now().UTC())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := repo.Insert(ctx, user); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := repo.GetByEmail(ctx, "root@scimesh.local")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.ID != user.ID || got.Role != domain.RoleUser || got.Verified {
|
||||
t.Errorf("user = %+v", got)
|
||||
}
|
||||
duplicate, err := domain.NewUser("ROOT@scimesh.local", "hash2", time.Now().UTC())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := repo.Insert(ctx, duplicate); !errors.Is(err, usecase.ErrEmailExists) {
|
||||
t.Errorf("duplicate email err = %v, want ErrEmailExists", err)
|
||||
}
|
||||
if _, err := repo.GetByID(ctx, uuid.New()); !errors.Is(err, usecase.ErrUserNotFound) {
|
||||
t.Errorf("missing user err = %v, want ErrUserNotFound", err)
|
||||
}
|
||||
if err := repo.SetVerified(ctx, user.ID, true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err = repo.GetByID(ctx, user.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !got.Verified {
|
||||
t.Error("verified flag did not persist")
|
||||
}
|
||||
if err := repo.SetRole(ctx, user.ID, domain.RoleAdmin); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err = repo.GetByID(ctx, user.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Role != domain.RoleAdmin {
|
||||
t.Errorf("role = %q, want admin", got.Role)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWorkerKeyRepoLifecycle(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
repo := NewWorkerKeyRepo(db)
|
||||
|
||||
user, err := domain.NewUser("worker@scimesh.local", "hash", time.Now().UTC())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := NewUserRepo(db).Insert(ctx, user); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
key, plaintext, err := domain.NewWorkerKey(user.ID, "my machine", time.Now().UTC())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if plaintext == "" || key.TokenHash == "" {
|
||||
t.Fatal("key must carry a hash and return a plaintext")
|
||||
}
|
||||
if err := repo.Insert(ctx, key); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
found, err := repo.GetActiveByHash(ctx, key.TokenHash)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if found.ID != key.ID {
|
||||
t.Errorf("key = %+v", found)
|
||||
}
|
||||
keys, err := repo.ListByUser(ctx, user.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(keys) != 1 {
|
||||
t.Errorf("keys = %d, want 1", len(keys))
|
||||
}
|
||||
if err := repo.TouchLastUsed(ctx, key.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := repo.Revoke(ctx, key.ID, user.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := repo.GetActiveByHash(ctx, key.TokenHash); !errors.Is(err, usecase.ErrWorkerKeyNotFound) {
|
||||
t.Errorf("revoked key err = %v, want ErrWorkerKeyNotFound", err)
|
||||
}
|
||||
if err := repo.Revoke(ctx, key.ID, user.ID); !errors.Is(err, usecase.ErrWorkerKeyNotFound) {
|
||||
t.Errorf("double revoke err = %v, want ErrWorkerKeyNotFound", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
package http
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
)
|
||||
|
||||
// registerRequest / loginRequest are the JSON bodies clients POST. Kept separate
|
||||
// from the domain so the wire format can evolve without touching the entity.
|
||||
type registerRequest struct {
|
||||
Email string `json:"email"`
|
||||
Password string `json:"password"`
|
||||
}
|
||||
|
||||
type loginRequest struct {
|
||||
Email string `json:"email"`
|
||||
Password string `json:"password"`
|
||||
}
|
||||
|
||||
// userResponse is the public view of a user. It never carries the password hash.
|
||||
type userResponse struct {
|
||||
ID string `json:"id"`
|
||||
Email string `json:"email"`
|
||||
Role string `json:"role"`
|
||||
Verified bool `json:"verified"`
|
||||
CreatedAt string `json:"created_at"`
|
||||
}
|
||||
|
||||
type loginResponse struct {
|
||||
Token string `json:"token"`
|
||||
User userResponse `json:"user"`
|
||||
}
|
||||
|
||||
func toUserResponse(u *domain.User) userResponse {
|
||||
return userResponse{
|
||||
ID: u.ID.String(),
|
||||
Email: u.Email,
|
||||
Role: string(u.Role),
|
||||
Verified: u.Verified,
|
||||
CreatedAt: u.CreatedAt.UTC().Format(time.RFC3339),
|
||||
}
|
||||
}
|
||||
|
||||
// createWorkerKeyRequest is the body for minting a worker key. Name is an
|
||||
// optional human label; the domain defaults it when blank.
|
||||
type createWorkerKeyRequest struct {
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
// exchangeWorkerKeyRequest trades a worker key for a short-lived JWT.
|
||||
type exchangeWorkerKeyRequest struct {
|
||||
Key string `json:"key"`
|
||||
}
|
||||
|
||||
type exchangeWorkerKeyResponse struct {
|
||||
Token string `json:"token"`
|
||||
ExpiresIn int `json:"expires_in"`
|
||||
}
|
||||
|
||||
// workerKeyResponse is the public view of a key. It never carries the secret —
|
||||
// only the non-secret prefix used to identify a row.
|
||||
type workerKeyResponse struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Prefix string `json:"prefix"`
|
||||
CreatedAt string `json:"created_at"`
|
||||
LastUsedAt string `json:"last_used_at,omitempty"`
|
||||
}
|
||||
|
||||
// createdWorkerKeyResponse extends the public view with the one-time plaintext,
|
||||
// returned only from the create call and never again.
|
||||
type createdWorkerKeyResponse struct {
|
||||
workerKeyResponse
|
||||
Key string `json:"key"`
|
||||
}
|
||||
|
||||
type workerKeysResponse struct {
|
||||
WorkerKeys []workerKeyResponse `json:"worker_keys"`
|
||||
}
|
||||
|
||||
func toWorkerKeyResponse(k *domain.WorkerKey) workerKeyResponse {
|
||||
resp := workerKeyResponse{
|
||||
ID: k.ID.String(),
|
||||
Name: k.Name,
|
||||
Prefix: k.Prefix,
|
||||
CreatedAt: k.CreatedAt.UTC().Format(time.RFC3339),
|
||||
}
|
||||
if k.LastUsedAt != nil {
|
||||
resp.LastUsedAt = k.LastUsedAt.UTC().Format(time.RFC3339)
|
||||
}
|
||||
return resp
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
package http
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/usecase"
|
||||
)
|
||||
|
||||
// maxJSONBody caps a request body. Credentials are tiny; anything larger is a
|
||||
// mistake or an attack, so reject it before allocating.
|
||||
const maxJSONBody = 1 << 20 // 1 MiB
|
||||
|
||||
type errorResponse struct {
|
||||
Error string `json:"error"`
|
||||
RequestID string `json:"request_id,omitempty"`
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, status int, v any) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
_ = json.NewEncoder(w).Encode(v)
|
||||
}
|
||||
|
||||
// writeError maps a domain or use-case error to an HTTP status and a safe
|
||||
// message, logging only genuine server faults (5xx). Client errors (4xx) are
|
||||
// expected and stay out of the error log.
|
||||
func writeError(w http.ResponseWriter, r *http.Request, log *slog.Logger, err error) {
|
||||
status, msg := statusForError(err)
|
||||
if status >= http.StatusInternalServerError {
|
||||
log.Error("request failed",
|
||||
"err", err,
|
||||
"request_id", requestIDFrom(r.Context()),
|
||||
"path", r.URL.Path,
|
||||
)
|
||||
}
|
||||
writeJSON(w, status, errorResponse{Error: msg, RequestID: requestIDFrom(r.Context())})
|
||||
}
|
||||
|
||||
func statusForError(err error) (int, string) {
|
||||
switch {
|
||||
case errors.Is(err, usecase.ErrEmailExists):
|
||||
return http.StatusConflict, "email already registered"
|
||||
case errors.Is(err, usecase.ErrInvalidCredentials):
|
||||
return http.StatusUnauthorized, "invalid email or password"
|
||||
case errors.Is(err, usecase.ErrUserNotFound):
|
||||
return http.StatusNotFound, "user not found"
|
||||
case errors.Is(err, usecase.ErrWorkerKeyNotFound):
|
||||
return http.StatusNotFound, "worker key not found"
|
||||
case errors.Is(err, usecase.ErrInvalidWorkerKey):
|
||||
return http.StatusUnauthorized, "invalid worker key"
|
||||
case errors.Is(err, domain.ErrWorkerKeyNameTooLong):
|
||||
return http.StatusBadRequest, "worker key name is too long"
|
||||
case errors.Is(err, usecase.ErrPasswordTooShort):
|
||||
return http.StatusBadRequest, "password must be at least 8 characters"
|
||||
case errors.Is(err, usecase.ErrPasswordTooLong):
|
||||
return http.StatusBadRequest, "password must be at most 72 bytes"
|
||||
case errors.Is(err, usecase.ErrInvalidRole):
|
||||
return http.StatusBadRequest, "invalid role"
|
||||
case errors.Is(err, domain.ErrEmptyEmail), errors.Is(err, domain.ErrInvalidEmail):
|
||||
return http.StatusBadRequest, "email is not a valid address"
|
||||
default:
|
||||
// Don't leak internals; the real error is in the log under request_id.
|
||||
return http.StatusInternalServerError, "internal error"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,231 @@
|
||||
package http
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/usecase"
|
||||
)
|
||||
|
||||
// Handlers holds the use cases each endpoint drives.
|
||||
type Handlers struct {
|
||||
register *usecase.Register
|
||||
login *usecase.Login
|
||||
setVerified *usecase.SetVerified
|
||||
setRole *usecase.SetRole
|
||||
createWorkerKey *usecase.CreateWorkerKey
|
||||
listWorkerKeys *usecase.ListWorkerKeys
|
||||
revokeWorkerKey *usecase.RevokeWorkerKey
|
||||
exchangeWorkerKey *usecase.ExchangeWorkerKey
|
||||
users usecase.UserRepository
|
||||
log *slog.Logger
|
||||
}
|
||||
|
||||
// handleHealth is an unauthenticated liveness probe for the container and load
|
||||
// balancer.
|
||||
func (h *Handlers) handleHealth(w http.ResponseWriter, _ *http.Request) {
|
||||
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
|
||||
}
|
||||
|
||||
// handleRegister creates an account. It returns 201 with the public user view,
|
||||
// 409 if the email is taken, or 400 on a malformed body / weak password.
|
||||
func (h *Handlers) handleRegister(w http.ResponseWriter, r *http.Request) {
|
||||
var req registerRequest
|
||||
if !decodeJSON(w, r, &req) {
|
||||
return
|
||||
}
|
||||
u, err := h.register.Execute(r.Context(), req.Email, req.Password)
|
||||
if err != nil {
|
||||
writeError(w, r, h.log, err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusCreated, toUserResponse(u))
|
||||
}
|
||||
|
||||
// handleLogin verifies credentials and returns a signed token plus the user.
|
||||
func (h *Handlers) handleLogin(w http.ResponseWriter, r *http.Request) {
|
||||
var req loginRequest
|
||||
if !decodeJSON(w, r, &req) {
|
||||
return
|
||||
}
|
||||
token, u, err := h.login.Execute(r.Context(), req.Email, req.Password)
|
||||
if err != nil {
|
||||
writeError(w, r, h.log, err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, loginResponse{Token: token, User: toUserResponse(u)})
|
||||
}
|
||||
|
||||
// handleMe returns the caller's own account, proving the token works end to end.
|
||||
// It reads the user id the JWT middleware stashed in the context.
|
||||
func (h *Handlers) handleMe(w http.ResponseWriter, r *http.Request) {
|
||||
id, ok := userIDFrom(r.Context())
|
||||
if !ok {
|
||||
unauthorized(w, r)
|
||||
return
|
||||
}
|
||||
u, err := h.users.GetByID(r.Context(), id)
|
||||
if err != nil {
|
||||
writeError(w, r, h.log, err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, toUserResponse(u))
|
||||
}
|
||||
|
||||
// handleSetVerified grants (verified=true) or revokes (false) the trusted-
|
||||
// contributor badge for the user in the path. Admin-only; the withAdmin
|
||||
// middleware has already enforced the role by the time this runs.
|
||||
func (h *Handlers) handleSetVerified(verified bool) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
id, err := uuid.Parse(r.PathValue("id"))
|
||||
if err != nil {
|
||||
writeJSON(w, http.StatusBadRequest, errorResponse{
|
||||
Error: "invalid user id",
|
||||
RequestID: requestIDFrom(r.Context()),
|
||||
})
|
||||
return
|
||||
}
|
||||
if err := h.setVerified.Execute(r.Context(), id, verified); err != nil {
|
||||
writeError(w, r, h.log, err)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
}
|
||||
|
||||
// handleSetRole promotes (admin) or demotes (user) the user in the path. Admin-
|
||||
// only; the withAdmin middleware has already enforced the caller's role.
|
||||
func (h *Handlers) handleSetRole(role domain.Role) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
id, err := uuid.Parse(r.PathValue("id"))
|
||||
if err != nil {
|
||||
writeJSON(w, http.StatusBadRequest, errorResponse{
|
||||
Error: "invalid user id",
|
||||
RequestID: requestIDFrom(r.Context()),
|
||||
})
|
||||
return
|
||||
}
|
||||
if err := h.setRole.Execute(r.Context(), id, role); err != nil {
|
||||
writeError(w, r, h.log, err)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
}
|
||||
|
||||
// handleCreateWorkerKey mints a long-lived worker key for the authenticated
|
||||
// caller and returns it once, plaintext included. The user copies it into their
|
||||
// worker's SCIMESH_WORKER_KEY; it is never retrievable again.
|
||||
func (h *Handlers) handleCreateWorkerKey(w http.ResponseWriter, r *http.Request) {
|
||||
id, ok := userIDFrom(r.Context())
|
||||
if !ok {
|
||||
unauthorized(w, r)
|
||||
return
|
||||
}
|
||||
// A signed JWT can outlive a demo reset or an account deletion. Check that
|
||||
// its subject still exists before attempting the insert, otherwise the
|
||||
// worker_keys foreign key would turn a stale session into an internal error.
|
||||
if _, err := h.users.GetByID(r.Context(), id); err != nil {
|
||||
if errors.Is(err, usecase.ErrUserNotFound) {
|
||||
unauthorized(w, r)
|
||||
return
|
||||
}
|
||||
writeError(w, r, h.log, err)
|
||||
return
|
||||
}
|
||||
var req createWorkerKeyRequest
|
||||
if !decodeJSON(w, r, &req) {
|
||||
return
|
||||
}
|
||||
key, raw, err := h.createWorkerKey.Execute(r.Context(), id, req.Name)
|
||||
if err != nil {
|
||||
writeError(w, r, h.log, err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusCreated, createdWorkerKeyResponse{
|
||||
workerKeyResponse: toWorkerKeyResponse(key),
|
||||
Key: raw,
|
||||
})
|
||||
}
|
||||
|
||||
// handleListWorkerKeys returns the caller's live keys (no secrets) for display
|
||||
// and revocation.
|
||||
func (h *Handlers) handleListWorkerKeys(w http.ResponseWriter, r *http.Request) {
|
||||
id, ok := userIDFrom(r.Context())
|
||||
if !ok {
|
||||
unauthorized(w, r)
|
||||
return
|
||||
}
|
||||
keys, err := h.listWorkerKeys.Execute(r.Context(), id)
|
||||
if err != nil {
|
||||
writeError(w, r, h.log, err)
|
||||
return
|
||||
}
|
||||
out := make([]workerKeyResponse, 0, len(keys))
|
||||
for _, k := range keys {
|
||||
out = append(out, toWorkerKeyResponse(k))
|
||||
}
|
||||
writeJSON(w, http.StatusOK, workerKeysResponse{WorkerKeys: out})
|
||||
}
|
||||
|
||||
// handleRevokeWorkerKey retires one of the caller's keys. The repository scopes
|
||||
// the delete to the owner, so a mismatched id is a clean 404, not another user's
|
||||
// key.
|
||||
func (h *Handlers) handleRevokeWorkerKey(w http.ResponseWriter, r *http.Request) {
|
||||
userID, ok := userIDFrom(r.Context())
|
||||
if !ok {
|
||||
unauthorized(w, r)
|
||||
return
|
||||
}
|
||||
keyID, err := uuid.Parse(r.PathValue("id"))
|
||||
if err != nil {
|
||||
writeJSON(w, http.StatusBadRequest, errorResponse{
|
||||
Error: "invalid worker key id",
|
||||
RequestID: requestIDFrom(r.Context()),
|
||||
})
|
||||
return
|
||||
}
|
||||
if err := h.revokeWorkerKey.Execute(r.Context(), userID, keyID); err != nil {
|
||||
writeError(w, r, h.log, err)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
|
||||
// handleExchangeWorkerKey trades a worker key for a short-lived JWT. It is
|
||||
// unauthenticated: the key itself is the credential. A worker calls this on
|
||||
// startup and again to refresh before the JWT expires.
|
||||
func (h *Handlers) handleExchangeWorkerKey(w http.ResponseWriter, r *http.Request) {
|
||||
var req exchangeWorkerKeyRequest
|
||||
if !decodeJSON(w, r, &req) {
|
||||
return
|
||||
}
|
||||
token, expiresIn, err := h.exchangeWorkerKey.Execute(r.Context(), req.Key)
|
||||
if err != nil {
|
||||
writeError(w, r, h.log, err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, exchangeWorkerKeyResponse{Token: token, ExpiresIn: expiresIn})
|
||||
}
|
||||
|
||||
// decodeJSON reads a size-capped JSON body into dst, rejecting unknown fields.
|
||||
// It writes a 400 and returns false on any problem, so callers can `if
|
||||
// !decodeJSON(...) { return }`.
|
||||
func decodeJSON(w http.ResponseWriter, r *http.Request, dst any) bool {
|
||||
r.Body = http.MaxBytesReader(w, r.Body, maxJSONBody)
|
||||
dec := json.NewDecoder(r.Body)
|
||||
dec.DisallowUnknownFields()
|
||||
if err := dec.Decode(dst); err != nil {
|
||||
writeJSON(w, http.StatusBadRequest, errorResponse{
|
||||
Error: "invalid JSON body",
|
||||
RequestID: requestIDFrom(r.Context()),
|
||||
})
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
package http
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/auth"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
)
|
||||
|
||||
type ctxKey string
|
||||
|
||||
const (
|
||||
requestIDKey ctxKey = "request_id"
|
||||
userIDKey ctxKey = "user_id"
|
||||
roleKey ctxKey = "role"
|
||||
)
|
||||
|
||||
// withRequestID stamps every request with an ID for correlated logs and error
|
||||
// bodies. It wraps the auth middleware rather than the other way round, so even
|
||||
// a rejected request carries an ID the caller can quote in a bug report.
|
||||
func withRequestID(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
id := newRequestID()
|
||||
w.Header().Set("X-Request-ID", id)
|
||||
next.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), requestIDKey, id)))
|
||||
})
|
||||
}
|
||||
|
||||
func requestIDFrom(ctx context.Context) string {
|
||||
if v, ok := ctx.Value(requestIDKey).(string); ok {
|
||||
return v
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func newRequestID() string {
|
||||
var b [8]byte
|
||||
_, _ = rand.Read(b[:])
|
||||
return hex.EncodeToString(b[:])
|
||||
}
|
||||
|
||||
// tokenVerifier is the slice of auth.Issuer the JWT middleware needs. Taking an
|
||||
// interface keeps the middleware testable with a stub verifier.
|
||||
type tokenVerifier interface {
|
||||
Verify(token string) (*auth.Claims, error)
|
||||
}
|
||||
|
||||
// withJWT verifies the Bearer token and stashes the caller's id and role in the
|
||||
// request context. It rejects any request without a valid, unexpired HS256
|
||||
// token — this is what protects endpoints that act on a specific user.
|
||||
func withJWT(v tokenVerifier) func(http.Handler) http.Handler {
|
||||
return func(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
raw := strings.TrimPrefix(r.Header.Get("Authorization"), "Bearer ")
|
||||
if raw == "" {
|
||||
unauthorized(w, r)
|
||||
return
|
||||
}
|
||||
claims, err := v.Verify(raw)
|
||||
if err != nil {
|
||||
unauthorized(w, r)
|
||||
return
|
||||
}
|
||||
id, err := uuid.Parse(claims.Subject)
|
||||
if err != nil {
|
||||
unauthorized(w, r)
|
||||
return
|
||||
}
|
||||
ctx := context.WithValue(r.Context(), userIDKey, id)
|
||||
ctx = context.WithValue(ctx, roleKey, claims.Role)
|
||||
next.ServeHTTP(w, r.WithContext(ctx))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func unauthorized(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("WWW-Authenticate", "Bearer")
|
||||
writeJSON(w, http.StatusUnauthorized, errorResponse{
|
||||
Error: "unauthorized",
|
||||
RequestID: requestIDFrom(r.Context()),
|
||||
})
|
||||
}
|
||||
|
||||
// userIDFrom returns the authenticated caller's id, set by withJWT.
|
||||
func userIDFrom(ctx context.Context) (uuid.UUID, bool) {
|
||||
id, ok := ctx.Value(userIDKey).(uuid.UUID)
|
||||
return id, ok
|
||||
}
|
||||
|
||||
// withAdmin rejects any caller whose token role is not admin. It must sit inside
|
||||
// withJWT, which stamps the role after verifying the token.
|
||||
func withAdmin(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if role, ok := r.Context().Value(roleKey).(domain.Role); !ok || role != domain.RoleAdmin {
|
||||
writeJSON(w, http.StatusForbidden, errorResponse{
|
||||
Error: "admin role required",
|
||||
RequestID: requestIDFrom(r.Context()),
|
||||
})
|
||||
return
|
||||
}
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
// statusRecorder captures the status code for the access log.
|
||||
type statusRecorder struct {
|
||||
http.ResponseWriter
|
||||
status int
|
||||
}
|
||||
|
||||
func (s *statusRecorder) WriteHeader(code int) {
|
||||
s.status = code
|
||||
s.ResponseWriter.WriteHeader(code)
|
||||
}
|
||||
|
||||
// withAccessLog records one structured line per request — the minimum needed to
|
||||
// debug a distributed system after the fact.
|
||||
func withAccessLog(log *slog.Logger) func(http.Handler) http.Handler {
|
||||
return func(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
start := time.Now()
|
||||
rec := &statusRecorder{ResponseWriter: w, status: http.StatusOK}
|
||||
next.ServeHTTP(rec, r)
|
||||
log.Info("request",
|
||||
"request_id", requestIDFrom(r.Context()),
|
||||
"method", r.Method,
|
||||
"path", r.URL.Path,
|
||||
"status", rec.status,
|
||||
"duration_ms", time.Since(start).Milliseconds(),
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// chain applies middleware so that the first argument is the outermost layer.
|
||||
func chain(h http.Handler, mw ...func(http.Handler) http.Handler) http.Handler {
|
||||
for i := len(mw) - 1; i >= 0; i-- {
|
||||
h = mw[i](h)
|
||||
}
|
||||
return h
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
// Package http exposes the userservice over HTTP: registration, login, and a
|
||||
// token-protected /me. It owns routing, request decoding, and error mapping;
|
||||
// business rules live in the usecase layer.
|
||||
package http
|
||||
|
||||
import (
|
||||
"log/slog"
|
||||
"net/http"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/auth"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/usecase"
|
||||
)
|
||||
|
||||
// UseCases bundles the application services the handlers drive.
|
||||
type UseCases struct {
|
||||
Register *usecase.Register
|
||||
Login *usecase.Login
|
||||
SetVerified *usecase.SetVerified
|
||||
SetRole *usecase.SetRole
|
||||
CreateWorkerKey *usecase.CreateWorkerKey
|
||||
ListWorkerKeys *usecase.ListWorkerKeys
|
||||
RevokeWorkerKey *usecase.RevokeWorkerKey
|
||||
ExchangeWorkerKey *usecase.ExchangeWorkerKey
|
||||
Users usecase.UserRepository
|
||||
}
|
||||
|
||||
// NewServer wires the routes and the middleware stack and returns the handler.
|
||||
// The issuer verifies tokens for the JWT-protected routes.
|
||||
func NewServer(log *slog.Logger, uc UseCases, issuer auth.Issuer) http.Handler {
|
||||
h := &Handlers{
|
||||
register: uc.Register,
|
||||
login: uc.Login,
|
||||
setVerified: uc.SetVerified,
|
||||
setRole: uc.SetRole,
|
||||
createWorkerKey: uc.CreateWorkerKey,
|
||||
listWorkerKeys: uc.ListWorkerKeys,
|
||||
revokeWorkerKey: uc.RevokeWorkerKey,
|
||||
exchangeWorkerKey: uc.ExchangeWorkerKey,
|
||||
users: uc.Users,
|
||||
log: log,
|
||||
}
|
||||
|
||||
mux := http.NewServeMux()
|
||||
// Method-aware patterns (Go 1.22+): a GET to /register is a 405, not a match.
|
||||
mux.HandleFunc("GET /health", h.handleHealth)
|
||||
mux.HandleFunc("POST /register", h.handleRegister)
|
||||
mux.HandleFunc("POST /login", h.handleLogin)
|
||||
// /me proves a token round-trips; it sits behind JWT auth.
|
||||
mux.Handle("GET /me", chain(http.HandlerFunc(h.handleMe), withJWT(issuer)))
|
||||
|
||||
// Worker keys: a user mints a long-lived key (JWT-protected), and a worker
|
||||
// trades it for a short-lived JWT on the public exchange endpoint — the key
|
||||
// itself is the credential there, so no prior token is required.
|
||||
mux.HandleFunc("POST /worker-tokens/exchange", h.handleExchangeWorkerKey)
|
||||
mux.Handle("POST /worker-keys", chain(http.HandlerFunc(h.handleCreateWorkerKey), withJWT(issuer)))
|
||||
mux.Handle("GET /worker-keys", chain(http.HandlerFunc(h.handleListWorkerKeys), withJWT(issuer)))
|
||||
mux.Handle("DELETE /worker-keys/{id}", chain(http.HandlerFunc(h.handleRevokeWorkerKey), withJWT(issuer)))
|
||||
|
||||
// Admin-only: grant or revoke the trusted-contributor badge. withAdmin sits
|
||||
// inside withJWT so the role is available from the verified token.
|
||||
mux.Handle("POST /users/{id}/verify",
|
||||
chain(h.handleSetVerified(true), withJWT(issuer), withAdmin))
|
||||
mux.Handle("POST /users/{id}/unverify",
|
||||
chain(h.handleSetVerified(false), withJWT(issuer), withAdmin))
|
||||
mux.Handle("POST /users/{id}/promote",
|
||||
chain(h.handleSetRole(domain.RoleAdmin), withJWT(issuer), withAdmin))
|
||||
mux.Handle("POST /users/{id}/demote",
|
||||
chain(h.handleSetRole(domain.RoleUser), withJWT(issuer), withAdmin))
|
||||
|
||||
// Outermost first: every request gets an ID and an access-log line.
|
||||
return chain(mux, withRequestID, withAccessLog(log))
|
||||
}
|
||||
@@ -0,0 +1,372 @@
|
||||
package http_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/auth"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/memstore"
|
||||
apihttp "github.com/emil28092005/SciMesh/coordinator/internal/userservice/transport/http"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/usecase"
|
||||
)
|
||||
|
||||
const secret = "server-test-secret-32-bytes-long!!!!"
|
||||
|
||||
func newTestServer() http.Handler {
|
||||
users := memstore.NewUserRepo()
|
||||
hasher := auth.NewHasher(4)
|
||||
clk := memstore.Clock{T: time.Date(2026, 7, 26, 0, 0, 0, 0, time.UTC)}
|
||||
// Real clock for the issuer so tokens are valid at verification time.
|
||||
issuer := auth.NewIssuer(secret, time.Hour, nil)
|
||||
|
||||
uc := apihttp.UseCases{
|
||||
Register: usecase.NewRegister(users, hasher, clk),
|
||||
Login: usecase.NewLogin(users, hasher, issuer),
|
||||
SetVerified: usecase.NewSetVerified(users),
|
||||
SetRole: usecase.NewSetRole(users),
|
||||
Users: users,
|
||||
}
|
||||
log := slog.New(slog.NewTextHandler(io.Discard, nil))
|
||||
return apihttp.NewServer(log, uc, issuer)
|
||||
}
|
||||
|
||||
func do(t *testing.T, h http.Handler, method, path, token string, body any) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
var buf bytes.Buffer
|
||||
if body != nil {
|
||||
if err := json.NewEncoder(&buf).Encode(body); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
req, err := http.NewRequestWithContext(context.Background(), method, path, &buf)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
return rec
|
||||
}
|
||||
|
||||
func TestRegisterThenLoginThenMe(t *testing.T) {
|
||||
h := newTestServer()
|
||||
creds := map[string]string{"email": "flow@example.com", "password": "password123"}
|
||||
|
||||
// Register -> 201
|
||||
rec := do(t, h, http.MethodPost, "/register", "", creds)
|
||||
if rec.Code != http.StatusCreated {
|
||||
t.Fatalf("register: got %d, body %s", rec.Code, rec.Body)
|
||||
}
|
||||
|
||||
// Login -> 200 with a token
|
||||
rec = do(t, h, http.MethodPost, "/login", "", creds)
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("login: got %d, body %s", rec.Code, rec.Body)
|
||||
}
|
||||
var lr struct {
|
||||
Token string `json:"token"`
|
||||
User struct {
|
||||
Email string `json:"email"`
|
||||
Role string `json:"role"`
|
||||
} `json:"user"`
|
||||
}
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), &lr); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if lr.Token == "" || lr.User.Email != "flow@example.com" || lr.User.Role != "user" {
|
||||
t.Fatalf("unexpected login body: %+v", lr)
|
||||
}
|
||||
|
||||
// /me with the token -> 200, same user
|
||||
rec = do(t, h, http.MethodGet, "/me", lr.Token, nil)
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("me: got %d, body %s", rec.Code, rec.Body)
|
||||
}
|
||||
var me struct {
|
||||
Email string `json:"email"`
|
||||
}
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), &me); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if me.Email != "flow@example.com" {
|
||||
t.Errorf("me email = %q", me.Email)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRegisterDuplicate(t *testing.T) {
|
||||
h := newTestServer()
|
||||
creds := map[string]string{"email": "dup@example.com", "password": "password123"}
|
||||
_ = do(t, h, http.MethodPost, "/register", "", creds)
|
||||
|
||||
rec := do(t, h, http.MethodPost, "/register", "", creds)
|
||||
if rec.Code != http.StatusConflict {
|
||||
t.Errorf("duplicate register: got %d, want 409", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRegisterValidation(t *testing.T) {
|
||||
h := newTestServer()
|
||||
cases := []struct {
|
||||
name string
|
||||
body map[string]string
|
||||
}{
|
||||
{"weak password", map[string]string{"email": "a@b.com", "password": "short"}},
|
||||
{"bad email", map[string]string{"email": "nope", "password": "password123"}},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
rec := do(t, h, http.MethodPost, "/register", "", tc.body)
|
||||
if rec.Code != http.StatusBadRequest {
|
||||
t.Errorf("got %d, want 400", rec.Code)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRegisterRejectsUnknownFields(t *testing.T) {
|
||||
h := newTestServer()
|
||||
rec := do(t, h, http.MethodPost, "/register", "", map[string]string{
|
||||
"email": "a@b.com", "password": "password123", "role": "admin",
|
||||
})
|
||||
if rec.Code != http.StatusBadRequest {
|
||||
t.Errorf("unknown field must be rejected: got %d", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoginWrongPassword(t *testing.T) {
|
||||
h := newTestServer()
|
||||
_ = do(t, h, http.MethodPost, "/register", "", map[string]string{
|
||||
"email": "x@example.com", "password": "password123",
|
||||
})
|
||||
rec := do(t, h, http.MethodPost, "/login", "", map[string]string{
|
||||
"email": "x@example.com", "password": "wrongpass1",
|
||||
})
|
||||
if rec.Code != http.StatusUnauthorized {
|
||||
t.Errorf("got %d, want 401", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMeRequiresToken(t *testing.T) {
|
||||
h := newTestServer()
|
||||
if rec := do(t, h, http.MethodGet, "/me", "", nil); rec.Code != http.StatusUnauthorized {
|
||||
t.Errorf("no token: got %d, want 401", rec.Code)
|
||||
}
|
||||
if rec := do(t, h, http.MethodGet, "/me", "garbage.token.here", nil); rec.Code != http.StatusUnauthorized {
|
||||
t.Errorf("bad token: got %d, want 401", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHealth(t *testing.T) {
|
||||
h := newTestServer()
|
||||
if rec := do(t, h, http.MethodGet, "/health", "", nil); rec.Code != http.StatusOK {
|
||||
t.Errorf("health: got %d", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
// failingUsers is a UserRepository whose reads fail with an unexpected (non-
|
||||
// sentinel) error, so the handler must map it to 500 and not leak internals.
|
||||
type failingUsers struct{ usecase.UserRepository }
|
||||
|
||||
func (failingUsers) GetByID(context.Context, uuid.UUID) (*domain.User, error) {
|
||||
return nil, errors.New("db exploded")
|
||||
}
|
||||
|
||||
func TestMeInternalError(t *testing.T) {
|
||||
hasher := auth.NewHasher(4)
|
||||
clk := memstore.Clock{T: time.Date(2026, 7, 26, 0, 0, 0, 0, time.UTC)}
|
||||
issuer := auth.NewIssuer(secret, time.Hour, nil)
|
||||
|
||||
users := failingUsers{UserRepository: memstore.NewUserRepo()}
|
||||
uc := apihttp.UseCases{
|
||||
Register: usecase.NewRegister(users, hasher, clk),
|
||||
Login: usecase.NewLogin(users, hasher, issuer),
|
||||
SetVerified: usecase.NewSetVerified(users),
|
||||
SetRole: usecase.NewSetRole(users),
|
||||
Users: users,
|
||||
}
|
||||
h := apihttp.NewServer(slog.New(slog.NewTextHandler(io.Discard, nil)), uc, issuer)
|
||||
|
||||
// A structurally valid token for a caller the failing repo can't load.
|
||||
token, err := issuer.Issue(&domain.User{ID: uuid.New(), Role: domain.RoleUser})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
rec := do(t, h, http.MethodGet, "/me", token, nil)
|
||||
if rec.Code != http.StatusInternalServerError {
|
||||
t.Errorf("got %d, want 500", rec.Code)
|
||||
}
|
||||
// The body must not disclose the underlying error.
|
||||
if bytes.Contains(rec.Body.Bytes(), []byte("db exploded")) {
|
||||
t.Error("internal error leaked to the client")
|
||||
}
|
||||
}
|
||||
|
||||
// mintToken issues a token with the package secret for a synthetic caller of the
|
||||
// given role — enough to drive the admin-gated endpoints.
|
||||
func mintToken(t *testing.T, role domain.Role) string {
|
||||
t.Helper()
|
||||
token, err := auth.NewIssuer(secret, time.Hour, nil).Issue(&domain.User{ID: uuid.New(), Role: role})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return token
|
||||
}
|
||||
|
||||
// registerUser creates an account and returns its id.
|
||||
func registerUser(t *testing.T, h http.Handler, email string) string {
|
||||
t.Helper()
|
||||
rec := do(t, h, http.MethodPost, "/register", "", map[string]string{"email": email, "password": "password123"})
|
||||
if rec.Code != http.StatusCreated {
|
||||
t.Fatalf("register: %d", rec.Code)
|
||||
}
|
||||
var reg struct {
|
||||
ID string `json:"id"`
|
||||
}
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), ®); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return reg.ID
|
||||
}
|
||||
|
||||
func TestAdminVerifiesUserEndToEnd(t *testing.T) {
|
||||
h := newTestServer()
|
||||
id := registerUser(t, h, "contrib@example.com")
|
||||
|
||||
// Admin grants the badge.
|
||||
rec := do(t, h, http.MethodPost, "/users/"+id+"/verify", mintToken(t, domain.RoleAdmin), nil)
|
||||
if rec.Code != http.StatusNoContent {
|
||||
t.Fatalf("admin verify: got %d, body %s", rec.Code, rec.Body)
|
||||
}
|
||||
|
||||
// The change is visible when the contributor logs in.
|
||||
rec = do(t, h, http.MethodPost, "/login", "", map[string]string{"email": "contrib@example.com", "password": "password123"})
|
||||
var lr struct {
|
||||
User struct {
|
||||
Verified bool `json:"verified"`
|
||||
} `json:"user"`
|
||||
}
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), &lr); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !lr.User.Verified {
|
||||
t.Error("verified badge not reflected after admin granted it")
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyRequiresAdminRole(t *testing.T) {
|
||||
h := newTestServer()
|
||||
id := registerUser(t, h, "someone@example.com")
|
||||
|
||||
// A plain user token must not be able to grant the badge.
|
||||
rec := do(t, h, http.MethodPost, "/users/"+id+"/verify", mintToken(t, domain.RoleUser), nil)
|
||||
if rec.Code != http.StatusForbidden {
|
||||
t.Errorf("plain user: got %d, want 403", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyRequiresAuth(t *testing.T) {
|
||||
h := newTestServer()
|
||||
rec := do(t, h, http.MethodPost, "/users/"+uuid.NewString()+"/verify", "", nil)
|
||||
if rec.Code != http.StatusUnauthorized {
|
||||
t.Errorf("no token: got %d, want 401", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyInvalidID(t *testing.T) {
|
||||
h := newTestServer()
|
||||
rec := do(t, h, http.MethodPost, "/users/not-a-uuid/verify", mintToken(t, domain.RoleAdmin), nil)
|
||||
if rec.Code != http.StatusBadRequest {
|
||||
t.Errorf("bad id: got %d, want 400", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyUnknownUser(t *testing.T) {
|
||||
h := newTestServer()
|
||||
rec := do(t, h, http.MethodPost, "/users/"+uuid.NewString()+"/verify", mintToken(t, domain.RoleAdmin), nil)
|
||||
if rec.Code != http.StatusNotFound {
|
||||
t.Errorf("unknown user: got %d, want 404", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdminPromotesAndDemotes(t *testing.T) {
|
||||
h := newTestServer()
|
||||
id := registerUser(t, h, "promote@example.com")
|
||||
admin := mintToken(t, domain.RoleAdmin)
|
||||
|
||||
if rec := do(t, h, http.MethodPost, "/users/"+id+"/promote", admin, nil); rec.Code != http.StatusNoContent {
|
||||
t.Fatalf("promote: got %d, body %s", rec.Code, rec.Body)
|
||||
}
|
||||
// The promoted user now logs in as an admin.
|
||||
rec := do(t, h, http.MethodPost, "/login", "", map[string]string{"email": "promote@example.com", "password": "password123"})
|
||||
var lr struct {
|
||||
User struct {
|
||||
Role string `json:"role"`
|
||||
} `json:"user"`
|
||||
}
|
||||
_ = json.Unmarshal(rec.Body.Bytes(), &lr)
|
||||
if lr.User.Role != "admin" {
|
||||
t.Errorf("role after promote = %q, want admin", lr.User.Role)
|
||||
}
|
||||
|
||||
if rec := do(t, h, http.MethodPost, "/users/"+id+"/demote", admin, nil); rec.Code != http.StatusNoContent {
|
||||
t.Fatalf("demote: got %d", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPromoteRequiresAdmin(t *testing.T) {
|
||||
h := newTestServer()
|
||||
id := registerUser(t, h, "target@example.com")
|
||||
|
||||
if rec := do(t, h, http.MethodPost, "/users/"+id+"/promote", mintToken(t, domain.RoleUser), nil); rec.Code != http.StatusForbidden {
|
||||
t.Errorf("plain user promote: got %d, want 403", rec.Code)
|
||||
}
|
||||
if rec := do(t, h, http.MethodPost, "/users/"+id+"/promote", "", nil); rec.Code != http.StatusUnauthorized {
|
||||
t.Errorf("no token: got %d, want 401", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPromoteUnknownUser(t *testing.T) {
|
||||
h := newTestServer()
|
||||
rec := do(t, h, http.MethodPost, "/users/"+uuid.NewString()+"/promote", mintToken(t, domain.RoleAdmin), nil)
|
||||
if rec.Code != http.StatusNotFound {
|
||||
t.Errorf("unknown user promote: got %d, want 404", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUnverifyRevokes(t *testing.T) {
|
||||
h := newTestServer()
|
||||
id := registerUser(t, h, "revoke@example.com")
|
||||
admin := mintToken(t, domain.RoleAdmin)
|
||||
|
||||
if rec := do(t, h, http.MethodPost, "/users/"+id+"/verify", admin, nil); rec.Code != http.StatusNoContent {
|
||||
t.Fatalf("verify: %d", rec.Code)
|
||||
}
|
||||
if rec := do(t, h, http.MethodPost, "/users/"+id+"/unverify", admin, nil); rec.Code != http.StatusNoContent {
|
||||
t.Fatalf("unverify: %d", rec.Code)
|
||||
}
|
||||
|
||||
rec := do(t, h, http.MethodPost, "/login", "", map[string]string{"email": "revoke@example.com", "password": "password123"})
|
||||
var lr struct {
|
||||
User struct {
|
||||
Verified bool `json:"verified"`
|
||||
} `json:"user"`
|
||||
}
|
||||
_ = json.Unmarshal(rec.Body.Bytes(), &lr)
|
||||
if lr.User.Verified {
|
||||
t.Error("verified should be false after unverify")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
package usecase
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
)
|
||||
|
||||
// BootstrapAdmin seeds the first admin account. It exists because there is no
|
||||
// other way to create one: /register always makes a plain user, and promoting a
|
||||
// user to admin requires an already-existing admin. Running it at startup with
|
||||
// operator-supplied credentials breaks that chicken-and-egg.
|
||||
type BootstrapAdmin struct {
|
||||
users UserRepository
|
||||
hasher PasswordHasher
|
||||
clk Clock
|
||||
}
|
||||
|
||||
func NewBootstrapAdmin(users UserRepository, hasher PasswordHasher, clk Clock) *BootstrapAdmin {
|
||||
return &BootstrapAdmin{users: users, hasher: hasher, clk: clk}
|
||||
}
|
||||
|
||||
// Execute creates the admin if it does not already exist, reporting whether it
|
||||
// created one. It is idempotent: a second run (a restart) finds the account and
|
||||
// does nothing, so it is safe to call on every boot.
|
||||
func (uc *BootstrapAdmin) Execute(ctx context.Context, email, password string) (created bool, err error) {
|
||||
email = domain.NormalizeEmail(email)
|
||||
|
||||
if _, err := uc.users.GetByEmail(ctx, email); err == nil {
|
||||
return false, nil // already bootstrapped
|
||||
} else if !errors.Is(err, ErrUserNotFound) {
|
||||
return false, err
|
||||
}
|
||||
|
||||
if len(password) < minPasswordLen {
|
||||
return false, ErrPasswordTooShort
|
||||
}
|
||||
if len(password) > maxPasswordLen {
|
||||
return false, ErrPasswordTooLong
|
||||
}
|
||||
|
||||
hash, err := uc.hasher.Hash(password)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
u, err := domain.NewUser(email, hash, uc.clk.Now())
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
// Direct role assignment is safe here: this is a trusted server-side seed,
|
||||
// not a request. A root admin is also a trusted contributor.
|
||||
u.Role = domain.RoleAdmin
|
||||
u.Verified = true
|
||||
|
||||
if err := uc.users.Insert(ctx, u); err != nil {
|
||||
// A concurrent bootstrap (two replicas booting at once) is fine: whoever
|
||||
// lost the race just observes the account now exists.
|
||||
if errors.Is(err, ErrEmailExists) {
|
||||
return false, nil
|
||||
}
|
||||
return false, err
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
package usecase_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/auth"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/memstore"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/usecase"
|
||||
)
|
||||
|
||||
func newBootstrap() (*usecase.BootstrapAdmin, *memstore.UserRepo) {
|
||||
users := memstore.NewUserRepo()
|
||||
hasher := auth.NewHasher(4)
|
||||
clk := memstore.Clock{T: time.Date(2026, 7, 26, 0, 0, 0, 0, time.UTC)}
|
||||
return usecase.NewBootstrapAdmin(users, hasher, clk), users
|
||||
}
|
||||
|
||||
func TestBootstrapCreatesAdmin(t *testing.T) {
|
||||
bs, users := newBootstrap()
|
||||
|
||||
created, err := bs.Execute(context.Background(), "Root@Example.com", "rootpassword")
|
||||
if err != nil {
|
||||
t.Fatalf("bootstrap: %v", err)
|
||||
}
|
||||
if !created {
|
||||
t.Fatal("expected an admin to be created")
|
||||
}
|
||||
|
||||
u, err := users.GetByEmail(context.Background(), "root@example.com")
|
||||
if err != nil {
|
||||
t.Fatalf("admin not persisted: %v", err)
|
||||
}
|
||||
if u.Role != domain.RoleAdmin {
|
||||
t.Errorf("role = %q, want admin", u.Role)
|
||||
}
|
||||
if !u.Verified {
|
||||
t.Error("bootstrap admin should be verified")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBootstrapIsIdempotent(t *testing.T) {
|
||||
bs, users := newBootstrap()
|
||||
ctx := context.Background()
|
||||
|
||||
if _, err := bs.Execute(ctx, "root@example.com", "rootpassword"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
created, err := bs.Execute(ctx, "root@example.com", "rootpassword")
|
||||
if err != nil {
|
||||
t.Fatalf("second run: %v", err)
|
||||
}
|
||||
if created {
|
||||
t.Error("second run must not create a duplicate admin")
|
||||
}
|
||||
|
||||
// The account must still be a single admin.
|
||||
if u, _ := users.GetByEmail(ctx, "root@example.com"); u.Role != domain.RoleAdmin {
|
||||
t.Errorf("role changed: %q", u.Role)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBootstrapRejectsWeakPassword(t *testing.T) {
|
||||
bs, _ := newBootstrap()
|
||||
if _, err := bs.Execute(context.Background(), "root@example.com", "short"); !errors.Is(err, usecase.ErrPasswordTooShort) {
|
||||
t.Errorf("got %v, want ErrPasswordTooShort", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,109 @@
|
||||
package usecase_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/usecase"
|
||||
)
|
||||
|
||||
// These stubs let a test inject failures the happy-path memstore never produces,
|
||||
// so the use cases' error branches are exercised too.
|
||||
|
||||
var errBoom = errors.New("boom")
|
||||
|
||||
type stubRepo struct {
|
||||
getByEmail func() (*domain.User, error)
|
||||
insert func() error
|
||||
}
|
||||
|
||||
func (s stubRepo) Insert(context.Context, *domain.User) error { return s.insert() }
|
||||
func (s stubRepo) GetByEmail(context.Context, string) (*domain.User, error) {
|
||||
return s.getByEmail()
|
||||
}
|
||||
func (s stubRepo) GetByID(context.Context, uuid.UUID) (*domain.User, error) {
|
||||
return nil, usecase.ErrUserNotFound
|
||||
}
|
||||
func (s stubRepo) SetVerified(context.Context, uuid.UUID, bool) error {
|
||||
return usecase.ErrUserNotFound
|
||||
}
|
||||
func (s stubRepo) SetRole(context.Context, uuid.UUID, domain.Role) error {
|
||||
return usecase.ErrUserNotFound
|
||||
}
|
||||
|
||||
type stubHasher struct {
|
||||
hashErr error
|
||||
compareErr error
|
||||
}
|
||||
|
||||
func (s stubHasher) Hash(string) (string, error) {
|
||||
if s.hashErr != nil {
|
||||
return "", s.hashErr
|
||||
}
|
||||
return "hashed", nil
|
||||
}
|
||||
func (s stubHasher) Compare(string, string) error { return s.compareErr }
|
||||
|
||||
type stubIssuer struct{ err error }
|
||||
|
||||
func (s stubIssuer) Issue(*domain.User) (string, error) {
|
||||
if s.err != nil {
|
||||
return "", s.err
|
||||
}
|
||||
return "token", nil
|
||||
}
|
||||
|
||||
func TestRegisterPropagatesHasherError(t *testing.T) {
|
||||
clk := stubClock{time.Now()}
|
||||
reg := usecase.NewRegister(stubRepo{}, stubHasher{hashErr: errBoom}, clk)
|
||||
|
||||
_, err := reg.Execute(context.Background(), "a@b.com", "password123")
|
||||
if !errors.Is(err, errBoom) {
|
||||
t.Errorf("got %v, want errBoom", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRegisterPropagatesInsertError(t *testing.T) {
|
||||
clk := stubClock{time.Now()}
|
||||
repo := stubRepo{insert: func() error { return errBoom }}
|
||||
reg := usecase.NewRegister(repo, stubHasher{}, clk)
|
||||
|
||||
_, err := reg.Execute(context.Background(), "a@b.com", "password123")
|
||||
if !errors.Is(err, errBoom) {
|
||||
t.Errorf("got %v, want errBoom", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoginPropagatesRepoError(t *testing.T) {
|
||||
// A non-ErrUserNotFound repo error must surface as-is, not be masked as
|
||||
// ErrInvalidCredentials.
|
||||
repo := stubRepo{getByEmail: func() (*domain.User, error) { return nil, errBoom }}
|
||||
login := usecase.NewLogin(repo, stubHasher{}, stubIssuer{})
|
||||
|
||||
_, _, err := login.Execute(context.Background(), "a@b.com", "password123")
|
||||
if !errors.Is(err, errBoom) {
|
||||
t.Errorf("got %v, want errBoom", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoginPropagatesIssuerError(t *testing.T) {
|
||||
repo := stubRepo{getByEmail: func() (*domain.User, error) {
|
||||
return &domain.User{ID: uuid.New(), Email: "a@b.com", Role: domain.RoleUser}, nil
|
||||
}}
|
||||
// Hasher accepts the password (nil compareErr) so we reach token issuance.
|
||||
login := usecase.NewLogin(repo, stubHasher{}, stubIssuer{err: errBoom})
|
||||
|
||||
_, _, err := login.Execute(context.Background(), "a@b.com", "password123")
|
||||
if !errors.Is(err, errBoom) {
|
||||
t.Errorf("got %v, want errBoom", err)
|
||||
}
|
||||
}
|
||||
|
||||
type stubClock struct{ t time.Time }
|
||||
|
||||
func (c stubClock) Now() time.Time { return c.t }
|
||||
@@ -0,0 +1,27 @@
|
||||
package usecase
|
||||
|
||||
import "errors"
|
||||
|
||||
var (
|
||||
// Repository-contract errors, returned by UserRepository implementations.
|
||||
ErrEmailExists = errors.New("email already registered")
|
||||
ErrUserNotFound = errors.New("user not found")
|
||||
|
||||
// ErrWorkerKeyNotFound is returned by WorkerKeyRepository when no live key
|
||||
// matches (by id for revoke, by hash for exchange).
|
||||
ErrWorkerKeyNotFound = errors.New("worker key not found")
|
||||
// ErrInvalidWorkerKey is surfaced to the transport layer for a key that does
|
||||
// not exchange (unknown, revoked, or owner gone). Deliberately opaque so a
|
||||
// caller cannot distinguish the cases while probing.
|
||||
ErrInvalidWorkerKey = errors.New("invalid worker key")
|
||||
|
||||
// Use-case errors surfaced to the transport layer.
|
||||
//
|
||||
// ErrInvalidCredentials is deliberately returned for both an unknown email
|
||||
// and a wrong password, so an attacker cannot use the response to learn
|
||||
// which emails are registered.
|
||||
ErrInvalidCredentials = errors.New("invalid email or password")
|
||||
ErrPasswordTooShort = errors.New("password too short")
|
||||
ErrPasswordTooLong = errors.New("password too long")
|
||||
ErrInvalidRole = errors.New("invalid role")
|
||||
)
|
||||
@@ -0,0 +1,42 @@
|
||||
package usecase
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
)
|
||||
|
||||
// Login verifies credentials and issues a signed token.
|
||||
type Login struct {
|
||||
users UserRepository
|
||||
hasher PasswordHasher
|
||||
tokens TokenIssuer
|
||||
}
|
||||
|
||||
func NewLogin(users UserRepository, hasher PasswordHasher, tokens TokenIssuer) *Login {
|
||||
return &Login{users: users, hasher: hasher, tokens: tokens}
|
||||
}
|
||||
|
||||
// Execute returns a signed token and the user on success. It returns
|
||||
// ErrInvalidCredentials for both an unknown email and a wrong password so the
|
||||
// two cases are indistinguishable to a caller probing for valid accounts.
|
||||
func (l *Login) Execute(ctx context.Context, email, password string) (string, *domain.User, error) {
|
||||
u, err := l.users.GetByEmail(ctx, domain.NormalizeEmail(email))
|
||||
if err != nil {
|
||||
if errors.Is(err, ErrUserNotFound) {
|
||||
return "", nil, ErrInvalidCredentials
|
||||
}
|
||||
return "", nil, err
|
||||
}
|
||||
|
||||
if err := l.hasher.Compare(u.PasswordHash, password); err != nil {
|
||||
return "", nil, ErrInvalidCredentials
|
||||
}
|
||||
|
||||
token, err := l.tokens.Issue(u)
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
return token, u, nil
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
// Package usecase holds the application logic — registration and login — plus
|
||||
// the ports (interfaces) it depends on. The concrete adapters (PostgreSQL,
|
||||
// bcrypt, JWT) are injected from cmd, so this package never imports them.
|
||||
package usecase
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
)
|
||||
|
||||
// UserRepository persists and looks up users. Implementations return the
|
||||
// sentinel errors in errors.go so the use cases can react without knowing about
|
||||
// SQL or driver types.
|
||||
type UserRepository interface {
|
||||
// Insert stores a new user, returning ErrEmailExists if the email is taken.
|
||||
Insert(ctx context.Context, u *domain.User) error
|
||||
// GetByEmail returns the user with the (normalised) email, or ErrUserNotFound.
|
||||
GetByEmail(ctx context.Context, email string) (*domain.User, error)
|
||||
// GetByID returns the user with id, or ErrUserNotFound.
|
||||
GetByID(ctx context.Context, id uuid.UUID) (*domain.User, error)
|
||||
// SetVerified toggles the verified flag, returning ErrUserNotFound if no
|
||||
// such user exists.
|
||||
SetVerified(ctx context.Context, id uuid.UUID, verified bool) error
|
||||
// SetRole changes a user's role, returning ErrUserNotFound if no such user
|
||||
// exists.
|
||||
SetRole(ctx context.Context, id uuid.UUID, role domain.Role) error
|
||||
}
|
||||
|
||||
// WorkerKeyRepository persists and looks up the long-lived worker keys a user
|
||||
// creates to run a worker bound to their account. Implementations return the
|
||||
// sentinel errors in errors.go so the use cases stay free of SQL types.
|
||||
type WorkerKeyRepository interface {
|
||||
// Insert stores a freshly minted key.
|
||||
Insert(ctx context.Context, k *domain.WorkerKey) error
|
||||
// ListByUser returns a user's live (non-revoked) keys, newest first.
|
||||
ListByUser(ctx context.Context, userID uuid.UUID) ([]*domain.WorkerKey, error)
|
||||
// GetActiveByHash returns the non-revoked key with the given hash, or
|
||||
// ErrWorkerKeyNotFound.
|
||||
GetActiveByHash(ctx context.Context, tokenHash string) (*domain.WorkerKey, error)
|
||||
// Revoke retires a key the user owns, returning ErrWorkerKeyNotFound when no
|
||||
// live key with that id belongs to the user.
|
||||
Revoke(ctx context.Context, id, userID uuid.UUID) error
|
||||
// TouchLastUsed records a successful exchange. Best-effort: a failure here
|
||||
// must not fail the exchange itself.
|
||||
TouchLastUsed(ctx context.Context, id uuid.UUID) error
|
||||
}
|
||||
|
||||
// PasswordHasher hashes and verifies passwords. The bcrypt adapter satisfies it.
|
||||
type PasswordHasher interface {
|
||||
Hash(password string) (string, error)
|
||||
Compare(hash, password string) error
|
||||
}
|
||||
|
||||
// TokenIssuer mints a signed access token for an authenticated user. It takes
|
||||
// the whole user so trust-bearing claims (role, verified) travel in the token.
|
||||
type TokenIssuer interface {
|
||||
Issue(u *domain.User) (string, error)
|
||||
}
|
||||
|
||||
// Clock reads the current time; a fake one makes tests deterministic.
|
||||
type Clock interface {
|
||||
Now() time.Time
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
package usecase
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
)
|
||||
|
||||
const (
|
||||
// minPasswordLen is a floor, not a policy engine — enough to reject the
|
||||
// obviously weak without pretending to measure real strength.
|
||||
minPasswordLen = 8
|
||||
// maxPasswordLen is bcrypt's hard input limit: it ignores bytes past 72, so
|
||||
// accepting a longer password would silently hash only its prefix.
|
||||
maxPasswordLen = 72
|
||||
)
|
||||
|
||||
// Register creates a new account: it validates the password, hashes it, builds
|
||||
// the domain user, and persists it.
|
||||
type Register struct {
|
||||
users UserRepository
|
||||
hasher PasswordHasher
|
||||
clk Clock
|
||||
}
|
||||
|
||||
func NewRegister(users UserRepository, hasher PasswordHasher, clk Clock) *Register {
|
||||
return &Register{users: users, hasher: hasher, clk: clk}
|
||||
}
|
||||
|
||||
// Execute registers email/password and returns the persisted user. The returned
|
||||
// user carries no plaintext password, only its hash.
|
||||
func (r *Register) Execute(ctx context.Context, email, password string) (*domain.User, error) {
|
||||
if len(password) < minPasswordLen {
|
||||
return nil, ErrPasswordTooShort
|
||||
}
|
||||
if len(password) > maxPasswordLen {
|
||||
return nil, ErrPasswordTooLong
|
||||
}
|
||||
|
||||
hash, err := r.hasher.Hash(password)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// NewUser normalises the email and enforces its shape; it returns a domain
|
||||
// validation error the transport layer maps to 400.
|
||||
u, err := domain.NewUser(email, hash, r.clk.Now())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := r.users.Insert(ctx, u); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return u, nil
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
package usecase
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
)
|
||||
|
||||
// SetRole promotes or demotes a user. Only an admin may call this (enforced in
|
||||
// the transport layer); the use case validates the target role and applies it.
|
||||
type SetRole struct {
|
||||
users UserRepository
|
||||
}
|
||||
|
||||
func NewSetRole(users UserRepository) *SetRole {
|
||||
return &SetRole{users: users}
|
||||
}
|
||||
|
||||
// Execute assigns role to the user, returning ErrInvalidRole for an unknown role
|
||||
// or ErrUserNotFound if the user does not exist.
|
||||
func (uc *SetRole) Execute(ctx context.Context, id uuid.UUID, role domain.Role) error {
|
||||
if !role.Valid() {
|
||||
return ErrInvalidRole
|
||||
}
|
||||
return uc.users.SetRole(ctx, id, role)
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
package usecase_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/auth"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/memstore"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/usecase"
|
||||
)
|
||||
|
||||
const secret = "usecase-test-secret-32-bytes-long!!!"
|
||||
|
||||
func newFixtures() (*usecase.Register, *usecase.Login, *memstore.UserRepo) {
|
||||
users := memstore.NewUserRepo()
|
||||
hasher := auth.NewHasher(4) // low cost keeps tests fast
|
||||
clk := memstore.Clock{T: time.Date(2026, 7, 26, 0, 0, 0, 0, time.UTC)}
|
||||
// The issuer uses the real clock (nil): token expiry is validated against
|
||||
// wall-clock time, so a fixed issue-time would make tokens instantly stale.
|
||||
issuer := auth.NewIssuer(secret, time.Hour, nil)
|
||||
|
||||
reg := usecase.NewRegister(users, hasher, clk)
|
||||
login := usecase.NewLogin(users, hasher, issuer)
|
||||
return reg, login, users
|
||||
}
|
||||
|
||||
func TestRegisterSuccess(t *testing.T) {
|
||||
reg, _, users := newFixtures()
|
||||
|
||||
u, err := reg.Execute(context.Background(), "Alice@Example.com", "password123")
|
||||
if err != nil {
|
||||
t.Fatalf("register: %v", err)
|
||||
}
|
||||
if u.Email != "alice@example.com" {
|
||||
t.Errorf("email not normalised: %q", u.Email)
|
||||
}
|
||||
if u.Role != domain.RoleUser {
|
||||
t.Errorf("role = %q, want user", u.Role)
|
||||
}
|
||||
if strings.Contains(u.PasswordHash, "password123") {
|
||||
t.Error("password stored in cleartext")
|
||||
}
|
||||
if _, err := users.GetByEmail(context.Background(), "alice@example.com"); err != nil {
|
||||
t.Errorf("user not persisted: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRegisterDuplicateEmail(t *testing.T) {
|
||||
reg, _, _ := newFixtures()
|
||||
ctx := context.Background()
|
||||
|
||||
if _, err := reg.Execute(ctx, "dup@example.com", "password123"); err != nil {
|
||||
t.Fatalf("first register: %v", err)
|
||||
}
|
||||
_, err := reg.Execute(ctx, "Dup@example.com", "password123") // different case, same email
|
||||
if !errors.Is(err, usecase.ErrEmailExists) {
|
||||
t.Errorf("got %v, want ErrEmailExists", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRegisterPasswordPolicy(t *testing.T) {
|
||||
reg, _, _ := newFixtures()
|
||||
ctx := context.Background()
|
||||
|
||||
if _, err := reg.Execute(ctx, "a@b.com", "short"); !errors.Is(err, usecase.ErrPasswordTooShort) {
|
||||
t.Errorf("short password: got %v", err)
|
||||
}
|
||||
long := strings.Repeat("x", 73)
|
||||
if _, err := reg.Execute(ctx, "a@b.com", long); !errors.Is(err, usecase.ErrPasswordTooLong) {
|
||||
t.Errorf("long password: got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRegisterInvalidEmail(t *testing.T) {
|
||||
reg, _, _ := newFixtures()
|
||||
_, err := reg.Execute(context.Background(), "not-an-email", "password123")
|
||||
if !errors.Is(err, domain.ErrInvalidEmail) {
|
||||
t.Errorf("got %v, want ErrInvalidEmail", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoginSuccess(t *testing.T) {
|
||||
reg, login, _ := newFixtures()
|
||||
ctx := context.Background()
|
||||
if _, err := reg.Execute(ctx, "user@example.com", "password123"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
token, u, err := login.Execute(ctx, "User@Example.com", "password123")
|
||||
if err != nil {
|
||||
t.Fatalf("login: %v", err)
|
||||
}
|
||||
if token == "" {
|
||||
t.Error("empty token")
|
||||
}
|
||||
if u.Email != "user@example.com" {
|
||||
t.Errorf("wrong user returned: %q", u.Email)
|
||||
}
|
||||
|
||||
// The token must verify and carry this user's id.
|
||||
claims, err := auth.NewIssuer(secret, time.Hour, nil).Verify(token)
|
||||
if err != nil {
|
||||
t.Fatalf("issued token does not verify: %v", err)
|
||||
}
|
||||
if claims.Subject != u.ID.String() {
|
||||
t.Errorf("token sub = %q, want %q", claims.Subject, u.ID.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoginWrongPassword(t *testing.T) {
|
||||
reg, login, _ := newFixtures()
|
||||
ctx := context.Background()
|
||||
if _, err := reg.Execute(ctx, "user@example.com", "password123"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_, _, err := login.Execute(ctx, "user@example.com", "wrongpass1")
|
||||
if !errors.Is(err, usecase.ErrInvalidCredentials) {
|
||||
t.Errorf("got %v, want ErrInvalidCredentials", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoginUnknownEmailIsIndistinguishable(t *testing.T) {
|
||||
_, login, _ := newFixtures()
|
||||
_, _, err := login.Execute(context.Background(), "ghost@example.com", "password123")
|
||||
if !errors.Is(err, usecase.ErrInvalidCredentials) {
|
||||
t.Errorf("unknown email must return ErrInvalidCredentials, got %v", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
package usecase
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// SetVerified grants or revokes a user's trusted-contributor badge. Only an
|
||||
// admin may call this (enforced in the transport layer); the use case itself
|
||||
// just applies the change.
|
||||
type SetVerified struct {
|
||||
users UserRepository
|
||||
}
|
||||
|
||||
func NewSetVerified(users UserRepository) *SetVerified {
|
||||
return &SetVerified{users: users}
|
||||
}
|
||||
|
||||
// Execute sets the verified flag on the target user, returning ErrUserNotFound
|
||||
// if the user does not exist.
|
||||
func (uc *SetVerified) Execute(ctx context.Context, id uuid.UUID, verified bool) error {
|
||||
return uc.users.SetVerified(ctx, id, verified)
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
package usecase_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/memstore"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/usecase"
|
||||
)
|
||||
|
||||
func TestSetVerifiedGrantsAndRevokes(t *testing.T) {
|
||||
reg, _, users := newFixtures()
|
||||
ctx := context.Background()
|
||||
|
||||
u, err := reg.Execute(ctx, "contrib@example.com", "password123")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if u.Verified {
|
||||
t.Fatal("a fresh account must be unverified")
|
||||
}
|
||||
|
||||
sv := usecase.NewSetVerified(users)
|
||||
|
||||
if err := sv.Execute(ctx, u.ID, true); err != nil {
|
||||
t.Fatalf("grant: %v", err)
|
||||
}
|
||||
got, _ := users.GetByID(ctx, u.ID)
|
||||
if !got.Verified {
|
||||
t.Error("verified flag not set")
|
||||
}
|
||||
|
||||
if err := sv.Execute(ctx, u.ID, false); err != nil {
|
||||
t.Fatalf("revoke: %v", err)
|
||||
}
|
||||
got, _ = users.GetByID(ctx, u.ID)
|
||||
if got.Verified {
|
||||
t.Error("verified flag not cleared")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetVerifiedUnknownUser(t *testing.T) {
|
||||
users := memstore.NewUserRepo()
|
||||
sv := usecase.NewSetVerified(users)
|
||||
|
||||
if err := sv.Execute(context.Background(), uuid.New(), true); !errors.Is(err, usecase.ErrUserNotFound) {
|
||||
t.Errorf("got %v, want ErrUserNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoginTokenCarriesVerified(t *testing.T) {
|
||||
reg, login, users := newFixtures()
|
||||
ctx := context.Background()
|
||||
|
||||
u, err := reg.Execute(ctx, "trusted@example.com", "password123")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := usecase.NewSetVerified(users).Execute(ctx, u.ID, true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_, loggedIn, err := login.Execute(ctx, "trusted@example.com", "password123")
|
||||
if err != nil {
|
||||
t.Fatalf("login: %v", err)
|
||||
}
|
||||
if !loggedIn.Verified {
|
||||
t.Error("login must reflect the granted verified flag")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
package usecase
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
)
|
||||
|
||||
// CreateWorkerKey mints a long-lived worker key for a user and returns the
|
||||
// one-time plaintext to show once.
|
||||
type CreateWorkerKey struct {
|
||||
keys WorkerKeyRepository
|
||||
clock Clock
|
||||
}
|
||||
|
||||
func NewCreateWorkerKey(keys WorkerKeyRepository, clock Clock) *CreateWorkerKey {
|
||||
return &CreateWorkerKey{keys: keys, clock: clock}
|
||||
}
|
||||
|
||||
// Execute returns the stored key (hash only) and the plaintext secret. The
|
||||
// secret is never persisted, so this is the sole moment it can be surfaced.
|
||||
func (uc *CreateWorkerKey) Execute(ctx context.Context, userID uuid.UUID, name string) (*domain.WorkerKey, string, error) {
|
||||
key, raw, err := domain.NewWorkerKey(userID, name, uc.clock.Now())
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
if err := uc.keys.Insert(ctx, key); err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
return key, raw, nil
|
||||
}
|
||||
|
||||
// ListWorkerKeys returns a user's live keys for display and management.
|
||||
type ListWorkerKeys struct {
|
||||
keys WorkerKeyRepository
|
||||
}
|
||||
|
||||
func NewListWorkerKeys(keys WorkerKeyRepository) *ListWorkerKeys {
|
||||
return &ListWorkerKeys{keys: keys}
|
||||
}
|
||||
|
||||
func (uc *ListWorkerKeys) Execute(ctx context.Context, userID uuid.UUID) ([]*domain.WorkerKey, error) {
|
||||
return uc.keys.ListByUser(ctx, userID)
|
||||
}
|
||||
|
||||
// RevokeWorkerKey retires one of the caller's keys.
|
||||
type RevokeWorkerKey struct {
|
||||
keys WorkerKeyRepository
|
||||
}
|
||||
|
||||
func NewRevokeWorkerKey(keys WorkerKeyRepository) *RevokeWorkerKey {
|
||||
return &RevokeWorkerKey{keys: keys}
|
||||
}
|
||||
|
||||
func (uc *RevokeWorkerKey) Execute(ctx context.Context, userID, id uuid.UUID) error {
|
||||
return uc.keys.Revoke(ctx, id, userID)
|
||||
}
|
||||
|
||||
// ExchangeWorkerKey trades a valid worker key for a short-lived JWT. The JWT
|
||||
// carries the owner's current role and verified flag, so a worker that refreshes
|
||||
// after an admin verifies the owner picks up the upgraded trust on its next
|
||||
// registration.
|
||||
type ExchangeWorkerKey struct {
|
||||
keys WorkerKeyRepository
|
||||
users UserRepository
|
||||
tokens TokenIssuer
|
||||
ttl time.Duration
|
||||
}
|
||||
|
||||
func NewExchangeWorkerKey(keys WorkerKeyRepository, users UserRepository, tokens TokenIssuer, ttl time.Duration) *ExchangeWorkerKey {
|
||||
return &ExchangeWorkerKey{keys: keys, users: users, tokens: tokens, ttl: ttl}
|
||||
}
|
||||
|
||||
// Execute returns a signed token and its lifetime in seconds. Every failure to
|
||||
// resolve the key to a usable owner collapses to ErrInvalidWorkerKey so a caller
|
||||
// cannot tell an unknown key from a revoked one or a deleted owner.
|
||||
func (uc *ExchangeWorkerKey) Execute(ctx context.Context, rawKey string) (string, int, error) {
|
||||
if rawKey == "" {
|
||||
return "", 0, ErrInvalidWorkerKey
|
||||
}
|
||||
|
||||
key, err := uc.keys.GetActiveByHash(ctx, domain.HashWorkerKey(rawKey))
|
||||
if err != nil {
|
||||
if errors.Is(err, ErrWorkerKeyNotFound) {
|
||||
return "", 0, ErrInvalidWorkerKey
|
||||
}
|
||||
return "", 0, err
|
||||
}
|
||||
|
||||
u, err := uc.users.GetByID(ctx, key.UserID)
|
||||
if err != nil {
|
||||
if errors.Is(err, ErrUserNotFound) {
|
||||
return "", 0, ErrInvalidWorkerKey
|
||||
}
|
||||
return "", 0, err
|
||||
}
|
||||
|
||||
token, err := uc.tokens.Issue(u)
|
||||
if err != nil {
|
||||
return "", 0, err
|
||||
}
|
||||
|
||||
// Best-effort: a failed timestamp update must not sink an otherwise valid
|
||||
// exchange the worker depends on to keep running.
|
||||
_ = uc.keys.TouchLastUsed(ctx, key.ID)
|
||||
|
||||
return token, int(uc.ttl.Seconds()), nil
|
||||
}
|
||||
@@ -0,0 +1,186 @@
|
||||
package usecase_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/auth"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/domain"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/memstore"
|
||||
"github.com/emil28092005/SciMesh/coordinator/internal/userservice/usecase"
|
||||
)
|
||||
|
||||
// fakeKeyRepo is an in-memory WorkerKeyRepository for the use-case tests.
|
||||
type fakeKeyRepo struct {
|
||||
byHash map[string]*domain.WorkerKey
|
||||
byID map[uuid.UUID]*domain.WorkerKey
|
||||
touched []uuid.UUID
|
||||
}
|
||||
|
||||
func newFakeKeyRepo() *fakeKeyRepo {
|
||||
return &fakeKeyRepo{byHash: map[string]*domain.WorkerKey{}, byID: map[uuid.UUID]*domain.WorkerKey{}}
|
||||
}
|
||||
|
||||
func (r *fakeKeyRepo) Insert(_ context.Context, k *domain.WorkerKey) error {
|
||||
r.byHash[k.TokenHash] = k
|
||||
r.byID[k.ID] = k
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *fakeKeyRepo) ListByUser(_ context.Context, userID uuid.UUID) ([]*domain.WorkerKey, error) {
|
||||
var out []*domain.WorkerKey
|
||||
for _, k := range r.byID {
|
||||
if k.UserID == userID && !k.Revoked() {
|
||||
out = append(out, k)
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (r *fakeKeyRepo) GetActiveByHash(_ context.Context, hash string) (*domain.WorkerKey, error) {
|
||||
k, ok := r.byHash[hash]
|
||||
if !ok || k.Revoked() {
|
||||
return nil, usecase.ErrWorkerKeyNotFound
|
||||
}
|
||||
return k, nil
|
||||
}
|
||||
|
||||
func (r *fakeKeyRepo) Revoke(_ context.Context, id, userID uuid.UUID) error {
|
||||
k, ok := r.byID[id]
|
||||
if !ok || k.UserID != userID || k.Revoked() {
|
||||
return usecase.ErrWorkerKeyNotFound
|
||||
}
|
||||
now := time.Now()
|
||||
k.RevokedAt = &now
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *fakeKeyRepo) TouchLastUsed(_ context.Context, id uuid.UUID) error {
|
||||
r.touched = append(r.touched, id)
|
||||
return nil
|
||||
}
|
||||
|
||||
func newKeyFixtures(t *testing.T) (*usecase.CreateWorkerKey, *usecase.ExchangeWorkerKey, *usecase.RevokeWorkerKey, *usecase.ListWorkerKeys, *fakeKeyRepo, *domain.User) {
|
||||
t.Helper()
|
||||
users := memstore.NewUserRepo()
|
||||
hasher := auth.NewHasher(4)
|
||||
clk := memstore.Clock{T: time.Date(2026, 7, 26, 0, 0, 0, 0, time.UTC)}
|
||||
issuer := auth.NewIssuer(secret, time.Hour, nil)
|
||||
keys := newFakeKeyRepo()
|
||||
|
||||
u, err := usecase.NewRegister(users, hasher, clk).Execute(context.Background(), "worker@example.com", "password123")
|
||||
if err != nil {
|
||||
t.Fatalf("seed user: %v", err)
|
||||
}
|
||||
|
||||
return usecase.NewCreateWorkerKey(keys, clk),
|
||||
usecase.NewExchangeWorkerKey(keys, users, issuer, time.Hour),
|
||||
usecase.NewRevokeWorkerKey(keys),
|
||||
usecase.NewListWorkerKeys(keys),
|
||||
keys, u
|
||||
}
|
||||
|
||||
func TestCreateAndExchangeWorkerKey(t *testing.T) {
|
||||
create, exchange, _, _, keys, u := newKeyFixtures(t)
|
||||
ctx := context.Background()
|
||||
|
||||
key, raw, err := create.Execute(ctx, u.ID, "home-desktop")
|
||||
if err != nil {
|
||||
t.Fatalf("create: %v", err)
|
||||
}
|
||||
if key.Name != "home-desktop" || raw == "" {
|
||||
t.Fatalf("unexpected key %+v raw=%q", key, raw)
|
||||
}
|
||||
|
||||
token, expiresIn, err := exchange.Execute(ctx, raw)
|
||||
if err != nil {
|
||||
t.Fatalf("exchange: %v", err)
|
||||
}
|
||||
if expiresIn != int((time.Hour).Seconds()) {
|
||||
t.Errorf("expires_in = %d, want 3600", expiresIn)
|
||||
}
|
||||
|
||||
claims, err := auth.NewIssuer(secret, time.Hour, nil).Verify(token)
|
||||
if err != nil {
|
||||
t.Fatalf("issued token does not verify: %v", err)
|
||||
}
|
||||
if claims.Subject != u.ID.String() {
|
||||
t.Errorf("token sub = %q, want owner %q", claims.Subject, u.ID)
|
||||
}
|
||||
if len(keys.touched) != 1 || keys.touched[0] != key.ID {
|
||||
t.Errorf("exchange did not record last-used, touched=%v", keys.touched)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExchangeUnknownKeyIsInvalid(t *testing.T) {
|
||||
_, exchange, _, _, _, _ := newKeyFixtures(t)
|
||||
if _, _, err := exchange.Execute(context.Background(), "scimesh_wk_live_nope"); !errors.Is(err, usecase.ErrInvalidWorkerKey) {
|
||||
t.Errorf("got %v, want ErrInvalidWorkerKey", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExchangeEmptyKeyIsInvalid(t *testing.T) {
|
||||
_, exchange, _, _, _, _ := newKeyFixtures(t)
|
||||
if _, _, err := exchange.Execute(context.Background(), ""); !errors.Is(err, usecase.ErrInvalidWorkerKey) {
|
||||
t.Errorf("got %v, want ErrInvalidWorkerKey", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExchangeRevokedKeyIsInvalid(t *testing.T) {
|
||||
create, exchange, revoke, _, _, u := newKeyFixtures(t)
|
||||
ctx := context.Background()
|
||||
|
||||
key, raw, err := create.Execute(ctx, u.ID, "laptop")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := revoke.Execute(ctx, u.ID, key.ID); err != nil {
|
||||
t.Fatalf("revoke: %v", err)
|
||||
}
|
||||
if _, _, err := exchange.Execute(ctx, raw); !errors.Is(err, usecase.ErrInvalidWorkerKey) {
|
||||
t.Errorf("revoked key still exchanges: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRevokeIsScopedToOwner(t *testing.T) {
|
||||
create, _, revoke, _, _, u := newKeyFixtures(t)
|
||||
ctx := context.Background()
|
||||
|
||||
key, _, err := create.Execute(ctx, u.ID, "laptop")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// A different user must not be able to revoke this key.
|
||||
if err := revoke.Execute(ctx, uuid.New(), key.ID); !errors.Is(err, usecase.ErrWorkerKeyNotFound) {
|
||||
t.Errorf("cross-owner revoke returned %v, want ErrWorkerKeyNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestListReturnsOnlyLiveKeys(t *testing.T) {
|
||||
create, _, revoke, list, _, u := newKeyFixtures(t)
|
||||
ctx := context.Background()
|
||||
|
||||
live, _, err := create.Execute(ctx, u.ID, "keep")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
dead, _, err := create.Execute(ctx, u.ID, "drop")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := revoke.Execute(ctx, u.ID, dead.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
got, err := list.Execute(ctx, u.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("list: %v", err)
|
||||
}
|
||||
if len(got) != 1 || got[0].ID != live.ID {
|
||||
t.Errorf("list = %d keys, want only the live one", len(got))
|
||||
}
|
||||
}
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
# SciMesh installer for Windows: downloads the coordinator binary for this
|
||||
# machine from the latest GitHub release and installs it under %LOCALAPPDATA%.
|
||||
#
|
||||
# powershell -ExecutionPolicy Bypass -c "irm https://raw.githubusercontent.com/emil28092005/SciMesh/main/install.ps1 | iex"
|
||||
#
|
||||
# Then run:
|
||||
#
|
||||
# coordinator serve --open
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
$Repo = "emil28092005/SciMesh"
|
||||
$Version = if ($env:SCIMESH_VERSION) { $env:SCIMESH_VERSION } else { "latest" }
|
||||
$InstallDir = if ($env:SCIMESH_INSTALL_DIR) {
|
||||
$env:SCIMESH_INSTALL_DIR
|
||||
} else {
|
||||
Join-Path $env:LOCALAPPDATA "SciMesh"
|
||||
}
|
||||
|
||||
$Arch = switch ($env:PROCESSOR_ARCHITECTURE) {
|
||||
"AMD64" { "amd64" }
|
||||
"ARM64" { "arm64" }
|
||||
default { throw "unsupported architecture: $env:PROCESSOR_ARCHITECTURE" }
|
||||
}
|
||||
|
||||
if ($Version -eq "latest") {
|
||||
Write-Host "Resolving the latest SciMesh release..."
|
||||
$Url = "https://github.com/$Repo/releases/latest/download/coordinator-windows-$Arch.exe"
|
||||
} else {
|
||||
$Url = "https://github.com/$Repo/releases/download/$Version/coordinator-windows-$Arch.exe"
|
||||
}
|
||||
|
||||
New-Item -ItemType Directory -Force -Path $InstallDir | Out-Null
|
||||
$Target = Join-Path $InstallDir "coordinator.exe"
|
||||
|
||||
Write-Host "Downloading $Url"
|
||||
Invoke-WebRequest -Uri $Url -OutFile "$Target.tmp"
|
||||
Move-Item -Force "$Target.tmp" $Target
|
||||
|
||||
Write-Host ""
|
||||
Write-Host "SciMesh installed: $Target"
|
||||
& $Target --version
|
||||
Write-Host ""
|
||||
Write-Host "Start the platform (one command, everything embedded):"
|
||||
Write-Host " $Target serve --open"
|
||||
Write-Host ""
|
||||
Write-Host "Your data lives in ~\.scimesh. The admin login is printed on first start."
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env bash
|
||||
# SciMesh installer: downloads the coordinator binary for this OS/architecture
|
||||
# from the latest GitHub release and installs it locally. One command, no
|
||||
# picking from a list of files:
|
||||
#
|
||||
# curl -fsSL https://raw.githubusercontent.com/emil28092005/SciMesh/main/install.sh | bash
|
||||
#
|
||||
# Installed to ~/.local/bin/coordinator (Linux/macOS). Then run:
|
||||
#
|
||||
# coordinator serve --open
|
||||
set -eu
|
||||
|
||||
REPO="emil28092005/SciMesh"
|
||||
VERSION="${SCIMESH_VERSION:-latest}"
|
||||
INSTALL_DIR="${SCIMESH_INSTALL_DIR:-$HOME/.local/bin}"
|
||||
|
||||
case "$(uname -s)" in
|
||||
Linux) OS="linux" ;;
|
||||
Darwin) OS="darwin" ;;
|
||||
*) echo "unsupported OS: $(uname -s)" >&2; exit 1 ;;
|
||||
esac
|
||||
|
||||
case "$(uname -m)" in
|
||||
x86_64|amd64) ARCH="amd64" ;;
|
||||
aarch64|arm64) ARCH="arm64" ;;
|
||||
*) echo "unsupported architecture: $(uname -m)" >&2; exit 1 ;;
|
||||
esac
|
||||
|
||||
if [ "$VERSION" = "latest" ]; then
|
||||
echo "Resolving the latest SciMesh release..."
|
||||
URL="https://github.com/${REPO}/releases/latest/download/coordinator-${OS}-${ARCH}"
|
||||
else
|
||||
URL="https://github.com/${REPO}/releases/download/${VERSION}/coordinator-${OS}-${ARCH}"
|
||||
fi
|
||||
|
||||
mkdir -p "$INSTALL_DIR"
|
||||
TARGET="$INSTALL_DIR/coordinator"
|
||||
|
||||
echo "Downloading $URL"
|
||||
curl -fsSL -o "$TARGET.tmp" "$URL"
|
||||
chmod +x "$TARGET.tmp"
|
||||
mv "$TARGET.tmp" "$TARGET"
|
||||
|
||||
echo
|
||||
echo "SciMesh installed: $TARGET"
|
||||
"$TARGET" --version
|
||||
echo
|
||||
echo "Start the platform (one command, everything embedded):"
|
||||
echo " $TARGET serve --open"
|
||||
echo
|
||||
echo "Your data lives in ~/.scimesh. The admin login is printed on first start."
|
||||
+35
-22
@@ -38,15 +38,32 @@ The two halves of the project:
|
||||
|
||||
## Quick start
|
||||
|
||||
The fastest path for a scientist: install the platform with one command and
|
||||
start it. Everything — the coordinator, its databases, the userservice, and
|
||||
local workers — is embedded in a single binary; no PostgreSQL, no Docker, no
|
||||
Python setup.
|
||||
|
||||
```bash
|
||||
# Linux / macOS
|
||||
curl -fsSL https://raw.githubusercontent.com/emil28092005/SciMesh/main/install.sh | bash
|
||||
coordinator serve --open
|
||||
|
||||
# Windows (PowerShell)
|
||||
powershell -ExecutionPolicy Bypass -c "irm https://raw.githubusercontent.com/emil28092005/SciMesh/main/install.ps1 | iex"
|
||||
coordinator serve --open
|
||||
```
|
||||
|
||||
The first start prints the admin login (also stored under `~/.scimesh`), and
|
||||
`--open` opens the UI in the browser. `coordinator serve --workers 2`
|
||||
spawns two local workers; `SCIMESH_PIP_PACKAGE` points the managed venv at
|
||||
your scimesh wheel so scientific workloads can run.
|
||||
|
||||
For development, install the Python SDK and run workloads locally:
|
||||
|
||||
```bash
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -e '.[dev]'
|
||||
```
|
||||
|
||||
List the installed SDK workloads and run one locally:
|
||||
|
||||
```bash
|
||||
scimesh workload list
|
||||
scimesh workload run molwt-filter \
|
||||
--input molecules.tsv \
|
||||
@@ -54,12 +71,6 @@ scimesh workload run molwt-filter \
|
||||
-o filtered.csv
|
||||
```
|
||||
|
||||
Run the local scientific CLI workloads:
|
||||
|
||||
```bash
|
||||
scimesh help
|
||||
```
|
||||
|
||||
Start the full demo (PostgreSQL, coordinator, UI, two workers):
|
||||
|
||||
```bash
|
||||
@@ -71,8 +82,8 @@ make demo-ui
|
||||
|
||||
Every `v*` tag pushes a GitHub Release with static binaries for
|
||||
`coordinator` and `worker-agent` on linux/darwin/windows × amd64/arm64
|
||||
(plus SHA-256 checksums) and the `coordinator` image on GHCR. Download and
|
||||
run:
|
||||
(plus SHA-256 checksums), the installer scripts above, and the `coordinator`
|
||||
image on GHCR. Download and run:
|
||||
|
||||
```bash
|
||||
curl -L -o coordinator https://github.com/emil28092005/SciMesh/releases/latest/download/coordinator-linux-amd64
|
||||
@@ -82,15 +93,17 @@ chmod +x coordinator
|
||||
- **worker-agent** runs anywhere with Python: it spawns
|
||||
`python -m scimesh.worker.task`, so the machine needs the `scimesh`
|
||||
package in a venv (`pip install scimesh`) and the usual environment:
|
||||
`COORDINATOR_URL`, `WORKER_AUTH_TOKEN`, `WORK_DIR`.
|
||||
- **coordinator** needs PostgreSQL running (`DATABASE_URL`,
|
||||
`COORDINATOR_STORAGE_DIR`, `JWT_SECRET`); the binary applies its embedded
|
||||
schema migrations itself on startup (`AUTO_MIGRATE=false` opts out), so no
|
||||
separate migration step is needed. The interactive wizard provisions the
|
||||
rest: it checks the database, creates it when missing, generates a
|
||||
`JWT_SECRET`, and writes a `.env` file (`coordinator setup --help`, or
|
||||
`make setup`; `--yes` for non-interactive runs). The UI login additionally
|
||||
requires `USERSERVICE_URL`.
|
||||
`COORDINATOR_URL`, `WORKER_AUTH_TOKEN`, `WORK_DIR`. The same binary can run
|
||||
it via `coordinator agent`.
|
||||
- **coordinator** needs no external services at all in its default mode:
|
||||
`coordinator serve` embeds SQLite (both databases), the userservice, and
|
||||
local workers. The `SCIMESH_DB=postgres` engine remains for cluster
|
||||
deployments (`DATABASE_URL`, `COORDINATOR_STORAGE_DIR`, `JWT_SECRET`); the
|
||||
binary applies its embedded schema migrations itself on startup
|
||||
(`AUTO_MIGRATE=false` opts out), and `coordinator setup` provisions a
|
||||
PostgreSQL deployment interactively. The UI login uses the userservice
|
||||
(`USERSERVICE_URL`) — embedded by `serve`, or the standalone `users/`
|
||||
service otherwise.
|
||||
`coordinator --version` / `worker-agent --version` print the build tag.
|
||||
|
||||
Build and serve this documentation site:
|
||||
|
||||
Reference in New Issue
Block a user