Document the admin console and the worker setup wizard
This commit is contained in:
@@ -1066,10 +1066,14 @@ workload enable/disable, settings (read-only), and storage metrics.
|
|||||||
**Depends on:** CTX-11 (UI patterns, session/roles), CTX-15 (userservice
|
**Depends on:** CTX-11 (UI patterns, session/roles), CTX-15 (userservice
|
||||||
proxy), the sqlite/postgres storage pair.
|
proxy), the sqlite/postgres storage pair.
|
||||||
|
|
||||||
**Acceptance criteria:** full plan in
|
**Status: implemented.** Admin console at `/ui/admin`: system/storage/health,
|
||||||
[`docs/ui-admin-worker-plan.md`](docs/ui-admin-worker-plan.md) (section 3);
|
jobs (filter + pagination + owner resolution), workers with trust controls,
|
||||||
all `/ui/admin/*` routes are admin-only, bounded read model, both engines
|
users & worker keys (userservice admin endpoints), workload enable/disable
|
||||||
supported, unit + permission + integration tests green.
|
(`workload_settings` migration in both engines, enforced at submit time and
|
||||||
|
hidden from the job form), metrics (7-day buckets, failure rate), and
|
||||||
|
settings with an audited worker-token reveal. All `/ui/admin/*` routes are
|
||||||
|
admin-only and backed by bounded read models; sqlite + postgres parity;
|
||||||
|
unit/permission/integration tests green.
|
||||||
|
|
||||||
### CTX-20 — Worker Setup UI
|
### CTX-20 — Worker Setup UI
|
||||||
|
|
||||||
@@ -1081,6 +1085,15 @@ have only the worker installed — no coordinator needed.
|
|||||||
|
|
||||||
**Depends on:** the agent daemon; the worker-key exchange (CTX-15).
|
**Depends on:** the agent daemon; the worker-key exchange (CTX-15).
|
||||||
|
|
||||||
|
**Status: implemented.** `worker-agent setup` serves the local wizard on
|
||||||
|
127.0.0.1 (default port 12700): coordinator URL + token or worker key,
|
||||||
|
work dir and name, preflight check (coordinator/health, python3, scimesh),
|
||||||
|
config saved to `~/.scimesh-worker/config.json` (0600), start/stop of the
|
||||||
|
worker as a background process, and a live status page with the agent log.
|
||||||
|
The daemon also gained `--config <path>` (environment still wins) and
|
||||||
|
`--check`. End-to-end verified: the wizard started a real worker that
|
||||||
|
registered with a coordinator and appeared in the admin console.
|
||||||
|
|
||||||
**Acceptance criteria:** full plan in
|
**Acceptance criteria:** full plan in
|
||||||
[`docs/ui-admin-worker-plan.md`](docs/ui-admin-worker-plan.md) (section 4);
|
[`docs/ui-admin-worker-plan.md`](docs/ui-admin-worker-plan.md) (section 4);
|
||||||
the agent gains `setup`, `--config <path>`, and `--check`; end-to-end: the
|
the agent gains `setup`, `--config <path>`, and `--check`; end-to-end: the
|
||||||
|
|||||||
@@ -70,6 +70,16 @@ PostgreSQL, no Docker, no environment variables. The scientific runtime is a
|
|||||||
managed venv (`~/.scimesh/venv`); point `SCIMESH_PIP_PACKAGE` at your scimesh
|
managed venv (`~/.scimesh/venv`); point `SCIMESH_PIP_PACKAGE` at your scimesh
|
||||||
wheel to install it automatically.
|
wheel to install it automatically.
|
||||||
|
|
||||||
|
The coordinator serves two operator surfaces: the **control room** (jobs,
|
||||||
|
workloads, docs) and the **admin console** at `/ui/admin` — cluster health
|
||||||
|
and storage, paginated job table, worker fleet with trust controls, users and
|
||||||
|
worker keys, workload enable/disable, metrics and the worker token. The
|
||||||
|
**worker** binary (`worker-agent`) carries its own local setup wizard for
|
||||||
|
machines that run only a worker: `worker-agent setup` opens a browser wizard
|
||||||
|
at `127.0.0.1` that collects the coordinator URL and credential, runs a
|
||||||
|
preflight check, saves `~/.scimesh-worker/config.json` and starts/stops the
|
||||||
|
worker with a live log (see the [standalone docs](mkdocs/standalone.md)).
|
||||||
|
|
||||||
Manual download and run of a release binary:
|
Manual download and run of a release binary:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# План реализации: Coordinator Admin UI и Worker Setup UI
|
# План реализации: Coordinator Admin UI и Worker Setup UI
|
||||||
|
|
||||||
> Статус: **в работе**. Визуал утверждён — мокапы:
|
> Статус: **реализовано (M1–M3), E2E проверено**. Визуал утверждён — мокапы:
|
||||||
> [`ui-mockups/coordinator-admin.html`](ui-mockups/coordinator-admin.html),
|
> [`ui-mockups/coordinator-admin.html`](ui-mockups/coordinator-admin.html),
|
||||||
> [`ui-mockups/worker-setup.html`](ui-mockups/worker-setup.html).
|
> [`ui-mockups/worker-setup.html`](ui-mockups/worker-setup.html).
|
||||||
> Два UI живут в **разных бинарниках**: админка кластера — в `coordinator`
|
> Два UI живут в **разных бинарниках**: админка кластера — в `coordinator`
|
||||||
@@ -145,22 +145,22 @@ Spawner — интерфейс, в тестах подменяется.
|
|||||||
|
|
||||||
## 5. Милестоуны
|
## 5. Милестоуны
|
||||||
|
|
||||||
- **M1. Admin-фундамент**: оболочка `admin.html` по мокапу + `GET /system`
|
- **M1. Admin-фундамент** — ✅ done: оболочка `admin.html` по мокапу +
|
||||||
(storage stats, health, node) + `GET /jobs` (фильтр+пагинация+счётчики) +
|
`GET /system` (storage stats, health, node) + `GET /jobs`
|
||||||
`GET /metrics`. Тесты: pagination/metrics/storage-stats на sqlite,
|
(фильтр+пагинация+счётчики) + `GET /metrics`; тесты на sqlite, 403 для
|
||||||
permission-тесты 403.
|
не-админа.
|
||||||
- **M2. Access & Platform**: userservice `ListUsers`/`ListWorkerKeysAll`
|
- **M2. Access & Platform** — ✅ done: userservice `ListUsers`/
|
||||||
(mem+sqlite+http), `SetTrust` (оба движка), users/keys/trust API и
|
`ListWorkerKeysAll` + admin revoke (mem+sqlite+http), `SetTrust` (оба
|
||||||
разделы, `workload_settings` миграция 0002 + enable/disable + enforcement
|
движка), users/keys/trust разделы, `workload_settings` (миграции 0002/
|
||||||
в `SubmitDataset` и форме, settings + token reveal (audit). Тесты на каждый
|
0014) + enforcement в `SubmitDataset` и форме, settings + audited token
|
||||||
слой.
|
reveal. Все секции админки на живых данных.
|
||||||
- **M3. Worker Setup**: `configfile.go`, `--config`, `--check`, `setupui`
|
- **M3. Worker Setup** — ✅ done: `configfile.go` (0600), `--config` (env
|
||||||
(server + шаблон по мокапу, 6 API), тесты (roundtrip конфига, права 0600,
|
wins), `--check`, `setupui` (сервер на 127.0.0.1, 6 API, шаблон по мокапу),
|
||||||
check с httptest, API визарда с подменённым spawner).
|
тесты (roundtrip, права, check, API с fake supervisor).
|
||||||
- **M4. E2E + docs**: браузерный E2E (admin видит реальные данные; визард
|
- **M4. E2E + docs** — ✅ done: браузерный E2E (admin видит реальные
|
||||||
запускает реального воркера против тестового координатора → регистрация →
|
данные; визард запустил реального воркера → `wizard-machine online` в
|
||||||
джоб), обновить `mkdocs/index.md`, `mkdocs/standalone.md`
|
админке), обновлены `mkdocs/index.md` и `mkdocs/standalone.md`
|
||||||
(`worker-agent setup` вместо ручных export), README, чекбоксы CTX-19/20.
|
(`worker-agent setup`), README, статусы CTX-19/CTX-20 в `PLAN.md`.
|
||||||
|
|
||||||
## 6. Тестирование
|
## 6. Тестирование
|
||||||
|
|
||||||
|
|||||||
+16
-3
@@ -33,8 +33,10 @@ The two halves of the project:
|
|||||||
resource reservation, and allowlist-driven workload discovery.
|
resource reservation, and allowlist-driven workload discovery.
|
||||||
- **An operator UI** served by the coordinator: the control room, a workload
|
- **An operator UI** served by the coordinator: the control room, a workload
|
||||||
library page, a workload-agnostic "new computation" form whose controls come
|
library page, a workload-agnostic "new computation" form whose controls come
|
||||||
from each workload's own `UIElement` declarations, and this documentation
|
from each workload's own `UIElement` declarations, an **admin console**
|
||||||
site at `/ui/docs/`.
|
(`/ui/admin`) for cluster operators — system/storage/health, jobs,
|
||||||
|
worker trust, users and worker keys, workload enable/disable, metrics and
|
||||||
|
the worker token — and this documentation site at `/ui/docs/`.
|
||||||
|
|
||||||
## Quick start
|
## Quick start
|
||||||
|
|
||||||
@@ -90,10 +92,21 @@ curl -L -o coordinator https://github.com/emil28092005/SciMesh/releases/latest/d
|
|||||||
chmod +x coordinator
|
chmod +x coordinator
|
||||||
```
|
```
|
||||||
|
|
||||||
- **worker-agent** is installed separately and joins an existing coordinator:
|
- **worker-agent** is installed separately and joins an existing coordinator.
|
||||||
|
Point its **local setup wizard** at the cluster — no need to have the
|
||||||
|
coordinator on this machine:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
curl -fsSL https://raw.githubusercontent.com/emil28092005/SciMesh/main/install.sh | bash -s worker
|
curl -fsSL https://raw.githubusercontent.com/emil28092005/SciMesh/main/install.sh | bash -s worker
|
||||||
|
worker-agent setup # opens http://127.0.0.1:12700 in your browser
|
||||||
|
```
|
||||||
|
|
||||||
|
The wizard collects the coordinator URL and token (or worker key), runs a
|
||||||
|
preflight check, saves the configuration under `~/.scimesh-worker/` and
|
||||||
|
starts the worker as a background process — with a live status page and log.
|
||||||
|
Everything can also be done by hand:
|
||||||
|
|
||||||
|
```bash
|
||||||
export COORDINATOR_URL=http://COORDINATOR_HOST:8080
|
export COORDINATOR_URL=http://COORDINATOR_HOST:8080
|
||||||
export WORKER_AUTH_TOKEN=<worker token from the coordinator>
|
export WORKER_AUTH_TOKEN=<worker token from the coordinator>
|
||||||
export WORK_DIR=~/scimesh-worker
|
export WORK_DIR=~/scimesh-worker
|
||||||
|
|||||||
+14
-3
@@ -50,10 +50,18 @@ BOOTSTRAP_ADMIN_PASSWORD='choose-a-strong-password' \
|
|||||||
docker compose -f docker-compose.yml -f docker-compose.users.yml up -d --build
|
docker compose -f docker-compose.yml -f docker-compose.users.yml up -d --build
|
||||||
```
|
```
|
||||||
|
|
||||||
Then install workers on any machines with Python:
|
Then install workers on any machines with Python. The worker's own setup
|
||||||
|
wizard walks through the rest — URL, token or worker key, work directory —
|
||||||
|
and starts the worker for you:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
curl -fsSL https://raw.githubusercontent.com/emil28092005/SciMesh/main/install.sh | bash -s worker
|
curl -fsSL https://raw.githubusercontent.com/emil28092005/SciMesh/main/install.sh | bash -s worker
|
||||||
|
worker-agent setup # local wizard at http://127.0.0.1:12700
|
||||||
|
```
|
||||||
|
|
||||||
|
Or configure by hand:
|
||||||
|
|
||||||
|
```bash
|
||||||
export COORDINATOR_URL=http://COORDINATOR_HOST:8080
|
export COORDINATOR_URL=http://COORDINATOR_HOST:8080
|
||||||
export WORKER_AUTH_TOKEN="$COORDINATOR_TOKEN" # the coordinator's shared token
|
export WORKER_AUTH_TOKEN="$COORDINATOR_TOKEN" # the coordinator's shared token
|
||||||
export WORK_DIR=~/scimesh-worker
|
export WORK_DIR=~/scimesh-worker
|
||||||
@@ -112,8 +120,11 @@ worker-agent
|
|||||||
The binary applies its embedded schema migrations on startup
|
The binary applies its embedded schema migrations on startup
|
||||||
(`AUTO_MIGRATE=false` to disable when you manage them out of band).
|
(`AUTO_MIGRATE=false` to disable when you manage them out of band).
|
||||||
|
|
||||||
4. **Attach workers** as in Option A. The UI login uses the userservice
|
4. **Attach workers** as in Option A (or via `worker-agent setup`). The UI
|
||||||
session; the workers use `COORDINATOR_TOKEN`.
|
login uses the userservice session; the workers use `COORDINATOR_TOKEN` or
|
||||||
|
a worker key. The coordinator's admin console (`/ui/admin`) shows the
|
||||||
|
whole cluster: jobs, worker fleet with trust controls, accounts and keys,
|
||||||
|
workload switches and metrics.
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user