Compare commits

...
Author SHA1 Message Date
Emil 15dd9651a8 Ship the scimesh wheel in releases; wizard and serve install it version-locked
coordinator / test (push) Waiting to run
python / test (push) Waiting to run
release / binaries (amd64, darwin) (push) Waiting to run
release / binaries (amd64, linux) (push) Waiting to run
release / binaries (amd64, windows) (push) Waiting to run
release / binaries (arm64, darwin) (push) Waiting to run
release / binaries (arm64, linux) (push) Waiting to run
release / binaries (arm64, windows) (push) Waiting to run
release / wheel (push) Waiting to run
release / release (push) Blocked by required conditions
release / image (push) Waiting to run
users / test (push) Waiting to run
2026-08-03 02:59:22 +03:00
Emil 30c441a7e9 Require an explicit scimesh source in the worker wizard (PyPI name is another project)
coordinator / test (push) Waiting to run
python / test (push) Waiting to run
release / binaries (amd64, darwin) (push) Waiting to run
release / binaries (amd64, linux) (push) Waiting to run
release / binaries (amd64, windows) (push) Waiting to run
release / binaries (arm64, darwin) (push) Waiting to run
release / binaries (arm64, linux) (push) Waiting to run
release / binaries (arm64, windows) (push) Waiting to run
release / release (push) Blocked by required conditions
release / image (push) Waiting to run
users / test (push) Waiting to run
2026-08-03 02:39:24 +03:00
Emil cb51172885 Let the worker wizard install the scimesh runtime into a managed venv
coordinator / test (push) Waiting to run
python / test (push) Waiting to run
release / binaries (amd64, darwin) (push) Waiting to run
release / binaries (amd64, linux) (push) Waiting to run
release / binaries (amd64, windows) (push) Waiting to run
release / binaries (arm64, darwin) (push) Waiting to run
release / binaries (arm64, linux) (push) Waiting to run
release / binaries (arm64, windows) (push) Waiting to run
release / release (push) Blocked by required conditions
release / image (push) Waiting to run
users / test (push) Waiting to run
2026-08-03 02:32:06 +03:00
Emil 029b26e6ae Explain the admin-required error and offer account switching on the login page
coordinator / test (push) Waiting to run
python / test (push) Waiting to run
release / binaries (amd64, darwin) (push) Waiting to run
release / binaries (amd64, linux) (push) Waiting to run
release / binaries (amd64, windows) (push) Waiting to run
release / binaries (arm64, darwin) (push) Waiting to run
release / binaries (arm64, linux) (push) Waiting to run
release / binaries (arm64, windows) (push) Waiting to run
release / release (push) Blocked by required conditions
release / image (push) Waiting to run
users / test (push) Waiting to run
2026-08-03 02:06:45 +03:00
Emil 12499d2d7b Remove the demo control room; /ui lands on the admin console
coordinator / test (push) Waiting to run
python / test (push) Waiting to run
release / binaries (amd64, darwin) (push) Waiting to run
release / binaries (amd64, linux) (push) Waiting to run
release / binaries (amd64, windows) (push) Waiting to run
release / binaries (arm64, darwin) (push) Waiting to run
release / binaries (arm64, linux) (push) Waiting to run
release / binaries (arm64, windows) (push) Waiting to run
release / release (push) Blocked by required conditions
release / image (push) Waiting to run
users / test (push) Waiting to run
2026-08-03 01:58:02 +03:00
Emil c4d88c7ffc Satisfy the linter gate: errors.Is, context-aware exec/listen, gosec nolints
coordinator / test (push) Waiting to run
python / test (push) Waiting to run
release / binaries (amd64, darwin) (push) Waiting to run
release / binaries (amd64, linux) (push) Waiting to run
release / binaries (amd64, windows) (push) Waiting to run
release / binaries (arm64, darwin) (push) Waiting to run
release / binaries (arm64, linux) (push) Waiting to run
release / binaries (arm64, windows) (push) Waiting to run
release / release (push) Blocked by required conditions
release / image (push) Waiting to run
users / test (push) Waiting to run
2026-08-03 01:47:53 +03:00
Emil 44247bd94e Give every wizard test its own port and disable keep-alive pooling
coordinator / test (push) Waiting to run
python / test (push) Waiting to run
release / binaries (amd64, darwin) (push) Waiting to run
release / binaries (amd64, linux) (push) Waiting to run
release / binaries (amd64, windows) (push) Waiting to run
release / binaries (arm64, darwin) (push) Waiting to run
release / binaries (arm64, linux) (push) Waiting to run
release / binaries (arm64, windows) (push) Waiting to run
release / release (push) Blocked by required conditions
release / image (push) Waiting to run
users / test (push) Waiting to run
2026-08-03 01:43:48 +03:00
Emil 7c1d0dc568 Tell non-admins why the console is off-limits instead of bouncing them to the dashboard
coordinator / test (push) Waiting to run
python / test (push) Waiting to run
release / binaries (amd64, darwin) (push) Waiting to run
release / binaries (amd64, linux) (push) Waiting to run
release / binaries (amd64, windows) (push) Waiting to run
release / binaries (arm64, darwin) (push) Waiting to run
release / binaries (arm64, linux) (push) Waiting to run
release / binaries (arm64, windows) (push) Waiting to run
release / release (push) Blocked by required conditions
release / image (push) Waiting to run
users / test (push) Waiting to run
2026-08-03 01:34:22 +03:00
40 changed files with 656 additions and 534 deletions
+37 -1
View File
@@ -54,8 +54,39 @@ jobs:
path: coordinator/dist/*
if-no-files-found: error
wheel:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: build the scimesh wheel
env:
VERSION: ${{ github.ref_name }}
run: |
# The tag (v1.1.0-alpha.10) becomes the package version in its
# PEP 440 form (1.1.0a10); the wheel is then version-locked to the
# binaries of the same release.
WHEEL_VERSION="${VERSION#v}"
WHEEL_VERSION="${WHEEL_VERSION/-alpha./a}"
WHEEL_VERSION="${WHEEL_VERSION/-beta./b}"
WHEEL_VERSION="${WHEEL_VERSION/-rc./rc}"
sed -i "s/^version = .*/version = \"${WHEEL_VERSION}\"/" pyproject.toml
python -m pip install --quiet build
python -m build --wheel --outdir dist
ls -la dist/
- uses: actions/upload-artifact@v4
with:
name: wheel
path: dist/*.whl
if-no-files-found: error
release:
needs: binaries
needs: [binaries, wheel]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
@@ -68,6 +99,11 @@ jobs:
pattern: binaries-*
merge-multiple: true
- uses: actions/download-artifact@v4
with:
name: wheel
path: artifacts
- name: checksums
working-directory: artifacts
run: sha256sum * > SHA256SUMS.txt
+7 -6
View File
@@ -60,7 +60,8 @@ curl -fsSL https://raw.githubusercontent.com/emil28092005/SciMesh/main/install.s
powershell -ExecutionPolicy Bypass -c "irm https://raw.githubusercontent.com/emil28092005/SciMesh/main/install.ps1 | iex"
```
Set `SCIMESH_AUTO_START=0` to install without starting anything. A standalone
Set `SCIMESH_AUTO_START=0` to install without starting anything. The old demo
control room was removed: `/ui` is the admin console. A standalone
worker is installed the same way (`bash -s worker`, or
`SCIMESH_COMPONENT=worker` on Windows); its installer opens the local setup
wizard (`worker-agent setup`) in the browser automatically.
@@ -73,12 +74,12 @@ PostgreSQL, no Docker, no environment variables. The scientific runtime is a
managed venv (`~/.scimesh/venv`); point `SCIMESH_PIP_PACKAGE` at your scimesh
wheel to install it automatically.
The coordinator serves two operator surfaces: the **control room** (jobs,
workloads, docs) and the **admin console** at `/ui/admin` — cluster health
The coordinator's UI is the **admin console** at `/ui/admin` — cluster health
and storage, paginated job table, worker fleet with trust controls, users and
worker keys, workload enable/disable, metrics and the worker token
(`serve --open` lands on the admin console; login returns you to the page you
asked for). The **worker** binary (`worker-agent`) carries its own local setup
worker keys, workload enable/disable, metrics and the worker token. The job
form (`/ui/jobs/new`), job detail pages and the workload library complete the
operator surface; `/ui` redirects to the console, `serve --open` lands on it,
and login returns you to the page you asked for. The **worker** binary (`worker-agent`) carries its own local setup
wizard for machines that run only a worker: `worker-agent setup` opens a
browser wizard at `127.0.0.1` that collects the coordinator URL and
credential, runs a preflight check, saves `~/.scimesh-worker/config.json` and
+20 -5
View File
@@ -2,6 +2,9 @@ package main
import (
"context"
"github.com/emil28092005/SciMesh/coordinator/internal/agent"
"crypto/rand"
"encoding/hex"
"flag"
@@ -259,13 +262,25 @@ func ensureRuntime(log *slog.Logger, dataDir, venvPython string) {
}
pip := filepath.Join(venvDir, binName("bin/pip"))
// The scimesh package is installed from an explicit source only: the PyPI
// name is not ours yet, so `pip install scimesh` would fetch a stranger's
// package. Operators publish a wheel or index via SCIMESH_PIP_PACKAGE.
// name belongs to an unrelated project, so `pip install scimesh` would
// fetch a stranger's package. Default: download the wheel attached to our
// own GitHub release for this binary version; SCIMESH_PIP_PACKAGE
// overrides with a custom wheel, checkout or index.
source := os.Getenv("SCIMESH_PIP_PACKAGE")
if source == "" {
log.Warn("scientific runtime venv created, but scimesh is not installed",
"hint", pip+" install <your scimesh wheel or index> (or set SCIMESH_PIP_PACKAGE)")
return
url, _, err := agent.ReleaseWheelURL(version)
if err != nil {
log.Warn("scientific runtime venv created, but scimesh is not installed",
"hint", "set SCIMESH_PIP_PACKAGE to your wheel or index")
return
}
downloaded, err := agent.DownloadWheel(context.Background(), url, venvDir)
if err != nil {
log.Warn("could not download the scimesh wheel for this release",
"err", err, "hint", "set SCIMESH_PIP_PACKAGE to your wheel or index")
return
}
source = downloaded
}
// #nosec G204,G702 -- pip and source are operator-configured paths.
install := exec.CommandContext(context.Background(), pip, "install", source)
+5 -3
View File
@@ -7,6 +7,7 @@ package main
import (
"context"
"errors"
"flag"
"fmt"
"log/slog"
@@ -93,7 +94,7 @@ func loadConfig(configPath string) (*agent.Config, error) {
}
envPath := os.Getenv("SCIMESH_WORKER_CONFIG")
if envPath != "" {
if _, err := os.Stat(envPath); err == nil {
if _, err := os.Stat(envPath); err == nil { //nolint:gosec // G703: path is the operator's own env var
return agent.LoadConfigFile(envPath)
}
}
@@ -153,7 +154,7 @@ func runSetup(args []string) int {
// Block until the signal arrives (never returns an error that matters: a
// cancelled context is the normal exit path).
err = server.Serve(ctx, listener)
if err != nil && err != http.ErrServerClosed {
if err != nil && !errors.Is(err, http.ErrServerClosed) {
logger.Error("setup wizard stopped", "err", err)
return 1
}
@@ -172,7 +173,8 @@ func openBrowser(url string) {
if err != nil {
continue
}
_ = exec.Command(binary, candidate[1:]...).Start()
//nolint:gosec // G204: candidates are our own fixed list; the url is a loopback literal
_ = exec.CommandContext(context.Background(), binary, candidate[1:]...).Start()
return
}
}
+6 -1
View File
@@ -80,10 +80,15 @@ func CheckEnvironment(ctx context.Context) CheckReport {
return report
}
report.Python = CheckItem{Name: "python", OK: true, Detail: python}
//nolint:gosec // G204: python comes from LookPath, the argument list is constant
cmd := exec.CommandContext(ctx, python, "-c", "import scimesh; print(scimesh.__version__ if hasattr(scimesh, '__version__') else 'installed')")
out, err := cmd.Output()
if err != nil {
report.Scimesh = CheckItem{Name: "scimesh", OK: false, Detail: "install with: pip install scimesh"}
// The worker executes workloads by spawning scimesh's task runner, so
// the package is a hard requirement, not an optimisation. The PyPI
// name belongs to a different project, so the wizard installs from
// SCIMESH_PIP_PACKAGE instead of suggesting a bare pip install.
report.Scimesh = CheckItem{Name: "scimesh", OK: false, Detail: "the worker runs workloads through scimesh — install it from your wheel or index (SCIMESH_PIP_PACKAGE)"}
return report
}
report.Scimesh = CheckItem{Name: "scimesh", OK: true, Detail: strings.TrimSpace(string(out))}
+1
View File
@@ -41,6 +41,7 @@ func DefaultConfigPath() string {
// created by the wizard with 0600 permissions, so no credential is exposed to
// other local users.
func LoadConfigFile(path string) (*Config, error) {
//nolint:gosec // G304: path is --config or SCIMESH_WORKER_CONFIG, operator-supplied
raw, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("read config file: %w", err)
@@ -0,0 +1,70 @@
package agent
import (
"context"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"strings"
"time"
)
// ReleaseWheelURL returns the download URL of the scimesh wheel attached to
// the GitHub release that matches the given binary version (for example
// "1.1.0-alpha.10"), plus the wheel file name. The wheel is version-locked to
// the binary so a worker's catalog always matches its task runner.
func ReleaseWheelURL(version string) (string, string, error) {
if version == "" || version == "dev" {
return "", "", fmt.Errorf("no release wheel for build %q", version)
}
filename := fmt.Sprintf("scimesh-%s-py3-none-any.whl", NormalizePEP440(version))
return fmt.Sprintf("https://github.com/emil28092005/SciMesh/releases/download/v%s/%s", version, filename), filename, nil
}
// NormalizePEP440 turns our release tag suffixes into the PEP 440 form
// setuptools uses for wheel names: 1.1.0-alpha.10 -> 1.1.0a10,
// 1.1.0-beta.2 -> 1.1.0b2, 1.1.0-rc.1 -> 1.1.0rc1. Stable tags pass through.
func NormalizePEP440(version string) string {
for from, to := range map[string]string{"-alpha.": "a", "-beta.": "b", "-rc.": "rc"} {
version = strings.ReplaceAll(version, from, to)
}
return version
}
// DownloadWheel fetches the release wheel into dir (config directory of the
// wizard / serve data dir) and returns the local path. Best-effort download
// with a generous timeout: wheels can be several MB.
func DownloadWheel(ctx context.Context, url, dir string) (string, error) {
target := filepath.Join(dir, wheelNameFromURL(url))
ctx, cancel := context.WithTimeout(ctx, 10*time.Minute)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return "", err
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
return "", fmt.Errorf("download wheel: %w", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
return "", fmt.Errorf("download wheel: HTTP %d", resp.StatusCode)
}
//nolint:gosec // G304: target is our own config dir + a fixed wheel name
out, err := os.Create(target)
if err != nil {
return "", fmt.Errorf("download wheel: %w", err)
}
defer func() { _ = out.Close() }()
if _, err := io.Copy(out, resp.Body); err != nil {
return "", fmt.Errorf("download wheel: %w", err)
}
return target, nil
}
// wheelNameFromURL extracts the trailing file name of a wheel URL.
func wheelNameFromURL(url string) string {
return url[strings.LastIndex(url, "/")+1:]
}
@@ -0,0 +1,81 @@
package agent
import (
"context"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
)
func TestNormalizePEP440(t *testing.T) {
cases := map[string]string{
"1.1.0": "1.1.0",
"1.1.0-alpha.10": "1.1.0a10",
"1.1.0-beta.2": "1.1.0b2",
"1.1.0-rc.1": "1.1.0rc1",
"1.0.0": "1.0.0",
}
for in, want := range cases {
if got := NormalizePEP440(in); got != want {
t.Errorf("NormalizePEP440(%q) = %q, want %q", in, got, want)
}
}
}
func TestReleaseWheelURL(t *testing.T) {
url, name, err := ReleaseWheelURL("1.1.0-alpha.10")
if err != nil {
t.Fatal(err)
}
wantURL := "https://github.com/emil28092005/SciMesh/releases/download/v1.1.0-alpha.10/scimesh-1.1.0a10-py3-none-any.whl"
if url != wantURL {
t.Errorf("url = %q, want %q", url, wantURL)
}
if name != "scimesh-1.1.0a10-py3-none-any.whl" {
t.Errorf("name = %q", name)
}
// A dev build has no release wheel.
if _, _, err := ReleaseWheelURL("dev"); err == nil {
t.Error("dev build must not resolve a release wheel")
}
if _, _, err := ReleaseWheelURL(""); err == nil {
t.Error("empty version must not resolve a release wheel")
}
}
func TestDownloadWheel(t *testing.T) {
payload := []byte("fake wheel bytes")
stub := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write(payload)
}))
defer stub.Close()
dir := t.TempDir()
path, err := DownloadWheel(context.Background(), stub.URL+"/scimesh-1.1.0a10-py3-none-any.whl", dir)
if err != nil {
t.Fatal(err)
}
if !strings.HasSuffix(path, "scimesh-1.1.0a10-py3-none-any.whl") {
t.Errorf("path = %q", path)
}
got, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if string(got) != string(payload) {
t.Error("wheel bytes mismatch")
}
}
func TestDownloadWheelReportsHTTPErrors(t *testing.T) {
stub := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.NotFound(w, r)
}))
defer stub.Close()
if _, err := DownloadWheel(context.Background(), stub.URL+"/missing.whl", t.TempDir()); err == nil {
t.Error("404 must fail the download")
}
}
+139 -10
View File
@@ -107,12 +107,15 @@ func (s *PIDSupervisor) Start(configPath, logPath string) (int, error) {
if err != nil {
return 0, fmt.Errorf("resolve worker binary: %w", err)
}
//nolint:gosec // G304: logPath lives in the wizard's own config directory
logFile, err := os.OpenFile(logPath, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o600)
if err != nil {
return 0, fmt.Errorf("open worker log: %w", err)
}
defer func() { _ = logFile.Close() }()
cmd := exec.Command(exe, "--config", configPath)
//nolint:gosec // G204: exe is os.Executable, configPath is the wizard's own file;
// Background ctx: the child's lifecycle is managed by the supervisor, not the context
cmd := exec.CommandContext(context.Background(), exe, "--config", configPath)
cmd.Stdout = logFile
cmd.Stderr = logFile
cmd.Stdin = nil
@@ -173,13 +176,15 @@ func (s *PIDSupervisor) Stop() error {
// Server is the wizard HTTP server, bound to the loopback interface only.
type Server struct {
log *slog.Logger
cfgPath string
logPath string
dir string
sup Supervisor
openBrowser func(string)
port int
log *slog.Logger
cfgPath string
logPath string
dir string
sup Supervisor
openBrowser func(string)
port int
install func(ctx context.Context, venvPython, pkg string) error
downloadWheel func(ctx context.Context, url, dir string) (string, error)
}
// Options customises the wizard for tests and embedding.
@@ -189,6 +194,12 @@ type Options struct {
OpenBrowser func(url string)
Supervisor Supervisor
Dir string // directory for pid/log files; defaults to the config dir
// InstallScimesh overrides the pip step of the runtime installer (tests
// substitute a fake); nil uses the real pip inside the managed venv.
InstallScimesh func(ctx context.Context, venvPython, pkg string) error
// DownloadWheel overrides the release-wheel download (tests substitute a
// fake); nil downloads from the GitHub release matching the agent version.
DownloadWheel func(ctx context.Context, url, dir string) (string, error)
}
func New(log *slog.Logger, opts Options) *Server {
@@ -212,13 +223,21 @@ func New(log *slog.Logger, opts Options) *Server {
if port == 0 {
port = defaultPort
}
return &Server{log: log, cfgPath: cfgPath, logPath: filepath.Join(dir, logFileName), dir: dir, sup: sup, openBrowser: open, port: port}
install := opts.InstallScimesh
if install == nil {
install = installScimeshWithPip
}
downloadWheel := opts.DownloadWheel
if downloadWheel == nil {
downloadWheel = agent.DownloadWheel
}
return &Server{log: log, cfgPath: cfgPath, logPath: filepath.Join(dir, logFileName), dir: dir, sup: sup, openBrowser: open, port: port, install: install, downloadWheel: downloadWheel}
}
// Listen binds the loopback listener and returns it; Serve runs the server on
// it. Split so tests can inspect the actual ephemeral port.
func (s *Server) Listen() (net.Listener, error) {
return net.Listen("tcp", fmt.Sprintf("127.0.0.1:%d", s.port))
return (&net.ListenConfig{}).Listen(context.Background(), "tcp", fmt.Sprintf("127.0.0.1:%d", s.port))
}
// OpenBrowser hands the wizard URL to the configured opener (default: no-op).
@@ -231,6 +250,7 @@ func (s *Server) Serve(ctx context.Context, listener net.Listener) error {
mux.HandleFunc("GET /api/status", s.handleStatus)
mux.HandleFunc("POST /api/config", s.handleSaveConfig)
mux.HandleFunc("POST /api/test", s.handleTest)
mux.HandleFunc("POST /api/runtime/install", s.handleInstallRuntime)
mux.HandleFunc("POST /api/start", s.handleStart)
mux.HandleFunc("POST /api/stop", s.handleStop)
mux.HandleFunc("GET /api/logs", s.handleLogs)
@@ -409,3 +429,112 @@ func (s *Server) handleLogs(w http.ResponseWriter, r *http.Request) {
// ErrCanceled mirrors context.Canceled for callers that treat a cancelled
// wizard as a clean exit.
var ErrCanceled = errors.New("setup wizard cancelled")
type installRuntimeRequest struct {
// ScimeshPackage overrides where the scimesh wheel comes from: a local
// wheel/index path or the PyPI name. Defaults to SCIMESH_PIP_PACKAGE, then
// to the PyPI name.
ScimeshPackage string `json:"scimesh_package"`
}
type installRuntimeResponse struct {
OK bool `json:"ok"`
Python string `json:"python,omitempty"` // venv python to use as TASK_RUNNER[0]
Installed bool `json:"installed"`
}
// handleInstallRuntime creates a managed venv next to the worker config and
// installs the scimesh package into it, so the machine needs no manual pip
// step. The venv python path is returned for the wizard to bake into the
// task runner.
func (s *Server) handleInstallRuntime(w http.ResponseWriter, r *http.Request) {
var req installRuntimeRequest
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<20)).Decode(&req); err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid JSON body"})
return
}
pkg := strings.TrimSpace(req.ScimeshPackage)
if pkg == "" {
pkg = os.Getenv("SCIMESH_PIP_PACKAGE")
}
if pkg == "" {
// No PyPI default on purpose: the PyPI name "scimesh" belongs to an
// unrelated project. Instead we ship the wheel in our own GitHub
// release, version-locked to this binary, and download it from there.
url, _, err := agent.ReleaseWheelURL(agent.Version)
if err != nil {
writeJSON(w, http.StatusConflict, map[string]string{
"error": "no scimesh source configured: set SCIMESH_PIP_PACKAGE to your wheel, checkout or index, then retry",
})
return
}
local, err := s.downloadWheel(r.Context(), url, s.dir)
if err != nil {
s.log.Error("download release wheel", "err", err, "url", url)
writeJSON(w, http.StatusConflict, map[string]string{
"error": "could not download the scimesh wheel for this release: " + err.Error() + ". Set SCIMESH_PIP_PACKAGE to your wheel, checkout or index and retry.",
})
return
}
pkg = local
}
python3, err := exec.LookPath("python3")
if err != nil {
writeJSON(w, http.StatusConflict, map[string]string{"error": "python3 is not installed on this machine"})
return
}
venvDir := filepath.Join(s.dir, "venv")
venvPython := filepath.Join(venvDir, "bin", "python")
if _, err := os.Stat(venvPython); err != nil {
// Windows layout: Scripts/python.exe.
if win := filepath.Join(venvDir, "Scripts", "python.exe"); stat(win) {
venvPython = win
}
}
if _, err := os.Stat(venvPython); err != nil {
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Minute)
defer cancel()
cmd := exec.CommandContext(ctx, python3, "-m", "venv", venvDir) //nolint:gosec // G204: python3 from LookPath, venvDir is our own dir
if out, err := cmd.CombinedOutput(); err != nil {
s.log.Error("create runtime venv", "err", err, "out", truncate(string(out), 500))
writeJSON(w, http.StatusConflict, map[string]string{"error": "could not create the python venv"})
return
}
}
if err := s.install(r.Context(), venvPython, pkg); err != nil {
s.log.Error("install scimesh runtime", "err", err)
writeJSON(w, http.StatusConflict, map[string]string{
"error": "pip install " + pkg + " failed: " + err.Error() +
". Set SCIMESH_PIP_PACKAGE to your scimesh wheel or index and retry.",
})
return
}
writeJSON(w, http.StatusOK, installRuntimeResponse{OK: true, Python: venvPython, Installed: true})
}
// installScimeshWithPip installs the package with the venv's own pip,
// streaming into the agent log so a long build is not silent.
func installScimeshWithPip(ctx context.Context, venvPython, pkg string) error {
pip := filepath.Join(filepath.Dir(venvPython), "pip")
if _, err := os.Stat(pip); err != nil {
pip += ".exe"
}
ctx, cancel := context.WithTimeout(ctx, 10*time.Minute)
defer cancel()
cmd := exec.CommandContext(ctx, pip, "install", pkg) //nolint:gosec // G204: pip from our venv, pkg is operator-set or a fixed default
return cmd.Run()
}
func stat(path string) bool {
_, err := os.Stat(path)
return err == nil
}
func truncate(s string, n int) string {
if len(s) <= n {
return s
}
return s[:n] + "…"
}
@@ -3,8 +3,10 @@ package setupui
import (
"context"
"encoding/json"
"errors"
"io"
"log/slog"
"net"
"net/http"
"net/http/httptest"
"os"
@@ -22,14 +24,29 @@ func testLogger() *slog.Logger {
}
func newTestServer(t *testing.T, sup Supervisor) (*Server, string) {
t.Helper()
return newTestServerWithInstall(t, sup, nil)
}
func newTestServerWithInstall(t *testing.T, sup Supervisor, install func(ctx context.Context, venvPython, pkg string) error) (*Server, string) {
t.Helper()
return newTestServerWithInstallAndWheel(t, sup, install, nil)
}
func newTestServerWithInstallAndWheel(t *testing.T, sup Supervisor, install func(ctx context.Context, venvPython, pkg string) error, wheel func(ctx context.Context, url, dir string) (string, error)) (*Server, string) {
t.Helper()
dir := t.TempDir()
server := New(testLogger(), Options{
Port: 0, // ephemeral: tests must never collide on the default 12700
ConfigPath: filepath.Join(dir, "config.json"),
Dir: dir,
Supervisor: sup,
OpenBrowser: func(string) {},
// A distinct random port per test: Port 0 means "the default 12700" in
// the server, which would let the shared http.Client pool reuse a stale
// keep-alive connection across tests (EOF after a Shutdown).
Port: freePort(t),
ConfigPath: filepath.Join(dir, "config.json"),
Dir: dir,
Supervisor: sup,
OpenBrowser: func(string) {},
InstallScimesh: install,
DownloadWheel: wheel,
})
listener, err := server.Listen()
if err != nil {
@@ -76,12 +93,14 @@ func (f *fakeSup) Alive() bool {
func postJSON(t *testing.T, base, path string, body any) (*httptest.ResponseRecorder, map[string]any) {
t.Helper()
req, err := http.NewRequest(http.MethodPost, base+path, strings.NewReader(mustJSON(t, body)))
req, err := http.NewRequestWithContext(context.Background(), http.MethodPost, base+path, strings.NewReader(mustJSON(t, body)))
if err != nil {
t.Fatal(err)
}
req.Header.Set("Content-Type", "application/json")
client := http.Client{}
// No keep-alive pooling: a pooled connection to a shut-down test server
// would surface as an EOF instead of a fresh dial.
client := http.Client{Transport: &http.Transport{DisableKeepAlives: true}}
resp, err := client.Do(req)
if err != nil {
t.Fatal(err)
@@ -94,6 +113,20 @@ func postJSON(t *testing.T, base, path string, body any) (*httptest.ResponseReco
return rec, data
}
// freePort reserves an ephemeral port and returns it. The listener is closed
// immediately; the tiny reuse window is acceptable for tests and each test
// gets a different port, so nothing can collide or share pooled connections.
func freePort(t *testing.T) int {
t.Helper()
listener, err := (&net.ListenConfig{}).Listen(context.Background(), "tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
port := listener.Addr().(*net.TCPAddr).Port
_ = listener.Close()
return port
}
func mustJSON(t *testing.T, v any) string {
t.Helper()
raw, err := json.Marshal(v)
@@ -174,7 +207,7 @@ func TestWizardStartStopLifecycle(t *testing.T) {
}
// Status reflects the running state.
req, _ := http.NewRequest(http.MethodGet, base+"/api/status", nil)
req, _ := http.NewRequestWithContext(context.Background(), http.MethodGet, base+"/api/status", nil)
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatal(err)
@@ -198,7 +231,7 @@ func TestWizardStatusPrefillsSavedConfig(t *testing.T) {
postJSON(t, base, "/api/config", map[string]any{
"coordinator_url": "http://10.0.0.5:8080", "worker_key": "smk_abc", "work_dir": "/w", "worker_name": "n1",
})
req, _ := http.NewRequest(http.MethodGet, base+"/api/status", nil)
req, _ := http.NewRequestWithContext(context.Background(), http.MethodGet, base+"/api/status", nil)
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatal(err)
@@ -271,3 +304,108 @@ func TestConfigFileDefaultsAndEnvOverride(t *testing.T) {
t.Errorf("cpu = %d", config.CPUCount)
}
}
func TestInstallRuntimeCreatesVenvAndReportsPython(t *testing.T) {
var installedPkg string
sup := &fakeSup{}
_, base := newTestServerWithInstall(t, sup, func(ctx context.Context, venvPython, pkg string) error {
installedPkg = pkg
// Prove the venv python path really exists by creating a marker file
// where the real venv python would be.
_ = os.MkdirAll(filepath.Dir(venvPython), 0o755)
_ = os.WriteFile(venvPython, []byte("#!/bin/sh\nexit 0\n"), 0o755)
return nil
})
rec, data := postJSON(t, base, "/api/runtime/install", map[string]any{"scimesh_package": "/wheels/scimesh.whl"})
if rec.Code != http.StatusOK || data["ok"] != true {
t.Fatalf("install: got %d %v, want 200 ok", rec.Code, data)
}
if installedPkg != "/wheels/scimesh.whl" {
t.Errorf("package = %q, want the requested wheel", installedPkg)
}
if !strings.HasSuffix(data["python"].(string), "venv/bin/python") {
t.Errorf("python = %v, want the venv python", data["python"])
}
}
func TestInstallRuntimeRequiresASource(t *testing.T) {
sup := &fakeSup{}
_, base := newTestServerWithInstall(t, sup, func(ctx context.Context, venvPython, pkg string) error {
t.Fatal("install must not run without a package source")
return nil
})
// No source anywhere (SCIMESH_PIP_PACKAGE unset, request empty): 409 with
// guidance. The PyPI name is another project, so no silent fallback.
rec, data := postJSON(t, base, "/api/runtime/install", map[string]any{})
if rec.Code != http.StatusConflict {
t.Fatalf("install without source: got %d, want 409", rec.Code)
}
if !strings.Contains(data["error"].(string), "SCIMESH_PIP_PACKAGE") {
t.Errorf("error = %v, want a hint about SCIMESH_PIP_PACKAGE", data["error"])
}
}
func TestInstallRuntimeFailureIsExplained(t *testing.T) {
sup := &fakeSup{}
_, base := newTestServerWithInstall(t, sup, func(ctx context.Context, venvPython, pkg string) error {
return errors.New("no matching distribution found")
})
rec, data := postJSON(t, base, "/api/runtime/install", map[string]any{"scimesh_package": "/wheels/scimesh.whl"})
if rec.Code != http.StatusConflict {
t.Fatalf("install failure: got %d, want 409", rec.Code)
}
if !strings.Contains(data["error"].(string), "SCIMESH_PIP_PACKAGE") {
t.Errorf("error = %v, want a hint about SCIMESH_PIP_PACKAGE", data["error"])
}
}
func TestInstallRuntimeDownloadsReleaseWheelWhenNoSource(t *testing.T) {
oldVersion := agent.Version
agent.Version = "1.1.0-alpha.10"
t.Cleanup(func() { agent.Version = oldVersion })
var downloadedURL, installedPkg string
sup := &fakeSup{}
_, base := newTestServerWithInstallAndWheel(t, sup,
func(ctx context.Context, venvPython, pkg string) error {
installedPkg = pkg
return nil
},
func(ctx context.Context, url, dir string) (string, error) {
downloadedURL = url
return filepath.Join(dir, "scimesh-1.1.0a10-py3-none-any.whl"), nil
})
rec, data := postJSON(t, base, "/api/runtime/install", map[string]any{})
if rec.Code != http.StatusOK || data["ok"] != true {
t.Fatalf("install: got %d %v, want 200 ok", rec.Code, data)
}
if !strings.Contains(downloadedURL, "releases/download/v1.1.0-alpha.10/scimesh-1.1.0a10-py3-none-any.whl") {
t.Errorf("download url = %q, want the release wheel of this version", downloadedURL)
}
if !strings.HasSuffix(installedPkg, "scimesh-1.1.0a10-py3-none-any.whl") {
t.Errorf("pip received %q, want the downloaded wheel", installedPkg)
}
}
func TestInstallRuntimeWheelDownloadFailureIsExplained(t *testing.T) {
oldVersion := agent.Version
agent.Version = "1.1.0-alpha.10"
t.Cleanup(func() { agent.Version = oldVersion })
sup := &fakeSup{}
_, base := newTestServerWithInstallAndWheel(t, sup,
func(ctx context.Context, venvPython, pkg string) error { t.Fatal("pip must not run"); return nil },
func(ctx context.Context, url, dir string) (string, error) {
return "", errors.New("HTTP 404")
})
rec, data := postJSON(t, base, "/api/runtime/install", map[string]any{})
if rec.Code != http.StatusConflict {
t.Fatalf("got %d, want 409", rec.Code)
}
if !strings.Contains(data["error"].(string), "SCIMESH_PIP_PACKAGE") {
t.Errorf("error = %v, want a SCIMESH_PIP_PACKAGE hint", data["error"])
}
}
@@ -183,7 +183,7 @@ code{font-family:var(--mono);font-size:.86em}
<script>
const $=id=>document.getElementById(id);
let state={mode:'token',cpu:'auto'};
let state={mode:'token',cpu:'auto',venvPython:null};
let checksOk=false;
function err(msg){$('error-box').innerHTML=msg?'<div class="error-strip">'+msg+'</div>':''}
@@ -215,7 +215,7 @@ async function postJSON(path,body){
return {status:r.status,data};
}
function draftConfig(){
return {
const cfg={
coordinator_url:$('in-url').value.trim(),
token:state.mode==='token'?$('in-token').value.trim():'',
worker_key:state.mode==='key'?$('in-key').value.trim():'',
@@ -224,6 +224,8 @@ function draftConfig(){
worker_name:$('in-name').value.trim(),
cpu_count:state.cpu==='custom'?parseInt($('in-cpu').value||'1',10):0
};
if(state.venvPython)cfg.task_runner=[state.venvPython,'-m','scimesh.worker.task'];
return cfg;
}
$('b1').onclick=()=>{
@@ -145,20 +145,20 @@ func TestUIReadRepoListsReducerFields(t *testing.T) {
if claimed, err := jobs.ClaimReduction(ctx, job.ID, time.Now().UTC()); err != nil || !claimed {
t.Fatalf("claim reduction = (%v, %v)", claimed, err)
}
listed, err := NewUIReadRepo(pool).ListJobs(ctx, nil, 20)
listed, _, err := NewAdminReadRepo(pool).ListJobsPaginated(ctx, "", 20, 0)
if err != nil {
t.Fatalf("list UI jobs: %v", err)
t.Fatalf("list admin jobs: %v", err)
}
for _, item := range listed {
if item.ID != job.ID {
continue
}
if item.Status != domain.JobReducing || item.ReducerStartedAt == nil {
t.Fatalf("UI reducer projection = %+v", item)
t.Fatalf("admin reducer projection = %+v", item)
}
return
}
t.Fatalf("seeded job %s is missing from UI list", job.ID)
t.Fatalf("seeded job %s is missing from the admin list", job.ID)
}
// A job must land whole or not at all: a half-created job leaves chunks no
@@ -24,40 +24,6 @@ func (r *UIReadRepo) GetJob(ctx context.Context, id uuid.UUID) (*domain.Job, err
return job, err
}
func (r *UIReadRepo) ListJobs(ctx context.Context, owner *uuid.UUID, limit int) ([]domain.Job, error) {
if limit < 1 || limit > 100 {
return nil, domain.ErrInvalidInput
}
q := psql.Select(jobColumns...).From("jobs")
if owner != nil {
q = q.Where(sq.Eq{"owner_id": *owner})
}
sql, args, err := q.OrderBy("created_at DESC", "id DESC").Limit(uint64(limit)).ToSql()
if err != nil {
return nil, err
}
rows, err := conn(ctx, r.pool).Query(ctx, sql, args...)
if err != nil {
return nil, fmt.Errorf("list jobs: %w", err)
}
defer rows.Close()
jobs := make([]domain.Job, 0)
for rows.Next() {
var j domain.Job
var status string
if err := rows.Scan(
&j.ID, &j.Workload, &j.InputURI, &j.Parameters, &status, &j.CreatedAt, &j.CompletedAt,
&j.InputArtifactID, &j.ResultArtifactID, &j.ErrorCode, &j.ErrorMessage, &j.ReducerStartedAt,
&j.OwnerID,
); err != nil {
return nil, err
}
j.Status = domain.JobStatus(status)
jobs = append(jobs, j)
}
return jobs, rows.Err()
}
func (r *UIReadRepo) ListTasksByJob(ctx context.Context, jobID uuid.UUID) ([]domain.Task, error) {
sql, args, err := psql.Select(taskColumns...).From("tasks").Where(sq.Eq{"job_id": jobID}).OrderBy("chunk_index ASC").ToSql()
if err != nil {
@@ -79,68 +45,18 @@ func (r *UIReadRepo) ListTasksByJob(ctx context.Context, jobID uuid.UUID) ([]dom
return tasks, rows.Err()
}
func (r *UIReadRepo) ListTasksByJobs(ctx context.Context, jobIDs []uuid.UUID) (map[uuid.UUID][]domain.Task, error) {
out := make(map[uuid.UUID][]domain.Task, len(jobIDs))
if len(jobIDs) == 0 {
return out, nil
}
sql, args, err := psql.Select(taskColumns...).From("tasks").
Where(sq.Eq{"job_id": jobIDs}).OrderBy("job_id ASC", "chunk_index ASC").ToSql()
if err != nil {
return nil, err
}
rows, err := conn(ctx, r.pool).Query(ctx, sql, args...)
if err != nil {
return nil, fmt.Errorf("list tasks by jobs: %w", err)
}
defer rows.Close()
for rows.Next() {
task, err := scanTask(rows)
if err != nil {
return nil, err
}
out[task.JobID] = append(out[task.JobID], *task)
}
return out, rows.Err()
}
func (r *UIReadRepo) ListWorkers(ctx context.Context, limit int) ([]domain.Worker, error) {
if limit < 1 || limit > 100 {
return nil, domain.ErrInvalidInput
}
sql, args, err := psql.Select(workerColumns...).From("workers").OrderBy("last_heartbeat_at DESC", "id DESC").Limit(uint64(limit)).ToSql()
if err != nil {
return nil, err
}
rows, err := conn(ctx, r.pool).Query(ctx, sql, args...)
if err != nil {
return nil, fmt.Errorf("list workers: %w", err)
}
defer rows.Close()
workers := make([]domain.Worker, 0)
for rows.Next() {
worker, err := scanWorker(rows)
if err != nil {
return nil, err
}
workers = append(workers, *worker)
}
return workers, rows.Err()
}
func (r *UIReadRepo) ListWorkersByOwner(ctx context.Context, owner uuid.UUID, limit int) ([]domain.Worker, error) {
if limit < 1 || limit > 100 {
return nil, domain.ErrInvalidInput
}
sql, args, err := psql.Select(workerColumns...).From("workers").
Where(sq.Eq{"owner_id": owner}).
OrderBy("last_heartbeat_at DESC", "id DESC").Limit(uint64(limit)).ToSql()
if err != nil {
return nil, err
}
rows, err := conn(ctx, r.pool).Query(ctx, sql, args...)
if err != nil {
return nil, fmt.Errorf("list workers by owner: %w", err)
return nil, fmt.Errorf("list workers: %w", err)
}
defer rows.Close()
workers := make([]domain.Worker, 0)
@@ -2,6 +2,7 @@ package sqlite
import (
"context"
"errors"
"testing"
"time"
@@ -83,7 +84,7 @@ func TestWorkerSetTrust(t *testing.T) {
if err := repo.SetTrust(ctx, worker.ID, domain.WorkerTrusted); err != nil {
t.Fatal(err)
}
if err := repo.SetTrust(ctx, uuid.New(), domain.WorkerTrusted); err != domain.ErrWorkerNotFound {
if err := repo.SetTrust(ctx, uuid.New(), domain.WorkerTrusted); !errors.Is(err, domain.ErrWorkerNotFound) {
t.Errorf("unknown worker trust err = %v, want ErrWorkerNotFound", err)
}
}
@@ -4,7 +4,6 @@ import (
"context"
"database/sql"
"fmt"
"strings"
"github.com/google/uuid"
@@ -20,34 +19,6 @@ func (r *UIReadRepo) GetJob(ctx context.Context, id uuid.UUID) (*domain.Job, err
return NewJobRepo(r.db).Get(ctx, id)
}
func (r *UIReadRepo) ListJobs(ctx context.Context, owner *uuid.UUID, limit int) ([]domain.Job, error) {
if limit < 1 || limit > 100 {
return nil, domain.ErrInvalidInput
}
query := "SELECT " + jobColumns + " FROM jobs"
args := []any{}
if owner != nil {
query += " WHERE owner_id = ?"
args = append(args, owner.String())
}
query += " ORDER BY created_at DESC, id DESC LIMIT ?"
args = append(args, limit)
rows, err := conn(ctx, r.db).QueryContext(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("list jobs: %w", err)
}
defer func() { _ = rows.Close() }()
jobs := make([]domain.Job, 0)
for rows.Next() {
job, err := scanJob(rows)
if err != nil {
return nil, err
}
jobs = append(jobs, *job)
}
return jobs, rows.Err()
}
func (r *UIReadRepo) ListTasksByJob(ctx context.Context, jobID uuid.UUID) ([]domain.Task, error) {
rows, err := conn(ctx, r.db).QueryContext(ctx,
"SELECT "+taskColumns+" FROM tasks WHERE job_id = ? ORDER BY chunk_index ASC", jobID.String())
@@ -66,50 +37,12 @@ func (r *UIReadRepo) ListTasksByJob(ctx context.Context, jobID uuid.UUID) ([]dom
return tasks, rows.Err()
}
func (r *UIReadRepo) ListTasksByJobs(ctx context.Context, jobIDs []uuid.UUID) (map[uuid.UUID][]domain.Task, error) {
out := make(map[uuid.UUID][]domain.Task, len(jobIDs))
if len(jobIDs) == 0 {
return out, nil
}
placeholders := make([]string, 0, len(jobIDs))
args := make([]any, 0, len(jobIDs))
for _, id := range jobIDs {
placeholders = append(placeholders, "?")
args = append(args, id.String())
}
rows, err := conn(ctx, r.db).QueryContext(ctx,
"SELECT "+taskColumns+" FROM tasks WHERE job_id IN ("+strings.Join(placeholders, ", ")+") ORDER BY job_id ASC, chunk_index ASC",
args...)
if err != nil {
return nil, fmt.Errorf("list tasks by jobs: %w", err)
}
defer func() { _ = rows.Close() }()
for rows.Next() {
task, err := scanTask(rows)
if err != nil {
return nil, err
}
out[task.JobID] = append(out[task.JobID], *task)
}
return out, rows.Err()
}
func (r *UIReadRepo) ListWorkers(ctx context.Context, limit int) ([]domain.Worker, error) {
return r.listWorkers(ctx, "", nil, limit)
}
func (r *UIReadRepo) ListWorkersByOwner(ctx context.Context, owner uuid.UUID, limit int) ([]domain.Worker, error) {
return r.listWorkers(ctx, " WHERE owner_id = ?", []any{owner.String()}, limit)
}
func (r *UIReadRepo) listWorkers(ctx context.Context, clause string, args []any, limit int) ([]domain.Worker, error) {
if limit < 1 || limit > 100 {
return nil, domain.ErrInvalidInput
}
query := "SELECT " + workerColumns + " FROM workers" + clause +
" ORDER BY last_heartbeat_at DESC, id DESC LIMIT ?"
fullArgs := append(args, limit)
rows, err := conn(ctx, r.db).QueryContext(ctx, query, fullArgs...)
rows, err := conn(ctx, r.db).QueryContext(ctx,
"SELECT "+workerColumns+" FROM workers ORDER BY last_heartbeat_at DESC, id DESC LIMIT ?", limit)
if err != nil {
return nil, fmt.Errorf("list workers: %w", err)
}
@@ -147,7 +147,6 @@ func (s *Server) Handler(token string, uiToken ...string) http.Handler {
{"GET /ui/docs", s.handleUIDocsIndex},
{"GET /ui/docs/{path...}", s.handleUIDocs},
{"GET /ui/jobs/{job_id}", s.handleUIJob},
{"GET /ui/api/overview", s.handleUIOverviewJSON},
{"GET /ui/api/jobs/{job_id}", s.handleUIJobJSON},
{"POST /ui/api/jobs/{job_id}/cancel", s.handleCancelJob},
{"POST /ui/api/jobs/upload", s.handleUploadDataset},
@@ -159,6 +158,7 @@ func (s *Server) Handler(token string, uiToken ...string) http.Handler {
// Public auth pages — reachable without a session so a user can log in.
ui.HandleFunc("GET /ui/login", s.handleUILoginForm)
ui.HandleFunc("POST /ui/login", s.handleUILogin)
ui.HandleFunc("GET /ui/logout-form", s.handleUILogoutForm)
ui.HandleFunc("GET /ui/register", s.handleUIRegisterForm)
ui.HandleFunc("POST /ui/register", s.handleUIRegister)
ui.HandleFunc("POST /ui/logout", s.handleUILogout)
@@ -145,42 +145,17 @@ func TestUIRequiresDistinctCredentialAndRendersDashboard(t *testing.T) {
req = request()
req.SetBasicAuth("operator", uiToken)
resp, err = http.DefaultClient.Do(req)
// Do not follow the redirect: we assert it, not its target.
client := &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
resp, err = client.Do(req)
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("UI status: %d", resp.StatusCode)
}
body, _ := io.ReadAll(resp.Body)
if !strings.Contains(string(body), "SciMesh control room") {
t.Errorf("dashboard body missing title")
}
}
func TestUIOverviewReturnsLiveSafeProjection(t *testing.T) {
e := newEnv(t, healthy)
code, _ := e.do(t, "POST", "/jobs", `{"workload":"w","input_uri":"s3://in","chunks":[{"chunk_index":0,"input_uri":"s3://c","input_sha256":"sha"}]}`)
if code != http.StatusCreated {
t.Fatalf("create job: %d", code)
}
req, _ := http.NewRequestWithContext(context.Background(), "GET", e.ts.URL+"/ui/api/overview", nil)
req.SetBasicAuth("operator", uiToken)
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
var overview map[string]any
if err := json.NewDecoder(resp.Body).Decode(&overview); err != nil {
t.Fatal(err)
}
if resp.StatusCode != http.StatusOK || overview["active_jobs"].(float64) != 1 || overview["online_workers"].(float64) != 1 {
t.Fatalf("overview = (%d, %v)", resp.StatusCode, overview)
}
if _, leaked := overview["worker_auth_token"]; leaked {
t.Fatal("overview must not expose authentication configuration")
// In basic-auth mode (no userservice) /ui lands on the job form: the admin
// console exists only in session mode.
if resp.StatusCode != http.StatusSeeOther || resp.Header.Get("Location") != "/ui/jobs/new" {
t.Fatalf("UI status: %d -> %s, want 303 -> /ui/jobs/new", resp.StatusCode, resp.Header.Get("Location"))
}
}
@@ -11,7 +11,7 @@
</head>
<body data-coordinator="{{.CoordinatorURL}}" data-userservice="{{.UserserviceURL}}">
<main class="page">
<a class="back" href="/ui">← Back to control room</a><p class="eyebrow">Contribute compute</p><h1>Turn this computer into a worker</h1><p class="lead">Create a key, install the worker, and run one command. The worker binds to your account and pulls tasks whenever it is online.</p>
<a class="back" href="/ui/admin">← Back to control room</a><p class="eyebrow">Contribute compute</p><h1>Turn this computer into a worker</h1><p class="lead">Create a key, install the worker, and run one command. The worker binds to your account and pulls tasks whenever it is online.</p>
<div class="layout">
<section class="card">
<h2 style="margin:0 0 4px;color:#f1f6ff">Your worker keys</h2>
@@ -29,7 +29,7 @@
<h2>Set it up</h2>
<ol>
<li><strong>Create a key</strong><br>Use the form; copy the command it generates.</li>
<li><strong>Paste it in a terminal</strong><br>The command installs the worker, points it at this coordinator, and starts it. The machine then appears under <a href="/ui">My machines</a>.</li>
<li><strong>Paste it in a terminal</strong><br>The command installs the worker, points it at this coordinator, and starts it. The machine then appears under <a href="/ui/admin">Workers</a>.</li>
</ol>
<h2 style="margin-top:24px">Single-binary mode?</h2>
<p>If the coordinator runs as <code>coordinator serve</code>, skip the key:
@@ -143,14 +143,14 @@ tbody tr:hover{background:var(--panel-2)}
</nav>
<div class="side-foot">
<div class="user-chip"><div class="avatar">{{.Role}}</div><div><b>Signed in as {{.Role}}</b><span>cluster administrator</span></div></div>
<a class="back-link" href="/ui">← Back to control room</a>
<a class="back-link" href="/ui/jobs/new"> New computation</a>
</div>
</aside>
<div class="main">
<header class="topbar">
<div><h1 id="page-title">System</h1><p id="page-sub">Cluster state and node information</p></div>
<div class="env-badge"><i></i><span id="env-label">admin console</span></div>
<div style="display:flex;gap:10px;align-items:center"><a class="btn btn-primary" href="/ui/jobs/new"> New computation</a><div class="env-badge"><i></i><span id="env-label">admin console</span></div></div>
</header>
<div class="content">
@@ -198,7 +198,7 @@ tbody tr:hover{background:var(--panel-2)}
<div class="tabs" id="job-tabs"></div>
<div class="card">
<table>
<thead><tr><th>Job</th><th>Workload</th><th>Owner</th><th>Status</th><th>Progress</th><th>Submitted</th></tr></thead>
<thead><tr><th>Job</th><th>Workload</th><th>Owner</th><th>Status</th><th>Progress</th><th>Submitted</th><th></th></tr></thead>
<tbody id="job-rows"></tbody>
</table>
<div class="footer-row"><span id="job-range"></span><div class="pager"><button id="pg-prev" aria-label="previous"></button><button id="pg-next" aria-label="next"></button></div></div>
@@ -380,7 +380,8 @@ async function loadJobs(){
'<td style="color:var(--text-2)">'+esc(j.owner)+'</td>'+
'<td>'+pill(statusLabel[j.status]||j.status,statusClass[j.status]||'pill-waiting',null)+'</td>'+
'<td><div class="bar"><span style="width:'+pct+'%"></span></div><div class="bar-label">'+j.completed+' / '+j.total+' shards'+(j.failed?' · '+j.failed+' failed':'')+'</div></td>'+
'<td style="color:var(--text-2);font-size:12.5px">'+fmtTime(j.created_at)+'</td>';
'<td style="color:var(--text-2);font-size:12.5px">'+fmtTime(j.created_at)+'</td>'+
'<td><a class="btn btn-ghost btn-sm" href="/ui/jobs/'+encodeURIComponent(j.id)+'">Open</a></td>';
rows.append(tr);
}
const from=(v.page-1)*v.per_page+1,to=Math.min(v.page*v.per_page,v.total);
@@ -1,42 +0,0 @@
{{define "dashboard.html"}}
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>SciMesh control room</title>
<style>
:root{color:#dce8ff;background:#08111f;font:16px/1.5 Inter,ui-sans-serif,system-ui,sans-serif;color-scheme:dark}*{box-sizing:border-box}body{margin:0;background:radial-gradient(circle at 15% -10%,#163d77 0,transparent 32rem),radial-gradient(circle at 95% 5%,#123e39 0,transparent 29rem),#08111f}.page{max-width:1240px;margin:auto;padding:28px 22px 64px}.top{display:flex;align-items:flex-start;justify-content:space-between;gap:22px}.eyebrow{margin:0;color:#78a9ff;font-size:.77rem;font-weight:750;letter-spacing:.14em;text-transform:uppercase}.title{max-width:700px;margin:8px 0;font-size:clamp(2rem,5vw,3.6rem);line-height:1.04;letter-spacing:-.055em}.lead{max-width:690px;margin:0;color:#aabbd5;font-size:1.05rem}.button{display:inline-flex;align-items:center;gap:8px;border:0;border-radius:10px;padding:12px 16px;background:#4f8cff;color:#071224;font:inherit;font-weight:800;text-decoration:none;box-shadow:0 12px 30px #163d7766}.live{display:inline-flex;align-items:center;gap:7px;margin-top:18px;color:#8ba2c2;font-size:.87rem}.pulse{width:8px;height:8px;border-radius:50%;background:#5ee6a6;box-shadow:0 0 0 5px #5ee6a622}.summary{display:grid;grid-template-columns:1.4fr repeat(3,1fr);gap:13px;margin:32px 0}.panel,.metric,.flow-step,.job,.worker{border:1px solid #26415f;border-radius:15px;background:#0d1a2cdd;box-shadow:0 20px 45px #0000001f}.panel{padding:20px}.pipeline{display:grid;grid-template-columns:repeat(4,1fr);gap:10px;margin-top:16px}.flow-step{position:relative;padding:14px;min-height:116px}.flow-step:not(:last-child):after{content:"";position:absolute;top:35px;right:-10px;width:10px;height:2px;background:#3c5d83}.flow-step b{display:block;color:#f2f7ff}.flow-step small{display:block;margin-top:6px;color:#91a8c6}.flow-step .dot{display:inline-block;width:9px;height:9px;margin-right:7px;border-radius:50%;background:#5ee6a6}.metric{padding:16px}.metric b{display:block;margin-top:7px;color:#f5f8ff;font-size:2rem;line-height:1}.metric span{color:#9bb0cc;font-size:.84rem}.section-head{display:flex;align-items:baseline;justify-content:space-between;gap:12px;margin:36px 0 12px}.section-head h2{margin:0;color:#f3f7ff;font-size:1.18rem}.section-head p{margin:0;color:#8fa6c3;font-size:.9rem}.jobs{display:grid;gap:10px}.job{display:grid;grid-template-columns:minmax(210px,1.35fr) minmax(150px,.9fr) minmax(180px,1fr) auto;gap:18px;align-items:center;padding:17px 18px;text-decoration:none;color:inherit;transition:border-color .15s,transform .15s}.job:hover{border-color:#5d96ee;transform:translateY(-1px)}.job-name{color:#f3f7ff;font-weight:750}.job-id{margin-top:3px;color:#8196b3;font-family:ui-monospace,SFMono-Regular,monospace;font-size:.76rem}.badge{display:inline-flex;align-items:center;border-radius:999px;padding:4px 9px;font-size:.78rem;font-weight:800}.badge-waiting{background:#23344d;color:#b9cce9}.badge-active{background:#173d77;color:#9fc7ff}.badge-success{background:#123f34;color:#76efb5}.badge-danger{background:#552334;color:#ff9bad}.bar{height:7px;margin-top:8px;overflow:hidden;border-radius:999px;background:#20344e}.bar span{display:block;height:100%;border-radius:inherit;background:linear-gradient(90deg,#5a92ff,#61e1bd)}.numbers{color:#afc0d9;font-size:.88rem}.arrow{color:#79aaff;font-size:1.35rem}.workers{display:grid;grid-template-columns:repeat(3,1fr);gap:11px}.worker{padding:15px}.worker-head{display:flex;justify-content:space-between;gap:8px}.worker strong{color:#f3f7ff}.worker small,.worker p{color:#95a9c4}.worker p{margin:12px 0 0}.cap{display:inline-block;margin:4px 5px 0 0;border:1px solid #365576;border-radius:5px;padding:2px 6px;color:#a9c9f4;font:.75rem ui-monospace,SFMono-Regular,monospace}.empty{padding:30px;border:1px dashed #35516f;border-radius:14px;color:#9ab0cb;text-align:center}.offline{color:#faafbd}.sr{position:absolute;width:1px;height:1px;clip:rect(0,0,0,0);overflow:hidden;white-space:nowrap}@media(max-width:820px){.top,.section-head{display:block}.button{margin-top:18px}.summary{grid-template-columns:1fr 1fr}.panel{grid-column:span 2}.pipeline{grid-template-columns:1fr 1fr}.flow-step:not(:last-child):after{display:none}.job{grid-template-columns:1fr 1fr}.arrow{display:none}.workers{grid-template-columns:1fr 1fr}}@media(max-width:540px){.page{padding:22px 14px}.summary,.workers{grid-template-columns:1fr}.panel{grid-column:auto}.pipeline,.job{grid-template-columns:1fr}.title{font-size:2.35rem}}
</style>
</head>
<body>
<main class="page">
<header class="top">
<div><p class="eyebrow">Local scientific compute</p><h1 class="title">SciMesh control room</h1><p class="lead">Follow the real path from a molecular TSV to a globally reduced similarity result—without reading coordinator logs.</p><div class="live"><i class="pulse"></i><span id="refresh-state">Live overview · refreshes every 2 seconds</span></div></div>
<div style="display:flex;gap:10px;align-items:center;flex-wrap:wrap">{{if .Session}}<span class="live" style="margin-top:0">Signed in · {{.Session.Role}}</span>{{end}}{{if .Session}}<a class="button" href="/ui/profile" style="background:#23344d;color:#dce8ff;box-shadow:none">Profile</a>{{end}}{{if and .Session (eq .Session.Role "admin")}}<a class="button" href="/ui/admin" style="background:#23344d;color:#dce8ff;box-shadow:none">Admin</a>{{end}}{{if .Session}}<a class="button" href="/ui/workloads" style="background:#23344d;color:#dce8ff;box-shadow:none">Workloads</a>{{end}}{{if .Session}}<a class="button" href="/ui/docs/" style="background:#23344d;color:#dce8ff;box-shadow:none">Docs</a>{{end}}{{if .Session}}<a class="button" href="/ui/workers/new" style="background:#23344d;color:#dce8ff;box-shadow:none">🖥 Add your machine</a>{{end}}<a class="button" href="/ui/jobs/new"> New computation</a>{{if .Session}}<form method="post" action="/ui/logout" style="margin:0"><button class="button" type="submit" style="background:#23344d;color:#dce8ff;box-shadow:none">Log out</button></form>{{end}}</div>
</header>
<section class="summary" aria-label="Pipeline summary">
<div class="panel"><strong>How a search becomes a result</strong><div class="pipeline"><div class="flow-step"><span><i class="dot"></i>01</span><b>Upload TSV</b><small>The coordinator validates and slices the dataset.</small></div><div class="flow-step"><span><i class="dot"></i>02</span><b>Run shards</b><small>Workers fingerprint molecules and return shard top-k CSVs.</small></div><div class="flow-step"><span><i class="dot"></i>03</span><b>Merge exactly</b><small>The coordinator ranks retained candidates deterministically.</small></div><div class="flow-step"><span><i class="dot"></i>04</span><b>Download CSV</b><small>A checksum-protected global result is ready.</small></div></div></div>
<div class="metric"><span>Active runs</span><b id="active-jobs">{{.ActiveJobs}}</b><small>waiting, running, or merging</small></div>
<div class="metric"><span>Available workers</span><b id="online-workers">{{.OnlineWorkers}}</b><small>recently registered</small></div>
<div class="metric"><span>Finished runs</span><b id="finished-jobs">{{.FinishedJobs}}</b><small>in the latest 20</small></div>
</section>
<section><div class="section-head"><h2>Recent computations</h2><p id="job-count">{{len .Jobs}} shown · newest first</p></div><div id="jobs" class="jobs">{{range .Jobs}}<a class="job" href="/ui/jobs/{{.ID}}"><div><div class="job-name">{{workloadLabel .Workload}}</div><div class="job-id">{{.ID}}</div></div><div><span class="badge badge-{{statusClass .Status}}">{{statusLabel .Status}}</span><div class="job-id">{{statusHint .Status}}</div></div><div><div class="numbers"><b>{{.Completed}}</b> / {{.Total}} shards complete{{if gt .Failed 0}} · <span class="offline">{{.Failed}} failed</span>{{end}}</div><div class="bar"><span style="width:{{progressPercent .Completed .Failed .Cancelled .Total}}%"></span></div></div><span class="arrow" aria-hidden="true"></span></a>{{else}}<div class="empty"><strong>No computations yet.</strong><br>Start a computation, then keep one or more workers running to watch this dashboard come alive.</div>{{end}}</div></section>
{{if and .Session (ne .Session.Role "admin")}}<section><div class="section-head"><h2>My machines</h2><p>Workers you registered. <a href="/ui/workers/new" style="color:#79aaff">Add your machine →</a></p></div><div id="my-workers" class="workers">{{range .MyWorkers}}<article class="worker"><div class="worker-head"><strong>{{.Name}}</strong><span class="badge badge-{{workerStatusClass .Status}}">{{workerStatusLabel .Status}}</span></div><small>{{.ID}}</small><p>{{range .Capabilities}}<span class="cap">{{.}}</span>{{end}}</p><p>Last signal · {{time .LastHeartbeatAt}}</p></article>{{else}}<div class="empty"><strong>No machine of yours is connected.</strong><br><a href="/ui/workers/new" style="color:#79aaff">Turn this computer into a worker →</a></div>{{end}}</div></section>{{end}}
<section><div class="section-head"><h2>Worker fleet</h2><p>Workers register themselves; this page never controls their processes.</p></div><div id="workers" class="workers">{{range .Workers}}<article class="worker"><div class="worker-head"><strong>{{.Name}}</strong><span class="badge badge-{{workerStatusClass .Status}}">{{workerStatusLabel .Status}}</span></div><small>{{.ID}}</small><p>{{range .Capabilities}}<span class="cap">{{.}}</span>{{end}}</p><p>Last signal · {{time .LastHeartbeatAt}}</p></article>{{else}}<div class="empty"><strong>No worker is registered.</strong><br>Start <code>scimesh-worker</code> in another terminal, then return here.</div>{{end}}</div></section>
</main>
<script>
const statusInfo={pending:['Waiting for a worker','waiting'],leased:['Assigned to a worker','active'],running:['Running','active'],reducing:['Merging results','active'],completed:['Completed','success'],failed:['Needs attention','danger'],cancelled:['Stopped','waiting']};
const pct=j=>j.total?Math.min(100,Math.floor((j.completed+j.failed+j.cancelled)*100/j.total)):0;
const node=(tag,text,cls)=>{const n=document.createElement(tag);if(text!==undefined)n.textContent=text;if(cls)n.className=cls;return n};
const renderJobs=jobs=>{const box=document.querySelector('#jobs');box.replaceChildren();if(!jobs.length){const empty=node('div',undefined,'empty');empty.append(node('strong','No computations yet.'),document.createElement('br'),document.createTextNode('Start a computation, then keep one or more workers running to watch this dashboard come alive.'));box.append(empty);return}for(const job of jobs){const info=statusInfo[job.status]||[job.status,'waiting'],link=node('a',undefined,'job');link.href='/ui/jobs/'+encodeURIComponent(job.id);const intro=node('div');intro.append(node('div',job.workload==='similarity-search'?'Molecule similarity search':job.workload,'job-name'),node('div',job.id,'job-id'));const state=node('div');state.append(node('span',info[0],'badge badge-'+info[1]),node('div',job.status==='reducing'?'Every shard is complete; coordinator is ranking the global top-k.':'Live coordinator state','job-id'));const progress=node('div'),numbers=node('div',undefined,'numbers');numbers.append(node('b',String(job.completed)),document.createTextNode(' / '+job.total+' shards complete'));if(job.failed){numbers.append(document.createTextNode(' · '),node('span',job.failed+' failed','offline'))}const bar=node('div',undefined,'bar'),fill=node('span');fill.style.width=pct(job)+'%';bar.append(fill);progress.append(numbers,bar);link.append(intro,state,progress,node('span','→','arrow'));box.append(link)}};
const workerCard=worker=>{const card=node('article',undefined,'worker'),head=node('div',undefined,'worker-head'),left=node('div'),workerInfo=worker.status==='online'?['Available','success']:worker.status==='busy'?['Busy','active']:['Offline','waiting'];left.append(node('strong',worker.name),node('small',worker.id));head.append(left,node('span',workerInfo[0],'badge badge-'+workerInfo[1]));const caps=node('p');for(const capability of worker.capabilities||[])caps.append(node('span',capability,'cap'));card.append(head,caps,node('p','Last signal · '+new Date(worker.last_heartbeat_at).toLocaleString()));return card};
const renderWorkers=workers=>{const box=document.querySelector('#workers');box.replaceChildren();if(!workers.length){const empty=node('div',undefined,'empty');empty.append(node('strong','No worker is registered.'),document.createElement('br'),document.createTextNode('Start scimesh-worker in another terminal, then return here.'));box.append(empty);return}for(const worker of workers)box.append(workerCard(worker))};
const renderMyWorkers=workers=>{const box=document.querySelector('#my-workers');if(!box)return;box.replaceChildren();if(!workers.length){const empty=node('div',undefined,'empty'),link=node('a','Turn this computer into a worker →');link.href='/ui/workers/new';link.style.color='#79aaff';empty.append(node('strong','No machine of yours is connected.'),document.createElement('br'),link);box.append(empty);return}for(const worker of workers)box.append(workerCard(worker))};
let timer;const refresh=async()=>{try{const response=await fetch('/ui/api/overview',{headers:{Accept:'application/json'}});if(!response.ok)throw Error();const view=await response.json();document.querySelector('#active-jobs').textContent=view.active_jobs;document.querySelector('#online-workers').textContent=view.online_workers;document.querySelector('#finished-jobs').textContent=view.finished_jobs;document.querySelector('#job-count').textContent=view.jobs.length+' shown · newest first';renderJobs(view.jobs);renderWorkers(view.workers);renderMyWorkers(view.my_workers||[]);document.querySelector('#refresh-state').textContent='Live overview · updated just now'}catch(_){document.querySelector('#refresh-state').textContent='Connection interrupted · trying again automatically'}};
const start=()=>{if(!timer&&!document.hidden)timer=setInterval(refresh,2000)};document.addEventListener('visibilitychange',()=>{if(document.hidden&&timer){clearInterval(timer);timer=undefined}else start()});start();
</script>
</body>
</html>
{{end}}
@@ -16,7 +16,7 @@
<div class="card">
<p>The documentation site has not been built or the coordinator has not been pointed at it. From the repository root, run:</p>
<p><code>make docs</code> &nbsp;then restart the coordinator with <code>SCIMESH_DOCS_DIR</code> set to the generated <code>site/</code> directory (the <code>make demo-ui</code> demo does this automatically).</p>
<a class="back" href="/ui">← Back to the control room</a>
<a class="back" href="/ui/admin">← Back to the control room</a>
</div>
</main>
</body>
@@ -12,7 +12,7 @@
</head>
<body>
<main class="page">
<div style="display:flex;justify-content:space-between;align-items:center;gap:12px"><a class="back" href="/ui">← Back to control room</a>{{if .Session}}<div style="display:flex;gap:10px;align-items:center"><a class="back" href="/ui/profile">Profile</a><form method="post" action="/ui/logout" style="margin:0"><button type="submit" style="border:0;border-radius:10px;padding:9px 14px;background:#23344d;color:#dce8ff;font:inherit;font-weight:800;cursor:pointer">Log out</button></form></div>{{end}}</div>
<div style="display:flex;justify-content:space-between;align-items:center;gap:12px"><a class="back" href="/ui/admin">← Back to control room</a>{{if .Session}}<div style="display:flex;gap:10px;align-items:center"><a class="back" href="/ui/profile">Profile</a><form method="post" action="/ui/logout" style="margin:0"><button type="submit" style="border:0;border-radius:10px;padding:9px 14px;background:#23344d;color:#dce8ff;font:inherit;font-weight:800;cursor:pointer">Log out</button></form></div>{{end}}</div>
<div class="top"><div><p class="eyebrow">{{workloadLabel .Workload}}</p><h1 class="title">Live pipeline</h1><p class="subtitle">One job, shown from accepted input through its final coordinator-owned scientific result.</p></div><div class="live" id="refresh-state">Live · refreshes every 2 seconds</div></div>
<section class="panel summary"><div class="summary-top"><div><span id="status" class="badge badge-{{statusClass .Status}}">{{statusLabel .Status}}</span><p id="hint" class="hint">{{statusHint .Status}}</p></div><div id="stop-wrap" {{if not (cancellable .Status)}}class="hidden"{{end}}><button id="stop-job" class="stop" type="button">Stop unfinished shards</button><div class="live">Completed shards are preserved.</div></div></div><div class="bar"><span id="progress-bar" style="width:{{progressPercent .Completed .Failed .Cancelled .Total}}%"></span></div><p class="progress-line" id="progress">{{.Completed}} of {{.Total}} shards complete</p><div class="metrics"><div class="metric"><b id="total">{{.Total}}</b><small>total shards</small></div><div class="metric"><b id="completed">{{.Completed}}</b><small>completed</small></div><div class="metric"><b id="pending">{{.Pending}}</b><small>waiting</small></div><div class="metric"><b id="active">{{add .Leased .Running}}</b><small>with workers</small></div><div class="metric"><b id="failed">{{.Failed}}</b><small>failed</small></div><div class="metric"><b id="cancelled">{{.Cancelled}}</b><small>stopped</small></div></div></section>
@@ -20,7 +20,10 @@
<input id="password" name="password" type="password" autocomplete="current-password" required>
<button class="button" type="submit">Sign in</button>
</form>
{{if .Error}}<p class="error">{{.Error}}</p>{{end}}
{{if eq .Error "admin role required"}}
<p class="error">The admin console is reserved for the cluster administrator.</p>
<p class="alt">You are signed in as a non-admin. <a href="/ui/logout-form">Log out</a>, then sign in with the admin account — its login is printed by <code>coordinator serve</code> on first start and stored in <code>~/.scimesh/admin.password</code>.</p>
{{else if .Error}}<p class="error">{{.Error}}</p>{{end}}
<p class="alt">No account? <a href="/ui/register">Register</a></p>
</main>
</body>
@@ -0,0 +1,24 @@
{{define "logout-form.html"}}
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Sign out · SciMesh</title>
<style>
:root{color:#e5efff;background:#08111f;font:16px/1.5 Inter,ui-sans-serif,system-ui,sans-serif;color-scheme:dark}*{box-sizing:border-box}body{margin:0;min-height:100vh;display:grid;place-items:center;background:radial-gradient(circle at 10% -8%,#183f77 0,transparent 32rem),#08111f}a{color:#94bdff}.card{width:min(92vw,380px);border:1px solid #294662;border-radius:15px;background:#0d1a2cdd;box-shadow:0 20px 45px #00000021;padding:28px}.eyebrow{margin:0 0 4px;color:#7baaff;font-size:.78rem;font-weight:800;letter-spacing:.14em;text-transform:uppercase}h1{margin:0 0 12px;color:#f4f8ff;font-size:1.5rem;letter-spacing:-.03em}p{margin:0 0 18px;color:#9fb3cf;font-size:.92rem}code{color:#cfe0ff}.button{display:block;width:100%;margin-top:4px;border:0;border-radius:10px;padding:12px 16px;background:#ff7d92;color:#230810;font:inherit;font-weight:850;cursor:pointer}.alt{margin:18px 0 0;color:#9fb3cf;font-size:.92rem}
</style>
</head>
<body>
<main class="card">
<p class="eyebrow">SciMesh</p>
<h1>Sign out</h1>
<p>End the current session so you can sign in with a different account (for example the cluster administrator).</p>
<form method="post" action="/ui/logout">
<button class="button" type="submit">Log out</button>
</form>
<p class="alt">Changed your mind? <a href="/ui/login">Back to sign in</a></p>
</main>
</body>
</html>
{{end}}
@@ -11,7 +11,7 @@
</head>
<body>
<main class="page">
<a class="back" href="/ui">← Back to control room</a><p class="eyebrow">New computation</p><h1>Any workload, end to end</h1><p class="lead">Choose a workload from the installed library. Upload its dataset; workers compute shards; the coordinator reduces the partials into one final artifact.</p>
<a class="back" href="/ui/admin">← Back to control room</a><p class="eyebrow">New computation</p><h1>Any workload, end to end</h1><p class="lead">Choose a workload from the installed library. Upload its dataset; workers compute shards; the coordinator reduces the partials into one final artifact.</p>
<div class="layout"><form id="run" class="card" novalidate><label for="workload">Workload</label><select id="workload" name="workload"></select><p id="workload-meta" class="workload-meta hidden"></p><div id="params"></div><div class="split"><div><label for="chunk-rows">Rows per shard</label><input id="chunk-rows" name="chunk_rows" type="number" min="1" max="100000" value="1000" required><p class="hint">Smaller shards make more visible tasks.</p></div><div><label for="max-rows">Maximum dataset rows <small>(optional)</small></label><input id="max-rows" name="max_rows" type="number" min="1" max="10000000" placeholder="For example: 500"><p class="hint">Only the first N data rows become shards; the upload stays stored.</p></div></div><label for="file">Dataset file</label><input id="file" type="file" name="file" required accept=".tsv,.txt,.csv,text/tab-separated-values,text/csv"><p class="hint">A delimited table with a header row. The workload defines the required columns.</p><div id="preview" class="run-preview"><strong>Ready to plan a run.</strong><br>Select a dataset to see the file that will be sent to the coordinator.</div><button class="button" id="submit" type="submit">Create pipeline run →</button><p id="working" class="working hidden" aria-live="polite">Uploading dataset and creating coordinator-owned shards…</p><p id="error" class="error" role="alert"></p></form><aside class="aside"><h2>What you will observe</h2><ol><li><strong>Input accepted</strong><br>Dataset is validated and split into durable shard artifacts.</li><li><strong>Workers claim tasks</strong><br>Each worker downloads one shard, computes, and uploads a partial result.</li><li><strong>Global reduction</strong><br>The coordinator reduces all partials into one final artifact.</li><li><strong>Final download</strong><br>The result page exposes a checksum-protected result file.</li></ol><h2 style="margin-top:26px">Before you submit</h2><p>Keep at least one worker running in another terminal. The browser cannot start or control worker processes.</p><p>The form controls come from the workload's own declarations in the SDK library.</p></aside></div>
</main>
<script>
@@ -12,7 +12,7 @@
<main class="page">
<header class="top">
<div><p class="eyebrow">Account</p><h1>Your profile</h1></div>
<div style="display:flex;gap:10px;align-items:center"><a href="/ui">← Dashboard</a><form method="post" action="/ui/logout" style="margin:0"><button class="btn btn-muted" type="submit">Log out</button></form></div>
<div style="display:flex;gap:10px;align-items:center"><a href="/ui/admin">← Dashboard</a><form method="post" action="/ui/logout" style="margin:0"><button class="btn btn-muted" type="submit">Log out</button></form></div>
</header>
{{if .Error}}<div class="err">{{.Error}}</div>{{end}}
@@ -48,7 +48,7 @@
<div class="empty"><strong>No workloads are installed.</strong><br>Install an SDK workload package and run <code>scimesh workload export</code> to republish this catalog.</div>
{{end}}
</div>
<p class="lead" style="margin-top:26px"><a class="back" href="/ui">← Back to the control room</a></p>
<p class="lead" style="margin-top:26px"><a class="back" href="/ui/admin">← Back to the control room</a></p>
</main>
</body>
</html>
+9 -21
View File
@@ -217,29 +217,17 @@ func (s *Server) renderUI(w http.ResponseWriter, name string, data any) {
}
}
// handleUIHome lands the operator on the real UI. In session mode that is the
// admin console; under basic auth (no userservice, no roles) it is the job
// form, since the console does not exist there. The old demo control room is
// gone; /ui is a plain redirect so stale bookmarks still arrive somewhere
// useful.
func (s *Server) handleUIHome(w http.ResponseWriter, r *http.Request) {
ctx, cancel := s.reqCtx(r)
defer cancel()
view, err := s.uc.Dashboard.Overview(ctx, 20)
if err != nil {
s.writeError(w, r, err)
return
target := "/ui/jobs/new"
if s.uiSessionMode() {
target = "/ui/admin"
}
s.renderUI(w, "dashboard.html", view)
}
// handleUIOverviewJSON is the bounded polling projection used by the operator
// dashboard. It intentionally returns only the safe UI read model, never
// worker tokens, storage keys, or database entities.
func (s *Server) handleUIOverviewJSON(w http.ResponseWriter, r *http.Request) {
ctx, cancel := s.reqCtx(r)
defer cancel()
view, err := s.uc.Dashboard.Overview(ctx, 20)
if err != nil {
s.writeError(w, r, err)
return
}
writeJSON(w, http.StatusOK, view)
http.Redirect(w, r, target, http.StatusSeeOther)
}
func (s *Server) handleUINewJob(w http.ResponseWriter, r *http.Request) {
@@ -26,12 +26,19 @@ var adminUserActions = map[string]bool{
}
// requireAdmin gates a route on the session caller being an admin. It runs
// inside withUISession, which has already stamped the requester. A non-admin is
// sent back to the dashboard rather than shown the panel.
// inside withUISession, which has already stamped the requester. A signed-in
// non-admin is told why (and bounced to the login with the message); an
// unauthenticated caller never gets here — the gate has already sent them to
// the login page with the intended destination.
func requireAdmin(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if req, ok := authctx.From(r.Context()); !ok || !req.IsAdmin() {
http.Redirect(w, r, "/ui", http.StatusSeeOther)
target := "/ui/login?error=admin+role+required"
// Keep the destination so a successful login lands straight back.
if strings.HasPrefix(r.URL.Path, "/ui/") {
target += "&next=" + url.QueryEscape(r.URL.Path)
}
http.Redirect(w, r, target, http.StatusSeeOther)
return
}
next.ServeHTTP(w, r)
@@ -11,7 +11,6 @@ import (
"github.com/google/uuid"
"github.com/emil28092005/SciMesh/coordinator/internal/authctx"
"github.com/emil28092005/SciMesh/coordinator/internal/usecase"
)
func adminReq(t *testing.T, role string) *http.Request {
@@ -30,15 +29,15 @@ func TestRequireAdminAllowsAdminOnly(t *testing.T) {
t.Error("admin must reach the handler")
}
// Plain user is redirected to the dashboard.
// Plain user is redirected to the login with the reason and the destination.
reached = false
rec := httptest.NewRecorder()
h.ServeHTTP(rec, adminReq(t, "user"))
if reached {
t.Error("non-admin must not reach the handler")
}
if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/ui" {
t.Errorf("non-admin got %d -> %q, want 303 -> /ui", rec.Code, rec.Header().Get("Location"))
if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/ui/login?error=admin+role+required&next=%2Fui%2Fadmin" {
t.Errorf("non-admin got %d -> %q, want 303 -> login with the admin-required error and next", rec.Code, rec.Header().Get("Location"))
}
}
@@ -101,13 +100,24 @@ func TestAdminUserActionRejectsBadID(t *testing.T) {
}
}
func TestDashboardAdminLinkOnlyForAdmin(t *testing.T) {
admin := render(t, "dashboard.html", usecase.DashboardView{Session: &usecase.SessionView{Role: "admin"}})
if !strings.Contains(admin, "/ui/admin") {
t.Error("admin must see the Admin link")
func TestLoginPageExplainsAdminRequiredError(t *testing.T) {
html := render(t, "login.html", map[string]any{"Error": "admin role required"})
if !strings.Contains(html, "/ui/logout-form") {
t.Error("the admin-required error must offer a logout path to switch accounts")
}
user := render(t, "dashboard.html", usecase.DashboardView{Session: &usecase.SessionView{Role: "user"}})
if strings.Contains(user, "/ui/admin") {
t.Error("a plain user must not see the Admin link")
if !strings.Contains(html, "cluster administrator") {
t.Error("the admin-required error must name the admin account")
}
// Other errors keep the plain message, no logout teaser.
plain := render(t, "login.html", map[string]any{"Error": "invalid email or password"})
if strings.Contains(plain, "/ui/logout-form") {
t.Error("plain login errors must not advertise logout")
}
}
func TestLogoutFormRendersPostButton(t *testing.T) {
html := render(t, "logout-form.html", map[string]any{})
if !strings.Contains(html, `action="/ui/logout"`) || !strings.Contains(html, "Log out") {
t.Error("logout form must POST /ui/logout")
}
}
@@ -58,6 +58,13 @@ func (s *Server) handleUILoginForm(w http.ResponseWriter, r *http.Request) {
s.renderUI(w, "login.html", map[string]any{"Error": r.URL.Query().Get("error"), "Next": r.URL.Query().Get("next")})
}
// handleUILogoutForm renders a small confirm page for ending the current
// session. The actual logout stays a POST (/ui/logout); this page exists so a
// signed-in non-admin who hit an admin-only page can switch accounts.
func (s *Server) handleUILogoutForm(w http.ResponseWriter, r *http.Request) {
s.renderUI(w, "logout-form.html", map[string]any{})
}
func (s *Server) handleUIRegisterForm(w http.ResponseWriter, r *http.Request) {
s.renderUI(w, "register.html", map[string]any{"Error": r.URL.Query().Get("error")})
}
@@ -91,8 +98,9 @@ func (s *Server) handleUILogin(w http.ResponseWriter, r *http.Request) {
// value that escapes the UI prefix — that would be an open redirect.
next := strings.TrimSpace(r.FormValue("next"))
if next == "" || !strings.HasPrefix(next, "/ui/") {
next = "/ui"
next = "/ui/admin"
}
//nolint:gosec // G710: next is validated to start with /ui/ just above
http.Redirect(w, r, next, http.StatusSeeOther)
}
@@ -115,8 +115,8 @@ func TestHandleUILoginSetsCookieOnSuccess(t *testing.T) {
rec := httptest.NewRecorder()
s.handleUILogin(rec, postForm("/ui/login", url.Values{"email": {"a@b.com"}, "password": {"password123"}}))
if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/ui" {
t.Fatalf("got %d -> %q, want 303 -> /ui", rec.Code, rec.Header().Get("Location"))
if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/ui/admin" {
t.Fatalf("got %d -> %q, want 303 -> /ui/admin", rec.Code, rec.Header().Get("Location"))
}
cookies := rec.Result().Cookies()
if len(cookies) == 0 || cookies[0].Name != sessionCookie || cookies[0].Value != "issued.jwt.here" {
@@ -193,8 +193,8 @@ func TestHandleUILoginRedirectsToNext(t *testing.T) {
for _, next := range []string{"https://evil.example", "/", "//evil.example", "/api/jobs"} {
rec = httptest.NewRecorder()
s.handleUILogin(rec, postForm("/ui/login", url.Values{"email": {"a@b.com"}, "password": {"p"}, "next": {next}}))
if loc := rec.Header().Get("Location"); loc != "/ui" {
t.Errorf("next=%q landed on %q, want /ui (no open redirect)", next, loc)
if loc := rec.Header().Get("Location"); loc != "/ui/admin" {
t.Errorf("next=%q landed on %q, want /ui/admin (no open redirect)", next, loc)
}
}
}
@@ -17,15 +17,15 @@ func render(t *testing.T, name string, data any) string {
return buf.String()
}
func TestDashboardLogoutOnlyInSession(t *testing.T) {
withSession := render(t, "dashboard.html", usecase.DashboardView{Session: &usecase.SessionView{Role: "admin"}})
func TestJobPageLogoutOnlyInSession(t *testing.T) {
withSession := render(t, "job.html", usecase.JobDetailView{Session: &usecase.SessionView{Role: "admin"}})
if !strings.Contains(withSession, "/ui/logout") || !strings.Contains(withSession, "Log out") {
t.Error("dashboard must show a logout control in session mode")
t.Error("job page must show a logout control in session mode")
}
noSession := render(t, "dashboard.html", usecase.DashboardView{})
noSession := render(t, "job.html", usecase.JobDetailView{})
if strings.Contains(noSession, "/ui/logout") {
t.Error("dashboard must not show logout under basic auth (no session)")
t.Error("job page must not show logout under basic auth (no session)")
}
}
-12
View File
@@ -20,18 +20,6 @@ func ownerFromContext(ctx context.Context) *uuid.UUID {
return nil
}
// uiOwnerFilter returns the owner a UI listing must be restricted to: nil for an
// operator/admin or an unauthenticated (basic-auth) session, which see all jobs,
// or the caller's id for a plain user, who sees only their own.
func uiOwnerFilter(ctx context.Context) *uuid.UUID {
r, ok := authctx.From(ctx)
if !ok || r.IsAdmin() {
return nil
}
id := r.UserID
return &id
}
// authorizeJobAccess enforces that a non-admin user may only act on their own
// job. It returns ErrJobNotFound — not a 403 — on a mismatch, so the response
// never reveals that another user's job exists.
+7 -88
View File
@@ -18,15 +18,8 @@ import (
// It intentionally exposes no storage paths or credentials.
type UIReadRepository interface {
GetJob(ctx context.Context, jobID uuid.UUID) (*domain.Job, error)
// ListJobs returns the most recent jobs. A non-nil owner restricts the list
// to that user's jobs; nil returns all (operator/admin view).
ListJobs(ctx context.Context, owner *uuid.UUID, limit int) ([]domain.Job, error)
ListTasksByJob(ctx context.Context, jobID uuid.UUID) ([]domain.Task, error)
ListTasksByJobs(ctx context.Context, jobIDs []uuid.UUID) (map[uuid.UUID][]domain.Task, error)
ListWorkers(ctx context.Context, limit int) ([]domain.Worker, error)
// ListWorkersByOwner returns the most recent workers registered by one user,
// for the "my machines" section of the dashboard.
ListWorkersByOwner(ctx context.Context, owner uuid.UUID, limit int) ([]domain.Worker, error)
ListArtifactsByJob(ctx context.Context, jobID uuid.UUID) ([]domain.Artifact, error)
}
@@ -88,18 +81,13 @@ type WorkerCard struct {
LastHeartbeatAt time.Time `json:"last_heartbeat_at"`
}
type DashboardView struct {
Jobs []JobCard `json:"jobs"`
Workers []WorkerCard `json:"workers"`
// MyWorkers is the signed-in user's own registered workers. Empty for an
// admin or a basic-auth operator, who instead see the whole fleet in Workers.
MyWorkers []WorkerCard `json:"my_workers"`
ActiveJobs int `json:"active_jobs"`
FinishedJobs int `json:"finished_jobs"`
OnlineWorkers int `json:"online_workers"`
// Session is the signed-in user, when the UI runs in session mode. nil under
// basic auth. Template-only, never serialised to the polling JSON.
Session *SessionView `json:"-"`
type JobDetailView struct {
JobCard
Tasks []TaskCard `json:"tasks"`
Artifacts []ArtifactCard `json:"artifacts"`
Parameters []ParameterCard `json:"parameters"`
FinalResultAvailable bool `json:"final_result_available"`
Session *SessionView `json:"-"`
}
// SessionView is the minimal identity the UI header needs to show who is signed
@@ -119,15 +107,6 @@ func sessionViewFrom(ctx context.Context) *SessionView {
return &SessionView{Role: r.Role, Verified: r.Verified}
}
type JobDetailView struct {
JobCard
Tasks []TaskCard `json:"tasks"`
Artifacts []ArtifactCard `json:"artifacts"`
Parameters []ParameterCard `json:"parameters"`
FinalResultAvailable bool `json:"final_result_available"`
Session *SessionView `json:"-"`
}
type Dashboard struct {
read UIReadRepository
catalog *workloads.Catalog
@@ -137,66 +116,6 @@ func NewDashboard(read UIReadRepository, catalog *workloads.Catalog) *Dashboard
return &Dashboard{read: read, catalog: catalog}
}
func (d *Dashboard) Overview(ctx context.Context, limit int) (DashboardView, error) {
jobs, err := d.read.ListJobs(ctx, uiOwnerFilter(ctx), limit)
if err != nil {
return DashboardView{}, err
}
workers, err := d.read.ListWorkers(ctx, limit)
if err != nil {
return DashboardView{}, err
}
out := DashboardView{Jobs: make([]JobCard, 0, len(jobs)), Workers: make([]WorkerCard, 0, len(workers))}
jobIDs := make([]uuid.UUID, 0, len(jobs))
for _, job := range jobs {
jobIDs = append(jobIDs, job.ID)
}
tasksByJob, err := d.read.ListTasksByJobs(ctx, jobIDs)
if err != nil {
return DashboardView{}, err
}
for _, job := range jobs {
card := jobCard(job, tasksByJob[job.ID])
out.Jobs = append(out.Jobs, card)
switch card.Status {
case string(domain.JobCompleted), string(domain.JobFailed), string(domain.JobCancelled):
out.FinishedJobs++
default:
out.ActiveJobs++
}
}
for _, worker := range workers {
out.Workers = append(out.Workers, workerCard(worker))
if worker.Status == domain.WorkerOnline || worker.Status == domain.WorkerBusy {
out.OnlineWorkers++
}
}
// A plain user also gets a dedicated "my machines" list scoped to their own
// registrations; an admin/operator sees only the fleet above.
if owner := uiOwnerFilter(ctx); owner != nil {
mine, err := d.read.ListWorkersByOwner(ctx, *owner, limit)
if err != nil {
return DashboardView{}, err
}
out.MyWorkers = make([]WorkerCard, 0, len(mine))
for _, worker := range mine {
out.MyWorkers = append(out.MyWorkers, workerCard(worker))
}
}
out.Session = sessionViewFrom(ctx)
return out, nil
}
func workerCard(w domain.Worker) WorkerCard {
return WorkerCard{
ID: w.ID.String(),
Name: w.Name,
Status: string(w.Status),
Capabilities: w.Capabilities,
LastHeartbeatAt: w.LastHeartbeatAt,
}
}
func (d *Dashboard) JobDetail(ctx context.Context, jobID uuid.UUID) (JobDetailView, error) {
job, err := d.read.GetJob(ctx, jobID)
if err != nil {
@@ -36,32 +36,6 @@ func userCtx(id uuid.UUID, role string) context.Context {
return authctx.With(context.Background(), authctx.Requester{UserID: id, Role: role})
}
func TestOverviewScopesJobsByOwner(t *testing.T) {
dash, jobs := newDashboard()
alice, bob := uuid.New(), uuid.New()
ownedJob(t, jobs, alice)
ownedJob(t, jobs, bob)
// A plain user sees only their own job.
v, err := dash.Overview(userCtx(alice, "user"), 20)
if err != nil {
t.Fatal(err)
}
if len(v.Jobs) != 1 {
t.Errorf("alice sees %d jobs, want 1", len(v.Jobs))
}
// An admin sees every job.
if v, _ := dash.Overview(userCtx(uuid.New(), "admin"), 20); len(v.Jobs) != 2 {
t.Errorf("admin sees %d jobs, want 2", len(v.Jobs))
}
// No requester (basic-auth operator) sees every job — unchanged behaviour.
if v, _ := dash.Overview(context.Background(), 20); len(v.Jobs) != 2 {
t.Errorf("operator sees %d jobs, want 2", len(v.Jobs))
}
}
func TestJobDetailRejectsAnotherUsersJob(t *testing.T) {
dash, jobs := newDashboard()
alice, bob := uuid.New(), uuid.New()
@@ -1,66 +0,0 @@
package usecase_test
import (
"context"
"testing"
"time"
"github.com/google/uuid"
"github.com/emil28092005/SciMesh/coordinator/internal/domain"
"github.com/emil28092005/SciMesh/coordinator/internal/memstore"
"github.com/emil28092005/SciMesh/coordinator/internal/usecase"
)
func newDashboardWithWorkers() (*usecase.Dashboard, *memstore.WorkerRepo) {
jobs := memstore.NewJobRepo()
tasks := memstore.NewTaskRepo()
workers := memstore.NewWorkerRepo()
artifacts := memstore.NewArtifactRepo()
return usecase.NewDashboard(memstore.NewUIReadRepo(jobs, tasks, workers, artifacts), testCatalog()), workers
}
func seedWorker(t *testing.T, workers *memstore.WorkerRepo, owner *uuid.UUID, name string) {
t.Helper()
w := &domain.Worker{
ID: uuid.New(),
Name: name,
Capabilities: []string{"similarity-search"},
Status: domain.WorkerOnline,
OwnerID: owner,
LastHeartbeatAt: time.Now().UTC(),
}
if err := workers.Insert(context.Background(), w); err != nil {
t.Fatalf("insert worker: %v", err)
}
}
func TestOverviewSplitsMyWorkers(t *testing.T) {
dash, workers := newDashboardWithWorkers()
alice, bob := uuid.New(), uuid.New()
seedWorker(t, workers, &alice, "alice-box")
seedWorker(t, workers, &bob, "bob-box")
seedWorker(t, workers, nil, "lab-shared") // owner-less shared-token worker
// A plain user sees the whole fleet, but MyWorkers holds only their own.
v, err := dash.Overview(userCtx(alice, "user"), 20)
if err != nil {
t.Fatal(err)
}
if len(v.Workers) != 3 {
t.Errorf("fleet shows %d workers, want 3", len(v.Workers))
}
if len(v.MyWorkers) != 1 || v.MyWorkers[0].Name != "alice-box" {
t.Errorf("MyWorkers = %+v, want only alice-box", v.MyWorkers)
}
// An admin is not owner-scoped: they get the fleet and no personal list.
if av, _ := dash.Overview(userCtx(uuid.New(), "admin"), 20); len(av.MyWorkers) != 0 || len(av.Workers) != 3 {
t.Errorf("admin MyWorkers=%d Workers=%d, want 0 and 3", len(av.MyWorkers), len(av.Workers))
}
// A basic-auth operator (no requester) also gets no personal list.
if ov, _ := dash.Overview(context.Background(), 20); len(ov.MyWorkers) != 0 {
t.Errorf("operator MyWorkers=%d, want 0", len(ov.MyWorkers))
}
}
+3 -2
View File
@@ -65,7 +65,7 @@ Write-Host "SciMesh $Component installed: $Target"
Write-Host ""
if ($Component -eq "coordinator") {
if ($AutoStart -eq "1") {
Write-Host "Starting the platform and opening the control room in your browser..."
Write-Host "Starting the platform and opening the admin console in your browser..."
Write-Host "(stop it with Ctrl-C; it keeps your data in ~\.scimesh)"
Write-Host ""
& $Target serve --open
@@ -97,6 +97,7 @@ if ($Component -eq "coordinator") {
Write-Host "For a coordinator started with 'coordinator serve', the worker token is"
Write-Host "in ~\.scimesh\worker.token on that machine. Set SCIMESH_PIP_PACKAGE to"
Write-Host "install scimesh into a managed venv, or install it yourself:"
Write-Host " pip install scimesh"
Write-Host " set SCIMESH_PIP_PACKAGE=<your wheel or index>"
Write-Host " $Target setup"
}
}
+2 -2
View File
@@ -78,7 +78,7 @@ fi
if [ "$COMPONENT" = "coordinator" ]; then
if [ "$AUTO_START" = "1" ]; then
echo
echo "Starting the platform and opening the control room in your browser..."
echo "Starting the platform and opening the admin console in your browser..."
echo "(stop it with Ctrl-C; it keeps your data in ~/.scimesh)"
echo
exec "$TARGET" serve --open
@@ -112,5 +112,5 @@ else
echo "For a coordinator started with 'coordinator serve', the worker token is"
echo "in ~/.scimesh/worker.token on that machine. Set SCIMESH_PIP_PACKAGE to"
echo "install scimesh into a managed venv, or install it yourself:"
echo " pip install scimesh"
echo " SCIMESH_PIP_PACKAGE=<your wheel or index> worker-agent setup"
fi
+12 -10
View File
@@ -31,12 +31,12 @@ The two halves of the project:
- **A distributed worker** that executes the same SDK workload handlers on
tasks claimed from the coordinator, with digest-pinned `TaskSpec`s,
resource reservation, and allowlist-driven workload discovery.
- **An operator UI** served by the coordinator: the control room, a workload
library page, a workload-agnostic "new computation" form whose controls come
from each workload's own `UIElement` declarations, an **admin console**
- **An operator UI** served by the coordinator: the **admin console**
(`/ui/admin`) for cluster operators — system/storage/health, jobs,
worker trust, users and worker keys, workload enable/disable, metrics and
the worker token — and this documentation site at `/ui/docs/`.
the worker token — plus a workload-agnostic "new computation" form whose
controls come from each workload's own `UIElement` declarations, job detail
pages, a workload library page, and this documentation site at `/ui/docs/`.
## Quick start
@@ -46,14 +46,14 @@ local workers — is embedded in a single binary; no PostgreSQL, no Docker, no
Python setup.
```bash
# Linux / macOS — installs and opens the control room automatically
# Linux / macOS — installs and opens the admin console automatically
curl -fsSL https://raw.githubusercontent.com/emil28092005/SciMesh/main/install.sh | bash
# Windows (PowerShell)
powershell -ExecutionPolicy Bypass -c "irm https://raw.githubusercontent.com/emil28092005/SciMesh/main/install.ps1 | iex"
```
The installer starts the platform and opens the control room in your browser
The installer starts the platform and opens the admin console in your browser
(set `SCIMESH_AUTO_START=0` to install only). The first start prints the admin
login (also stored under `~/.scimesh`). `coordinator serve --workers 2`
spawns two local workers; `SCIMESH_PIP_PACKAGE` points the managed venv at
@@ -113,10 +113,12 @@ chmod +x coordinator
```
It spawns `python -m scimesh.worker.task`, so the machine needs Python 3
with the `scimesh` package (`pip install scimesh`, or let the managed venv
do it via `SCIMESH_PIP_PACKAGE`). For a `coordinator serve` instance, the
worker token is in `~/.scimesh/worker.token`. On Windows set
`SCIMESH_COMPONENT=worker` for `install.ps1`.
with the `scimesh` package. The wizard installs it into its own venv; the
package must come from your wheel, checkout or index — point
`SCIMESH_PIP_PACKAGE` at it (the PyPI name `scimesh` belongs to an unrelated
project). For a `coordinator serve` instance, the worker token is in
`~/.scimesh/worker.token`. On Windows set `SCIMESH_COMPONENT=worker` for
`install.ps1`.
- **coordinator** needs no external services at all in its default mode:
`coordinator serve` embeds SQLite (both databases), the userservice, and
local workers. The `SCIMESH_DB=postgres` engine remains for cluster