Serve documentation from the operator UI

2026-08-02 15:50:07 +03:00
parent 284aef5d6f
commit f20cc7fe00
72 changed files with 3509 additions and 698 deletions
@@ -0,0 +1,86 @@
+# SciMesh
+
+SciMesh is a local-first platform for scientific computation on molecular
+datasets. It turns a scientific run into independent tasks, dispatches them
+to Python workers, and deterministically combines the partial results into a
+checksum-protected final artifact.
+
+The two halves of the project:
+
+- **The Workload SDK (`scimesh.sdk`)** — a strict Python framework for
+  authoring scientific workloads. Workloads are ordinary user scripts built
+  on the SDK; they run locally, in the conformance harness, and on claimed
+  coordinator tasks without touching any other part of the program.
+- **The coordinator and worker** — a Go/PostgreSQL coordinator with an
+  operator UI and Python worker agents that execute SDK-built workloads over
+  an HTTP contract.
+
+## What is implemented
+
+- **SDK-built workloads**: `similarity-search` (exact top-k Tanimoto search),
+  `similarity-graph` (exact sparse similarity graph with pair-coverage),
+  `descriptor-batch` (pinned RDKit 2D descriptors), and `molwt-filter`
+  (molecular-weight filter — the minimal authoring example).
+- **`MapReduceWorkload`**: the primary authoring scaffold. A subclass
+  declares identity, parameters, ports, and scientific hooks; the SDK
+  assembles the manifest, map/reduce stages, the digest-pinned
+  planner/runner/reducer, and the exact-artifact verifier.
+- **A local conformance runtime** (`LocalCoreBatchExecutor`): a trusted,
+  in-process harness that validates scientific parity, sealed outputs,
+  provenance, and limits.
+- **A distributed worker** that executes the same SDK workload handlers on
+  tasks claimed from the coordinator, with digest-pinned `TaskSpec`s,
+  resource reservation, and allowlist-driven workload discovery.
+- **An operator UI** served by the coordinator: the control room, a workload
+  library page, and this documentation site at `/ui/docs/`.
+
+## Quick start
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+pip install -e '.[dev]'
+```
+
+List the installed SDK workloads and run one locally:
+
+```bash
+scimesh workload list
+scimesh workload run molwt-filter \
+  --input molecules.tsv \
+  --params '{"min_molwt": 40.0}' \
+  -o filtered.csv
+```
+
+Run the local scientific CLI workloads:
+
+```bash
+scimesh help
+```
+
+Start the full demo (PostgreSQL, coordinator, UI, two workers):
+
+```bash
+make demo-ui
+# open http://localhost:18080/ui  (root@scimesh.local / rootpassword)
+```
+
+Build and serve this documentation site:
+
+```bash
+make docs
+make docs-serve      # http://localhost:8000
+```
+
+## Where to go next
+
+- [SDK overview](sdk/overview.md) — what the SDK is and is not.
+- [Authoring workloads](sdk/authoring-workloads.md) — write your first
+  workload with `MapReduceWorkload`.
+- [Workload CLI](sdk/cli.md) — list, run, and export workloads from the
+  command line.
+- [Worker integration](sdk/worker-integration.md) — how the distributed
+  worker executes SDK workloads.
+- [API reference](api/index.md) — the complete `scimesh.sdk` API, generated
+  from docstrings.
+- [Documentation approach](approach.md) — the rules this site is written by.