Update workload SDK roadmap
This commit is contained in:
Emil
@@ -0,0 +1,123 @@
|
||||
# SciMesh Workload SDK roadmap
|
||||
|
||||
**Status:** future design and sequencing document. No SDK package, commands, or
|
||||
general verifier abstraction described here is implemented yet.
|
||||
|
||||
## Purpose and boundaries
|
||||
|
||||
The SDK should let a scientific developer add an allowlisted workload without
|
||||
learning coordinator internals or writing SQL, while keeping one scientific
|
||||
implementation usable locally and in distributed execution:
|
||||
|
||||
```text
|
||||
scientific implementation -> local adapter -> planner/tasks -> reducer -> verifier
|
||||
```
|
||||
|
||||
It must not execute arbitrary code or shell commands supplied by a coordinator.
|
||||
SDK v1 is not a public marketplace, generic container/job runner, cross-language
|
||||
SDK, automatic correctness-proof system, or immediate route to stochastic ML or
|
||||
GPU workloads.
|
||||
|
||||
The current foundation is the Python `DistributedWorkload` protocol and registry
|
||||
under `scimesh/distributed/`, the Worker Agent under `scimesh/worker/`, and the
|
||||
coordinator API contract. See [CTX-07](ctx-07-distributed-workload-protocol.md),
|
||||
[worker-building guide](building-workers.md), and [API contract](api-contract.md).
|
||||
|
||||
## Proposed public concepts
|
||||
|
||||
| Concept | Responsibility |
|
||||
| --- | --- |
|
||||
| `WorkloadDefinition` / `WorkloadManifest` | Name, versions, schemas, execution and verification metadata. |
|
||||
| `ParameterSchema`, `InputSpec`, `OutputSpec`, `ArtifactRef` | Typed public inputs and durable artifact shapes. |
|
||||
| `TaskPlan`, `Planner`, `Runner`, `Reducer` | Validate, split, execute, and deterministically combine work. |
|
||||
| `Verifier` | Accept or reject result evidence; never silently downgrade checks. |
|
||||
| `ResourceRequirements`, `ExecutionProfile`, `ReproducibilityProfile` | Bounded resource needs and pinned execution assumptions. |
|
||||
|
||||
A manifest should include workload/version and SDK compatibility versions,
|
||||
description, parameter/input/output schemas, planner/runner/reducer/verifier
|
||||
types, determinism and trust profiles, resource/output limits, worker
|
||||
capabilities, and environment or image digest. Compatibility must be explicit
|
||||
among SDK, coordinator protocol, worker runtime, workload, output schema, and
|
||||
verifier versions.
|
||||
|
||||
## Contracts
|
||||
|
||||
**Planner:** validates before durable Job/Task creation; produces versioned,
|
||||
JSON-serializable plans that refer only to durable artifacts; gives stable task
|
||||
order and expected resources/outputs; fails transactionally without a partial
|
||||
task graph.
|
||||
|
||||
**Runner:** receives typed parameters and owned artifact references; runs only
|
||||
allowlisted SDK code; writes to its attempt directory; produces output manifest,
|
||||
metrics, and sanitized failures; respects cancellation/lease loss when platform
|
||||
support exists; never uses `shell=True` or unnecessarily exposes credentials.
|
||||
|
||||
**Reducer:** consumes only accepted partial artifacts in stable order; is
|
||||
idempotent or coordinator-state protected; creates a versioned final manifest;
|
||||
defines missing, duplicate, and malformed-shard failures.
|
||||
|
||||
**Verifier:** is versioned with the workload and states one of byte-exact,
|
||||
canonical, numeric, domain-specific, or trust-policy comparison. It processes
|
||||
structured manifests and bounded streams where practical, records sanitized
|
||||
evidence, and rejects inconsistent results.
|
||||
|
||||
Current untrusted quorum is only `ExactArtifactVerifier`: distinct owners must
|
||||
produce whole files with identical SHA-256. Future modes are
|
||||
`CanonicalRecordVerifier`, `NumericToleranceVerifier`,
|
||||
`DomainSpecificVerifier`, and `TrustedWorkerPolicy`. Canonical mode requires a
|
||||
specified parser/schema/order/encoding/serialization; numeric mode compares
|
||||
structured values, not CSV text.
|
||||
|
||||
## Resources and reproducibility
|
||||
|
||||
The extensible requirement model is `cpu_cores`, `memory_mb`, `scratch_mb`,
|
||||
`gpu_count`, `gpu_memory_mb`, `accelerator_kind`, `exclusive_device`, and
|
||||
`estimated_output_bytes`. It must not imply one Task equals one CPU core.
|
||||
|
||||
Untrusted byte-exact workloads require a pinned image/environment digest,
|
||||
runtime and dependency versions, fixed locale/timezone/UTF-8/newlines/dialect,
|
||||
explicit invalid-row policy and algorithm options, canonical ordering, stable
|
||||
archive metadata, golden fixtures, two independently provisioned workers, and
|
||||
local/distributed plus retry/completion-order parity tests.
|
||||
|
||||
## Compatibility evolution
|
||||
|
||||
The stable release retains the current one-input/one-result task contract.
|
||||
First, a composite manifest artifact may reference multiple logical inputs;
|
||||
later, ordered input and output artifact collections can become first-class.
|
||||
The transition must be versioned and retain old workload compatibility.
|
||||
|
||||
Discovery should use an installed Python package, manifest, pinned environment
|
||||
metadata, explicit entry points, and golden fixtures. It must be allowlisted;
|
||||
never scan or execute user-provided module paths.
|
||||
|
||||
## Delivery sequence
|
||||
|
||||
1. Finish distributed `similarity-graph` and reliability/cross-language CI.
|
||||
2. Stabilize manifest, schema, planner/runner/reducer interfaces, exact verifier,
|
||||
compatibility metadata, and an author guide.
|
||||
3. Deliver `descriptor-batch` as the reference workload: pinned RDKit 2D
|
||||
descriptors; canonical one-row-per-input CSV; shard-index concatenation with
|
||||
one header; byte-identical local/distributed output and two-worker quorum.
|
||||
4. Add standardization, SMARTS screening, fingerprint export, fixed-template
|
||||
reaction enumeration, then reaction validation/descriptors.
|
||||
5. Generalize composite artifacts, process slots, resource requests, and richer
|
||||
verifier policies.
|
||||
6. Only then consider pinned, trusted/domain-verified numeric, ML, docking, QM,
|
||||
MD, and GPU workloads.
|
||||
|
||||
Future developer tooling may include `scimesh workload init`, `validate`,
|
||||
`test-local`, `test-distributed`, `golden`, and `package`; these commands do not
|
||||
exist today. A template should generate a manifest, schemas, planner, runner,
|
||||
reducer, verifier, unit tests, golden fixture, two-worker integration test, and
|
||||
documentation.
|
||||
|
||||
## Open decisions
|
||||
|
||||
- Is the SDK part of `scimesh` or a separately versioned Python distribution?
|
||||
- What stable bridge connects Go orchestration to Python planners/reducers?
|
||||
- Where do future verifiers execute, and how are environments attested?
|
||||
- Which trust modes may run each verifier profile?
|
||||
- Who may install/enable workloads in multi-user deployments?
|
||||
- How are composite I/O, version negotiation, output-growth limits, and
|
||||
numeric-tolerance access governed without breaking the existing API?
|
||||
Reference in New Issue
Block a user