Emil_Shanaty/SciMesh
1
0
Fork 0
Code Issues Pull Requests Actions 121 Packages Projects Releases Wiki Activity
56 Commits 13 Branches 28 Tags
f953112cfd723f2995eb9fea052d44d1a24582ef
Commit Graph
5 Commits
Author SHA1 Message Date
Emil 19cbf7f113 Harden distributed pipeline 2026-07-24 14:16:42 +03:00
Emil 7547a30bde Add job cancellation and dataset row limit
coordinator / test (push) Waiting to run
Details
2026-07-23 23:14:25 +03:00
Emil e83e0b5e1f Add local operator web interface 2026-07-23 22:08:58 +03:00
Efremenko Arhip 3b41455b20 feat(coordinator): running state, worker liveness, request-size limits 
Polish pass hardening the queue and closing plan gaps.

- Task state machine gains `running`: the first heartbeat moves a task from
  leased to running (migrations 0006/0007 add the enum value and extend the
  lease-integrity check). verifyLease, ExpireLease, the reaper SQL, and job
  progress all treat leased and running alike.
- Worker liveness: a heartbeat from a registered worker (UUID worker_id) bumps
  its last_heartbeat_at online; a second background reaper marks workers offline
  after WORKER_OFFLINE_AFTER of silence (RunReaper generalized to RunPeriodic).
- Request-size limits: JSON bodies capped at 1 MiB; dataset/artifact uploads
  capped at MAX_UPLOAD_BYTES (default 1 GiB) via http.MaxBytesReader.
- Tests cover the running transition, liveness + offline reaper (unit over
  memstore and integration over Postgres).
 2026-07-23 17:36:26 +03:00
Efremenko Arhip e5ba27951a test(coordinator): unit + integration coverage across every layer 
- domain: NewJobWithTasks, DeriveStatus, NewUploadedJob, NewShardTask,
  NewWorker, NewArtifact/SetContent (domain 47% -> 88%).
- internal/memstore: in-memory implementations of every usecase port, so
  orchestration can be tested without Postgres or a filesystem.
- usecase: claim/renew/complete/fail/create/register/upload/submit-dataset
  flows over memstore, including rule-10 cross-task rejection, idempotent
  replay, lease sweep-on-claim, and dataset chunking (usecase 0% -> 73%).
- transport: httptest end-to-end over real use cases + memstore — auth,
  readiness, full lifecycle, multipart upload + shard input, error mappings
  (0% -> 70%).
- postgres integration: fix the tests broken by the artifact_id switch and add
  worker-repo, artifact-repo, and shard-task (nullable input_uri) round-trips.

go test -race ./... is clean; golangci-lint (incl. integration tag) reports 0.
 2026-07-23 16:47:23 +03:00