Efremenko Arhip
58da6ef139
Task results are now coordinator-owned artifacts end to end.
- domain.Task carries ResultArtifactID instead of ResultURI/ResultSHA256;
CompleteWith and its idempotency key are keyed on the artifact id.
- CompleteTask verifies the referenced artifact was stored for this exact
task (rule 10): a worker cannot finish task B with task A's artifact, nor
name an id that isn't a partial_result. Mismatch → 409.
- POST /tasks/{id}/result takes {result:{artifact_id,...}}; ListResults and
ResultManifest follow.
- migration 0004 drops result_uri/result_sha256 and requires a completed task
to reference its result_artifact_id.
- smoke and requests.http exercise upload → complete-by-id → replay → conflict.
200 lines
5.5 KiB
HTTP
200 lines
5.5 KiB
HTTP
# SciMesh Coordinator — API requests
|
|
#
|
|
# Runnable from any editor with a REST client (VSCodium/VS Code "REST Client",
|
|
# JetBrains HTTP Client). Click "Send Request" above each block, top to bottom:
|
|
# later requests reuse ids captured from earlier responses.
|
|
#
|
|
# Start the stack first: docker compose up -d
|
|
|
|
@host = http://localhost:8080
|
|
@token = change-me
|
|
@worker = worker-1
|
|
|
|
### Readiness — the only unauthenticated endpoint (probes the database)
|
|
GET {{host}}/health
|
|
|
|
### Auth check — no token must be rejected with 401
|
|
POST {{host}}/tasks/claim
|
|
Content-Type: application/json
|
|
|
|
{ "worker_id": "{{worker}}" }
|
|
|
|
### 0. Register a worker (201)
|
|
# @name register
|
|
POST {{host}}/workers/register
|
|
Authorization: Bearer {{token}}
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"name": "lab-worker-01",
|
|
"capabilities": ["similarity_search"],
|
|
"cpu_count": 8,
|
|
"memory_mb": 16384
|
|
}
|
|
|
|
@workerId = {{register.response.body.worker_id}}
|
|
|
|
### 1. Create a job and its chunks (201)
|
|
# The coordinator splits the submission into one task per chunk, transactionally.
|
|
# @name createJob
|
|
POST {{host}}/jobs
|
|
Authorization: Bearer {{token}}
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"workload": "similarity_search",
|
|
"input_uri": "s3://chembl/full.sdf",
|
|
"parameters": { "top_k": 10 },
|
|
"chunks": [
|
|
{ "chunk_index": 0, "input_uri": "s3://chembl/shard-0.sdf", "input_sha256": "aaa", "max_attempts": 3 },
|
|
{ "chunk_index": 1, "input_uri": "s3://chembl/shard-1.sdf", "input_sha256": "bbb", "max_attempts": 3 },
|
|
{ "chunk_index": 2, "input_uri": "s3://chembl/shard-2.sdf", "input_sha256": "ccc", "max_attempts": 3 }
|
|
]
|
|
}
|
|
|
|
@jobId = {{createJob.response.body.id}}
|
|
|
|
### 2. Claim a task (200, or 204 when the queue is empty)
|
|
# Each call leases a different task; run it repeatedly to see chunk_index advance.
|
|
# @name claim
|
|
POST {{host}}/tasks/claim
|
|
Authorization: Bearer {{token}}
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"worker_id": "{{worker}}",
|
|
"capabilities": ["similarity_search"],
|
|
"max_concurrency": 1
|
|
}
|
|
|
|
@taskId = {{claim.response.body.task_id}}
|
|
@attempt = {{claim.response.body.attempt}}
|
|
|
|
### 3. Heartbeat — renew the lease while the task is still running (200)
|
|
POST {{host}}/tasks/{{taskId}}/heartbeat
|
|
Authorization: Bearer {{token}}
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"worker_id": "{{worker}}",
|
|
"attempt": {{attempt}}
|
|
}
|
|
|
|
### 3a. Upload a partial-result artifact (200) — while the task is leased
|
|
# Identity travels in headers per the contract; the body is streamed as-is.
|
|
# @name uploadArtifact
|
|
PUT {{host}}/tasks/{{taskId}}/artifacts/result.csv
|
|
Authorization: Bearer {{token}}
|
|
Content-Type: text/csv
|
|
X-Worker-ID: {{worker}}
|
|
X-Task-Attempt: {{attempt}}
|
|
|
|
query,match,score
|
|
CHEMBL25,CHEMBL139,0.87
|
|
|
|
@artifactId = {{uploadArtifact.response.body.artifact_id}}
|
|
|
|
### 3b. Download the artifact by id (200)
|
|
GET {{host}}/artifacts/{{artifactId}}/download
|
|
Authorization: Bearer {{token}}
|
|
|
|
### 3c. Upload a second artifact — used by the conflict check below (200)
|
|
# @name uploadArtifact2
|
|
PUT {{host}}/tasks/{{taskId}}/artifacts/secondary.csv
|
|
Authorization: Bearer {{token}}
|
|
Content-Type: text/csv
|
|
X-Worker-ID: {{worker}}
|
|
X-Task-Attempt: {{attempt}}
|
|
|
|
query,match,score
|
|
CHEMBL25,CHEMBL521,0.42
|
|
|
|
@artifactId2 = {{uploadArtifact2.response.body.artifact_id}}
|
|
|
|
### 4. Submit the result, referencing the uploaded artifact (200)
|
|
POST {{host}}/tasks/{{taskId}}/result
|
|
Authorization: Bearer {{token}}
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"worker_id": "{{worker}}",
|
|
"attempt": {{attempt}},
|
|
"result": { "artifact_id": "{{artifactId}}", "content_type": "text/csv" },
|
|
"metrics": { "elapsed_ms": 1234, "candidates": 50000 }
|
|
}
|
|
|
|
### 4a. Replay the same result — must be idempotent (200, not 409)
|
|
POST {{host}}/tasks/{{taskId}}/result
|
|
Authorization: Bearer {{token}}
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"worker_id": "{{worker}}",
|
|
"attempt": {{attempt}},
|
|
"result": { "artifact_id": "{{artifactId}}" }
|
|
}
|
|
|
|
### 4b. A different artifact for the same task — conflict (409)
|
|
POST {{host}}/tasks/{{taskId}}/result
|
|
Authorization: Bearer {{token}}
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"worker_id": "{{worker}}",
|
|
"attempt": {{attempt}},
|
|
"result": { "artifact_id": "{{artifactId2}}" }
|
|
}
|
|
|
|
### 4c. Another worker submitting for this task — conflict (409)
|
|
POST {{host}}/tasks/{{taskId}}/result
|
|
Authorization: Bearer {{token}}
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"worker_id": "impostor",
|
|
"attempt": {{attempt}},
|
|
"result": { "artifact_id": "{{artifactId}}" }
|
|
}
|
|
|
|
### 5. Report a failure instead (200)
|
|
# retryable=true returns the task to the queue while attempts remain;
|
|
# retryable=false fails it terminally.
|
|
POST {{host}}/tasks/{{taskId}}/failure
|
|
Authorization: Bearer {{token}}
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"worker_id": "{{worker}}",
|
|
"attempt": {{attempt}},
|
|
"error_code": "download_failed",
|
|
"error_message": "checksum mismatch on shard",
|
|
"retryable": true
|
|
}
|
|
|
|
### 6. Job progress (200)
|
|
GET {{host}}/jobs/{{jobId}}
|
|
Authorization: Bearer {{token}}
|
|
|
|
### --- error cases -------------------------------------------------------
|
|
|
|
### Malformed UUID in the path (400)
|
|
POST {{host}}/tasks/not-a-uuid/result
|
|
Authorization: Bearer {{token}}
|
|
Content-Type: application/json
|
|
|
|
{ "worker_id": "{{worker}}", "attempt": 1, "result_uri": "s3://x", "result_sha256": "x" }
|
|
|
|
### Unknown field in the body (400) — a misspelled key must not pass silently
|
|
POST {{host}}/tasks/claim
|
|
Authorization: Bearer {{token}}
|
|
Content-Type: application/json
|
|
|
|
{ "worker_ID": "{{worker}}" }
|
|
|
|
### Unknown job (404)
|
|
GET {{host}}/jobs/00000000-0000-0000-0000-000000000000
|
|
Authorization: Bearer {{token}}
|
|
|
|
### Stitching is not implemented yet (501)
|
|
# Any endpoint whose use case is still a stub answers 501.
|