feat(coordinator): artifact upload/download endpoints (CTX-05, part 2)
Wire the artifact storage foundation to HTTP.
- PUT /tasks/{id}/artifacts/{filename}: a worker streams a partial result;
the coordinator verifies lease ownership (foreign worker → 409), streams
the bytes to blob storage while hashing, and records the metadata. An
orphaned blob from a failed metadata insert is cleaned up.
- GET /artifacts/{id}/download: streams an artifact back with its content
type, length, and checksum.
- Ownership is read with a new non-locking TaskRepository.Get, so no row lock
is held across a long upload. Identity travels in X-Worker-ID / X-Task-Attempt
headers per the contract; upload/download bypass the short request timeout.
- docker-compose mounts ./data for durable artifact storage; smoke and
requests.http exercise an upload → foreign-409 → download round-trip.
This commit is contained in:
@@ -101,6 +101,22 @@ fi
|
||||
|
||||
check "heartbeat" 200 -X POST "${HOST}/tasks/${task_id}/heartbeat" "${auth[@]}" \
|
||||
-d "{\"worker_id\":\"w1\",\"attempt\":${attempt}}"
|
||||
|
||||
# --- artifacts (while the task is still leased) ---------------------------
|
||||
bearer=(-H "Authorization: Bearer ${TOKEN}")
|
||||
check "upload artifact" 200 -X PUT "${HOST}/tasks/${task_id}/artifacts/result.csv" "${bearer[@]}" \
|
||||
-H 'Content-Type: text/csv' -H 'X-Worker-ID: w1' -H "X-Task-Attempt: ${attempt}" \
|
||||
--data-binary $'query,match,score\nA,B,0.9\n'
|
||||
check "foreign worker upload → 409" 409 -X PUT "${HOST}/tasks/${task_id}/artifacts/x.csv" "${bearer[@]}" \
|
||||
-H 'Content-Type: text/csv' -H 'X-Worker-ID: impostor' -H "X-Task-Attempt: ${attempt}" \
|
||||
--data-binary 'x'
|
||||
|
||||
# Round-trip: upload one more, then download it by id and confirm the bytes.
|
||||
art=$(curl -sS -X PUT "${HOST}/tasks/${task_id}/artifacts/dl.csv" "${bearer[@]}" \
|
||||
-H 'Content-Type: text/csv' -H 'X-Worker-ID: w1' -H "X-Task-Attempt: ${attempt}" --data-binary 'a,b,c')
|
||||
art_id=$(printf '%s' "$art" | python3 -c 'import json,sys;print(json.load(sys.stdin)["artifact_id"])' 2>/dev/null)
|
||||
check "download artifact" 200 "${HOST}/artifacts/${art_id}/download" "${bearer[@]}"
|
||||
|
||||
check "foreign worker submits → 409" 409 -X POST "${HOST}/tasks/${task_id}/result" "${auth[@]}" \
|
||||
-d "{\"worker_id\":\"impostor\",\"attempt\":${attempt},\"result_uri\":\"s3://x\",\"result_sha256\":\"x\"}"
|
||||
check "submit result" 200 -X POST "${HOST}/tasks/${task_id}/result" "${auth[@]}" \
|
||||
|
||||
Reference in New Issue
Block a user