Ingest and one-shot runs
Packets, uploads and the one-shot parse / extract runs of the server (T17). Every route follows
the API conventions: RFC 9457 problems, Idempotency-Key on POST, keyset
pagination, rate classes and audit events. Resources of another tenant (or project) are always
404.
Flows
Multipart packet
curl -sS -X POST "$DX/v1/packets" -H "Authorization: Bearer $KEY" \
-F files=@invoice.pdf -F files=@delivery-note.pdf \
-F 'metadata={"customer":"ACME"}' -F notes="March batch"
201 with the packet: {id: pkt_…, status: ready, file_count, content_hash, files[{id: file_…,
name, sha256, size, media_type, page_count, source, warnings}]}. 1 to max_files_per_packet
files; duplicate names become a.pdf, a (2).pdf. metadata is a JSON object of at most
64 KiB stored with the packet; it is never injected into a prompt (citable metadata arrives
with T24). GET /v1/packets/{id} lists the files; DELETE hides the packet (certified deletion
is T50).
URL import
curl -sS -X POST "$DX/v1/packets" -H "Authorization: Bearer $KEY" \
-H 'Content-Type: application/json' \
-d '{"urls": ["https://files.example.com/a.pdf", "https://files.example.com/b.pdf"]}'
202 with the packet in importing; an import run downloads every URL and the packet becomes
ready or, if any URL fails, failed with errors[{url_hash, code}] and no files (atomic).
POST /v1/parse and POST /v1/extract also accept {"url": …} (downloaded inside the request).
SSRF defence: only https, port 443 (DX_IMPORT_ALLOWED_PORTS), no credentials in the URL; the
host is resolved with getaddrinfo and every address must be public (no private, loopback,
link-local, multicast, reserved, unspecified, CGNAT 100.64.0.0/10, metadata 169.254.169.254,
ULA fc00::/7, IPv4-mapped IPv6; IPs written in decimal, octal, hex or short form are decoded
first). The connection is pinned to the validated IP (URL with the IP, Host header and the TLS
sni_hostname, so the certificate is still verified), trust_env=False, at most 3 redirects —
each one validated again —, 5 s to connect and DX_IMPORT_TIMEOUT_S (30 s) in total, and the body
is counted while streaming. Logs and errors hold only scheme, host and a hash of the URL; the
URL list travels to the worker as an encrypted blob. The tested forbidden targets live in
tests/fixtures/ingest/ssrf_targets.json (43 entries).
Direct (presigned) upload
curl -sS -X POST "$DX/v1/uploads" -H "Authorization: Bearer $KEY" \
-H 'Content-Type: application/json' \
-d '{"filename": "scan.pdf", "size": 63000000, "checksum_sha256": "…"}'
# → {id: upl_…, upload_url, method: PUT, headers, expires_at}
curl -sS -X PUT "$UPLOAD_URL" -H 'Content-Type: application/pdf' --data-binary @scan.pdf
curl -sS -X POST "$DX/v1/uploads/upl_…/complete" -H "Authorization: Bearer $KEY"
With S3 the upload_url is a presigned PUT (15 min) to t/{tenant_id}/incoming/{upload_id};
with local storage it is PUT /v1/uploads/{id}/content?token=… (send the API key too), a token
base64url(upload_id|exp) signed with HMAC-SHA256 for 15 minutes. The staging area holds the
bytes in clear until complete: use bucket encryption and a lifecycle rule that expires
t/*/incoming/ after 24 h. complete reads the staged object in streaming, checks the declared
size (upload-size-mismatch), checksum_sha256 (upload-checksum-mismatch), type and limits,
encrypts it (DXE1) into the content-addressed store, creates the file (a one-file packet) and
deletes the staged object; repeating it returns the same file_id. purge_stale_uploads (every
15 minutes) expires sessions not completed within 24 h. A single S3 PUT accepts up to 5 GiB,
above every plan limit.
One-shot runs
# parse
curl -sS -X POST "$DX/v1/parse" -H "Authorization: Bearer $KEY" -F file=@invoice.pdf
# extract with an inline schema and rules, waiting up to 30 s
curl -sS -X POST "$DX/v1/extract?wait=30" -H "Authorization: Bearer $KEY" \
-F file=@invoice.pdf -F "schema=$(cat invoice.schema.json)" -F rules=@rules.json
# or JSON with a packet
curl -sS -X POST "$DX/v1/extract" -H "Authorization: Bearer $KEY" \
-H 'Content-Type: application/json' \
-d '{"packet_id": "pkt_…", "schema": {…}, "tier": "standard", "dedup": "reuse"}'
202 with the Run and Location: /v1/runs/{id}. Fields: file (multipart) or packet_id or
url; schema (JSON Schema with x-dx, required for extract), rules (JSON rules of T09: a
list or {rules: […]}), tier (fast|standard|agentic), model, priority (0-9), dedup.
The schema and the rules are validated before anything is created: 400 with a JSON Pointer
(/schema/…, /rules/0/check/equals). A packet with more than one file is 422
packet-multi-file-unsupported (T24 treats a packet as one document).
The extract run executes five durable steps — prepare (load, convert with T16 when the file
is not a PDF; the derivative is a file with source.kind = derived), parse (DocIR stored
gzip-compressed, page PNGs), extract (model call and citations, T06), verify_score
(normalization, verification, rules, confidence and assembly — finalize of T08) and persist
(result blob + summary). parse runs prepare, parse, persist. Completed steps are
checkpointed: :resume re-executes only the unfinished ones.
?wait=N (N ≤ 60) polls every 250 ms with short transactions after the request transaction
is committed: a finished run answers 200 ({run, result} for extract, the Run for parse), an
expired wait 202 with the current state. At most 50 waits per project run at once; beyond that
the wait is ignored and the answer is 202.
Runs and results
| Route | |
|---|---|
GET /v1/runs?status=&kind=&created_from= |
keyset listing |
GET /v1/runs/{id}?wait=N |
{id, kind, status, packet_id, steps[{key, status, attempt, duration_ms, cost_usd}], usage, error, timings, links} |
POST /v1/runs/{id}:cancel |
queued → never starts; running → stops at the next step boundary; cancelled → 200 (idempotent); other terminal → 409 |
POST /v1/runs/{id}:resume |
failed/cancelled → 202, from the first unfinished step; anything else → 409 run-not-resumable |
GET /v1/runs/{id}/result?format=full\|data\|fields\|flat |
full = {run_id, kind, packet_id, file_id, document{pdoc_id, pages[{page, width_px, height_px, dpi, route}]}, output} where output is the output contract of T08 (schemas/output-1.0.json) |
GET /v1/runs/{id}/evidence?path=total |
the field and its evidence |
GET /v1/runs/{id}/document?format=docir\|markdown\|blocks |
DocIR of T02, anchored markdown (one anchor per line), layout blocks |
GET /v1/runs/{id}/pages/{n}.png |
ETag = sha256, Cache-Control: private, max-age=3600, If-None-Match → 304 |
A result above 5 MiB answers {result_url, expires_at, size}: GET
/v1/runs/{id}/result/download?token=… with the API key, valid for 15 minutes (then 403). Blobs
are encrypted with the tenant's DEK, so even on S3 the URL goes through the API (a presigned
object URL would only return ciphertext).
Limits per plan
From project_limits (T12; projects without a row use the plan of DX_PLAN):
| Limit | Default | Problem |
|---|---|---|
max_file_mb |
25 (MiB) | 413 payload-too-large with max_bytes (26214400) |
max_files_per_packet |
100 | 413 too-many-files |
max_pages_per_file |
unlimited in default (2000 if unset) |
413 too-many-pages |
max_concurrent_runs, max_queued_runs |
runtime defaults (10, 10000) | queue tier and 429 quota-exceeded |
Content-Length is checked before the body is read, and bytes are counted while copying, so a
lying client is cut off at the limit. project_limits is also the RuntimeLimits provider of the
runtime (T14) for runs submitted by the API.
Deduplication
content_hash = sha256(sorted sha256 of the files); the run fingerprint adds the run kind, schema
hash, rules hash, tier and model. The blob is always shared inside the tenant (one object per
sha256, never across tenants).
dedup |
Same document, schema, tier and policy already succeeded |
|---|---|
reprocess (default) |
new run |
reuse |
200 with the previous Run and Deduplicated: true |
reject |
409 duplicate-document with existing_run_id |
Problems
| Code | Status | When |
|---|---|---|
payload-too-large |
413 | a file above max_file_mb (max_bytes, file) |
too-many-files |
413 | more than max_files_per_packet |
too-many-pages |
413 | a PDF above max_pages_per_file |
unsupported-media-type |
415 | the content is not a supported format (ZIP without Office structure, executables…); a mismatching extension with a supported content only warns extension_mismatch |
pdf-encrypted |
422 | password-protected PDF (file, errors[]) |
invalid-document |
422 | corrupt PDF |
url-not-allowed |
422 | SSRF policy |
import-timeout, import-failed |
422 | download too slow / HTTP error |
upload-token-expired |
403 | upload token or result URL forged, foreign or expired |
upload-checksum-mismatch, upload-size-mismatch, upload-not-found |
422 | complete checks |
duplicate-document |
409 | dedup=reject |
packet-multi-file-unsupported |
422 | /extract with several files |
packet-not-ready |
409 | packet still importing or failed |
run-not-resumable |
409 | :resume of a run that is not failed or cancelled |
run-not-finished |
409 | result or document of a run that has none yet |