API conventions
Every endpoint of the ExtractIt REST API follows the conventions below. They are implemented
once, in src/extractit/api/ (T12); a resource only declares what it needs. The contract is
openapi/openapi.json (OpenAPI 3.1), protected in CI by Spectral and oasdiff.
The examples use $DX for the base URL and $DX_API_KEY for a key.
Authentication and permissions
Requests carry a project API key: Authorization: Bearer dx_<env>_<key_id>_<secret>.
curl -s "$DX/v1/me" -H "Authorization: Bearer $DX_API_KEY"
- Keys belong to a project.
envisliveortest;key_id(12 base32 characters) is public; the 43-character secret is shown once, at creation, and only its HMAC-SHA256 (with a versioned pepper) is stored. The pattern for secret scanners isdocs/security/key-pattern.txt. - A key has scopes (
read,write,review,eval,admin) and a role cap: the role of whoever created it. Effective permissions = permissions of the scopes ∩ permissions of the role. The matrix of the six roles and 17 permissions isdocs/security/rbac.md(generated). - Keys expire (90 days by default, at most 365), rotate with an overlap (24 h by default, at most
168 h) and are revoked with
DELETE /v1/api-keys/{id}; a revocation reaches every replica in at most 5 seconds. A project has at most 20 active keys. - Missing or bad credentials: 401 with
WWW-Authenticate: Bearer; thecodesaysmissing-credentials,invalid-key,key-expiredorkey-revoked(the last two only for the holder of the secret). Scope or role too low: 403insufficient-scopeorpermission-denied. After 10 failed authentications in a minute, an address gets 429. - Resources of another organization or project are always 404, never 403.
The first key of an organization comes from the bootstrap command:
python -m extractit.api.admin create-api-key --tenant org_… --role owner --scopes admin
Errors (RFC 9457)
Every 4xx and 5xx is application/problem+json:
{
"type": "https://docs.extractit.dev/errors/validation-error",
"title": "Request validation failed",
"status": 400,
"code": "validation-error",
"retryable": false,
"request_id": "req_01jc…",
"errors": [{"pointer": "/slug", "rule": "string_pattern_mismatch", "message": "…"}],
"recovery": "Fix the fields listed in errors[]."
}
codeis stable and kebab-case;typeends with it.retryabletells clients whether the same request may succeed later.- Validation errors are 400
validation-error(never FastAPI's 422).errors[].pointeris a JSON Pointer into the body (/slug) or#/query/<name>,#/header/<name>,#/path/<name>for other inputs. Input values are never echoed. Unknown body fields and unknown query parameters are errors. - 422 is reserved for well-formed requests that cannot be processed (
idempotency-key-reuse,invalid-expiry).
Request id, version and tracing
X-Request-ID: the client's value is kept when it matches^[A-Za-z0-9_-]{8,64}$; otherwise the server generatesreq_…. It is in every response, problem, log line and audit event.DX-Version: 2026-11-01: every response carries it; a request may send it, and an unsupported value is 400unsupported-version. Major versions live in the path (/v1); within/v1only additive changes are allowed.traceparent(W3C): the trace id is propagated to the logs, the audit log and, whenOTEL_EXPORTER_OTLP_ENDPOINTis set, to OpenTelemetry spans.
curl -s "$DX/v1/projects" -H "Authorization: Bearer $DX_API_KEY" \
-H "DX-Version: 2026-11-01" -H "X-Request-ID: my-job-0042" \
-H "traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01" -i
Collections: keyset pagination
curl -s "$DX/v1/audit-events?limit=100&action=project.created" \
-H "Authorization: Bearer $DX_API_KEY"
# {"data": [...], "next_cursor": "eyJr…", "has_more": true}
curl -s "$DX/v1/audit-events?limit=100&action=project.created&cursor=eyJr…" \
-H "Authorization: Bearer $DX_API_KEY"
limitdefaults to 50; values above 200 are clipped andX-Limit-Appliedreports the applied value. There is nooffsetand no total count.- Order is
(created_at, id). The cursor is opaque and signed; it is only valid with the same filters. A tampered or foreign cursor is 400invalid-cursor. Rows inserted while a client walks a collection never make it repeat or skip an item.
Idempotency
Every POST with effects accepts Idempotency-Key (1–255 printable characters; 24 h; scoped to
the key's project):
curl -s -X POST "$DX/v1/projects" -H "Authorization: Bearer $DX_API_KEY" \
-H "Idempotency-Key: 7c1d0f7e-create-invoices" -H "Content-Type: application/json" \
-d '{"slug": "invoices", "name": "Invoices"}'
- A retry with the same key and the same request (method, path, query and canonical body) replays
the stored response with
Idempotent-Replayed: true. - The same key with a different request: 422
idempotency-key-reuse. While the original is still running, a duplicate waits briefly and then gets 409idempotency-key-in-flight(retryable). - A request that fails releases its key, so the client can retry. After 24 h the key is free.
This follows the IETF
Idempotency-Keydraft (422/409) rather than a single 409.
Concurrency: ETag and If-Match
Editable resources return a strong ETag ("v<row_version>").
curl -s -i "$DX/v1/projects/prj_…" -H "Authorization: Bearer $DX_API_KEY" # ETag: "v3"
curl -s -X PATCH "$DX/v1/projects/prj_…" -H "Authorization: Bearer $DX_API_KEY" \
-H 'If-Match: "v3"' -H "Content-Type: application/json" -d '{"name": "Invoices EU"}'
PUT,PATCHandDELETEof editable resources requireIf-Match: missing → 428precondition-required; stale → 412precondition-failed.GETwith a matchingIf-None-Match→ 304.
Rate limits and quotas
- Per project and class:
submit120/min,read600/min,review300/min (plan defaults,api/plans.toml, overridable per project inproject_limits). Responses carryRateLimit-Policy: "read";q=600;w=60andRateLimit: "read";r=599;t=1; exceeding a class is 429rate-limitedwithRetry-After. - Quotas (
max_runs_per_day,max_pages_per_month, …; NULL means unlimited) are consumed inside the request transaction; exceeding one is 429quota-exceededwithlimit,used,resets_atandRetry-After. - The rate state lives in each replica's memory: with N replicas the effective limit is N times the configured one. Quotas are exact (database).
Audit
Every mutation writes one audit_log event (actor, action, target, outcome, request_id,
trace_id, hashes of the before/after snapshots) in the same transaction as the change: if the
change rolls back, so does the event. dx_app can only insert and read audit_log (a trigger
rejects UPDATE and DELETE for every role). Denied requests with a known tenant are recorded as
auth.denied (at most one per credential and reason per minute).
curl -s "$DX/v1/audit-events?actor=key_…&since=2026-11-01T00:00:00Z" \
-H "Authorization: Bearer $DX_API_KEY"
Declaring a new endpoint
- Create
src/extractit/api/routers/<resource>.pywithrouter = dx_router(tags=[...])and add the module name toROUTER_MODULESinapi/routers/__init__.py(one line). - Declare each operation with an explicit snake_case
operation_id, asummary, adescriptionand aresponse_model, and require a permission:
@router.post(
"/v1/widgets",
operation_id="create_widget",
summary="Create a widget",
description="…",
status_code=201,
response_model=WidgetOut,
)
def create_widget(
body: WidgetCreate,
session: SessionDep,
audit: AuditDep,
principal: Annotated[Principal, require(Permission.WORKFLOW_WRITE, audit="widget.created")],
) -> WidgetOut:
...
audit.record("widget", out.id, after=snapshot(widget))
return out
require(permission, audit=..., rate_class=...)is mandatory on every route exceptPUBLIC_ROUTES(/healthz,/readyz,/openapi.json); non-GET routes must name an audit action (resource.verb). The route class refuses to build otherwise andtests/api/test_api_architecture.pychecks the inventory.- The rate class defaults to
readfor GET andsubmitotherwise (rate_class="review"for review actions). - Use
SessionDep(the request's tenant session: never open sessions in handlers),PageDep+paginate(...)for collections,ETaggedfor editable resources,QuotaDepto consume quotas, and raiseApiError("code", detail=...)for errors. - Organization-level resources use
require(..., tenant_wide=True). - Tag operations that the MCP server (T47) should expose with
mcp: they need a summary, a description and anoperationIdof at most 64 characters (Spectral enforces it). - Regenerate the snapshot and add a changelog entry:
uv run python -m extractit.api.openapi --out openapi/openapi.json
$EDITOR docs/changelog/api.md
Compatibility and deprecation
/v1only changes additively. CI (api-contract) regenerates the snapshot (no diff allowed), runs Spectral (0 errors), runsoasdiff breakingagainstmain(fails on removed endpoints or response fields, new required inputs…) and requires an entry indocs/changelog/api.mdwhenever the snapshot changes.- To deprecate an operation, add
**deprecated(on="2027-01-15", sunset="2027-07-15")to its route. Responses then carryDeprecation: @<epoch>(RFC 9745) andSunset: <HTTP-date>(RFC 8594); the sunset must be at least 180 days after the deprecation, checked when the route is built and bypython -m extractit.api.openapi --check.
Observability
- Logs are JSON (structlog) with
request_id,trace_id,tenant_idandproject_id. Values of sensitive keys (authorization,api_key,secret,token,password, …) and anything shaped like an API key are redacted; request and response bodies are never logged. - Traces: set
OTEL_EXPORTER_OTLP_ENDPOINTto export OTLP/HTTP spans (server spans withdx.request_id, database spans without parameters). Without it nothing is created.