Webhooks, event streams and notifications
How the server tells you that something happened in a run (T27): Server-Sent Events for live
progress, Standard Webhooks for events of the product, and the
notify node for alerts to Slack, Teams, email or a signed webhook. Every route follows the
API conventions: RFC 9457 problems, keyset pagination and audit events. Reading
needs project:read; changing anything needs webhook:manage.
Delivery is at least once
A webhook can arrive more than once (a worker died after sending, a manual redelivery). Every
delivery of one event carries the same webhook-id header: deduplicate by it.
Live progress of a run (SSE)
curl -N "$DX/v1/runs/$RUN/events" -H "Authorization: Bearer $KEY"
retry: 3000
id: 2
event: run.started
data: {"run_id":"run_…","seq":2,"type":"run.started","created_at":"…Z","data":{}}
idis the run's sequence number: 1..N without gaps. Reconnect withLast-Event-ID: k(or?since=k) and the server sendsk+1..Nexactly once.retry: 3000; a: pingcomment every 15 s; the stream closes afterrun.succeeded,run.partially_succeeded,run.failedorrun.cancelled.- A run that does not exist, or belongs to another project, is a
404problem. A project can keep 100 streams open in one API process; the 101st gets429withRetry-After: 3. - Behind a proxy turn buffering off (
X-Accel-Buffering: nois already sent).
Webhook endpoints
| Route | What it does |
|---|---|
POST /v1/webhooks |
Create an endpoint. The response carries the whsec_… secret once. |
GET /v1/webhooks, GET /v1/webhooks/{id} |
Endpoints with state and last delivery (never the secret). |
PATCH /v1/webhooks/{id} |
URL, events, workflow_ids, enabled, description. |
DELETE /v1/webhooks/{id} |
Deletes the endpoint, its deliveries and attempts. |
POST /v1/webhooks/{id}:rotate-secret |
New secret, with an overlap of 0–168 h (default 24). |
POST /v1/webhooks/{id}:test |
Delivers a webhook.ping. |
GET /v1/webhooks/{id}/deliveries |
Delivery log, newest first, keyset-paginated (state, event). |
GET /v1/webhooks/{id}/deliveries/{delivery} |
One delivery with its attempts and the signed body. |
POST /v1/webhooks/{id}/deliveries/{delivery}:redeliver |
A new delivery of the same event. |
GET /v1/webhook-events |
The event catalog with the JSON Schema of each data. |
The URL must be https and resolve only to public addresses (400 validation-error otherwise); it is checked again
before every attempt, so a DNS name that turns private after the endpoint was created is
blocked. Redirects are never followed. events accepts types and run.*-style prefixes (empty =
all); workflow_ids limits run events to those workflows (empty = all). An endpoint created after
an event does not receive it.
Event catalog
The body is {"id": "evt_…", "type", "api_version", "created_at", "data"}. data holds ids and
small enums only — never extracted values or document content.
| Type | data |
|---|---|
run.started |
run_id, workflow_id, status |
node.started, node.skipped |
run_id, workflow_id, node_id, node_type, scope_key |
node.completed |
the above and cache_hit |
node.failed |
the above and error_class |
run.succeeded, run.partially_succeeded, run.failed, run.cancelled |
run_id, workflow_id, status |
run.waiting_review |
run_id, workflow_id, status, packet_id |
review.completed |
run_id, workflow_id, packet_id, extraction_id, fields_reviewed (field ids) |
workflow.version.published |
workflow_id, version_id, number, change |
webhook.ping |
endpoint_id |
GET /v1/webhook-events returns the authoritative list and the schemas. Later features add types
with one line (register_event(type, schema)).
Verify the signature
Each request carries webhook-id, webhook-timestamp and webhook-signature (v1,<base64>;
two space-separated signatures while a rotation overlaps). Verify the raw body.
from standardwebhooks import Webhook, WebhookVerificationError
wh = Webhook("whsec_…") # the secret returned when you created the endpoint
def handle(raw_body: bytes, headers: dict[str, str]) -> None:
try:
event = wh.verify(raw_body, headers) # also rejects timestamps older than 5 minutes
except WebhookVerificationError:
raise PermissionError("bad signature")
if already_processed(headers["webhook-id"]): # at least once: deduplicate
return
process(event)
import { Webhook } from "standardwebhooks";
const wh = new Webhook(process.env.DX_WEBHOOK_SECRET ?? "");
export function handle(rawBody: string, headers: Record<string, string>) {
const event = wh.verify(rawBody, headers); // throws on a bad signature or an old timestamp
if (alreadyProcessed(headers["webhook-id"])) return;
process(event);
}
The Python and TypeScript SDKs ship the same helper (extractit_sdk.webhooks.verify).
Retries and failures
An attempt succeeds on a 2xx. Anything else is a failure. After a failure the delivery waits
(durably: it survives restarts) and tries again at these offsets from the first attempt, each with
a deterministic jitter of ±10 %:
| Attempt | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| Offset | 0 | 5 s | 5 min | 30 min | 2 h | 5 h | 10 h | 14 h | 20 h | 24 h |
After the tenth attempt the delivery is abandoned. A 3xx is a failure (Location is not
followed). An HTTP 410 ends the delivery and disables the endpoint (disabled_reason: gone);
five abandoned deliveries in a row disable it too (too_many_failures). Enabling it again from
PATCH or from Settings → Webhooks clears the reason and the streak. The log keeps deliveries and
attempts for 30 days.
Rotate a secret without losing events
POST /v1/webhooks/{id}:rotate-secretwith{"overlap_hours": 24}. The response has the new secret (shown once).- While the overlap lasts every delivery carries two signatures — new and previous — and a verifier with either secret accepts it.
- Deploy the new secret in your receiver at any time during the overlap.
- When the overlap ends only the new signature is sent.
overlap_hours: 0cuts over at once.
Notification channels and the notify node
An administrator creates channels (API only); a notify node names one in target and never holds
a URL.
| Route | |
|---|---|
POST /v1/notification-channels |
{name, kind, config} — slack, teams, email or webhook |
GET/PATCH/DELETE /v1/notification-channels/{id} |
Settings are shown without tokens, passwords or URLs |
POST /v1/notification-channels/{id}:test |
Sends [TEST] …; dry_run: true renders without any network call |
- Slack:
webhook_url(hooks.slack.com) orbot_token+channel. Teams: the Power Automate flow URL (*.logic.azure.com,*.webhook.office.com). Email:smtp_host,smtp_port,security(none|starttls|tls), optionalusername/password,senderand up to 10recipients. Webhook:url; the JSON{title, text, severity, run_id, workflow, document}is signed like any webhook and the response returns itswhsec_…secret once. - Templates are Jinja2 in a sandbox (at most 8 KiB). Context:
fields.<name>(also byfield_id),validation.<rule_id>.status|detail|severity,run,document,segment,severity. Values are data:<!channel>, HTML or{{ … }}inside a value are escaped for the channel, never evaluated. A variable that does not resolve renders as<missing: fields.total>. - The subject starts with
[INFO],[WARNING]or[CRITICAL], and the body ends with a link toDX_PUBLIC_URL/runs/{run_id}. - Limits per project and channel: Slack and Teams 60/minute and 500/hour, email 30 and 300,
webhook 120 and 2 000 (
DX_NOTIFY_LIMITS="slack=60/500,email=30/300"). Excess messages are dropped with outcomedropped_rate_limited(the node does not fail). - One message per
sha256(run_id, node_id, scope_key): a resumed run that already notified sends nothing again. A definitive failure fails the node (NOTIFY_FAILED) and the run endspartially_succeeded. - Publishing a workflow whose
notifynode names a channel that does not exist shows a warning (NOTIFY_CHANNEL_UNKNOWN); a channel of another kind thanchannelis an error (NOTIFY_CHANNEL_KIND_MISMATCH).
Operation
| Variable | Default | Meaning |
|---|---|---|
DX_WEBHOOK_ALLOW_PRIVATE |
off | 1 allows http and private addresses (development only). |
DX_WEBHOOK_ALLOWED_NETS |
empty | Comma-separated CIDRs an on-premises installation may call. |
DX_WEBHOOK_RETRY_SCALE |
1 |
Shrinks the retry calendar (tests: 0.001 turns 24 h into 86 s). |
DX_WEBHOOK_PARTITION_CONCURRENCY |
200 |
Concurrent deliveries per endpoint. |
DX_DELIVERY_DISPATCHER |
1 |
0 disables the dispatcher in this worker (one leader runs across workers). |
DX_PUBLIC_URL |
http://localhost:3000 |
Base of the run links in notifications. |
DX_NOTIFY_LIMITS |
see above | Per-kind overrides of the notify limits. |
The dispatcher reads the run outbox through a cursor that cannot skip a commit, creates one delivery per subscribed endpoint and starts one durable workflow per delivery; secrets are sealed with the organization key and never appear in logs.