Encryption and the key hierarchy
Data governance (T50) sits on the envelope encryption of the data platform (T11). This page states what is encrypted, with which keys, and where the boundary of the guarantee is.
Hierarchy
KEK of the tenant local file (HKDF per tenant) or OpenBao Transit key dx-t-<tenant>
└─ DEK of the tenant tenant_keys: versioned, wrapped by the KEK (T11; still used for old blobs)
└─ DEK of one blob data_key: AES-256, one per blob *content*, wrapped by the same KEK (T50)
└─ object key HKDF-SHA256(DEK, salt per object) → AES-256-GCM, 1 MiB chunks
- Blob key.
t/{tenant}/blobs/aa/bb/<sha256 of the plaintext>: one object per content and tenant (no deduplication across tenants). - Envelope
DXE1format 2. HeaderDXE1 | 2 | 0 | salt | chunk_log2 | data-key id (16 bytes). Every chunk is authenticated with the header, the tenant id, the plaintext sha256 and a last-chunk flag as AAD. An object copied to another tenant, to another blob key, truncated or reordered fails to decrypt (BlobIntegrityError). A stored object never contains%PDFor any other plaintext. - Format 1 (DEK per tenant version) stays readable. Objects written before T50 have no data key
of their own: deleting them is a plain delete (the certificate says
method: delete, notcrypto_shred). - Derived artifacts (DocIR, page images, results) are blobs like any other and get their own data key; deleting a packet destroys the keys of every artifact of the runs that processed it.
- Reference counts. A content that another packet (or a dataset item) still references is
retained: its key is not destroyed and the certificate lists it under
retained. The count is computed from the referencing tables, never from a counter that could drift. - Cache. Unwrapped data keys live in a per-process cache (5 minutes, 1 024 entries) that is emptied for a key as soon as it is destroyed.
Crypto shredding
Destroying a data key is erasing its wrapped bytes from Postgres (status = destroyed, wrapped_dek
IS NULL, enforced by a CHECK). From then on nobody — not even with the KEK — can decrypt the
object, wherever it was copied (object-store versions, replicas, exports of the bucket).
Database backups keep the wrapped key as it was when they were taken. That is why every deletion
writes its signed certificate to the object store under t/{tenant}/erasure/ (outside the database)
and extractit admin erasure replay --org <id> re-applies those certificates after any restore.
What the local KEK cannot do
With DX_KEY_PROVIDER=local every KEK is derived from one master key file (DX_KEK_FILE, mode
0600). Destroying a tenant KEK is not verifiable there: the master key still derives it, and
backups of the file keep it. Destroying the data keys still makes the blobs unreadable, but if
the master key and an old database backup both leak, the old wrapped keys can be unwrapped. For
strong erasure requirements use OpenBao Transit (DX_KEY_PROVIDER=openbao): the KEK never
leaves OpenBao and deleting the tenant key there is a verifiable destruction.
Boundary of the guarantee
Covered: Postgres rows, the encrypted objects of the object store (all versions and delete markers), the data keys and their in-memory cache, and the values of the review (see below).
Not covered, and not claimed by any certificate: copies held by model providers beyond the retention
class listed in provider_retention; exports, downloads and webhook payloads already delivered;
backups of the object store made outside the product; any storage that is not inventoried by the
operator.
Review values (review_decision, field_override)
What a reviewer saw or typed is extracted data of the document, but review_decision is append-only
(dx_app cannot update or delete it, and neither can dx_system). Its values are therefore
sealed: previous_value, new_value, the quoted evidence and the free-text reason of every
decision, and value and evidence of every field_override, are stored as AES-256-GCM envelopes
({"$sealed": 1, "kid", "n", "c"}) with a data key per run (a data_key row wrapped by the
tenant KEK, like the blob keys; its sha256 is derived from the run id). The associated data binds
tenant, run, table, row and column, so a ciphertext cannot be moved to another row. The rest of the
row (ids, action, path, by, at, reason_code, times) stays in the clear and the table stays
append-only.
The erasure saga (certified DELETE, retention TTL, ZDR) destroys that key together with the blobs
of the run: the certificate lists review_decision and field_override in destroyed[] with
method: crypto_shred and the dek_id, a step verifies that no active key is left, and the erasure
ledger re-applies the destruction after a restore. From then on the review API reads those values as
null with redacted: true (GET /v1/runs/{id}/review/history; GET /v1/runs/{id}/review of a
deleted run answers the decided fields with no value). Migration 0016_t50_review seals the rows
that existed before; it needs the same key provider the API uses (DX_KEY_PROVIDER, DX_KEK_FILE
or DX_OPENBAO_*) whenever there are rows to seal.
Still in the clear and outside this guarantee: review_document.final_result (the snapshot of a
verified review; the saga deletes the row, backups keep it until they expire), the field.corrected
and review.completed outbox events already delivered, and the audit event of a review or
reject that carries a reason in its metadata.
Rotating the KEK (python -m extractit.store.admin rewrap-kek) rewraps the tenant DEKs; wrapped
data keys keep their KEK version and stay decryptable (Transit keeps old versions) until a rewrap of
data_key is added.