Skip to content

Installing ExtractIt

Three supported routes, from smallest to largest. All of them run the same code and the same images; the Helm profiles differ only in values (ADR-017).

Route For Database Backups
pip install extractit the CLI and the Python library on one machine none (in-process) —
Docker Compose (deploy/compose/) evaluation and small single-host installs PostgreSQL 16 in a volume daily logical dump, no RPO guarantee
Helm (deploy/helm/extractit) production: saas-eu, vpc, onprem CloudNativePG or the cloud's managed PostgreSQL continuous WAL archiving to S3 (RPO ≤ 15 min, measured ≤ 60 s)

pip

pip install extractit
extractit --version
extractit invoice.pdf --schema examples/invoice.schema.json -o out.json

The Code node needs Deno and the Pyodide bundle: python -m extractit.nodes.code.install --with-deno.

Docker Compose (production profile)

cp deploy/compose/.env.production.example deploy/compose/.env.production
# replace every secret (openssl rand -base64 36) and set DX_IMAGE / DX_WEB_IMAGE to the release
# digests from images.yaml
docker compose -f deploy/compose/compose.yaml --env-file deploy/compose/.env.production \
  -p extractit up -d --wait

What starts: postgres (16, volume pg-data, roles of deploy/compose/postgres/init-roles.sql), gotenberg, the one-shot keys (creates the master key on the dx-home volume, mode 0600) and migrate (Alembic as dx_migrator), then api, worker and web. API and worker run python -m extractit.ops.preflight first: they refuse to start (exit 3) against a database migrated by a newer release. The web container serves the Studio and proxies /v1, /auth, /healthz and /readyz to the API, so put your TLS reverse proxy in front of port 8080 only.

Observability: add -f deploy/compose/compose.observability.yaml (Collector, Prometheus with the SLO rules, Jaeger) and, for Langfuse, -f deploy/compose/compose.langfuse.yaml (observability). Backups: backup and restore.

deploy/tests/compose_test.sh is the acceptance test of this route (healthy in < 90 s, sample extraction, /v1/status and the status page reacting to a stopped database, backup/restore).

Helm

Prerequisites, installed by the cluster operator (the chart refuses to install without them when the values need them):

Operator Version tested Needed for
CloudNativePG 1.30.1 postgresql.mode=managed
Barman Cloud plugin (+ cert-manager 1.21) 0.16.0 backup.method=plugin (default); inTree works without it until CNPG 1.31
KEDA 2.21.0 keda.enabled=true (otherwise a CPU HorizontalPodAutoscaler)
gVisor (runsc) on labelled nodes — sandbox.mode=ephemeral-pod in saas-eu
helm install extractit oci://ghcr.io/<owner>/charts/extractit --version <version> \
  -n extractit --create-namespace -f values-saas-eu.yaml -f my-overrides.yaml

Profiles

saas-eu vpc onprem
Tenancy multi-tenant one customer one customer
PostgreSQL CloudNativePG, 3 instances external (RDS, Cloud SQL…) CloudNativePG, 2 instances
Object storage external S3 (EU region) the cloud's S3 external S3-compatible (SeaweedFS, Ceph RGW…)
Worker scaling KEDA on the DBOS backlog KEDA CPU HPA (no KEDA dependency)
Code node ephemeral pods with gVisor (required) Deno inside the worker Deno inside the worker
Observability Collector → customer OTLP / Langfuse Collector → OTLP Collector + Jaeger (bundle)
Status page Gatus — Gatus

Values

values.schema.json is closed: an unknown key anywhere in a group, a remote $ref, an image without @sha256:, or a values file that does not choose managed or external for postgresql.mode and objectStorage.mode is rejected before anything is rendered.

Main groups: global (registry mirror, pull secrets), profile, images (overrides by digest), postgresql.{mode,cnpg.*,external.*}, objectStorage.{mode,external.*,managed.*}, backup.{method,schedule,retention,destinationPath,endpoint,existingSecret}, keda.*, hpa.*, telemetry.*, observability.{mode: otlp|bundle|none,…}, langfuse.{otlpEndpoint,secretName}, networkPolicy.*, ingress.*, httpRoute.*, sandbox.{mode,runtimeClassName,limits,…}.

Secrets: the chart generates the API keys material (<release>-secrets), the master key (<release>-kek) and the database role passwords; they survive upgrades and are kept on helm uninstall (losing the master key makes every stored document unreadable). Bring your own with secrets.existingSecret (keys DX_KEY_PEPPER, DX_CURSOR_KEY, DX_SESSION_KEY, DX_UPLOAD_SECRET, DX_CACHE_HMAC_KEY) and, for postgresql.mode=external, a Secret with DX_DATABASE_URL, DX_SYSTEM_DATABASE_URL, DX_MIGRATOR_DATABASE_URL and DX_DBOS_DATABASE_URL (roles created with deploy/compose/postgres/init-roles.sql).

Governance signing keys: with the default local signer each API replica keeps its own keys (DX_HOME is per pod). Production deployments with several replicas use OpenBao (keys.provider=openbao, DX_GOV_SIGNER=openbao in extraEnv).

Images and supply chain

Every image is referenced by digest from images.yaml, generated by the release from the build metadata and packaged inside the chart (python -m extractit.ops.images). Private registries: global.imageRegistry rewrites the registry host and global.imagePullSecrets adds pull secrets; the digests do not change. Verify before installing:

cosign verify ghcr.io/<owner>/charts/extractit@<digest> \
  --certificate-identity https://github.com/<owner>/extractit/.github/workflows/release.yml@refs/tags/v<version> \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

The API and web images are reproducible: two builds of one commit give the same digest (deploy/tests/reproducible_test.sh; SOURCE_DATE_EPOCH, rewrite-timestamp, bases pinned by digest, locked dependencies).

Security defaults

Every pod meets Pod Security restricted (non-root, read-only root filesystem, seccomp RuntimeDefault, all capabilities dropped, no hostPath); label the namespace pod-security.kubernetes.io/enforce=restricted. With networkPolicy.enabled (default) every pod of the chart is default-deny and gets only its flows: workers have no ingress and reach only DNS, PostgreSQL, the object store, Gotenberg, the Collector and the CIDRs in networkPolicy.workerEgressCIDRs (model providers); 169.254.169.254 is excluded from every CIDR rule. API and web carry the interactive-review PriorityClass, workers batch-ingest.

Health

/healthz (liveness), /readyz (readiness: the database answers) and /v1/status (database, queue, storage, workers; always 200, the state is in the body) — see observability.

Acceptance tests

Script Proves
deploy/tests/chart_test.sh lint, render and kubeconform (Kubernetes 1.35-1.37) of the three profiles, schema negatives, Pod Security, digests, promtool
deploy/tests/install_test.sh install in kind, every pod Ready < 5 min, probes, extractit FILE sample
deploy/tests/netpol_test.sh worker egress: metadata and external IPs blocked, PostgreSQL and S3 reachable
deploy/tests/keda_test.sh 100 queued jobs scale workers above 1 in < 120 s and back after the cooldown
deploy/tests/sandbox_test.sh Code node as ephemeral Jobs (sandbox)
deploy/tests/restore_test.sh point-in-time restore (backup and restore)
deploy/tests/upgrade_test.sh zero-downtime upgrade (upgrade)

They need kind, helm, kubectl and the images in a local registry (deploy/tests/build_images.sh); run them one at a time.