Security gates
ExtractIt has no remote CI and no required checks on main (decision of the maintainer,
2026-11-12). The gates below are local make targets and pytest markers; nothing here is enforced
by a server. A release is only as safe as the last time somebody ran them.
| Gate | Command | What it proves | Discovers by | Needs |
|---|---|---|---|---|
| ASVS list | make security-asvs |
security/asvs.yaml is one-to-one with the official ASVS 5.0.0 JSON (pinned by sha256), every verified entry names evidence and tests that exist, no L1 requirement of an assessed chapter is planned, no high/critical finding is open |
the official JSON | — |
| Injection | make security-injection |
Obedient fake model over 66 adversarial documents (6 classes): 0 tool calls, 0 connections; tool-hijack against every discovered registry (110 attempts); a purpose of a model call without a corpus case fails; CSV/XLSX writers quote-prefix formulas; the viewer renders hostile values as text; SSRF matrix of 103 destinations | AST of src/ (purposes, registries, outbound calls), writers and the OpenAPI |
viewer e2e needs pnpm install + Playwright |
| RLS | make security-rls |
Every table with tenant_id has RLS enabled and forced, a policy with USING and WITH CHECK, and a tenant cannot read or write another tenant's rows (also with an empty context); the app role is not superuser nor BYPASSRLS |
pg_catalog |
Postgres |
| API leak | make security-api-leak |
Every operation of openapi/openapi.json is called as tenant B with tenant A's ids: 403/404, never 2xx or 5xx, no identifier of A in an answer, A's rows unchanged |
the OpenAPI snapshot | Postgres |
| Sandbox | make security-sandbox |
42 escape attempts against the Code node sandbox (Deno + Pyodide) are refused with the expected error; no Deno process is left behind | a fixed list | Deno + the Pyodide bundle |
make security |
all of the above | the union |
Docker-based and on-demand gates (not in make security; they need Docker and several minutes):
| Gate | Command | What it proves | Needs |
|---|---|---|---|
| ZAP | make security-zap |
zap-api-scan.py (official zaproxy/zap-stable image, Apache-2.0) over every operation of openapi/openapi.json (sliced by tools/zap_spec.py, 25 paths per scan), authenticated with the API key of the local stack, plus zap-baseline.py; tools/zap_verdict.py fails on any High or Critical alert; accepted Medium alerts go in security/zap/rules.tsv with a reason |
Docker, DX_STACK_INFO of tests/load/stack.py running |
| Load | make load |
Locust (MIT, run through uvx) at 100 docs/min for 5 min with a fixed-latency model: error rate < 0.1 %, 0 lost runs; then the 1 000 burst (202, all terminal < 15 min) and 50 submissions with one Idempotency-Key (1 run, 49 replays) |
Postgres |
| Chaos | make chaos |
30 s of latency, then 60 s without Postgres (Toxiproxy) while 30 runs are submitted: all succeed, none lost, no step with two successes | Docker, Postgres |
Also run by hand: bandit -r src ee/src -lll, pip-audit (make audit), gitleaks detect
(Docker image zricethezav/gitleaks; the per-file allowlist is in .gitleaks.toml),
pnpm audit --audit-level high.
How a gate fails on something new
- A table with
tenant_idand no row factory (tests/tenancy/factories.pyortests/security/tenant_leak/fixtures_factory.py) failssecurity-rls. - An operation whose path parameter cannot be harvested as tenant A, with no
@resourceand no line inEXEMPT_OPERATIONS, failssecurity-api-leak(statically, with no database). - A component that sends a model request under a new
purposefailssecurity-injectionuntiltests/redteam/purpose_coverage.pyhas an exemption with its reason or the corpus reaches it. - A new module-level tool registry (
ALLOWED_TOOLS,*_REGISTRY,TOOLS) fails until it is inKNOWN_REGISTRIESoftests/redteam/tool_registries.py. - A new outbound call site fails
tests/security/ssrfuntil it is registered with its guard. - A new export format (found through the writers and the OpenAPI) is run through the 100 payloads of the formula gate.
NFR-08 (corrupted values stopped)
The strict NFR-08 gate (95 % of the corrupted values end unverified, contradicted or fail a
rule) is green and blocking since FIX-T07: 100 % (45/45). T55 measured 82.2 % (37/45); the eight
misses were findings SEC-005, SEC-007 and SEC-008, fixed in the T07 verifier. The list of misses is
pinned (empty), so a new miss fails the gate. A second gate is green and blocking: no corrupted value
ends in the high confidence band, with no exception.