Hardening report (T55)
State: PARTIAL. Verified on a cloud VM (4 vCPU, 15 GB, Docker): the leak gates, the injection
suite, SSRF, formula injection, the viewer XSS run in Chromium, ZAP (API scan and baseline),
Locust (5 min), the burst and idempotency scenarios, a local chaos run (database latency and outage
behind Toxiproxy) and the supply-chain scanners. Not run: the sandbox battery (Pyodide bundle is
unreachable from the VM), the Compose and kind chaos, the 30-minute staging load, gVisor, the ASVS
chapters other than V8. The NFR-08 criterion "95 % of corrupted values stopped" was not met
by T55 (82.2 %) and is met since FIX-T07 (100 %, 45/45). The full table criterion -> test -> result
of the cloud run is in DELIVERY-T55.md.
All gates are local (make security*, see Security gates): there is no remote CI.
Criteria
| Criterion | State | Evidence |
|---|---|---|
RLS on every tenant table, discovered from pg_catalog |
met | make security-rls: 12 tests, including negatives (no RLS, no FORCE, no WITH CHECK, no factory, permissive policy, BYPASSRLS, empty context) |
| Every API operation as tenant B leaks nothing | met | make security-api-leak: 175 operations of openapi/openapi.json walked with tenant A's ids; 403/404, no identifier of A, A's rows unchanged; one exemption (segment PDF, reason in EXEMPT_OPERATIONS) |
| Injection suite: 0 tool calls, 0 connections | met | tests/redteam: 66 documents in 6 classes; request has no tool channel; socket guard saw 0 attempts |
≥ 95 % of corrupted values unverified/contradicted/rule-failed |
met: 100 % (45/45) since FIX-T07 (T55 measured 82.2 %, 37/45) | test_at_least_95_percent_of_corrupted_values_are_unverified_contradicted_or_fail_a_rule (no xfail); test_the_misses_are_exactly_the_documented_findings pins an empty list |
No corrupted value in the high confidence band |
met, no exception | test_no_corrupted_value_reaches_the_high_confidence_band |
| A purpose of a model call without a corpus case fails | met | tests/redteam/purpose_coverage.py (AST); 9 exemptions with reasons |
| Tool hijack against discovered registries | met | tool_registries.py: 110 attempts, none resolves to a tool; every DELETE and publish/alias operation is x-dx-destructive |
| CSV/XLSX formula injection | met | 0 of 100 payloads survive; XLSX needed a fix (quote prefix) |
| SSRF on webhook and URL destinations | met | 103-destination matrix, outbound-surface registry; SEC-004 fixed |
| Viewer renders hostile values as text | met | packages/viewer/e2e/xss.spec.ts: 62 payloads (event handlers on 8 tags x 6 sinks, javascript: links, markdown, <base>, exfiltration images), Chromium: no script, handler, dialog or outside request |
| Sandbox escape battery ≥ 30 | met | 42 attempts + 3 side-channel checks, deno profile; container profile not run |
| H8 confirmed | confirmed for the deno profile |
ADR-015 |
| ASVS 5.0.0 list checked on every run | partial | security/asvs.yaml: 345 requirements, V8 assessed (5 verified, 2 planned, L3 planned), the other chapters planned under SEC-ASVS-PENDING |
| ZAP API scan + baseline | met (limits below) | make security-zap: 175 operations in 6 scans, 0 High/Critical; baseline found X-Content-Type-Options missing on API answers (SEC-012, fixed) |
| Load: 5 min at 100 docs/min | met | 500 runs, 499 succeeded and 1 request connection error at the end of the window (0.027 % < 0.1 %), 0 lost, end-to-end p95 6.7 s |
| 50 submissions, one Idempotency-Key | met | 1 run, 49 Idempotent-Replayed: true |
| Burst of 1 000 | partial | 1 000 x 202 and all terminal in 449 s (< 15 min); submit p95 4.4 s with 40 concurrent clients (criterion 2 s) and 1.4 s with 10 (200 documents); single-process stack on 4 vCPU (SEC-013) |
| Chaos: database 30 s latency + 60 s outage | met locally | make chaos: 30/30 runs succeed, 0 lost, 0 steps with two successes (after SEC-017: 4 of 4 runs; before it 2 of 5 had one failed run). Raw reports: docs/security/reports/ |
| Chaos on Compose (worker kill, provider cut), kind | not run | script written (tests/chaos/compose_chaos.sh); the image cannot be built here |
| 500-page document, 25 MB file, 30 min staging load | not run | parse fake knows one document; no staging |
| bandit, pip-audit, gitleaks, pnpm audit | met | 0 high: see findings SEC-014..016 |
| External pentest | pending action | Pentest checklist |
Findings
| Id | Severity | Finding | State | Owner |
|---|---|---|---|---|
| SEC-004 | medium | SSRF in the per-organization SSO issuer discovery (no address guard) | fixed (ee/tests/test_ee_sso_ssrf_unit.py) |
T49 |
| SEC-005 | medium | An injection that quotes the literal value is verified against its own instruction line |
fixed (FIX-T07: lines of an instruction addressed to the model are never evidence, ground/instructions.py; tests/ground/test_verifier_injection.py) |
T07 |
| SEC-006 | low | http://localhost accepted as SSO issuer without a flag (development exemption) |
open | T49 |
| SEC-007 | medium | An enum value the verifier cannot read (currency=USD) ends block_only (confidence 0.60), not unverified: 4 of the 8 misses |
fixed (FIX-T07: block_only only when the normalizer reads the option in a phrase of the cited text; tests/ground/test_verifier_injection.py) |
T07 |
| SEC-008 | medium | Fuzzy partial grounding accepts a short value that shares a token with the cited line (CALLED read_file): 3 of the 8 misses |
fixed (FIX-T07: a fuzzy match must read ≥ 80 % of the value's characters as whole words, ground/matching.py::word_coverage; tests/ground/test_verifier_injection.py) |
T07 |
| SEC-009 | low | GET /v1/workflows/{id}/documents/columns answered 200 (empty) for a workflow of another tenant; PUT already answered 404 |
fixed, found by the API-leak walk | T55 |
| SEC-010 | info | Deno.cwd() answers inside the sandbox (names only the throw-away directory) |
accepted | T30 |
| SEC-011 | info | os.system crashes the sandbox runner and a Worker hangs to the wall limit (both fail closed) |
open | T30 |
| SEC-012 | low | API answers had no X-Content-Type-Options: nosniff (ZAP baseline) |
fixed (api/middleware.py, tests/api/test_api_conventions.py) |
T55 |
| SEC-013 | low | Submit p95 4.4 s at 40 concurrent uploads on one process (4 vCPU, worker in process); PG pool 32 with no overflow timed out at 100 concurrent | open (sizing; staging run needed) | T53 |
| SEC-014 | high (dev tool) | js-yaml < 4.3.2 under @hey-api/openapi-ts (3 high advisories, build-time only) |
fixed (pnpm override js-yaml >=4.3.2) |
T55 |
| SEC-015 | low | gitleaks: 5 fake sentinels in test fixtures from develop | fixed (per-file allowlist in .gitleaks.toml, 0 leaks) |
T55 |
| SEC-016 | info | bandit B701 (Jinja autoescape off) in two Markdown renderers | accepted (# nosec, not HTML) |
T55 |
| SEC-017 | medium | 60 s without Postgres exhausted the 3 attempts of the persist step: the run ended failed (found by the chaos run, 2 of 5 runs of 30) |
fixed (runtime/steps.py: database-unreachable failures retry up to 9 attempts, ~2 min; 4 of 4 chaos runs green after) |
T55 |
| XLSX | medium | Cells starting with =, +, -, @, tab or CR were written as formulas |
fixed (quotePrefix; tests/security/formula_injection) |
T55 |
| x-dx-destructive | low | The mark did not exist: every DELETE and publish_workflow_version/set_workflow_alias now carry it |
fixed | T55 |
The eight corrupted values the strict criterion missed at T55: visible-12 (SEC-005);
image_ocr-04, invisible-03, invisible-11, visible-06 (SEC-007);
tool_hijack-06, -09, -11 (SEC-008). FIX-T07 fixed the three findings in the T07 verifier and
the list is now empty: a test pins it, so any new miss fails it. No legitimate value of the
repository's fixtures moved from verified to a worse state (verdict-by-verdict comparison of
tests/ground, tests/eval, tests/golden, tests/redteam, tests/extract, tests/pipeline,
tests/review, tests/nodes, tests/knowledge, tests/agent_golden and tests/rules against
develop c47ec6ca). Limit: the instruction markers are a lexicon; an instruction phrased without
any of them that also quotes the value would be grounded (see Verification).
Not done (declared, not run)
- Sandbox battery (
make security-sandbox): skipped on the cloud VM (46 skipped: the Pyodide bundle is on GitHub/jsDelivr, blocked by the VM proxy). It passed on the maintainer's laptop (46). Container profile (gVisor): never run. - Compose and kind chaos (kill the worker, cut the provider):
tests/chaos/compose_chaos.shis written and untested; the image needs the PyTorch CPU wheels (unreachable here). - Staging: the 30-minute load, the 500-page document and the 25 MB file.
- ZAP depth: ZAP substitutes placeholder ids, so most requests answer 400 (the 604 client errors
are all 400, none 401: the key works). Authenticated deep paths are covered by the leak gates,
not by ZAP. The Studio (SPA) is not served by the local stack: CSP and
frame-ancestorscome fromweb_static.pyandviewere2e (CSP without violations), not from the baseline. - ASVS: chapters other than V8 are
planned(SEC-ASVS-PENDING, owner: T55 follow-up). - Locust is not a project dependency (
uv lockcannot reach the PyTorch index from the VM):make loadruns it withuvx. Add it withuv add --dev locustwhere the index is reachable. - Python license check of the runtime group cannot run here (needs the torch wheels); the web check passes.
- Videos and screenshots of the DoD: not recorded.