Knowledge
A workflow with a knowledge node ({type: knowledge, index: fts}) keeps one corpus per
workflow (not per version: entries from v1 and v2 coexist, each with its workflow_version_id).
When a run finishes successfully its DocIR is compiled to anchored markdown (the same
p{n}_l{m} anchors the citation verifier uses), a YAML header with the metadata and the extracted
fields (path: value [anchor]), and chunked (~400 tokens) into Postgres full-text search
(ADR-013). The node itself is a marker: the interpreter treats it as
a no-op.
API
All routes live under /v1/workflows/{id}/. Rows marked mcp carry the mcp OpenAPI tag, which
T47 turns into agent tools; the same operations are available in-process as
extractit.knowledge.tools.KnowledgeTools (list_documents, search, read,
query_extractions; no side effects, no network).
| Route | mcp | Purpose |
|---|---|---|
GET knowledge |
status: indexed, pending, failed, stale, last_indexed_at |
|
GET knowledge/documents |
mcp | keyset list; filters by date, category, name, metadata |
GET knowledge/documents/{entry_id}?pages=&anchors= |
mcp | read only what is asked |
GET knowledge:search?q=&k= |
mcp | hits with entry, page, anchors, score, fragment ≤ 300 chars |
GET knowledge:snapshot |
deterministic tar.gz (one markdown per entry + index.json), ETag = sha256 |
|
POST knowledge:reindex {scope: all\|failed} |
background reindex of the history / of failed entries | |
POST knowledge:ask |
mcp | question with verified citations, threads |
POST knowledge:query {sql} |
mcp | restricted read-only SQL over extractions |
Query syntax
q is parsed by websearch_to_tsquery: words are ANDed, "quoted phrases", -excluded, and or.
Quotes or operators never raise a syntax error. Case and diacritics are ignored; amounts and
invoice numbers are kept literally (INV-2026-9001, 1.210,00). If the AND query returns fewer
than k hits, an OR over the terms completes the list.
Limits
- Reading: 64 KB per call (
truncated,next_anchor). - Search fragments: 300 characters;
k≤ 50. - More than 100 000 entries: search and list need a filter, otherwise
400 FILTER_REQUIRED. ask: 8 steps and 60 000 input tokens by default (incomplete: truewithreason), threads keep 6 turns on the server (the client's history is ignored; another workflow's thread is 404).query: oneSELECToverkb_documents/kb_extractions, allowed functionscount sum avg min max lower upper coalesce date_trunc cast, no subqueries, CTEs, set operations, comments or catalogs;LIMIT≤ 500; runs as roledx_knowledge_roin a READ ONLY transaction with 2 s timeout.
Errors (RFC 9457)
KNOWLEDGE_NOT_ENABLED (409), KNOWLEDGE_NOT_READY (409), FILTER_REQUIRED (400),
QUERY_NOT_ALLOWED (400). An invalid body is 400 validation-error.
Answers with citations
The answering loop uses only the read-only tools. Document text enters the prompt in delimited
data blocks the model is told to treat as data. Every citation goes through the grounding verifier
(verified | partial | block_only | unverified | contradicted, with normalized regions). With
strict=true (default) claims without a verified or partial citation are dropped and, if none
remain, abstained=true; with strict=false they are returned with supported=false.
Events
The index is refreshed by run.completed; extraction.corrected marks the entry stale and
packet.deleted removes it (emitted by T32 and T50; tests use synthetic events).
Evaluation
bench/knowledge_qa/ (30 questions) and bench/reports/knowledge-qa.md: exact match 1.00 and
verified citations 1.00 on literal questions (thresholds 0.8 and 0.9), with a deterministic oracle
policy standing in for the model. Search: top-5 hit rate 30/30 on 100 documents, p95 45 ms over
10 000 entries, 1 000 documents × 10 pages indexed in 50 s.