Classify and Split nodes
classify says what a document is; split cuts a packet into documents. Both read the pages through
one page classifier (extractit.nodes.pageclass), so they share the prompt, the output
schema, the evidence and the thresholds. Calibration of the confidence (T39), learning from
corrections (T42), datasets and managed evaluations (T38) and the canvas editors (T34) are other
tickets.
Classify
{"id": "kind", "type": "classify", "config": {
"categories": [
{"id": "invoice", "name": "Invoice", "description": "A bill with a total to pay"},
{"id": "contract", "name": "Contract", "description": "A legal agreement with clauses"}],
"instructions": "Tickets are invoices only when they carry a VAT id.",
"threshold": 0.7, "unclassified_branch": true}}
Edges leaving the node are labelled with a category id (or unclassified); an unlabelled edge
carries every document.
| Setting | Default | Meaning |
|---|---|---|
categories |
— | at least 2, ids unique, at most 3 levels; unclassified is a reserved id |
instructions |
— | the author's text, placed before the labels in the prompt |
threshold |
0.7 | a confidence at or above it classifies; below it the node abstains |
unclassified_branch |
false | below the threshold, activate the edge unclassified |
multi_label / max_labels |
false / 3 | every label at or above the threshold activates its branch, at most max_labels |
How it decides. The first pages of the scope (as many as fit 12 000 estimated tokens; the rest
is dropped with a input_truncated warning) go in one structured call with top_k 3 and two
quoted pieces of evidence per page. The page scores are averaged per label with weight 2 on the first
page and 1 on the others (a page that does not rank a label counts 0). The quotes pass through the
citation verifier of T07, so each evidence carries verified, partial, unverified… and the
regions to highlight.
Hierarchy. The model chooses among the leaf ids. hierarchy_path comes from the tree
(["finance", "invoice", "credit_note"]). A label leaves by the edge of its own id; when it has none,
by the edge of its nearest ancestor that has one. A category with no edge on itself or on any
ancestor runs nothing and the node warns (no_edge).
Confidence is heuristic, not a probability.
score = clamp(0.6 * s1 + 0.25 * clamp(s1 - s2, 0, 1) + 0.15 * g)
s1 is the aggregated score of the first label, s2 the second one's (0 if none) and g is 1 when
some evidence is verified. It orders documents for review (kind: "heuristic", signals
verbalized and grounding); T39 will calibrate it, and the default threshold of 0.7 is only a
starting point.
Abstention and unclassified. Below the threshold the classification has abstained: true,
label: null, and the edge unclassified runs when unclassified_branch is on. Without that edge
nothing runs, the node still succeeds and its warnings say so (the validator warns earlier:
classify_unclassified_missing). A label outside the enum is an abstention: the id is dropped,
the error is recorded in the node's warnings (page 1: label_not_in_enum:<id>) and the run goes
on.
Limit of categories. One call classifies reliably up to 40 leaf categories; above that the
validator warns (classify_many_leaves). Pages are summarised to 2 000 characters (head 1 500, tail
500) so a long packet stays cheap.
Classify inside a segment (a scope of a split) classifies that segment's pages only.
Split
{"id": "cut", "type": "split", "config": {
"rules": [
{"id": "invoice", "name": "Invoice", "description": "A sales invoice"},
{"id": "statement", "name": "Statement", "description": "A bank statement",
"partition_key": "account_number"}],
"allow_non_contiguous": true, "unassigned_branch": true, "min_confidence": 0.5}}
Edges leaving the node are labelled with a rule id (or unassigned). Every page goes through the
page classifier in windows of 25 pages with an overlap of 1 (up to 4 windows in parallel inside
the node's step; a page seen twice keeps the answer of the earlier window). The pure function
group_pages then builds the segments:
- Without
partition_keya document is a run of adjacent pages of the same rule; it ends where the rule changes or the next pagestarts_new_document(the model's flag with a score of at least 0.6, or a strong deterministic signal). - With
partition_keythe unit is (rule, normalized value). The value is read from the page and normalized (NFKC, trimmed, spaces collapsed, lower case; with 4 or more digits spaces and hyphens go too:ES91 2100 0418equalsES9121000418). A page of the rule with no value inherits the previous one's. Withallow_non_contiguousevery page of a unit forms one segment however far apart (pages 1, 2, 4, 6 of account A and 3, 5 of account B give[1,2,4,6]and[3,5]); without it, runs of adjacent pages ([1,2] [3] [4] [5] [6]). Note that the node's configuration default forallow_non_contiguousisfalse: set it for interleaved packets. - Every page is in exactly one segment (a property test with 200 generated packets).
unassigned. A page with no rule — or with a rule the node does not have — goes to the single segmentunassigned(unassigned_reason: "no_rule"). A segment whose confidence is belowmin_confidence(default 0.5) becomes unassigned too, withunassigned_reason: "low_confidence"and the rule it would have had assuggested_rule_id. The descendants of an unassigned segment run only withunassigned_branchon, by the edgeunassigned.
Boundary evidence. Each segment lists boundary_evidence (packet_page, reason, score) for
the page where it starts. reason is one of title, page_number_reset, new_header,
partition_change, blank_separator, model. The deterministic signals need no model:
| Signal | Rule | Score |
|---|---|---|
page_number_reset |
"page N of M" / "página N de M" / "N / M" with N = 1 after a page with N = M | 0.95 |
partition_change |
the normalized partition value differs from the previous page's | 0.90 |
blank_separator |
a page with fewer than 5 words, and the page after it | 0.80 |
new_header |
the hash of the first two lines (digits folded) differs from the previous page's and equals the one of a page that opened a document | 0.70 |
title |
the page starts with a title block of the layout | 0.60 |
The confidence of a segment is the minimum of its pages' confidences (the score of the assigned rule) and the score of its boundaries.
Fan-out. The node returns one scope per segment (ScopeSpec(kind="segment", branch=<rule id>)):
the interpreter runs the descendants once per segment, at most max_fanout at a time, and the scope
key is the segment id (seg_…, stable for the same pages of the same node). The extraction of a
segment sits at outputs.extractions[<node>].scopes[<segment id>] of the run result. In a
multi-file packet the interpreter runs the node once per file, so a segment does not cross files
inside a run; the grouping, the evaluation and the PDF route do handle segments that do.
The segments and classifications API
| Route | Returns |
|---|---|
GET /v1/runs/{id}/classifications |
the classifications (06 §8): label, label_name, labels, confidence, ranking, rationale, evidence, abstained, hierarchy_path |
GET /v1/runs/{id}/segments |
the segments: id, rule_id, partition_value, pages (file_id, file_page, packet_page), boundary_evidence, confidence, status, unassigned_reason, suggested_rule_id |
GET /v1/runs/{id}/segments/{segment_id}/pdf |
the segment as an independent PDF |
The PDF is made on the first request with pypdf from the packet's files (pages in segment order,
from several files if needed) and stored as a run artifact; asking again builds and stores nothing.
ETag is the sha256 of the PDF and If-None-Match answers 304. A segment that does not exist, or
belongs to another project, is 404. Both nodes are cacheable: a re-run with the same configuration,
model and DocIR reuses the outcome and still gets its rows.
Validation rules
| Rule | Severity | When |
|---|---|---|
classify_categories_invalid |
error | fewer than 2 categories or 2 leaves, duplicate ids, more than 3 levels, or the reserved id unclassified |
classify_unclassified_edge_unknown |
error | an unclassified edge while unclassified_branch is off |
classify_unclassified_missing |
warning | unclassified_branch is on but no edge uses it |
classify_many_leaves |
warning | more than 40 leaf categories |
Quality
classification_report (accuracy, per-class precision/recall/F1, macro-F1, confusion matrix with the
abstentions, abstention rate and coverage, bootstrap intervals of T10) and segmentation_report
(page-assignment accuracy, boundary precision/recall/F1, F1 of exact segments, Pk and WindowDiff) are
pure functions (T10 has no metric registry). The synthetic datasets are bench/datasets/classify-v0
(40 documents, 4 classes) and bench/datasets/split-v0 (6 packets of 12 to 40 pages), generated by
bench/gen_packets.py with seed 28 and shipped with a frozen DocIR so the prompts do not depend on
the platform. python bench/classify_split_eval.py eval replays the cassettes;
bench/reports/classify-split-v0.md has the numbers, including what a real model measured and
how the cassettes of CI were recorded (a deterministic lexical reader, not a model).