Skip to content

Classify and Split nodes

classify says what a document is; split cuts a packet into documents. Both read the pages through one page classifier (extractit.nodes.pageclass), so they share the prompt, the output schema, the evidence and the thresholds. Calibration of the confidence (T39), learning from corrections (T42), datasets and managed evaluations (T38) and the canvas editors (T34) are other tickets.

Classify

{"id": "kind", "type": "classify", "config": {
  "categories": [
    {"id": "invoice", "name": "Invoice", "description": "A bill with a total to pay"},
    {"id": "contract", "name": "Contract", "description": "A legal agreement with clauses"}],
  "instructions": "Tickets are invoices only when they carry a VAT id.",
  "threshold": 0.7, "unclassified_branch": true}}

Edges leaving the node are labelled with a category id (or unclassified); an unlabelled edge carries every document.

Setting Default Meaning
categories — at least 2, ids unique, at most 3 levels; unclassified is a reserved id
instructions — the author's text, placed before the labels in the prompt
threshold 0.7 a confidence at or above it classifies; below it the node abstains
unclassified_branch false below the threshold, activate the edge unclassified
multi_label / max_labels false / 3 every label at or above the threshold activates its branch, at most max_labels

How it decides. The first pages of the scope (as many as fit 12 000 estimated tokens; the rest is dropped with a input_truncated warning) go in one structured call with top_k 3 and two quoted pieces of evidence per page. The page scores are averaged per label with weight 2 on the first page and 1 on the others (a page that does not rank a label counts 0). The quotes pass through the citation verifier of T07, so each evidence carries verified, partial, unverified… and the regions to highlight.

Hierarchy. The model chooses among the leaf ids. hierarchy_path comes from the tree (["finance", "invoice", "credit_note"]). A label leaves by the edge of its own id; when it has none, by the edge of its nearest ancestor that has one. A category with no edge on itself or on any ancestor runs nothing and the node warns (no_edge).

Confidence is heuristic, not a probability.

score = clamp(0.6 * s1 + 0.25 * clamp(s1 - s2, 0, 1) + 0.15 * g)

s1 is the aggregated score of the first label, s2 the second one's (0 if none) and g is 1 when some evidence is verified. It orders documents for review (kind: "heuristic", signals verbalized and grounding); T39 will calibrate it, and the default threshold of 0.7 is only a starting point.

Abstention and unclassified. Below the threshold the classification has abstained: true, label: null, and the edge unclassified runs when unclassified_branch is on. Without that edge nothing runs, the node still succeeds and its warnings say so (the validator warns earlier: classify_unclassified_missing). A label outside the enum is an abstention: the id is dropped, the error is recorded in the node's warnings (page 1: label_not_in_enum:<id>) and the run goes on.

Limit of categories. One call classifies reliably up to 40 leaf categories; above that the validator warns (classify_many_leaves). Pages are summarised to 2 000 characters (head 1 500, tail 500) so a long packet stays cheap.

Classify inside a segment (a scope of a split) classifies that segment's pages only.

Split

{"id": "cut", "type": "split", "config": {
  "rules": [
    {"id": "invoice", "name": "Invoice", "description": "A sales invoice"},
    {"id": "statement", "name": "Statement", "description": "A bank statement",
     "partition_key": "account_number"}],
  "allow_non_contiguous": true, "unassigned_branch": true, "min_confidence": 0.5}}

Edges leaving the node are labelled with a rule id (or unassigned). Every page goes through the page classifier in windows of 25 pages with an overlap of 1 (up to 4 windows in parallel inside the node's step; a page seen twice keeps the answer of the earlier window). The pure function group_pages then builds the segments:

  • Without partition_key a document is a run of adjacent pages of the same rule; it ends where the rule changes or the next page starts_new_document (the model's flag with a score of at least 0.6, or a strong deterministic signal).
  • With partition_key the unit is (rule, normalized value). The value is read from the page and normalized (NFKC, trimmed, spaces collapsed, lower case; with 4 or more digits spaces and hyphens go too: ES91 2100 0418 equals ES9121000418). A page of the rule with no value inherits the previous one's. With allow_non_contiguous every page of a unit forms one segment however far apart (pages 1, 2, 4, 6 of account A and 3, 5 of account B give [1,2,4,6] and [3,5]); without it, runs of adjacent pages ([1,2] [3] [4] [5] [6]). Note that the node's configuration default for allow_non_contiguous is false: set it for interleaved packets.
  • Every page is in exactly one segment (a property test with 200 generated packets).
  • unassigned. A page with no rule — or with a rule the node does not have — goes to the single segment unassigned (unassigned_reason: "no_rule"). A segment whose confidence is below min_confidence (default 0.5) becomes unassigned too, with unassigned_reason: "low_confidence" and the rule it would have had as suggested_rule_id. The descendants of an unassigned segment run only with unassigned_branch on, by the edge unassigned.

Boundary evidence. Each segment lists boundary_evidence (packet_page, reason, score) for the page where it starts. reason is one of title, page_number_reset, new_header, partition_change, blank_separator, model. The deterministic signals need no model:

Signal Rule Score
page_number_reset "page N of M" / "página N de M" / "N / M" with N = 1 after a page with N = M 0.95
partition_change the normalized partition value differs from the previous page's 0.90
blank_separator a page with fewer than 5 words, and the page after it 0.80
new_header the hash of the first two lines (digits folded) differs from the previous page's and equals the one of a page that opened a document 0.70
title the page starts with a title block of the layout 0.60

The confidence of a segment is the minimum of its pages' confidences (the score of the assigned rule) and the score of its boundaries.

Fan-out. The node returns one scope per segment (ScopeSpec(kind="segment", branch=<rule id>)): the interpreter runs the descendants once per segment, at most max_fanout at a time, and the scope key is the segment id (seg_…, stable for the same pages of the same node). The extraction of a segment sits at outputs.extractions[<node>].scopes[<segment id>] of the run result. In a multi-file packet the interpreter runs the node once per file, so a segment does not cross files inside a run; the grouping, the evaluation and the PDF route do handle segments that do.

The segments and classifications API

Route Returns
GET /v1/runs/{id}/classifications the classifications (06 §8): label, label_name, labels, confidence, ranking, rationale, evidence, abstained, hierarchy_path
GET /v1/runs/{id}/segments the segments: id, rule_id, partition_value, pages (file_id, file_page, packet_page), boundary_evidence, confidence, status, unassigned_reason, suggested_rule_id
GET /v1/runs/{id}/segments/{segment_id}/pdf the segment as an independent PDF

The PDF is made on the first request with pypdf from the packet's files (pages in segment order, from several files if needed) and stored as a run artifact; asking again builds and stores nothing. ETag is the sha256 of the PDF and If-None-Match answers 304. A segment that does not exist, or belongs to another project, is 404. Both nodes are cacheable: a re-run with the same configuration, model and DocIR reuses the outcome and still gets its rows.

Validation rules

Rule Severity When
classify_categories_invalid error fewer than 2 categories or 2 leaves, duplicate ids, more than 3 levels, or the reserved id unclassified
classify_unclassified_edge_unknown error an unclassified edge while unclassified_branch is off
classify_unclassified_missing warning unclassified_branch is on but no edge uses it
classify_many_leaves warning more than 40 leaf categories

Quality

classification_report (accuracy, per-class precision/recall/F1, macro-F1, confusion matrix with the abstentions, abstention rate and coverage, bootstrap intervals of T10) and segmentation_report (page-assignment accuracy, boundary precision/recall/F1, F1 of exact segments, Pk and WindowDiff) are pure functions (T10 has no metric registry). The synthetic datasets are bench/datasets/classify-v0 (40 documents, 4 classes) and bench/datasets/split-v0 (6 packets of 12 to 40 pages), generated by bench/gen_packets.py with seed 28 and shipped with a frozen DocIR so the prompts do not depend on the platform. python bench/classify_split_eval.py eval replays the cassettes; bench/reports/classify-split-v0.md has the numbers, including what a real model measured and how the cassettes of CI were recorded (a deterministic lexical reader, not a model).