Skip to content

Tables and logical tables

extractit.parse.tables (T19) completes the tables of the DocIR: cell structure with spans, multilevel headers with a canonical path per column, a confidence per cell, borderless tables, and logical tables that stitch the tables of consecutive pages into one verified table. No language model is used. parse() runs it right after building the DocIR (refine_document), so every ParsedDocument it returns already carries refined tables[] and logical_tables[].

extractit parse statement.pdf --format tables   # tables[], logical_tables[] (rows and parts), stitch decisions

Cell structure (refine_table)

from extractit.parse.tables import refine_table, header_paths

table = refine_table(doc.tables[0], doc.words)  # -> TableStructure
header_paths(table)  # ['Código', 'Importe / Neto', 'Importe / IVA']

Starting from TableFormer's grid (T04; the model is not called again):

  1. Grid. Cells are clamped to n_rows x n_cols; when two cells claim a position the larger one stays; positions nobody covers become empty cells.
  2. Words. Text and boxes always come from the DocIR words, never from the model. A word goes to the cell whose box contains its center (1 % of the cell height of tolerance; ties go to the largest overlap); a word outside every cell goes to the nearest cell within 1.5 row heights, otherwise it is listed in extra.orphan_words and the document gets a table_orphan_words warning. Text joins the words by visual line (vertical overlap) and x0 — the ticket's (line_idx, x0), but engine lines can split one visual line of a cell out of order; word_idxs lists them and word_range is set only when they are contiguous.
  3. Geometry. Column and row separators are the midpoints between the word extents of neighbouring columns and rows; column_x[j] is the median right edge of the single-column cells of column j (page coordinates).
  4. Headers (up to 3 rows). TableFormer's header flag when it exists (a flagged row that is at least 40 % numbers is not a header); otherwise leading rows where 70 % of the cells are non-empty and non-numeric, or with a col_span > 1 cell, or bold, followed by a row with at least 40 % numbers. cells[].is_header is exactly row in header_rows. header_paths joins, per leaf column, the header cells covering it from top to bottom without repeats: 'Importe / Neto'.
  5. Confidence. struct_conf = 0.4 * word_fit + 0.3 * grid_alignment + 0.3 * structure_score: word_fit is the share of the cell's words with 80 % of their box inside the cell's grid box, grid_alignment = 1 - d / (w / 2) (clipped) with d the distance from the cell's right edge to column_x of its last column and w that column's width, structure_score TableFormer's score or the documented constant 0.7 (Docling does not expose one). cell.conf = min(text_conf, struct_conf) with text_conf the mean OCR confidence of its words (native text: left out); null only for empty cells. TableStructure.confidence is the mean over cells with text.
  6. Borderless tables (method='hybrid'). A table with fewer than two columns, a confidence below 0.5, or a table block without structure is rebuilt from its words: rows by vertical overlap >= 50 %, columns by valleys of the horizontal projection wider than max(1.2 * median space between words, 1.2 % of the page width); a run of words over two columns becomes one cell with col_span. A gap counts as a space only when it is narrower than one word height and 1.5 median character widths, and the 1.2 % floor is capped at those 1.5 characters (T21): in a 6 pt borderless table a 5 mm gutter is under one word height, and counting it as a space, or a floor of two characters, merged an amount column with the text column next to it. Text of about 10 pt or more keeps the 1.2 % floor.
  7. Nested tables keep a flat structure and are marked extra.nested = true.
  8. Corrections found on real Docling output (long-tables-v0, tables-v0):
  9. dense rows: when TableFormer's rows disagree with the text lines (two lines in two or more cells of a row, or one line spread over two rows) the body rows are rebuilt from the lines, keeping the model's columns; a line with words only in text columns continues the row above (wrapped cell);
  10. merged columns: when two or more cells hide a column-wide gap between words (more than 4 median spaces and 2.5 % of the page) the table is rebuilt by gap grouping;
  11. multilevel headers: a header of two or more rows (and any gap-grouped header) is rebuilt from its words on the body's column bands — a label belongs to the first column whose band ends at or after its left edge — and group spans are inferred (a group header spans up to the next header cell of its row; a cell with nothing below spans down the header rows);
  12. OCR pages: RapidOCR pages come without layout, so their tables are found by gap grouping (rows of three or more columns separated by valleys, header rows above them).

Refining a refined table returns it unchanged (a refined table keeps its word assignment).

HTML and anchors

Block.markdown of a table block is HTML with row and cell anchors:

<table>
<thead>
<tr data-r="p3_t1_r0"><th scope="col" data-c="p3_t1_r0_c0" rowspan="2">Código</th><th scope="col" data-c="p3_t1_r0_c1" colspan="2">Importe</th></tr>
…
</thead>
<tbody>
<tr data-r="p3_t1_r17"><td data-c="p3_t1_r17_c0">A-1023</td>…<td data-c="p3_t1_r17_c4">1.234,56</td></tr>
</tbody>
</table>

doc.resolve_anchor("p3_t1_r17_c4") returns the word_ids and box of that cell, doc.resolve_anchor("p3_t1_r17") the union of the row, and an unknown anchor raises AnchorNotFound. The anchored markdown (to_anchored_markdown, T04) is unchanged: inside tables it keeps one anchor per line plus the <tr data-r> row anchors, so the citations and cassettes of T06–T08 do not move; cell anchors are resolved through the DocIR.

Logical tables (stitch_tables)

from extractit.parse.tables import load_config, stitch_tables
from extractit.parse.tables.stitch import logical_row_texts

logical = stitch_tables(doc, load_config(), overrides={(table_a.id, table_b.id): True})
lt = doc.logical_tables[0]
lt.row_ref(37).parts  # (RowPart(table_id, page, physical_row, packet_page), …)
logical_row_texts(doc, lt)[37]

For every pair (last table of page i, first table of page i+1) of one file, six signals in [0, 1]:

Signal Weight 1 when
columns 0.30 same column lines (column_x): mean error <= 1.5 % of the page width (Hungarian matching if the column count differs)
geometry 0.25 the first table reaches its bottom limit (top of the page_footer) and the second starts under the page_header, nothing in between
header_or_numbering 0.20 the second table repeats the header (rapidfuzz >= 90 on >= 80 % of the cells) or its first column continues the numbering
types 0.10 each column keeps its dominant type (number, date, text)
no_total 0.10 the first table does not end with a total row ('Suma y sigue', 'carried forward', 'Übertrag'… are markers, not totals)
marker 0.05 a continuation marker near the page break

s >= 0.60 and no veto stitches; 0.45 <= s < 0.60 does not stitch and emits ambiguous_table_stitch; overrides (ids or anchors) wins over both (T21 uses it after VLM confirmation). Vetoes: column error > 3 %, a body block between the tables, a final total row, and a first table that ends more than 25 % of the page above its bottom limit (added after tuning on the negative pairs: with a repeated header such a pair would otherwise reach 0.70). A page without a table breaks the chain; different files of a packet are never stitched (stitch_packet stitches each DocIR alone and numbers packet_page across the packet).

Every physical table belongs to exactly one LogicalTable (one segment when nothing is stitched). row_offset accumulates data rows after removing repeated header rows and continuation-marker rows ('Suma y sigue … 12.345,67'); the canonical header is the first segment's (header_levels keeps its rows). Split rows: when a stitched segment's last row ends within 3 % of the bottom margin and the next segment's first row fills the complementary columns (fill_complement >= 0.8), both are one logical row (split_rows, row_ref(i).parts has two parts, texts joined with a space). Stitched tables get continues.from/to.

Verification (stitch_checks): rows_conserved, column_types_consistent (numeric/date columns >= 70 % in the first segment stay >= 90 % in the others), numbering_continuous and carry_forward_balance (a 'suma y sigue' amount equals the sum above it, tolerance 0.01). A failed check emits suspect_table_stitch and keeps the stitch for T21 or review.

Weights and thresholds live in parse/tables/stitch_config.toml (version and sha256 in StitchConfig, reported in bench/reports/long-tables-v0.md); they were tuned on long-tables-v0 and its negatives and are frozen before T52.

Datasets and evaluation

Dataset Content Build
bench/datasets/tables-v0 45 one-page tables: 15 ruled with merged cells, 15 borderless, 15 scanned at 200 DPI; 12 with 2–3 header levels; seed 20261116 uv run python bench/datasets/tables-v0/build.py [--verify]
bench/datasets/long-tables-v0 36 documents (1, 2, 5, 10, 20, 50 pages x variants a–f) + 12 negative pairs; seed 20261117 uv run python bench/datasets/long-tables-v0/build.py [--verify]

Both are rendered with WeasyPrint from HTML (generator bench/generators/tablegen.py, bundled Noto Sans), with ground truth read from the layout box tree. Only GT and manifest are versioned (with sha256 per file and a global hash); PDFs are rebuilt by build.py, and --verify compares the GT bytes and, where recorded, the platform's PDF hash.

uv run python bench/scorers/tables.py tables-v0 --out bench/reports/tables-v0.md
uv run python bench/scorers/tables.py long-tables-v0 --out bench/reports/long-tables-v0.md
uv run python bench/scorers/tables.py long-tables-v0 --engine fake --sizes 1,2,5   # no weights

The scorer reuses the row metrics of the evaluation harness (Hungarian matching on the row key, recall, precision, duplicates, Kendall's tau, cell accuracy) and adds the structural F1 of cells (IoU >= 0.5 and equal spans, non-empty cells), text exactness over matched cells, header accuracy and stitch recall. --engine fake replaces Docling with a deterministic structure provider (docling-parse words + the GT grids) so the CI of a PR runs refinement and stitching without weights; the Docling runs are the docling marker and the reports.