Tables and logical tables
extractit.parse.tables (T19) completes the tables of the DocIR: cell structure with spans,
multilevel headers with a canonical path per column, a confidence per cell, borderless tables,
and logical tables that stitch the tables of consecutive pages into one verified table.
No language model is used. parse() runs it right after building the DocIR
(refine_document), so every ParsedDocument it returns already carries refined tables[] and
logical_tables[].
extractit parse statement.pdf --format tables # tables[], logical_tables[] (rows and parts), stitch decisions
Cell structure (refine_table)
from extractit.parse.tables import refine_table, header_paths
table = refine_table(doc.tables[0], doc.words) # -> TableStructure
header_paths(table) # ['Código', 'Importe / Neto', 'Importe / IVA']
Starting from TableFormer's grid (T04; the model is not called again):
- Grid. Cells are clamped to
n_rows x n_cols; when two cells claim a position the larger one stays; positions nobody covers become empty cells. - Words. Text and boxes always come from the DocIR words, never from the model. A word goes to
the cell whose box contains its center (1 % of the cell height of tolerance; ties go to the
largest overlap); a word outside every cell goes to the nearest cell within 1.5 row heights,
otherwise it is listed in
extra.orphan_wordsand the document gets atable_orphan_wordswarning. Text joins the words by visual line (vertical overlap) andx0— the ticket's(line_idx, x0), but engine lines can split one visual line of a cell out of order;word_idxslists them andword_rangeis set only when they are contiguous. - Geometry. Column and row separators are the midpoints between the word extents of
neighbouring columns and rows;
column_x[j]is the median right edge of the single-column cells of columnj(page coordinates). - Headers (up to 3 rows). TableFormer's header flag when it exists (a flagged row that is
at least 40 % numbers is not a header); otherwise leading rows where 70 % of the cells are
non-empty and non-numeric, or with a
col_span > 1cell, or bold, followed by a row with at least 40 % numbers.cells[].is_headeris exactlyrow in header_rows.header_pathsjoins, per leaf column, the header cells covering it from top to bottom without repeats:'Importe / Neto'. - Confidence.
struct_conf = 0.4 * word_fit + 0.3 * grid_alignment + 0.3 * structure_score:word_fitis the share of the cell's words with 80 % of their box inside the cell's grid box,grid_alignment = 1 - d / (w / 2)(clipped) withdthe distance from the cell's right edge tocolumn_xof its last column andwthat column's width,structure_scoreTableFormer's score or the documented constant 0.7 (Docling does not expose one).cell.conf = min(text_conf, struct_conf)withtext_confthe mean OCR confidence of its words (native text: left out);nullonly for empty cells.TableStructure.confidenceis the mean over cells with text. - Borderless tables (
method='hybrid'). A table with fewer than two columns, a confidence below 0.5, or atableblock without structure is rebuilt from its words: rows by vertical overlap >= 50 %, columns by valleys of the horizontal projection wider thanmax(1.2 * median space between words, 1.2 % of the page width); a run of words over two columns becomes one cell withcol_span. A gap counts as a space only when it is narrower than one word height and 1.5 median character widths, and the 1.2 % floor is capped at those 1.5 characters (T21): in a 6 pt borderless table a 5 mm gutter is under one word height, and counting it as a space, or a floor of two characters, merged an amount column with the text column next to it. Text of about 10 pt or more keeps the 1.2 % floor. - Nested tables keep a flat structure and are marked
extra.nested = true. - Corrections found on real Docling output (long-tables-v0, tables-v0):
- dense rows: when TableFormer's rows disagree with the text lines (two lines in two or more cells of a row, or one line spread over two rows) the body rows are rebuilt from the lines, keeping the model's columns; a line with words only in text columns continues the row above (wrapped cell);
- merged columns: when two or more cells hide a column-wide gap between words (more than 4 median spaces and 2.5 % of the page) the table is rebuilt by gap grouping;
- multilevel headers: a header of two or more rows (and any gap-grouped header) is rebuilt from its words on the body's column bands — a label belongs to the first column whose band ends at or after its left edge — and group spans are inferred (a group header spans up to the next header cell of its row; a cell with nothing below spans down the header rows);
- OCR pages: RapidOCR pages come without layout, so their tables are found by gap grouping (rows of three or more columns separated by valleys, header rows above them).
Refining a refined table returns it unchanged (a refined table keeps its word assignment).
HTML and anchors
Block.markdown of a table block is HTML with row and cell anchors:
<table>
<thead>
<tr data-r="p3_t1_r0"><th scope="col" data-c="p3_t1_r0_c0" rowspan="2">Código</th><th scope="col" data-c="p3_t1_r0_c1" colspan="2">Importe</th></tr>
…
</thead>
<tbody>
<tr data-r="p3_t1_r17"><td data-c="p3_t1_r17_c0">A-1023</td>…<td data-c="p3_t1_r17_c4">1.234,56</td></tr>
</tbody>
</table>
doc.resolve_anchor("p3_t1_r17_c4") returns the word_ids and box of that cell,
doc.resolve_anchor("p3_t1_r17") the union of the row, and an unknown anchor raises
AnchorNotFound. The anchored markdown (to_anchored_markdown, T04) is unchanged: inside tables
it keeps one anchor per line plus the <tr data-r> row anchors, so the citations and cassettes of
T06–T08 do not move; cell anchors are resolved through the DocIR.
Logical tables (stitch_tables)
from extractit.parse.tables import load_config, stitch_tables
from extractit.parse.tables.stitch import logical_row_texts
logical = stitch_tables(doc, load_config(), overrides={(table_a.id, table_b.id): True})
lt = doc.logical_tables[0]
lt.row_ref(37).parts # (RowPart(table_id, page, physical_row, packet_page), …)
logical_row_texts(doc, lt)[37]
For every pair (last table of page i, first table of page i+1) of one file, six signals in [0, 1]:
| Signal | Weight | 1 when |
|---|---|---|
columns |
0.30 | same column lines (column_x): mean error <= 1.5 % of the page width (Hungarian matching if the column count differs) |
geometry |
0.25 | the first table reaches its bottom limit (top of the page_footer) and the second starts under the page_header, nothing in between |
header_or_numbering |
0.20 | the second table repeats the header (rapidfuzz >= 90 on >= 80 % of the cells) or its first column continues the numbering |
types |
0.10 | each column keeps its dominant type (number, date, text) |
no_total |
0.10 | the first table does not end with a total row ('Suma y sigue', 'carried forward', 'Übertrag'… are markers, not totals) |
marker |
0.05 | a continuation marker near the page break |
s >= 0.60 and no veto stitches; 0.45 <= s < 0.60 does not stitch and emits
ambiguous_table_stitch; overrides (ids or anchors) wins over both (T21 uses it after VLM
confirmation). Vetoes: column error > 3 %, a body block between the tables, a final total row,
and a first table that ends more than 25 % of the page above its bottom limit (added after tuning
on the negative pairs: with a repeated header such a pair would otherwise reach 0.70). A page
without a table breaks the chain; different files of a packet are never stitched
(stitch_packet stitches each DocIR alone and numbers packet_page across the packet).
Every physical table belongs to exactly one LogicalTable (one segment when nothing is stitched).
row_offset accumulates data rows after removing repeated header rows and continuation-marker
rows ('Suma y sigue … 12.345,67'); the canonical header is the first segment's
(header_levels keeps its rows). Split rows: when a stitched segment's last row ends within
3 % of the bottom margin and the next segment's first row fills the complementary columns
(fill_complement >= 0.8), both are one logical row (split_rows, row_ref(i).parts has two
parts, texts joined with a space). Stitched tables get continues.from/to.
Verification (stitch_checks): rows_conserved, column_types_consistent (numeric/date
columns >= 70 % in the first segment stay >= 90 % in the others), numbering_continuous and
carry_forward_balance (a 'suma y sigue' amount equals the sum above it, tolerance 0.01). A
failed check emits suspect_table_stitch and keeps the stitch for T21 or review.
Weights and thresholds live in parse/tables/stitch_config.toml (version and sha256 in
StitchConfig, reported in bench/reports/long-tables-v0.md); they were tuned on
long-tables-v0 and its negatives and are frozen before T52.
Datasets and evaluation
| Dataset | Content | Build |
|---|---|---|
bench/datasets/tables-v0 |
45 one-page tables: 15 ruled with merged cells, 15 borderless, 15 scanned at 200 DPI; 12 with 2–3 header levels; seed 20261116 | uv run python bench/datasets/tables-v0/build.py [--verify] |
bench/datasets/long-tables-v0 |
36 documents (1, 2, 5, 10, 20, 50 pages x variants a–f) + 12 negative pairs; seed 20261117 | uv run python bench/datasets/long-tables-v0/build.py [--verify] |
Both are rendered with WeasyPrint from HTML (generator bench/generators/tablegen.py, bundled Noto
Sans), with ground truth read from the layout box tree. Only GT and manifest are versioned (with
sha256 per file and a global hash); PDFs are rebuilt by build.py, and --verify compares the GT
bytes and, where recorded, the platform's PDF hash.
uv run python bench/scorers/tables.py tables-v0 --out bench/reports/tables-v0.md
uv run python bench/scorers/tables.py long-tables-v0 --out bench/reports/long-tables-v0.md
uv run python bench/scorers/tables.py long-tables-v0 --engine fake --sizes 1,2,5 # no weights
The scorer reuses the row metrics of the evaluation harness (Hungarian matching on the row key,
recall, precision, duplicates, Kendall's tau, cell accuracy) and adds the structural F1 of cells
(IoU >= 0.5 and equal spans, non-empty cells), text exactness over matched cells, header accuracy
and stitch recall. --engine fake replaces Docling with a deterministic structure provider
(docling-parse words + the GT grids) so the CI of a PR runs refinement and stitching without
weights; the Docling runs are the docling marker and the reports.