Skip to content

OCR, image quality and managed engines (tier Fast)

parse() reads a page by OCR when the PDF has no text layer for it, or when the layer is not reliable. The result is the same DocIR (words with boxes, lines, blocks, anchors) as for a digital page: the rest of the system tells a scanned page from a digital one only by its route and its confidence.

extractit parse scan.pdf -o out/                   # --ocr auto (default): OCR only where it is needed
extractit parse scan.pdf --ocr force               # every page through OCR
extractit parse scan.pdf --ocr off                 # never (pages without text keep the `needs_ocr` warning)
extractit parse scan.pdf --languages es,ru         # hint: picks the recognition model(s) up front
extractit parse scan.pdf --ocr-engine azure_di     # a cloud engine (needs credentials, see below)
extractit models pull ocr [--script greek]         # download the RapidOCR models (sha256 verified)
extractit models status ocr
extractit engines list                             # engines, `enabled = true|false`, price, region
extractit engines test rapidocr                    # read one synthetic page (a cloud engine needs --yes)

With the default engine (RapidOCR) nothing leaves the machine. A cloud engine receives only the pages the stage assigns to it (never a digital page with a reliable text layer), as a JPEG of the corrected page.

When a page goes to OCR

--ocr auto: a page enters OCR if T04 left it without words, or if textlayer.assess rejects its text layer:

  1. more than 10 % of the characters are U+FFFD, (cid:n), GLYPH<n> or private use (U+E000–U+F8FF) → text_layer_unreliable (corrupt_chars);
  2. the page shows content (more than 10 % non-white pixels at 50 DPI) but the word boxes cover less than 5 % of it → text_layer_unreliable (low_coverage);
  3. the page has a full-page image (≥ 90 % of the area) and a text layer whose sample (3 regions of 20 lines), re-read with RapidOCR, has a CER above 0.25 → text_layer_unreliable (hidden_layer_mismatch).

A good hidden layer (an old OCR that is right) is kept: route = text_layer. A blank page (paper and noise, no ink) is never sent to an engine: 0 words and the warning blank_page.

What the stage does to a page

  1. Render at 200 DPI (pypdfium2). A page that is one raster below 150 DPI is rendered at its own resolution and enlarged (bicubic): ocr_low_dpi_upscaled.
  2. Orientation (0/90/180/270): a 900 px thumbnail (the ticket says 1 024 px; 900 px keeps the check under 0.5 s) and a projection profile decide the order of the probes; OCR of the thumbnail scores each rotation by sum(conf × length) of its horizontal lines; the first rotation that reads well (mean conf ≥ 0.88) ends the search, otherwise the best wins if it beats the runner-up by 1.3×. The upright page costs one probe.
  3. Deskew: Otsu, a horizontal closing, HoughLinesP; corrected when 0.5° < |angle| ≤ 15°.
  4. Enhancement: CLAHE and sharpening on the grey copy the engine reads, only when the contrast is low.
  5. The page image that is stored and shown by the viewer is the corrected colour image, so every box is in the displayed space. Page.rotation_applied = the PDF's /Rotate plus the OCR rotation (clockwise), Page.deskew_angle_deg = the skew undone (counter-clockwise positive); when the OCR rotation is 90 or 270 the page's width_pt/height_pt are swapped. extractit parse -o rebuilds that image deterministically (parse.ocr.images.page_images).
  6. Engine (OcrEngine.recognize(image, hint) -> OcrPageResult), then build_document (T04): the same anchors p{n}_l{m} as a digital page, numbered from 1 in reading order. Word boxes of RapidOCR are snapped to the ink.
  7. Quality: Page.quality (parse_confidence, layout_confidence, ocr_mean_conf, blur_score, contrast_score, skew_deg) and ParsedDocument.quality (weighted by words; pages_flagged lists pages with layout < 0.5 or parse < 0.6, configurable in ParseProfile).

Pages read by OCR have no layout model: Docling gives no regions for an image without text cells, so the blocks are XY-cut paragraphs and layout_confidence is None until the VLM path (T20) adds types and tables.

Scripts and languages

Script (module regex, Unicode Script): dominant script with ≥ 60 % of the letters, second script ≥ 15 % → mixed_scripts. Language: lingua-language-detector built with es, en, fr, de, it, pt, pl, nl and ru, only for texts of 200+ characters.

Models (models_ocr.toml): latin (PP-OCRv6 small, multilingual Latin) and cyrillic (PP-OCRv5) are installed by extractit models pull ocr; greek, hangul and thai are optional (--script). Arabic, Indic scripts and CJK have no model: the page is not read, the warnings script_unsupported_ocr and needs_vlm appear (the signal for T20's router) and the page is flagged.

  • No hint: the first pass uses latin. If it reads badly (mean confidence under 0.85, ink coverage under 0.8, fewer characters than the line boxes hold, or long lines that read almost nothing) the page is read once more with the next model; the best line of each place is kept (ocr_model_switch, both attempts in provenance.engines). A page in a mixed script comes out of that merge line by line.
  • --languages ru: the model comes from the hint and nothing is switched. --languages es,ru runs both models and merges line by line. An unknown code is ignored with invalid_language_hint.
  • Unicode normalization (NFC; ligatures U+FB00–FB06 expanded; soft hyphen and zero-width characters removed; no-break space → space) is applied once when the words are built.

Managed engines

Install the extra and set the credentials (they are read from the environment and never logged). extractit engines list shows enabled = false until they are present; a missing engine is skipped for RapidOCR with the warning ocr_engine_disabled.

Engine id Extra Credentials Notes
azure_di extractit[azure] DX_AZURE_DI_ENDPOINT, DX_AZURE_DI_KEY; optional DX_AZURE_DI_MODEL (prebuilt-read default, prebuilt-layout adds tables and paragraph roles) polygons in pixels ÷ page size from the response
textract extractit[aws] the boto3 credential chain and AWS_REGION (or AWS_DEFAULT_REGION); optional DX_TEXTRACT_FEATURES=layout,tables for AnalyzeDocument BoundingBox already normalized; confidence 0–100 ÷ 100; MERGED_CELL carries the real span
google_docai extractit[gcp] GOOGLE_APPLICATION_CREDENTIALS, DX_DOCAI_PROCESSOR (projects/…/locations/…/processors/…) Enterprise Document OCR; normalized_vertices, token confidence
mistral_ocr extractit[mistral] MISTRAL_API_KEY text only: boxes come from RapidOCR through align_text_to_words (source = 'vlm_aligned')

engines.toml records, for each engine, the region, the data retention and the price per 1 000 pages with its date and source (list prices collected on 2026-11-09 in the research notes; re-verify them on the day an engine is announced). OcrPageResult.cost_usd uses them.

Failures: a 429 or 5xx is retried up to 3 times with exponential backoff (1, 2, 4 s) and the service's Retry-After when it sends one (capped at 30 s). If the engine keeps failing, the page is read by RapidOCR and the page gets the warning ocr_engine_fallback with fallback_from=<engine>: <error>. (DocIR has no fallback_from field: it lives in that warning.) Images over the service limit (10 MB) are re-encoded at a lower resolution; boxes are normalized, so nothing moves. Boxes that fall outside [0, 1] by rounding are clipped and counted (ocr_coordinates_clipped).

Nothing sensitive is logged: not the keys, the endpoint, the image, the response body or the document text. SDK loggers are raised to WARNING during a call (botocore and azure-core print bodies and headers at DEBUG).

Validation with real accounts: pending

There are no Azure, AWS, Google or Mistral accounts behind this project yet. The adapters are complete (_call() with the SDK, _convert() as a pure function) but the responses in tests/cassettes/ocr/ were written by hand from each service's public documentation, and the SDK calls are exercised with fake clients. Before an engine is announced:

  1. capture one real response per engine with a call marked external and replace (or add next to) the cassette;
  2. check the numeric cases of tests/ocr/test_ocr_managed.py against it (polygon → box, BoundingBox, confidence scale, spans);
  3. run extractit engines test <id> --yes and a scanned fixture with --ocr-engine <id>.

Settings

Variable Effect
DX_MODELS_DIR where the models live (<dir>/ocr for RapidOCR); default ~/.extractit/models
DX_OCR_WORKERS pages read in parallel (default: cores − 1)
DX_OCR_THREADS ONNX Runtime threads per page (default OMP_NUM_THREADS, else min(4, cores))

The parse cache key includes the OCR mode, engine, languages, the RapidOCR/ONNX versions, the model checksums and (for a managed engine) the SDK version.

Warnings

Code Meaning
text_layer_unreliable the text layer was rejected (reason in the message); the page was read by OCR
needs_ocr a page has no text and OCR did not run (--ocr off, or no engine)
blank_page no visible content: 0 words, no engine call
ocr_model_switch the first model read badly; the second contributed (latin -> cyrillic)
script_unsupported_ocr, needs_vlm no model reads this script: the page was not read
mixed_scripts two scripts on the page
ocr_engine_fallback, ocr_engine_disabled, ocr_failed engine problems, with the fallback used
ocr_low_dpi_upscaled, ocr_coordinates_clipped, ocr_empty, invalid_language_hint see above

Limits

OCR of photos, handwriting, stamps and signatures is out of scope (T20, T21); right-to-left reading order and Indic scripts are not planned; tables of scanned pages (cell structure, borderless, multi-page) come with T19. The fixtures are synthetic: the report in bench/reports/ocr-fixtures.md is a floor on clean input, not a promise on real scans.