Skip to content

Supported formats and conversion to PDF

extractit.convert turns every supported input into a citable PDF. It is a pure layer: it returns dataclasses and knows nothing about packages, files or the store (T17 maps them).

from extractit.convert import build_converter, sniff, convert_cache_key

fmt = sniff(data, "invoice.bin")  # DetectedFormat(kind="docx", convertible=True, ...)
result = build_converter().convert(data, "invoice.bin")
for part in result.parts:  # ConvertedPart(name, pdf, media_type, derivation, parent, warnings)
    ...
key = convert_cache_key(sha256_hex, result.converter_id, result.converter_version)

The real type comes from the content (magic bytes, ZIP member names, OLE2 streams), never from the extension; the extension only chooses inside a family (.tsv vs .csv, .md vs .txt, .key vs .pages). An executable renamed .docx, or a ZIP named .pdf without Office structure, raises UnsupportedFormat.

Format by result

Input Detected by Route derivation Notes
pdf %PDF- none passthrough Returned unchanged after pypdf checks it (not encrypted, ≤ DX_CONVERT_MAX_PAGES).
docx docm dotx doc dot rtf odt ott ZIP names / OLE2 WordDocument / {\rtf Gotenberg → LibreOffice office_to_pdf Macros are never executed.
pptx pptm ppt odp ZIP names / OLE2 PowerPoint Document Gotenberg → LibreOffice office_to_pdf Speaker notes and hidden slides are not exported. Complex decks: best effort.
pages key ZIP Index/, Index.zip or 2009 index.xml Gotenberg → LibreOffice (libetonyek) office_to_pdf Best effort: fidelity depends on the pinned LibreOffice; failures are ConversionFailed.
xlsx xlsm xls ods numbers ZIP names / OLE2 Workbook Gotenberg → LibreOffice, landscape office_to_pdf Only visible sheets are printed. numbers is best effort. A very wide sheet may split across pages.
csv tsv content (delimiter, consistent columns) Chromium text_to_pdf Table with the header repeated on every page; ≤ 10 000 rows and 200 columns (sheet_truncated). Landscape above 8 columns.
txt md content / extension Chromium text_to_pdf Monospaced, pre-wrap; Markdown is shown as text.
json xml content Chromium text_to_pdf Re-indented. XML is parsed with defusedxml; with entities or a DTD it is shown as is (xml_not_reformatted), never expanded.
html <!doctype html>, <html>… Chromium after nh3 html_to_pdf See "Safety".
png jpg webp bmp gif tif tiff filetype Pillow, no service image_to_pdf One page per TIFF frame (≤ 500). EXIF orientation applied; alpha over white; CMYK to RGB. Animated GIF/WebP keep the first frame (animation_first_frame_only).
eml msg headers / OLE2 __properties_version1.0 Chromium (body) + recursive email_body + email_attachment Header block (From, To, Cc, Date, Subject), then the body; attachments are child parts.

HEIC, .pst and .ost are out of scope (LGPL/GPL components). Audio and video are not transcribed.

E-mail

  • The body prefers HTML (nh3-sanitized) over plain text. cid: images of 5 KiB or more are embedded as data: URIs; smaller ones (tracking pixels, signature bullets) are dropped (inline_images_dropped). Images embedded in the body are not repeated as attachments.
  • Part order is stable: the body first, then attachments depth-first in message order. Names: mail.pdf, mail/01-report.xlsx.pdf, mail/02-photo.png.pdf… Each attachment part has parent = the part of the message that carried it; nested messages hang from their parent message.
  • Only the root body is email_body; everything below is email_attachment.
  • Limits: depth 3 (an e-mail inside an e-mail inside an e-mail converts; a fourth level raises UnsupportedFormat(reason="max-depth-exceeded")), 50 attachments and 100 MiB in total (ArchiveTooLarge).
  • An attachment that cannot be converted (an .exe, a ZIP, a corrupt file) does not fail the message: it is skipped with an attachment_skipped warning on the body part. Limits, bombs and an unavailable converter do abort.
  • .msg is read with a small reader over olefile (subject, sender, To/Cc display names, date, body, HTML, attachments and embedded messages). extract-msg is GPL-3.0 and is not used.

Safety

  • Macros are not executed: LibreOffice runs headless with the default (disabled) macro security. The test suite converts a document with an auto-open macro that writes a marker file and checks that the file does not exist.
  • No network: the service sits on an internal Docker network with no route out; Chromium has JavaScript off and denies every http(s) URL, every local file but /tmp and public/private IPs. HTML and e-mail bodies are also sanitized before they reach Chromium: nh3 removes script, style, iframe, object, forms, inline style attributes, remote src/href, and a Content-Security-Policy that only allows data: images is added. A local test server counts 0 requests.
  • Bombs: ZIP-based formats are checked before anything else (> 500 MiB expanded or ratio > 100: ArchiveTooLarge); images above 100 million pixels are rejected before decoding (image-too-large); TIFFs above 500 frames raise UnsupportedFormat(reason="too-many-frames").
  • The produced PDF is opened with pypdf; more than DX_CONVERT_MAX_PAGES pages is UnsupportedFormat(reason="too-many-pages").

Configuration

Variable Default Meaning
DX_GOTENBERG_URL http://localhost:3000 Gotenberg base URL (the development Compose stack publishes 53000; deploy/compose/gotenberg publishes 53016).
DX_CONVERT_TIMEOUT_S 120 Deadline of one Gotenberg call, including its retries.
DX_CONVERT_MAX_PAGES 2000 Maximum pages of a produced PDF.
DX_CONVERT_CONCURRENCY 4 Conversions in flight; the next one waits for a free slot.

ConvertOptions carries the per-call limits (rows, columns, frames, pixels, depth, attachments); they are part of the cache key.

Errors

Class code retryable When
UnsupportedFormat unsupported_format no Not convertible (disguises, unknown binary), too many frames/pages, depth exceeded, input too large. reason says why.
ArchiveTooLarge archive_too_large no Decompression bomb, too many or too large attachments.
ConversionFailed conversion_failed no Gotenberg rejected the document (corrupt file) or returned something that is not a PDF.
ConversionTimeout conversion_timeout yes The deadline passed (or Gotenberg answered 504).
ConverterUnavailable converter_unavailable yes Gotenberg unreachable, or still 503 after 2 retries with backoff. Images and PDF keep working.

Warnings

sheet_truncated (rows_dropped, columns_dropped), xml_not_reformatted (reason), csv_not_tabulated (a cell above the CSV field limit: shown as plain text), animation_first_frame_only (frames), inline_images_dropped (count), attachment_skipped (filename, reason).

Cache key

convert_cache_key(sha256, converter_id, converter_version, opts) is sha256(len‖sha256_original ‖ len‖converter_id ‖ len‖converter_version ‖ len‖options). The same input gives the same key; another converter_version (bumped when the output changes, in particular when the pinned Gotenberg digest changes) or other options give another one. T17 stores the derivatives under it.

Running the service

docker compose -f deploy/compose/gotenberg/docker-compose.yml -p dx-t16 up -d --wait
export DX_GOTENBERG_URL=http://127.0.0.1:53016
uv run pytest tests/convert -m requires_gotenberg   # real conversions of the corpus
docker compose -f deploy/compose/gotenberg/docker-compose.yml -p dx-t16 down -v

deploy/compose/gotenberg/run-integration.sh does the three steps and always tears down. The corpus (tests/fixtures/convert/corpus/, under 2 MiB) is rebuilt with tests/fixtures/convert/build_corpus.py [--libreoffice].

Results on the test corpus

Generated by pytest -m requires_gotenberg (DX_T16_REPORT=report.md) against the pinned Gotenberg image (LibreOffice 26.8) on a development laptop; times are indicative. Every PDF contains its DX-SENTINEL-<id> string (extracted with pypdf); raster images have no text layer, so their row checks the page count instead (3 TIFF frames, 1 WebP). key has no row: no free Keynote file could be built (hand-made 2009 XML packages convert without text in LibreOffice 26.8), so Keynote is covered by detection tests only.

File Derivation Parts PDF pages Time (ms)
csv.csv text_to_pdf 1 1 2539
doc.doc office_to_pdf 1 1 1530
docx.docx office_to_pdf 1 1 160
eml.eml email_body 3 3 501
html.html html_to_pdf 1 1 196
json.json text_to_pdf 1 1 116
md.md text_to_pdf 1 1 184
msg.msg email_body 3 3 231
numbers.numbers office_to_pdf 1 1 123
odp.odp office_to_pdf 1 1 116
ods.ods office_to_pdf 1 1 104
odt.odt office_to_pdf 1 1 77
pages.pages office_to_pdf 1 1 62
pdf.pdf passthrough 1 1 0
ppt.ppt office_to_pdf 1 1 84
pptx.pptx office_to_pdf 1 1 336
rtf.rtf office_to_pdf 1 1 89
tiff.tiff image_to_pdf 1 3 0
tsv.tsv text_to_pdf 1 1 181
txt.txt text_to_pdf 1 1 93
webp.webp image_to_pdf 1 1 0
xls.xls office_to_pdf 1 1 102
xlsx.xlsx office_to_pdf 1 2 102
xml.xml text_to_pdf 1 1 193

Pinned behaviour worth knowing: an .xlsx with 3 sheets (one hidden) gives 2 landscape pages; for xls, ods and numbers Gotenberg's landscape field only reaches the first sheet's page style, so later sheets of those formats may stay portrait.