Supported formats and conversion to PDF
extractit.convert turns every supported input into a citable PDF. It is a pure layer: it
returns dataclasses and knows nothing about packages, files or the store (T17 maps them).
from extractit.convert import build_converter, sniff, convert_cache_key
fmt = sniff(data, "invoice.bin") # DetectedFormat(kind="docx", convertible=True, ...)
result = build_converter().convert(data, "invoice.bin")
for part in result.parts: # ConvertedPart(name, pdf, media_type, derivation, parent, warnings)
...
key = convert_cache_key(sha256_hex, result.converter_id, result.converter_version)
The real type comes from the content (magic bytes, ZIP member names, OLE2 streams), never from
the extension; the extension only chooses inside a family (.tsv vs .csv, .md vs .txt,
.key vs .pages). An executable renamed .docx, or a ZIP named .pdf without Office structure,
raises UnsupportedFormat.
Format by result
| Input | Detected by | Route | derivation |
Notes |
|---|---|---|---|---|
pdf |
%PDF- |
none | passthrough |
Returned unchanged after pypdf checks it (not encrypted, ≤ DX_CONVERT_MAX_PAGES). |
docx docm dotx doc dot rtf odt ott |
ZIP names / OLE2 WordDocument / {\rtf |
Gotenberg → LibreOffice | office_to_pdf |
Macros are never executed. |
pptx pptm ppt odp |
ZIP names / OLE2 PowerPoint Document |
Gotenberg → LibreOffice | office_to_pdf |
Speaker notes and hidden slides are not exported. Complex decks: best effort. |
pages key |
ZIP Index/, Index.zip or 2009 index.xml |
Gotenberg → LibreOffice (libetonyek) | office_to_pdf |
Best effort: fidelity depends on the pinned LibreOffice; failures are ConversionFailed. |
xlsx xlsm xls ods numbers |
ZIP names / OLE2 Workbook |
Gotenberg → LibreOffice, landscape | office_to_pdf |
Only visible sheets are printed. numbers is best effort. A very wide sheet may split across pages. |
csv tsv |
content (delimiter, consistent columns) | Chromium | text_to_pdf |
Table with the header repeated on every page; ≤ 10 000 rows and 200 columns (sheet_truncated). Landscape above 8 columns. |
txt md |
content / extension | Chromium | text_to_pdf |
Monospaced, pre-wrap; Markdown is shown as text. |
json xml |
content | Chromium | text_to_pdf |
Re-indented. XML is parsed with defusedxml; with entities or a DTD it is shown as is (xml_not_reformatted), never expanded. |
html |
<!doctype html>, <html>… |
Chromium after nh3 |
html_to_pdf |
See "Safety". |
png jpg webp bmp gif tif tiff |
filetype |
Pillow, no service | image_to_pdf |
One page per TIFF frame (≤ 500). EXIF orientation applied; alpha over white; CMYK to RGB. Animated GIF/WebP keep the first frame (animation_first_frame_only). |
eml msg |
headers / OLE2 __properties_version1.0 |
Chromium (body) + recursive | email_body + email_attachment |
Header block (From, To, Cc, Date, Subject), then the body; attachments are child parts. |
HEIC, .pst and .ost are out of scope (LGPL/GPL components). Audio and video are not transcribed.
- The body prefers HTML (
nh3-sanitized) over plain text.cid:images of 5 KiB or more are embedded asdata:URIs; smaller ones (tracking pixels, signature bullets) are dropped (inline_images_dropped). Images embedded in the body are not repeated as attachments. - Part order is stable: the body first, then attachments depth-first in message order. Names:
mail.pdf,mail/01-report.xlsx.pdf,mail/02-photo.png.pdf… Each attachment part hasparent= the part of the message that carried it; nested messages hang from their parent message. - Only the root body is
email_body; everything below isemail_attachment. - Limits: depth 3 (an e-mail inside an e-mail inside an e-mail converts; a fourth level raises
UnsupportedFormat(reason="max-depth-exceeded")), 50 attachments and 100 MiB in total (ArchiveTooLarge). - An attachment that cannot be converted (an
.exe, a ZIP, a corrupt file) does not fail the message: it is skipped with anattachment_skippedwarning on the body part. Limits, bombs and an unavailable converter do abort. .msgis read with a small reader overolefile(subject, sender, To/Cc display names, date, body, HTML, attachments and embedded messages).extract-msgis GPL-3.0 and is not used.
Safety
- Macros are not executed: LibreOffice runs headless with the default (disabled) macro security. The test suite converts a document with an auto-open macro that writes a marker file and checks that the file does not exist.
- No network: the service sits on an internal Docker network with no route out; Chromium has
JavaScript off and denies every
http(s)URL, every local file but/tmpand public/private IPs. HTML and e-mail bodies are also sanitized before they reach Chromium:nh3removesscript,style,iframe,object, forms, inlinestyleattributes, remotesrc/href, and aContent-Security-Policythat only allowsdata:images is added. A local test server counts 0 requests. - Bombs: ZIP-based formats are checked before anything else (> 500 MiB expanded or ratio > 100:
ArchiveTooLarge); images above 100 million pixels are rejected before decoding (image-too-large); TIFFs above 500 frames raiseUnsupportedFormat(reason="too-many-frames"). - The produced PDF is opened with
pypdf; more thanDX_CONVERT_MAX_PAGESpages isUnsupportedFormat(reason="too-many-pages").
Configuration
| Variable | Default | Meaning |
|---|---|---|
DX_GOTENBERG_URL |
http://localhost:3000 |
Gotenberg base URL (the development Compose stack publishes 53000; deploy/compose/gotenberg publishes 53016). |
DX_CONVERT_TIMEOUT_S |
120 |
Deadline of one Gotenberg call, including its retries. |
DX_CONVERT_MAX_PAGES |
2000 |
Maximum pages of a produced PDF. |
DX_CONVERT_CONCURRENCY |
4 |
Conversions in flight; the next one waits for a free slot. |
ConvertOptions carries the per-call limits (rows, columns, frames, pixels, depth, attachments);
they are part of the cache key.
Errors
| Class | code |
retryable |
When |
|---|---|---|---|
UnsupportedFormat |
unsupported_format |
no | Not convertible (disguises, unknown binary), too many frames/pages, depth exceeded, input too large. reason says why. |
ArchiveTooLarge |
archive_too_large |
no | Decompression bomb, too many or too large attachments. |
ConversionFailed |
conversion_failed |
no | Gotenberg rejected the document (corrupt file) or returned something that is not a PDF. |
ConversionTimeout |
conversion_timeout |
yes | The deadline passed (or Gotenberg answered 504). |
ConverterUnavailable |
converter_unavailable |
yes | Gotenberg unreachable, or still 503 after 2 retries with backoff. Images and PDF keep working. |
Warnings
sheet_truncated (rows_dropped, columns_dropped), xml_not_reformatted (reason),
csv_not_tabulated (a cell above the CSV field limit: shown as plain text),
animation_first_frame_only (frames), inline_images_dropped (count), attachment_skipped
(filename, reason).
Cache key
convert_cache_key(sha256, converter_id, converter_version, opts) is
sha256(len‖sha256_original ‖ len‖converter_id ‖ len‖converter_version ‖ len‖options). The same
input gives the same key; another converter_version (bumped when the output changes, in
particular when the pinned Gotenberg digest changes) or other options give another one. T17 stores
the derivatives under it.
Running the service
docker compose -f deploy/compose/gotenberg/docker-compose.yml -p dx-t16 up -d --wait
export DX_GOTENBERG_URL=http://127.0.0.1:53016
uv run pytest tests/convert -m requires_gotenberg # real conversions of the corpus
docker compose -f deploy/compose/gotenberg/docker-compose.yml -p dx-t16 down -v
deploy/compose/gotenberg/run-integration.sh does the three steps and always tears down. The
corpus (tests/fixtures/convert/corpus/, under 2 MiB) is rebuilt with
tests/fixtures/convert/build_corpus.py [--libreoffice].
Results on the test corpus
Generated by pytest -m requires_gotenberg (DX_T16_REPORT=report.md) against the pinned Gotenberg
image (LibreOffice 26.8) on a development laptop; times are indicative. Every PDF contains its
DX-SENTINEL-<id> string (extracted with pypdf); raster images have no text layer, so their row
checks the page count instead (3 TIFF frames, 1 WebP). key has no row: no free Keynote file could
be built (hand-made 2009 XML packages convert without text in LibreOffice 26.8), so Keynote is
covered by detection tests only.
| File | Derivation | Parts | PDF pages | Time (ms) |
|---|---|---|---|---|
csv.csv |
text_to_pdf | 1 | 1 | 2539 |
doc.doc |
office_to_pdf | 1 | 1 | 1530 |
docx.docx |
office_to_pdf | 1 | 1 | 160 |
eml.eml |
email_body | 3 | 3 | 501 |
html.html |
html_to_pdf | 1 | 1 | 196 |
json.json |
text_to_pdf | 1 | 1 | 116 |
md.md |
text_to_pdf | 1 | 1 | 184 |
msg.msg |
email_body | 3 | 3 | 231 |
numbers.numbers |
office_to_pdf | 1 | 1 | 123 |
odp.odp |
office_to_pdf | 1 | 1 | 116 |
ods.ods |
office_to_pdf | 1 | 1 | 104 |
odt.odt |
office_to_pdf | 1 | 1 | 77 |
pages.pages |
office_to_pdf | 1 | 1 | 62 |
pdf.pdf |
passthrough | 1 | 1 | 0 |
ppt.ppt |
office_to_pdf | 1 | 1 | 84 |
pptx.pptx |
office_to_pdf | 1 | 1 | 336 |
rtf.rtf |
office_to_pdf | 1 | 1 | 89 |
tiff.tiff |
image_to_pdf | 1 | 3 | 0 |
tsv.tsv |
text_to_pdf | 1 | 1 | 181 |
txt.txt |
text_to_pdf | 1 | 1 | 93 |
webp.webp |
image_to_pdf | 1 | 1 | 0 |
xls.xls |
office_to_pdf | 1 | 1 | 102 |
xlsx.xlsx |
office_to_pdf | 1 | 2 | 102 |
xml.xml |
text_to_pdf | 1 | 1 | 193 |
Pinned behaviour worth knowing: an .xlsx with 3 sheets (one hidden) gives 2 landscape pages;
for xls, ods and numbers Gotenberg's landscape field only reaches the first sheet's page
style, so later sheets of those formats may stay portrait.