Writing a plugin
A plugin adds a node to workflows or a parser engine (OCR) to the parse path. It is an ordinary Python distribution with a manifest and an entry point. ExtractIt never imports plugin code in its own process: plugins run in an isolated child process with the limits they declare.
Two complete examples live in examples/plugins/:
dx_node_redact— node,Extraction -> Extraction: replaces the configured fields with***.dx_engine_tesseract— parser engine: runs thetesseractbinary and returns words with boxes.
Layout
my_plugin/
pyproject.toml
src/my_plugin/
__init__.py
plugin.py
dx_plugin.json # the manifest, shipped inside the package
[project.entry-points."extractit.node_plugins"] # or "extractit.parser_engines"
my_plugin = "my_plugin.plugin:MyPlugin" # named exactly like the manifest id
Only these two groups are read.
The manifest (dx_plugin.json)
{
"id": "dx_node_redact",
"version": "1.0.0",
"kind": "node",
"dx_plugin_api": ">=1.0,<2",
"ports": {
"inputs": [{"name": "extraction", "type": "Extraction"}],
"outputs": [{"name": "extraction", "type": "Extraction"}]
},
"config_schema": {"type": "object", "properties": {"field_ids": {"type": "array"}}},
"requires": {"network": "none", "secrets": [], "max_cpu_s": 10, "max_mem_mb": 256, "max_output_kb": 4096},
"license": "Apache-2.0"
}
| Field | Rule |
|---|---|
id |
^[a-z0-9][a-z0-9_.-]{2,63}$ |
version |
semver |
kind |
node or parser_engine |
dx_plugin_api |
PEP 440 range that must admit the host's plugin API (1.0) |
ports |
typed ports of T25: ParsedDoc, Segment, Extraction, Verdicts, File (a node needs at least one) |
config_schema |
JSON Schema 2020-12 of the node's config.config (an object) |
requires.network |
"none" or the list of hosts it connects to |
requires.secrets |
names of the secrets it may receive (only approved ones are passed) |
requires.max_cpu_s / max_mem_mb / max_output_kb |
enforced by the guardian (defaults 30 s, 512 MiB, 1 MiB) |
engine_capabilities |
required for parser_engine: words, lines, tables, languages, local |
license |
SPDX; a copyleft license is reported as an informational notice |
The JSON Schema of the manifest is schemas/plugins/manifest.schema.json.
The code
class MyNode:
def run(self, ctx, inputs: dict, config: dict) -> dict:
# inputs/outputs are keyed by port name
return {"extraction": do_something(inputs["extraction"], config)}
class MyEngine:
def parse_page(self, image: Path, page_meta: dict, options: dict) -> dict:
return {
"words": [{"text": "TOTAL", "bbox": [0.1, 0.2, 0.3, 0.25], "conf": 0.97, "line": 0}],
"lines": [{"id": 0, "text": "TOTAL", "bbox": [0.1, 0.2, 0.3, 0.25], "conf": 0.97}],
}
- An
Extractionvalue is theoutput-1.0object of T02 and is validated against it. - A
ParsedDocinput arrives as the path (relative toctx.workdir) of the DocIR JSON. - A
Fileoutput is{"name", "media_type", "path"}of a file you wrote underctx.workdir. - Engine boxes are normalized to the page image (origin top-left) with
0 <= x0 < x1 <= 1,0 <= y0 < y1 <= 1andconfin[0, 1]; anything else is aplugin_contract_violationwith the JSON pointers of the offending words.
What the plugin process can do
The child is python -I -m extractit.plugins.worker <entry point>:
- empty environment (only
PATH, a scratchHOME, and the secrets you declared and an administrator approved) — never database credentials or store access; - a fresh scratch directory per call (data crosses as files there);
- the guardian kills it past
max_mem_mb(plugin_resource_exceeded),max_cpu_s(plugin_timeout) ormax_output_kb(plugin_output_too_large); - with
network: "none", no network at all: an empty network namespace on Linux (unshare --user --map-root-user --net), asandbox-execdeny-network profile on macOS. Where neither exists the server profile refuses to run it (PLUGIN_ISOLATION_UNAVAILABLE) and the local profile runs it with a warning.
Process isolation is weaker than gVisor or a microVM; that is why the server profile only runs plugins whose distribution sha256 is explicitly allowed (below).
Check your plugin
$ extractit plugins validate path/to/my_plugin # manifest + entry point, every violation in one JSON
$ extractit plugins validate path/to/my_plugin --suite # + the contract suite
$ extractit plugins validate my-plugin-distribution # an installed distribution
validate exits with 1 when there is any error. The contract suite
(extractit.plugins.testing.contract_suite) runs the plugin in its child process and checks:
manifest, configuration against config_schema (the schema's first examples entry is the default
fixture), outputs against their port types, determinism (same fixture twice, same output),
limits, and that it uses no undeclared network (an audit hook records socket.connect and
getaddrinfo).
Installing, enabling, running
$ pip install dx_node_redact-1.0.0-py3-none-any.whl
$ extractit plugins list --json # id, version, kind, sha256 (of the distribution's RECORD)
$ extractit plugins inspect dx_node_redact
$ extractit plugins enable dx_node_redact
On a server, GET /v1/plugins, GET /v1/plugins/{id} and POST /v1/plugins/{id}:enable|:disable
manage the state per project (plugin_installs). Settings:
| Setting | Meaning |
|---|---|
DX_PLUGINS_MODE (plugins.mode) |
server (default): only plugins whose sha256 is in the allow list; local: trust what is installed |
DX_PLUGINS_ALLOWLIST (plugins.allowlist) |
comma-separated sha256 of approved distributions |
DX_ENGINES_ALLOW (engines.allow) |
parser-engine plugins the OCR stage may select |
Enabling records the exact version and sha256; a run checks them again, and that the installed
files still match the distribution's RECORD (plugin_not_allowlisted otherwise).
A workflow node uses a plugin by id and an exact version or a range:
{"id": "redact", "type": "plugin",
"config": {"plugin_id": "dx_node_redact", "version": ">=1,<2", "config": {"field_ids": ["supplier_iban"]}}}
Validation reports, all in one TOPOLOGY_INVALID response: PLUGIN_NOT_AVAILABLE (not installed
or not enabled in the project), PLUGIN_VERSION_INCOMPATIBLE, PORT_TYPE_MISMATCH (an edge brings
nothing the plugin accepts, or the node declares other ports than the manifest) and
PLUGIN_CONFIG_INVALID. Every run records the plugin's id, version and sha256 in
reproducibility.plugins and emits a plugin.run span.
A parser engine listed in engines.allow is registered in the OCR engine registry of T05: select it
with ParseProfile(ocr_engine="dx_engine_tesseract"); its version and sha256 enter
engine_versions (and the parse cache key).