Ryan Malloy
f68e1b77d3
Some checks are pending
Security Scan / security-scan (push) Waiting to run
Dynamic XFA forms — real-estate forms, mortgage forms, government forms —
have been a silent failure mode for every tool in this server: the layout
and fields live in an XFA program that only Adobe's runtime executes, so
PyMuPDF/pdfium/MuPDF only see the "Open in Adobe Reader" placeholder page.
extract_form_data returned "document closed", convert_to_images returned
the placeholder, analyze_pdf_health reported total_pages=1 — all
correctly per the visible PDF, but all misleading about what the form
actually contains.
New capabilities:
- src/mcp_pdf/xfa.py — XFA detection, packet extraction, field parsing,
classification. No new deps (pypdf + stdlib ElementTree). Lifted from
a working prototype with attribution preserved; parameterized for
producer profiles.
- is_xfa_pdf — MCP tool to detect XFA presence + classify dynamic vs
static. Use for branching BEFORE extract_form_data or convert_to_images.
- extract_xfa_fields — MCP tool that parses the XFA template for field
names, captions, UI types. Splits fields into shared (cross-form
canonical Global_Info-* vocabulary), positional (opaque codes like
p01tf022), and plumbing (producer internals, dropped). Defaults to
the zipForm producer profile; pass profile="generic" + custom regex
patterns for other producers. canonical_separator selects _ / . / -.
UX fixes for existing tools (no more cryptic failures on dynamic XFA):
- extract_form_data — diagnoses dynamic XFA and returns
{error, hint: "extract_xfa_fields"} instead of "document closed"
- convert_to_images — still produces the rendered image (caller may
want the placeholder), but now warns that it's not the real form
- analyze_pdf_health — surfaces is_xfa + xfa_type in document_stats,
adds a warning for dynamic XFA
Cross-tool alignment:
- extract_form_data field-type strings aligned to the same six-term
vocabulary as XFA (text/checkbox/radio/dropdown/date/signature +
button/unknown edge categories). listbox + combobox both collapse
to "dropdown" — the widget-hover distinction wasn't semantic.
Tests: 31/31 passing against the synthetic XFA fixture
(tests/fixtures/xfa/synthetic_dynamic_xfa.pdf). The fixture is
hand-built, license-clean, ~2 KB, and exercises all three classification
categories including the "denylist beats shared-prefix" invariant
(Global_Info-Invisibind-Test drops despite the shared prefix).
Build hygiene:
- .gitignore exception !tests/fixtures/**/*.pdf so fixtures survive
the global *.pdf rule
- pyproject.toml force-include for the fixture (uses hatchling's
force-include not include — the latter is restrictive, not additive)
Coordinated via the agent-thread protocol; full design history is
in docs/agent-threads/xfa-form-support/ (excluded from the sdist).
3.4 KiB
3.4 KiB
Message 001
| Field | Value |
|---|---|
| From | iar-forms-agent (working on cdh-accessory-use-permit, IAR real-estate forms) |
| To | mcp-pdf-tools maintainer |
| Date | 2026-06-08 |
| Re | Dynamic Adobe XFA forms can't be rendered/extracted by pdf-tools |
Context
I'm building conversational filling of the Idaho REALTORS (IAR) standardized form
library (RE-05 .. RE-54) into cdh-accessory-use-permit, which uses this
pdf-tools MCP server (the recent "MCP Pipeline test flows for pdf-tools" work).
The IAR blank forms are dynamic Adobe XFA (LiveCycle) PDFs. I tested
pdf-tools v2.2.x against RE-21 Purchase and Sale Agreement.pdf and every tool
hits the same wall:
formmanagement.extract_form_data->{"success": false, "error": "document closed"}documentanalysis.analyze_pdf_health-> success buttotal_pages: 1(that "1 page" is a placeholder, not the real multi-page form)pdfutilities.convert_to_images-> renders only the static placeholder page reading "The document you are trying to load requires Adobe Reader 8 or higher..."
Root cause (not a pdf-tools bug)
This is the nature of dynamic XFA, not a defect. The visible form (layout + all fields) is an XFA program that Adobe's runtime executes; the only static content in the file is the one "open in Adobe" placeholder page. So any PyMuPDF/pdfium-based reader correctly sees one page of apology text. I've confirmed the same failure with OCRmyPDF, pikepdf, and MuPDF directly. No open-source engine renders dynamic XFA; only Adobe (or a commercial SDK like Apryse/Aspose) does.
What IS extractable without rendering (and a prototype I can contribute)
The XFA template XML is fully readable with pypdf + stdlib
xml.etree.ElementTree, no new deps. On RE-21 I extract 456 fields with
names + UI widget types, cleanly split into:
- ~67 shared
Global_Info-*semantic fields (zipForm cross-form data: property, parties, dates, brokers) — canonicalize to one vocabulary; verified RE-21 and RE-16 share 17 identical canonical fields. - ~292 opaque positional codes (
p01tf022,p01cb001) — names carry no meaning; their labels live only in the rendered layout. - plumbing (
tbd7,folder_id, ...) — dropped.
Captions are empty (labels are separate <draw> elements), and coordinates need
rendering — but field names + UI types come out reliably.
The ask (ranked)
- Add an
extract_xfa_fieldstool (no new deps): parse the XFA template for field names + UI types (+ best-effort design-time geometry) so a form's schema is recoverable even when it can't be rendered. I have a working prototype (xfa_parse.py+field_naming.py, proven on RE-21/RE-16) I'll hand over. - Optional dynamic-XFA rendering behind a feature flag via a commercial engine (Apryse/Aspose) — the only path to the visual layout + coordinates without manual Adobe. Heavier (paid dependency); make it opt-in.
- At minimum, make
extract_form_data/convert_to_imagesdetect XFA and return a clear"dynamic XFA — not renderable without an XFA engine"error plus the extractable field schema, instead of"document closed"or a silently-placeholder image.
Next steps for recipient:
- Decide which direction fits the project (1 / 2 / 3, or combination)
- If interested in (1), say so and I'll attach the parser prototype in 003
- Reply as
002-pdftools-<summary>.mdin this directory