nigig-org/REVIEWS/adr/0039-pdf-xmp-and-conformance-decisions.md

ADR 0039: XMP metadata handling and the conformance/OCR decisions

• Status: Accepted
• Date: 2026-09-05
• Review item: NIGIG_PDF_FEATURE_PARITY_PLAN.md Phase 9, "PDF/A validation helpers", "PDF/UA accessibility checks", "XMP metadata read/write", "OCR layer"
• Supersedes: nothing
• Related: ADR 0011 (the PDF/UA structure tree, which the PDF/UA bullet of this phase is built on), ADR 0021 (the process note that drove the table format)

Context

Phase 9 has four bullets. Surveying the existing code before changing anything:

1. PDF/UA accessibility checks. Already done. ADR 0011 covers the structure tree (/StructTreeRoot, /K, /IDTree, /ParentTree, /RoleMap), custom-tag resolution, and typed AccessibilityIssues; pdf-document/tests/structure.rs is the acceptance suite. The module deliberately carries no is_pdf_ua verdict, and a test asserts that absence. Nothing to build.
2. XMP metadata read/write. Genuinely missing on the read side. The writer (pdf-cos::writer) accepts raw XMP bytes (set_xmp_metadata) but nothing parses a /Metadata stream back into fields, and there is no XMP packet generator.
3. PDF/A validation helpers. Missing. A PDF/A check is a certification claim, and this project has a standing rule (ADR 0011, ADR 0017) against mechanical checks wearing a certification hat.
4. OCR layer. Missing, and the plan itself flags it "decision required". dart-pdf binds an external engine; a Rust equivalent is a new heavyweight dependency.

Decision

XMP: read the packet, bridge to DocumentInfo, generate a packet

pdf-document::xmp (new module, no XML dependency):

• read_xmp locates the catalogue /Metadata stream (trailer /Root → catalog /Metadata → resolve_stream, so /Filter-decoded bytes arrive already decompressed) and parses it.
• parse_packet scavenges the fields a metadata packet actually carries: dc:title and dc:description as rdf:Alt containers (the x-default literal wins), dc:creator as an ordered rdf:Seq, dc:subject as a rdf:Bag, and the plain-element forms pdf:Producer, xmp:CreatorTool, xmp:CreateDate, xmp:ModifyDate.
• to_document_info merges onto the existing DocumentInfo (/Info), XMP winning for the fields /Info does not carry (creators list, keyword list).
• to_packet emits a canonical packet (with the xpacket PI wrapper and the x:xmpmeta envelope) so writer workflows have a real producer for set_xmp_metadata.

The parser is a targeted scavenger, not a general XML parser — it is tolerant (a foreign or malformed packet yields None fields, never an error opening the document) and it does not validate RDF. That is a deliberate line: full XMP/RDF parsing is a spec-sized project, and the fields above are what viewers and workflows actually read.

PDF/UA: mark done via ADR 0011, nothing new built

The bullet's substance — struct-tree completeness, tag map, alt text — is ADR 0011's deliverable. The plan now records it as done rather than re-deriving it.

PDF/A helpers: defer, do not hand-write

PDF/A conformance is a certification. A hand-rolled "is this PDF/A-1b?" check risks becoming a boolean downstream trust adopts while checking only the entries someone remembered. Doing it properly is a spec-grounded pass over required entries and value ranges, plus a fixture corpus. Not this pass's work; recorded as deferred with the reason.

OCR: defer with a recorded decision

nigig will not bind an OCR engine now. Reasons:

• It is a new heavyweight dependency (a Rust engine or a Tesseract binding) bolted onto a second pipeline, with a different failure model than extraction.
• The text-first surfaces already delivered (extraction, search, and Phase 8's document_ai structure) cover everything a reader can present for a text PDF; OCR only matters for scanned documents with no text layer.
• Deferring is reversible: the decision is "not yet", not "never". When a scanned-document workflow becomes a requirement, the seam is clear — an OcrLayer standing between image decode and PageText.

Consequences

• A reader can show canonical metadata from either /Info or /Metadata, and a writer can publish the same facts into XMP.
• No XML dependency was added, and the parser's tolerance means a hostile or foreign packet cannot fail document open.
• The PDF/A and OCR decisions are recorded, so the plan's "decision required" flag is answered rather than left dangling.
• The Phase 9 exit criterion (full conformance suite) is officially not fully met: PDF/UA and XMP are covered, PDF/A has no suite by decision.

Merge criteria

Enumerated from the plan bullets first, per ADR 0021.

Criterion	State
XMP packet round-trips through the parser	✅ packet_round_trips_through_the_parser
Empty/foreign packets are tolerated, not fatal	✅ empty_packet_parses_to_empty_fields, garbage_is_tolerated_not_fatal
x-default literal wins over other languages	✅ x_default_literal_wins_over_others
Plain (non-container) elements parse	✅ plain_element_without_container_parses
XmpMetadata → DocumentInfo bridge	✅ document_info_bridge_uses_first_creator
PDF/UA checks exist and are tested	✅ ADR 0011 + tests/structure.rs; no is_pdf_ua boolean
PDF/A validation helpers	❌ deferred — certification-level work; reason recorded
OCR layer	❌ deferred by decision — not binding an external OCR engine now; reason recorded
Conformance test suite over generated + corpus documents	⚠️ partial — PDF/UA + XMP covered; PDF/A suite not written (deferred)