nigig-org/REVIEWS/adr/0039-pdf-xmp-and-conformance-decisions.md

# ADR 0039: XMP metadata handling and the conformance/OCR decisions

- **Status:** Accepted
- **Date:** 2026-09-05
- **Review item:** `NIGIG_PDF_FEATURE_PARITY_PLAN.md` Phase 9, "PDF/A
  validation helpers", "PDF/UA accessibility checks", "XMP metadata
  read/write", "OCR layer"
- **Supersedes:** nothing
- **Related:** ADR 0011 (the PDF/UA structure tree, which the PDF/UA bullet
  of this phase is built on), ADR 0021 (the process note that drove the
  table format)

## Context

Phase 9 has four bullets. Surveying the existing code before changing
anything:

1. **PDF/UA accessibility checks.** Already done. ADR 0011 covers the
   structure tree (`/StructTreeRoot`, `/K`, `/IDTree`, `/ParentTree`,
   `/RoleMap`), custom-tag resolution, and typed `AccessibilityIssue`s;
   `pdf-document/tests/structure.rs` is the acceptance suite. The module
   deliberately carries no `is_pdf_ua` verdict, and a test asserts that
   absence. Nothing to build.
2. **XMP metadata read/write.** Genuinely missing on the read side. The
   writer (`pdf-cos::writer`) accepts raw XMP bytes (`set_xmp_metadata`) but
   nothing parses a `/Metadata` stream back into fields, and there is no
   XMP packet generator.
3. **PDF/A validation helpers.** Missing. A PDF/A *check* is a certification
   claim, and this project has a standing rule (ADR 0011, ADR 0017) against
   mechanical checks wearing a certification hat.
4. **OCR layer.** Missing, and the plan itself flags it "decision required".
   dart-pdf binds an external engine; a Rust equivalent is a new heavyweight
   dependency.

## Decision

### XMP: read the packet, bridge to `DocumentInfo`, generate a packet

`pdf-document::xmp` (new module, no XML dependency):

- `read_xmp` locates the catalogue `/Metadata` stream (trailer `/Root` →
  catalog `/Metadata` → `resolve_stream`, so `/Filter`-decoded bytes arrive
  already decompressed) and parses it.
- `parse_packet` scavenges the fields a metadata packet actually carries:
  `dc:title` and `dc:description` as `rdf:Alt` containers (the `x-default`
  literal wins), `dc:creator` as an ordered `rdf:Seq`, `dc:subject` as a
  `rdf:Bag`, and the plain-element forms `pdf:Producer`, `xmp:CreatorTool`,
  `xmp:CreateDate`, `xmp:ModifyDate`.
- `to_document_info` merges onto the existing `DocumentInfo` (`/Info`), XMP
  winning for the fields `/Info` does not carry (creators list, keyword
  list).
- `to_packet` emits a canonical packet (with the `xpacket` PI wrapper and
  the `x:xmpmeta` envelope) so writer workflows have a real producer for
  `set_xmp_metadata`.

The parser is **a targeted scavenger, not a general XML parser** — it is
tolerant (a foreign or malformed packet yields `None` fields, never an
error opening the document) and it does not validate RDF. That is a
deliberate line: full XMP/RDF parsing is a spec-sized project, and the
fields above are what viewers and workflows actually read.

### PDF/UA: mark done via ADR 0011, nothing new built

The bullet's substance — struct-tree completeness, tag map, alt text — is
ADR 0011's deliverable. The plan now records it as done rather than
re-deriving it.

### PDF/A helpers: defer, do not hand-write

PDF/A conformance is a certification. A hand-rolled "is this PDF/A-1b?"
check risks becoming a boolean downstream trust adopts while checking only
the entries someone remembered. Doing it properly is a spec-grounded pass
over required entries and value ranges, plus a fixture corpus. Not this
pass's work; recorded as deferred with the reason.

### OCR: defer with a recorded decision

nigig will not bind an OCR engine now. Reasons:

- It is a new heavyweight dependency (a Rust engine or a Tesseract binding)
  bolted onto a second pipeline, with a different failure model than
  extraction.
- The text-first surfaces already delivered (extraction, search, and Phase
  8's `document_ai` structure) cover everything a reader can present for a
  text PDF; OCR only matters for scanned documents with no text layer.
- Deferring is reversible: the decision is "not yet", not "never". When a
  scanned-document workflow becomes a requirement, the seam is clear — an
  `OcrLayer` standing between image decode and `PageText`.

## Consequences

- A reader can show canonical metadata from either `/Info` or `/Metadata`,
  and a writer can publish the same facts into XMP.
- No XML dependency was added, and the parser's tolerance means a hostile or
  foreign packet cannot fail document open.
- The PDF/A and OCR decisions are recorded, so the plan's "decision
  required" flag is answered rather than left dangling.
- The Phase 9 exit criterion (full conformance suite) is officially **not
  fully met**: PDF/UA and XMP are covered, PDF/A has no suite by decision.

## Merge criteria

Enumerated from the plan bullets first, per ADR 0021.

| Criterion | State |
|---|---|
| XMP packet round-trips through the parser | ✅ `packet_round_trips_through_the_parser` |
| Empty/foreign packets are tolerated, not fatal | ✅ `empty_packet_parses_to_empty_fields`, `garbage_is_tolerated_not_fatal` |
| `x-default` literal wins over other languages | ✅ `x_default_literal_wins_over_others` |
| Plain (non-container) elements parse | ✅ `plain_element_without_container_parses` |
| `XmpMetadata` → `DocumentInfo` bridge | ✅ `document_info_bridge_uses_first_creator` |
| PDF/UA checks exist and are tested | ✅ ADR 0011 + `tests/structure.rs`; no `is_pdf_ua` boolean |
| PDF/A validation helpers | ❌ **deferred** — certification-level work; reason recorded |
| OCR layer | ❌ **deferred by decision** — not binding an external OCR engine now; reason recorded |
| Conformance test suite over generated + corpus documents | ⚠️ partial — PDF/UA + XMP covered; PDF/A suite not written (deferred) |