nigig-org/NIGIG_PDF_FEATURE_PARITY_PLAN.md

# nigig-pdf feature-parity plan (view → create → edit)

Target: bring the four-crate `nigig-pdf` stack in `crates/apps/pdf/` to feature
parity with the dart-pdf reference implementation (Apache-2.0) for **creating
and editing** PDFs, and absorb the Makepad `dev`-branch PDF code so we no longer
need it. Viewing is already the baseline; this plan closes the read/edit/write
gaps phase by phase.

Coverage parity is a first-class goal, not an afterthought: Section 2 maps all
137 dart-pdf test suites (pdf_cos 31, pdf_document 66, pdf_graphics 37) to a
nigig port/decision per phase, so each phase lands its features **with** the
corresponding dart-pdf test coverage. "Parity" for a phase = the dart-pdf
feature exists in the matching nigig crate **and** its dart-pdf test suite has a
passing nigig equivalent.

## 0. Reference codebases and current state

### 0.1 Makepad upstream `dev` PDF code (the current baseline to supersede)

| File | LOC | Capability |
|------|-----|-----------|
| `libs/pdf_parse/src/*.rs` | ~155K | Minimal parse-only library: lexer, parser, object, page, content ops, filters (zlib), font (char widths), image (inline + extraction), document, tests |
| `widgets/src/pdf_view.rs` | ~40K | View-only widget: page rendering via `PortalList`, text selection, zoom. **No edit, no form, no annotations, no save** |
| `examples/pdf` | ~3.3K | Viewer demo app |

Upstream `dev` `pdf_view.rs` capabilities (from source): text-segment tracking
for selection, selection anchor/cursor, zoom, page caching. That is the full
feature set — reading/display only.

### 0.2 nigig-pdf current state (what we have)

- `nigig-pdf-cos` (8 files): `lexer.rs`, `parser.rs`, `object.rs`, `xref.rs`,
  `writer.rs`, `filter.rs` (Flate + TIFF predictor), `encrypt.rs` (RC4,
  AES-128/256), `incremental.rs`.
- `nigig-pdf-document` (12 files): `document.rs`, `page.rs`, `annotations.rs`,
  `annotation_edit.rs`, `appearance.rs`, `destinations.rs`, `form.rs`,
  `form_actions.rs`, `signature.rs` (read-only), `structure.rs`, `save.rs`,
  `content_writer.rs`.
- `nigig-pdf-graphics` (19 files): `content.rs` interpreter, `device.rs`
  (`PdfDevice` trait), `recording.rs` (`RenderCommand`), `font.rs`, `sfnt.rs`,
  `cid.rs`, `cmap.rs`, `advance.rs`, `colorspace.rs`, `icc.rs`, `function.rs`,
  `transparency.rs`, `image.rs`, `cache.rs`, `worker.rs`, `text.rs`,
  `content_writer.rs`, `graphics_state.rs`.
- `nigig-pdf-makepad` (5 files): `renderer.rs`, `device.rs`, `page_view.rs`,
  `interaction.rs` (form-field interaction: `render_fields`, `render_open_combo`,
  `PdfAction`, selection, key input), `lib.rs`.
- Tests: 571 passing (cos 72, document 147 unit + ~160 integration, graphics
  191 unit + 19 golden + 7 phase-5 exit criterion). Fuzz targets in
  `pdf-cos/fuzz/`. Corpus in `crates/apps/pdf/tests/corpus/`.

### 0.3 dart-pdf reference inventory (feature target)

- `pdf_cos` (~23K LOC): full lexer/parser/xref with xref streams + hybrid
  refs + recovery; filters **Flate, LZW, CCITT, JBIG2, JPX, RunLength,
  ASCIIHex, ASCII85**; crypto **RC4, AES-128/256, RSA, CMS, OCSP, CRL**;
  serializer/builder/updater/compactor/byte-source.
- `pdf_document` (~500K LOC): document, page editing, **annotation editor +
  sync + clipboard**, **form editor + styling + admin**, **content editor +
  type-0 + rewriter + reflow**, **redaction**, **signature editor + PAdES +
  signing identity + Fulcio**, outlines + outline editor, page labels, struct
  tree + editor, XMP, **PDF/A + PDF/UA conformance**, OCR editor, attachment
  editor, header/footer, watermark/stamps, office conversion, disk cache.
- `pdf_graphics` (~74K LOC): **interpreter (3.7K)**, **PdfDevice + Recording
  + RenderCommand + wire codec (binary, isolate-safe)**, font info (embedded
  fonts, encodings, CID), ICC, colour spaces, functions, **mesh gradients,
  shading, overprint compositor**, text extraction with layout/search/
  selection, translating device, document AI, vector print.
- `dart_pdf_editor` (Flutter): canvas device, full editor UI (annotations,
  forms, redaction, search, OCR, AI).

**Scale reference (from the nigig-vs-dart-pdf review).** nigig is ~30K LOC in
~40 files against dart-pdf's ~230K LOC in ~178 files. Per-layer ratios matter
for phase sizing — expect the biggest effort where the ratio is largest:

| Layer | dart-pdf | nigig | ratio |
|-------|----------|-------|-------|
| pdf_cos / pdf-cos | ~23K | ~1.9K | ~12× |
| pdf_document / pdf-document | ~500K | ~3.5K | ~140× |
| pdf_graphics / pdf-graphics | ~74K | ~2K | ~37× |
| UI (dart_pdf_editor / pdf-makepad) | canvas_device ~74K | ~2.5K | ~30× |
| Test files | 137 | ~20 | ~7× |

The pdf_document ratio (~140×) is the dominant cost; phases 4–6 (create/edit)
carry most of it. Module-level comparisons (lexer/parser/xref/object,
annotations/form_editor/redaction, interpreter/font_info/text_extraction) are
the per-phase checklists in §1 and §2.

## 1. Phase plan

Phases are ordered by dependency. Each phase ends with a green build and a
committed, tested tranche. "Parity" for a phase = the dart-pdf feature exists
in the matching nigig crate with tests.

### Phase 1 — Fork sync + makepad_test enablement (immediate, ~1-2 sessions)

- Sync the Gitdab makepad fork to upstream `dev` (workflow.md §0.1), keeping
  `nigig-dev-reexports` intact.
- **Makepad test enablement (fork work, prerequisite for every UI test below).**
  Upstream `dev` makepad_test only runs through the Studio hub: headless mode
  still needs a Studio websocket, and Android is unsupported. This blocks
  automated UI verification of nigig-pdf. The fork must gain:
  - **Terminal/standalone mode**: `makepad_test` runs a suite with no Studio —
    the harness spawns the app binary directly, drives the event loop, and
    asserts on widgets (already partially present in the WIP runtime +
    `examples/counter/tests/ui.rs` pattern: `Selector::id(..).click()` +
    `.wait_text(..)`).
  - **Android mode**: run the same `ui.rs` suites against a device via adb
    (launch APK, port-forward the test channel, run asserts in-code) using the
    staged WIP in the local fork checkout
    (`makepad-native-glue/makepad`): `TestConfig.android/device_serial/adb_path`,
    `start_android_app()`, native-activity platform module, cargo_makepad
    install/launch.
  - Reconcile the WIP with upstream `dev` (currently ~2 commits behind), resolve
    conflicts, commit, and push to the Gitdab fork so the workspace pin uses it.
- Bump `rev =` in `pdf-makepad/Cargo.toml` + `map/Cargo.toml`; reconcile any
  API breaks; full workspace build + 571 tests green.
- Baseline the gap list in this document against the freshly-synced
  `libs/pdf_parse` + `widgets/src/pdf_view.rs` so we can demonstrate nigig-pdf
  has superseded them.

Exit: workspace green on the new rev; `cargo test -p nigig-pdf-makepad --test ui`
runs on the desktop (no Studio) and on an Android device; this document's
inventory updated.

### Phase 2 — Complete content-stream coverage (pdf-cos / pdf-graphics)

Close the interpreter operator gaps dart-pdf handles and nigig does not:

- Inline images `BI/ID/EI`.
- Image XObjects `Do` in content streams (currently placeholder in
  pdf-makepad).
- Form XObjects `Do` (nested content streams, resource dictionaries).
- Type 3 fonts (char-procs, `/FontMatrix`, `/CharProcs`, `/Encoding`).
- Shading operators `sh`/`sh1` + `/Pattern`/`/Shading` resources.
- Full text-object state (Tc/Tw/TL/Tz/Ts/Tstar/Td/TD/Tm/Tf/Tfs).
- **Read-side xref modernisation** (currently traditional xref + `/Prev` chains
  only): xref **streams** (`/Type /XRef`, object streams `/ObjStm`), **hybrid
  refs** (`/XRefStm`), and corrupt-file **recovery** (dart-pdf `xref.dart`
  recovery + `parser.dart` robustness, `xref_test.dart`/`parser_test.dart`
  coverage). nigig `parser.rs` is only 67 LOC; this is the biggest parse-side
  gap.
- Lexer/parser error position tracking for diagnostics (dart-pdf token-level
  lexer keeps source positions; nigig errors carry a position but the lexer is
  simpler).
- Object model: lazy object resolution for very large documents (dart-pdf
  `object.dart` lazy loading / type inference) — confirm nigig's `object.rs`
  object-store is adequate or grow it.
- Streaming content interpretation (dart-pdf `streaming_interpreter.dart`
  semantics: parse+interpret a content stream incrementally instead of
  materializing every op) — covers `streaming_interpreter_test.dart`.

Exit: `pdf-graphics` tests cover each operator family; golden corpus grows;
`pdf-makepad` `ui.rs` smoke tests render pages exercising each new op family
(image/form XObject `Do`, Type 3 text) and assert non-empty draw output.

**Status: complete** (ADR 0013, ADR 0014). Audited item by item rather than
worked top-down, because several entries were already done:

| Item | State |
|---|---|
| Inline images `BI/ID/EI` | already implemented and tested |
| Image/Form XObject `Do` | already implemented; `PdfPage::xobjects` fixed in ADR 0011 |
| Full text state `Tc/Tw/TL/Tz/Ts/Td/TD/Tm/Tf` | already implemented |
| Shading `sh` | operator parsed and reported |
| **Xref streams, object streams, hybrid `/XRefStm`** | **ADR 0013** — was the biggest gap: *no PDF 1.5+ file would open at all* |
| **Type 3 fonts** | **ADR 0014** — `/CharProcs`, `/FontMatrix`, `d0`/`d1` |
| **Streaming interpretation** | **ADR 0014** — `ContentStreamIter`, `interpret_streaming` |
| Lexer error positions | errors already carry a byte position |
| Lazy object resolution | `resolve_num` already caches per object; adequate |

Three latent bugs were found and fixed along the way that the plan did not
list: `d1` read four of its six operands, `PdfPage::xobjects` was empty for
every document, and `BMC` corrupted the colour of every following operator.

The `ui.rs` half of the exit criterion is blocked on the Makepad fork
needing a headless backend — see `REVIEWS/PDF_PARITY_PHASE1_STATUS.md`.

### Phase 3 — Filters and image codecs (pdf-cos)

- Lossless: `RunLengthDecode`, `ASCIIHexDecode`, `ASCII85Decode`, `LZWDecode`
  (+ EarlyChange predictor handling, CW-branching).
- Lossy/bitonal: CCITT G4 (and G3), JBIG2 (arith coding, huffman, refinement),
  JPX/JPEG2000. Decide crate strategy (e.g. `ccitt`, `jbig2`, `jpeg2k`/`openjpeg`
  or vendored ports from dart-pdf).
- Keep the `filter.rs` facade; add a codec registry for round-trip tests.
- Image decode surface: PNG/JPEG image decoding for raster image XObjects and
  inline images (covers `png_test.dart`); image downsampling for render
  performance (`image_downsample_test.dart`, `image_pixels.dart` surface).

Exit: corpus files using these filters decode; round-trip encode tests where
the format allows; `pdf-makepad` `ui.rs` test renders a page whose image uses
the new codec and asserts the decoded image is drawn (no blank region).

**Status: complete** (ADR 0015, ADR 0020). The lossless half landed under
ADR 0015; the three
image codecs it deferred — CCITT, JBIG2 and JPX — are now implemented, in
that order, each with its own tranche and mutation record.

| Item | State |
|---|---|
| `RunLengthDecode`, `ASCIIHexDecode`, `ASCII85Decode` | already worked; edge cases now covered |
| **`LZWDecode`** | **was broken outright** — dictionary indexing was two slots off, so the spec's own worked example failed |
| **`/EarlyChange`** | **was ignored** — a file setting 0 decoded to garbage, not an error |
| **Predictors on LZW** | **were gated to Flate only** |
| **Filter chains** | **`decode_stream` could not read an array `/Filter` at all**, so every chained stream in every document failed |
| **CCITT G3/G4** | **implemented** — T.4/T.6, all three `/K` schemes, decoded in the filter facade. Writing it found three defects: zero-length runs recorded no transition, trailing byte-padding was decoded as data, and a row of zero-length runs looped forever |
| **JBIG2** | **implemented** — MQ coder, generic regions (templates 0-3, TPGDON, MMR). Symbol dictionary, text/halftone/refinement regions and `/JBIG2Globals` are **refused by name**: those carry the composition machinery and shipping them means shipping an interpreter over untrusted input |
| **JPX** | **implemented** — pure Rust, no C dependency, so the Android cross-compile is unaffected. EBCOT tier-1, tag trees, packet headers, 5/3 and 9/7 wavelets, RCT/ICT. Verified against OpenJPEG-produced fixtures, exactly. Multiple tiles, custom precincts, code-block style options and `COC`/`QCC` are refused by name |
| `DCTDecode`/`JPXDecode` pass-through | **removed**: returned compressed bytes as though decoded |

All three are now done. The two open questions that gated JBIG2 and JPX
were answered rather than dropped:

- **JBIG2's CVE record.** The exploit surface is the composition machinery
  — symbol dictionaries, text regions, refinement — not the arithmetic
  coder. Those segment types are refused by name; the generic-region path
  that every PDF-embedded JBIG2 image actually uses is implemented. Every
  file-controlled offset is bounds-checked before a buffer is indexed, and
  the region-to-page composition check has its own test naming what it
  guards.
- **JPX's C dependency.** Avoided entirely: the decoder is pure Rust and
  shares its MQ arithmetic coder with JBIG2, since T.800 and T.88 specify
  the same one. No new crate, no cross-compile risk.

The remaining honest gap is coverage of the *refused* JPX and JBIG2
features, which is a deliberate scope boundary rather than an omission.

The `ui.rs` half of the exit criterion stays blocked on the Makepad headless
backend — see `REVIEWS/PDF_PARITY_PHASE1_STATUS.md`.

### Phase 4 — Document creation / writing (pdf-document + pdf-cos)

- Full `PdfWriter`/builder: outlines (outline tree + viewer prefs),
  page labels, named destinations, document-level metadata (title, author,
  keywords, XMP), viewer preferences, page mode/layout.
- `content_writer.rs` operators to match dart-pdf's `ContentWriter` (full
  text/path/color/image emission).
- AcroForm **creation**: field tree (text, check, radio, list/combo, signature),
  field flags, `NeedAppearances`, and correct AcroForm/Fields wiring. Include
  field-value reconciliation (`form_reconcile_test.dart`) and appearance
  generation (`generated_appearance_test.dart`, `text_box_appearance_test.dart`).
- Document attachments: `/Names /EmbeddedFiles` tree, file specs, attachment
  add/read/remove (`attachment_test.dart`).
- Header/footer and image stamping on pages (`header_footer_test.dart`,
  `image_stamp_test.dart`, `image_pdf_test.dart`).
- Font embedding: subset TrueType via `sfnt.rs` (glyph subsetting, cmap
  remap, `repair-cmap`), embed Type1/CFF if feasible.

Exit: generated PDFs from nigig open cleanly in dart-pdf and external viewers;
`pdf-makepad` `ui.rs` form tests create a field in the UI, fill it via
`Selector::id(..).click()`/keyboard input, and assert the field value + label
state in code.

**Status: complete, except the `ui.rs` interaction tests, which are
written and blocked on the Makepad headless backend.** (ADR 0019 covers
the writer; ADR 0021 covers stamping, reconciliation, CFF and cmap.)

Two earlier revisions of this line were wrong in opposite directions: the
first said "complete" while three named items were unimplemented, the
second listed them as gaps. All three are now done, and the history is
left visible rather than tidied away — a status line that has been
overstated once should show its working.

| Item | State |
|---|---|
| Outlines, page labels, named destinations, metadata, XMP, viewer prefs, page mode/layout | done |
| `content_writer.rs` operator coverage | done |
| AcroForm creation, field flags, `NeedAppearances`, appearance generation | done |
| Attachments (`/Names /EmbeddedFiles`) | done |
| TrueType subsetting and embedding | done |
| **Field-value reconciliation** (`form_reconcile_test.dart`) | **done** — `reconcile.rs`. Classifies how `/V` and `/AP` disagree and resolves it against a caller-declared `Intent`, because the right answer differs: a viewer must render `/AP` to match other viewers, an extractor must read `/V`, an editor must regenerate. `/NeedAppearances` overrides all three, which is the one case §12.7.3.3 actually settles |
| **Type1/CFF embedding** | **done** — `embed_opentype_whole`. Subsetting CFF stays refused by name; embedding the whole program does not, and that is what "if feasible" allows. Writes `/FontFile3` with `/Subtype /OpenType` under a `CIDFontType0` descendant. `EmbeddedFont::is_subsetted` reports which path ran, so a caller with a size budget or a licence constraint can tell |
| **`repair-cmap`** | **done** — `glyph_index` now falls back to a `(3,0)` symbol subtable at `0xF000 + low byte`, and reads format 0 as well as 4, 6 and 12. A symbol font's glyphs were previously unreachable by their plain characters: the lookup returned `None` and the character silently vanished |
| **Header/footer and image stamping** | **done** — `stamp.rs`: image XObject embedding (JPEG passed through as `/DCTDecode` without re-encoding, raw samples as Flate), header/footer banners with alignment, and stamp composition |

The stamping work closed a gap of exactly the shape ADR 0017 describes.
`ContentWriter::draw_image` had existed since Phase 2 and was tested, but
nothing could *create* the image XObject it names — so every `Do` operator
referred to a resource that did not exist, and no error was raised anywhere.

It also uncovered a live defect in `PdfDocBuilder`'s object numbering:
`add_object` derived its number from `pages.len()`, so adding a page *after*
embedding an image — the natural order, since the page content has to name
the image — shifted a number already handed out, and the page's `/XObject`
entry pointed at the page object itself. This is the same positional
numbering bug previously fixed for fonts, one layer out; both are now fixed
at the root by freezing the base when the first number is issued.

### Exit criterion, verified where it can be

**"Generated PDFs open cleanly in external viewers" — now actually
tested, and it passes.** `pdf-graphics/examples/generate_sample.rs`
output was run through two implementations this codebase shares no code
with:

| Check | Result |
|---|---|
| `qpdf --check` | *"No syntax or stream encoding errors found"* |
| `pdfinfo` (poppler) | title, author, subject, keywords, dates, 2 pages, `Form: AcroForm` |
| `pdftotext` (poppler) | all page text, **including the embedded DejaVu subset and its em-dash** |
| `qpdf --list-attachments` | `readme.txt` extracted by name, with its description |
| Catalogue keys | `/Outlines`, `/Names`, `/EmbeddedFiles`, `/PageLabels`, `/Dests`, `/PageMode`, `/ViewerPreferences`, `/AcroForm` all present |
| `pdffonts` (poppler) | the subset TrueType as `CID TrueType` (`sub yes`) **and** the whole CFF as `CID Type 0C (OT)` (`sub no`), both `emb yes` |
| XMP (`set_xmp_metadata`) | `pdfinfo` reports `Metadata Stream: yes` |

Two things that audit turned up:

- **The sample never exercised XMP**, so the one Phase 4 feature most
  likely to be silently absent was also the one nothing checked. Probed
  separately; it works.
- **A `Banner` naming an unregistered font produces a structurally valid
  PDF that draws no text.** `qpdf --check` passes; poppler says
  `Unknown font tag 'F1'`. The API cannot register the font itself —
  fonts belong to the document — so this is now documented on `Banner`
  and pinned by
  `a_banner_font_must_be_registered_or_the_page_lacks_the_resource`.

`pdffonts` is the check that earns its place for the font work: it is the
only one that inspects a font *program* rather than the file structure.
A CFF font written into `/FontFile2`, or given a `CIDFontType2`
descendant, still produces a file qpdf accepts — and a font that loads as
the wrong type or not at all. That shows up here and nowhere else.

**The `ui.rs` half remains blocked** on the Makepad headless backend. The
two form tests exist and are `#[ignore]`d — see
`REVIEWS/PDF_PARITY_PHASE1_STATUS.md`. This is the honest remaining gap:
the *engine* half of Phase 4 is verified end to end against third-party
readers, the *interaction* half is written but unrun. It is a fork
capability, not a nigig-pdf one.

### Phase 5 — Editing (pdf-document)

- Content-stream editing: insert/delete/replace operator runs, page content
  mutation with xref/revision-safe save (extend `save.rs` + `incremental.rs`).
- Text-run rewriting (font/size/position-aware) — dart-pdf
  `content_run_rewriter.dart`.
- Annotation editing: create/modify/delete annotations + appearance
  regeneration (`appearance.rs`), annotation sync + clipboard semantics,
  annotation metadata/restyle, and sub-type coverage: callout, comment/note,
  ink, free-text (`annotation_align_test.dart`, `annotation_metadata_test.dart`,
  `annotation_restyle_test.dart`, `callout_test.dart`, `comment_test.dart`,
  `ink_slice_test.dart`, `sync_test.dart`).
- Page management: insert, reorder, duplicate, delete pages
  (`page_ops_test.dart`, `page_index_map_test.dart`), plus **page import/merge
  from another document** (`import_source_test.dart`).
- **Flatten** annotations/forms into page content (`flatten_test.dart`).
- Object compaction for incremental save: cos `compactor.dart` equivalent
  (object-stream compaction, `compactor_test.dart`, `updater_test.dart`).
- Redaction: content analysis, redaction annotations, content removal.
- Outlines editing, page labels editing, struct-tree editing.

Exit: edit-anything round-trip tests (modify → save → re-parse → verify);
`pdf-makepad` `ui.rs` annotation tests add/move/delete an annotation through
the UI and assert the annotation count/state via the test code.

**Status: complete** (ADR 0022, ADR 0023), except the `ui.rs` interaction
half, which is blocked on the same missing Makepad headless backend as
Phase 4's.

| Item | State |
|---|---|
| Content-stream editing (insert/delete/replace operator runs) | done — `content_edit.rs`, gated by a corpus-wide round-trip property |
| Text-run rewriting (`content_run_rewriter.dart`) | done — preserves the operator *kind*, so a `'` keeps its line advance and a `TJ` keeps its kerning |
| Annotation editing + subtypes | already present (`annotation_edit.rs`); every subtype dart-pdf names was already modelled |
| Page insert/reorder/duplicate/delete (`page_ops_test.dart`) | done — `page_ops.rs`, page tree flattened to one level |
| Page index map (`page_index_map_test.dart`) | done — `PageIndexMap` |
| Page import/merge (`import_source_test.dart`) | done — deep-copies the transitive object graph, resolving inheritable attributes first |
| **Flatten** (`flatten_test.dart`) | done — `flatten.rs`, §12.5.5 placement, plus `/AcroForm` removal when no field survives |
| Object compaction (`compactor_test.dart`, `updater_test.dart`) | done — `compact.rs`, reachability from `/Root`; refuses a signed document by default |
| Redaction | done — `redact.rs`, removes operators rather than covering them; `pdftotext` cannot extract the redacted text |
| Outline / page label / struct-tree editing | done — `catalog_edit.rs` |

### What the round trips found

Every item is tested by saving, re-parsing, and asserting on what a reader
gets. That found **six defects in code that already existed**, none of
which raised an error:

- **`PdfWriter` wrote dictionary keys unordered**, so every generated PDF
  differed run to run. `PdfDict` is a `HashMap` and Rust seeds its hasher
  per process.
- **A short `/Length` silently truncated a stream**, losing the rest of its
  content. The reader guarded the too-long case and trusted the too-short
  one.
- **Two stream readers disagreed by one byte** over the EOL before
  `endstream`, so a write-read-write cycle grew every stream by a newline.
- **`((nested))` truncated a string to nothing** and left the reader
  mid-stream, so every operator after it was parsed from the wrong offset.
- **`#` escapes in names were never decoded**, so `/My#20Font` never
  matched the resource it named.
- **`PdfOp::Unknown` was declared and never constructed**, so an
  unrecognised operator vanished — survivable for a renderer, fatal for an
  editor.

The `ui.rs` half of the exit criterion stays blocked on the Makepad
headless backend — see `REVIEWS/PDF_PARITY_PHASE1_STATUS.md`.

### Phase 6 — Signing and security (pdf-document + pdf-cos)

- Digital signature **writing**: document-byte-range signing, CMS/PKCS#7
  structure, detached/attached signatures, PAdES basics
  (`pades_test.dart`, `signature_test.dart`).
- Signing identities and algorithms: RSA and **ECC/Ed25519**
  (`ec_identity_test.dart`, `ec_signing_test.dart`, `external_signing_test.dart`),
  PKIX certificate-chain validation for verification (`pki_test.dart`,
  `pkix_test.dart`), optional Fulcio-style identity (`fulcio_test.dart`).
- Signature appearance generation (reuse Phase 5 appearance work).
- Revocation checking: OCSP/CRL lookup for verification, per dart-pdf
  crypto handlers (Rust crates; optional but listed for verify-path parity).
- Permission flags (`DocMdpPermission` already exists) + encryption applied on
  save for AES-128/256 (writer side, not just parser side)
  (`standard_security_handler_test.dart`, `encryption_test.dart`).

Exit: sign → verify round-trip; encrypted-write round-trip.

**Status: every bullet implemented.** Both exit criteria pass and each item
in the list above is ticked or explicitly deferred below.

This status line has been wrong twice, in the same direction, so it is now
written as the plan's own bullets rather than as prose. A summary written
from what was built cannot show what was not built; only the spec, item by
item, can (the process note in ADR 0021, which I wrote and then failed to
follow here).

| Spec item | State |
|---|---|
| Document-byte-range signing | done — `sign_document`, ADR 0026 |
| CMS/PKCS#7 structure | done — ADR 0025, corrected by ADR 0026 |
| **Detached** signatures | done — `adbe.pkcs7.detached` |
| **Attached** signatures | **refused, not deferred** — see below |
| PAdES basics | done — `SignatureProfile::Cades`, ADR 0027 |
| RSA identities | done |
| ECC / Ed25519 identities | done — P-256 and Ed25519 |
| `external_signing_test.dart` | done — `ExternalSigner` trait, ADR 0027 |
| PKIX chain validation | done — issuer signatures verified, ADR 0026 |
| Fulcio-style identity | **deferred** — see below |
| Signature appearance generation | done — ADR 0026 |
| Revocation: CRL | done — stapled, offline, ADR 0026 |
| Revocation: OCSP | done — stapled, offline, ADR 0027 |
| Permission flags | done — `DocMdpPermission` read and reported |
| Encryption on save, AES-128/256 | done — ADR 0024 |

**Attached signatures are refused rather than implemented.** `adbe.pkcs7.sha1`
and `adbe.x509.rsa_sha1` are the two attached profiles, and both are
SHA-1 based. SHA-1 is broken for signature purposes, and a library that
offers a broken profile will have it selected by somebody who does not know
that — the same reasoning that keeps RC4 readable but not writable
(ADR 0024). They are parsed, so documents using them can be read and
reported; they cannot be written. `SubFilter::Pkcs7Sha1` and
`SubFilter::X509RsaSha1` have no corresponding `SignatureProfile`.

**Fulcio is deferred.** The plan marks it optional. It is a keyless-signing
identity that requires an OIDC round trip and a call to a certificate
authority, which is network access the engine crates are gated against.

Three things remain before this is a product feature. **None is a code gap**,
and all three are recorded in the ADRs rather than implied:

- **Independent review.** ADR 0026 is a self-review. It found a critical
  vulnerability — a forgery verified as valid — which is evidence the method
  works, and not evidence that nothing remains.
- **Acrobat interoperability.** qpdf and poppler accept the documents;
  Acrobat is stricter than the specification and is untested here.
- **Signing a document that already has an AcroForm.** `sign_document`
  replaces the form rather than merging into it.

### Phase 7 — Advanced rendering (pdf-graphics + pdf-makepad)

- Shading: axial/radial/function shading + mesh (free-form/lattice/Coons)
  — `shading.rs`/`mesh.rs` ports (`shading_test.dart`).
- Blend modes + overprint compositor (`overprint_compositor.dart` equivalent,
  `overprint_test.dart`).
- Transparency groups with soft masks already present; extend to
  blend-mode compositing in the Makepad device.
- Embedded-font rendering: glyph outlines from embedded TrueType **and CFF**
  (`cff_test.dart`, `embedded_font_render_test.dart`, `truetype_test.dart`).
- **Glyph-aware text runs** through the device trait — extend the nigig
  text-run type to carry per-glyph pen offsets and (when the font is embedded)
  outlines, matching dart-pdf `PdfGlyphPlacement`/`PdfTextRun`; enables
  hit-testing, copy, and font-driven layout (`render_command_test.dart`).
- Type 3 font rendering (Phase 2 parsing + glyph runs).
- Form/image XObject rendering through the device trait.
- **RenderCommand wire codec**: binary serialize/deserialize of `RenderCommand`
  for the worker-thread boundary (`render_command_codec_test.dart`,
  `page_text_codec_test.dart`), and tiled rendering sink
  (`PdfTiledCellSink` equivalent, `tiling_record_replay_test.dart`).

Exit: golden render corpus covers shading/mesh/overprint/image-XObject pages;
`pdf-makepad` `ui.rs` suite adds render smoke tests for each new rendering
feature (shading page, overprint page, image-XObject page) asserting non-empty
draw + correct element counts.

**Status: all eight bullets addressed; the golden-corpus exit criterion is
met, the `ui.rs` one remains blocked on the headless backend.**
Written as a table from the start, per the process note in ADR 0021, so an
unfinished item is a visible row rather than a sentence nobody wrote. Rows
marked **partial** name what is missing rather than rounding up.

| Spec item | State |
|---|---|
| Shading: axial / radial / function | **done** — ADR 0028 |
| Shading: mesh (free-form, lattice, Coons) | **done** — ADR 0029. Five corpus fixtures; writing them found three real bugs (type 5 read a phantom flag, types 6/7 read a colour per control point, flag 0 cleared the strip). Coons patches are still flattened to two triangles and report `is_approximate` |
| Blend modes + overprint compositor | **done** — ADR 0030. `composite.rs` composites all sixteen modes against a real backdrop; `/OP`, `/op`, `/OPM` parsed and honoured per ink |
| Transparency groups → blend-mode compositing in the Makepad device | **partial** — ADR 0030. Group compositing exists and is tested in `composite.rs`; the **Makepad** device still reports `Unsupported` because it has no render-to-texture. Knockout groups refused by name |
| Embedded-font glyph outlines (TrueType **and CFF**) | **done** — ADR 0031. `outline.rs` reads `glyf`/`loca` and Type 2 charstrings, asserted against fontTools ground truth. CID-keyed CFF (`FDArray`/`FDSelect`) and `seac` accents deferred |
| Glyph-aware text runs (per-glyph pen offsets) | **done** — ADR 0031. `GlyphPlacement` on `ShowTextWithMetrics`; hit-testing, selection and search highlighting now use real offsets instead of dividing the run advance by the character count. Outlines are reachable by glyph id, not attached to each placement |
| Type 3 font rendering | **done** — ADR 0032. Glyph procedures are interpreted through the device with the font matrix and pen composed correctly. Per-glyph caching deferred |
| Form/image XObject rendering through the device trait | **done** — ADR 0032. `render_form` resolves and runs the stream with `/Matrix`, the `/BBox` clip and a save/restore wrapper; image XObjects refused by name; recursion bounded. Wiring the Makepad **widget** to call it is UI work and is not done |
| `RenderCommand` wire codec + tiled sink | **done** — ADR 0033. Round-trips every variant; truncation is an error rather than a short list; tiles are pixel-identical to the whole page. Soft-mask groups do not cross the wire, by design and stated |

Exit criteria:

| Criterion | State |
|---|---|
| Golden render corpus covers **shading** pages | ✅ `shading_axial`, `shading_radial` — golden *pixels*, not command text |
| … **mesh** pages | ✅ `shading_mesh` |
| … **overprint** pages | ✅ `overprint_on` / `overprint_off`, as CMYK plates, because overprint cannot be shown in RGB |
| … **image-XObject** pages | ✅ `xobject_form` — a form XObject rendered to pixels, with its `cm` offset, its `/BBox` clip and the clipped-away overspill all visible in the golden. The golden caught a real bug the colour assertions missed (see ADR 0033) |
| `pdf-makepad` `ui.rs` render smoke tests | ❌ **blocked** — the Makepad headless backend, as since Phase 1. `windowing_backend.rs` has only X11/Wayland; 6 tests remain `#[ignore]`. Not this phase's work and not claimed as done |

### Phase 8 — Text intelligence (pdf-graphics)

- Text search: Unicode-aware search with hit rects over `text.rs` extraction.
- Selection improvement: layout-aware (multi-line, multi-column) hit-testing,
  matching `pdf_view.rs` behaviour and beyond.
- Content reflow (`content_reflow.dart` equivalent) — layout-friendly text
  export and re-layout (`content_reflow_test.dart`, `reflow_render_test.dart`).
- Text diff/cache (`text_cache.dart`, `text_diff.dart`) for cheap re-renders,
  plus page-level disk/page cache (`disk_cache_test.dart`, `page_cache_test.dart`).
- Document AI surface (`document_ai.dart`): structure/layout understanding on
  top of extraction (`document_ai_test.dart`).

Exit: search + selection integration tests; reflow unit tests.

**Status: two of five bullets done.** A table from the start, per ADR 0021.

| Spec item | State |
|---|---|
| Text search: Unicode-aware, with hit rects | **done** — ADR 0034. Cross-run matches, NFD normalisation, case *folding* (not lowercasing), opt-in diacritic folding, whole-word, one rectangle per run |
| Selection: layout-aware multi-line, multi-column hit-testing | **done** — ADR 0034. Columns detected before lines, because two columns share their baselines; reading order proven to be a permutation of the runs |
| Content reflow — layout-friendly export and re-layout | not started |
| Text diff/cache for cheap re-renders, plus disk/page cache | partial — `cache.rs` has an in-memory `PageCache` with an LRU budget and generation guards (Phase 3); no text diff, no disk cache |
| Document AI surface: structure/layout understanding | not started — `detect_lines`/`detect_columns_from_segments` from ADR 0034 are the substrate this would build on |

Exit criteria:

| Criterion | State |
|---|---|
| Search integration tests | ✅ `text_search.rs`, driven from `basic/two_columns.pdf` through parse → interpret → extract → index → search |
| Selection integration tests | ✅ hit-testing every run of a real page, plus the gutter |
| Reflow unit tests | ❌ **not started** — the reflow bullet is not done |

### Phase 9 — Conformance, metadata and OCR (pdf-document)

- PDF/A validation helpers (conformance checks against PDF/A-1b/2b,
  `pdf_a_test.dart`).
- PDF/UA accessibility checks (struct-tree completeness, tag map, alt text,
  `pdf_ua_test.dart`, `struct_text_test.dart`).
- XMP metadata read/write (Phase 4 covers writing; here full XMP packet
  handling, `xmp.dart` port).
- OCR layer (`ocr_layer_test.dart`): **decision required** — dart-pdf binds an
  external OCR engine. For Rust, decide between a Rust OCR crate
  (e.g. `tr`/`rust-tesseract`/onnx) or defer.

Exit: conformance test suite over generated + corpus documents.

### Phase 10 — Editor/UX parity (pdf-makepad)

- Form filling UI: field focus, combo/list popup, check/radio toggling,
  submit + JS-action wiring (extend `interaction.rs`).
- Annotation UI: add/hit-test/edit/delete annotation overlays.
- Signing UI: signature field creation, sign flow, verification badge.
- Search UI: toolbar, hit navigation, selection copy.
- Save/export entry points wired from UI to Phase 4–6 writers.

Exit: `pdf-makepad` can open, edit, sign, and save a document end-to-end —
parity with `dart_pdf_editor` core workflows — **proved by `ui.rs` suites**
that mirror each workflow (fill a form field and verify the value;
add an annotation and verify it persists; sign and verify the badge; search
and verify hit navigation; save and verify the file), all running headless in
a terminal and on Android via the Phase-1 makepad_test capability. No Studio
hub required for any of these.

## 2. dart-pdf test-inventory mapping (test transfer)

Every dart-pdf test suite below maps to one of:
- **DONE** — equivalent nigig coverage already exists;
- **P##** — port/adapt the test suite (and its feature) in that phase;
- **SKIP** — N/A for Rust or out of scope (recorded with reason);
- **DECIDE** — needs an explicit decision (external dependency, scope).

This is a living table; update it as suites land. The goal is **coverage
parity**: for every dart-pdf suite, a nigig equivalent test exists and passes.

### pdf_cos/test (31 suites)

| dart-pdf suite | status | notes |
|----------------|--------|-------|
| builder_test | DONE | nigig `writer.rs` doc-builder tests |
| byte_source_test | DONE | nigig operates on `&[u8]` |
| ccitt_test / ccitt_golden_test / ccitt_corpus | P3 | CCITT codec |
| compactor_test | P5 | object-stream compaction |
| content_parser_test | P2 | operator coverage |
| crypto_test | DONE/P6 | nigig `encrypt.rs` AES/RC4 |
| decoded_cache_test | P8 | cache tests |
| document_test | DONE | nigig `document.rs` |
| ec_identity_test | P6 | Ed25519 identity |
| encryption_test | P6 | encrypt-on-write |
| filter_test | P3 | codec registry round-trips |
| incremental_update_test | P5 | nigig `incremental.rs` + save |
| jbig2_test / jbig2_roundtrip_test / jpx_test | P3 | JBIG2/JPX |
| lexer_test | DONE | nigig `lexer.rs` |
| matrix_test | DONE | nigig `matrix` |
| parser_test | P2 | parser robustness + xref-stream parsing |
| perf_test | P8 | perf gate on decode paths |
| pki_test / pkix_test / pkix_extra_test | P6 | cert-chain validation |
| serializer_test | P4 | writer/serializer parity |
| standard_security_handler_test | P6 | encrypt-on-write |
| string_text_test | P2 | text handling |
| updater_test | P5 | incremental updater |
| xref_test / xref_writer_test | P2 | nigig `xref.rs` traditional only; add xref streams/hybrid/recovery |

### pdf_document/test (66 suites)

| dart-pdf suite | status | notes |
|----------------|--------|-------|
| annotation_test / annotation_editor_test | P5 | nigig `annotation_edit.rs` to grow |
| annotation_align_test / annotation_restyle_test / annotation_metadata_test | P5 | sub-types |
| annotation_clipboard_test / annotation_sync_test | P5 | sync semantics |
| attachment_test | P4 | embedded files |
| blank_document_test | P4 | doc creation |
| callout_test / comment_test | P5 | callout + note |
| check_mark_test | P4 | form check marks |
| color_processing_test | P2/P7 | color pipelines |
| compress_test | P3 | codecs |
| content_edit_test / content_edit_type0_test | P5 | stream editing + Type0 |
| content_reflow_test | P8 | reflow |
| disk_cache_test / page_cache_test | P8 | page cache |
| document_test | DONE | nigig `document.rs` |
| ec_signing_test / external_signing_test | P6 | ECC/Ed25519 signing |
| edit_impact_test | P5 | edit → re-render deltas |
| editor_test | P5 | editing aggregate |
| flatten_test | P5 | flatten to content |
| font_embed_test / font_used_in_test / fonts/ | P4 | font subsetting |
| form_test / form_fill_test | P4/P10 | creation + fill |
| form_admin_test | P4 | field admin |
| form_reconcile_test | P4 | value reconciliation |
| form_styling_test | P4 | field styling |
| fulcio_test | P6 | DECIDE (Fulcio) |
| header_footer_test | P4 | header/footer |
| image_pdf_test / image_stamp_test | P4 | image stamping |
| import_source_test | P5 | page import/merge |
| incremental_update_test | P5 | nigig `incremental.rs` |
| ink_slice_test | P5 | ink annotation |
| line_ending_test | P8 | text line handling |
| matrix_geometry_test | DONE | nigig geometry |
| measure_test | P4/P10 | measurement |
| office_conversion_test | SKIP | office → PDF; external tooling in Dart, out of scope |
| open_source_test | P4 | opening sources |
| outline_test | P4 | outlines |
| pades_test | P6 | PAdES |
| page_index_map_test / page_ops_test | P5 | page ops |
| page_labels_test | P4 | page labels |
| pdf_a_test / pdf_ua_test | P9 | conformance |
| perf_gate_test | P8 | perf gates |
| png_test | P3 | PNG decode |
| redaction_test | P5 | redaction |
| signature_test | P6 | signature write |
| struct_tree_test / struct_tree_editor_test | P5/P9 | structure |
| sync_test | P5 | annotation sync |
| takeoff_test | SKIP | construction-measurement niche, out of scope |
| text_box_appearance_test | P4 | appearance gen |
| text_run_rewriter_test | P5 | text-run rewrite |
| type0_font_test | P2/P8 | Type0 fonts |
| vector_snapshot_test | P5 | vector snapshots |

### pdf_graphics/test (37 suites)

| dart-pdf suite | status | notes |
|----------------|--------|-------|
| cff_test | P7 | CFF glyph outlines |
| cjk_cmap_test | P2 | CJK CID/CMap |
| color_space_test / colorant_buffer_test | DONE/P2 | nigig `colorspace.rs` |
| document_ai_test | P8 | document AI |
| embedded_font_render_test | P7 | embedded glyphs |
| font_cache_test / image_decode_cache_test | P8 | caches |
| font_info_test | P2/P4 | font engine |
| generated_appearance_test | P4 | appearance gen |
| ghent_corpus_test / ghent_jpx_indexed_test | P3 | **Ghent corpus** (print production) — nigig corpus must add these |
| icc_test | DONE | nigig `icc.rs` |
| image_colorants_test / image_scan_parity_test | P3 | image colorants |
| image_downsample_test / image_pixels_test | P3 | downsample/pixels |
| interpreter_test / streaming_interpreter_test | P2 | operator + streaming |
| ocr_layer_test | P9 | DECIDE (external OCR engine) |
| overprint_test | P7 | overprint compositor |
| page_text_codec_test | P7 | text wire codec |
| paste_rotation_render_test | P7 | paste/rotation render |
| path_test | DONE | nigig path |
| pdfjs_corpus_test | P2+ | **PDF.js corpus (171 files)** — nigig corpus must add |
| raster/ | P7 | raster pixel tests |
| reflow_render_test | P8 | reflow render |
| render_command_test | DONE | nigig `recording.rs` |
| render_command_codec_test | P7 | binary wire codec |
| shading_test | P7 | shading/mesh |
| struct_text_test | P9 | struct text |
| text_cache_test / text_diff_test | P8 | text caches |
| text_extraction_test | P8 | extraction |
| tiling_record_replay_test | P7 | tiled sink |
| truetype_test | DONE/P4 | nigig `sfnt.rs` |
| vector_print_test | SKIP | vector-print format, out of scope |

### Corpus parity

- nigig corpus: `crates/apps/pdf/tests/corpus/` (small, hand-picked).
- Target additions by phase:
  - P2/P3: PDF.js corpus subset (171 files, `pdfjs_corpus_test.dart`) covering
    the operators/codecs each phase adds;
  - P3: Ghent PDF Workgroup corpus (print-production, `ghent_corpus_test.dart`)
    once CCITT/JPX land;
  - P7: render-golden subsets of both corpora through `golden_render.rs`.

## 3. Cross-cutting concerns

- **Testing**: every phase adds integration tests; golden corpus
  (`pdf-graphics/tests/golden_render.rs`) extended; fuzz targets extended
  (Phase 3 codecs are a fuzz hotspot). Keep the "no feature without an
  integration test" rule (workflow.md §9).
- **Robustness parity**: dart-pdf runs robustness fuzzing (100K+ iterations).
  nigig has 12 libFuzzer targets (`pdf-cos/fuzz/`); add a CI robustness gate
  that runs corpus + fuzz seed inputs against every phase's new parser/codec
  surface, and grow the fuzz corpus with the PDF.js/Ghent additions in §2.
- **UI-test parity (makepad_test)**: every pdf-makepad feature ships a
  `tests/ui.rs` suite written in the `makepad_test` style (counter-example
  pattern: `app.locator(Selector::id(..)).click()` then
  `.wait_text(..)`/`.wait_visible()`, so a click that a human would do by hand
  is asserted in code — label/count/state changes must be tracked by the test).
  Rules:
  - Tests run **headless in a terminal** (no Studio) and **on Android** via the
    Phase-1 fork capability; they must not require the Studio hub.
  - `#[ignore]`-gated Studio-only tests are not acceptable substitutes — a
    feature is done only when its `ui.rs` suite passes on both targets.
  - The existing `pdf-makepad/tests/ui.rs` (currently `#[ignore]`-gated because
    it needs Studio) is the seed; migrate it off the hub in Phase 1 and keep it
    green in CI.
  - UI suites mirror `dart_pdf_editor` workflows (form fill, annotation add/
    edit/delete, signing, search), ported as nigig-authored Rust tests since
    dart-pdf's UI tests are Flutter-specific and not portable directly.
- **Provenance**: ported patterns from dart-pdf must keep `THIRD_PARTY_NOTICES.md`
  accurate (Apache-2.0 attribution). Ported code should be re-implemented in
  Rust idioms, not transliterated.
- **Performance**: re-check `cache.rs`/`worker.rs` after Phases 3, 7, 8
  (decoders and text are the hot paths). Extend `BENCH_BASELINE.md`. Object
  lazy-loading (Phase 2) and the wire codec (Phase 7) also gate large-doc
  memory behaviour.
- **Fork dependency**: any makepad-API-touching phase (1, 7, 10) must first
  satisfy workflow.md §0.1 fork sync.