andodeki
7d6fc4cbbe
Phase 4 of NIGIG_PDF_FEATURE_PARITY_PLAN.md. ADR 0019. Almost none of it existed: Outlines, PageLabels, EmbeddedFiles and ViewerPreferences appeared nowhere in the workspace, in any crate. What did exist was a builder whose central method was pub fn add_page_with_content(&mut self, _width: f64, _height: f64, ...) which accepted a page size and discarded it. Asking for 200x400 and 300x500 gave two US Letter pages, because no /MediaBox was written at all. The test asserted the output contained the string "/Type /Page", which it did. Two more defects sat in the object writer, both producing files our own parser rejects: dictionary keys were written unescaped (a key with a space reparses as "expected number"), and f64::NAN was emitted as the literal token NaN, so one non-finite value anywhere made the document unreadable. Added: outline trees with the open/closed state in the sign of /Count, /PageLabels as a number tree with real roman and A..Z/AA..ZZ numbering, named destinations, attachments with file specs, /Info, XMP, viewer preferences, page mode and layout; AcroForm creation for text, checkbox, radio, choice and signature fields with generated appearances; and TrueType subsetting - DejaVu Sans goes from 759,720 bytes to 4,348 for twelve characters. cmap is deliberately not rebuilt: the subset is embedded as a CID font with Identity-H, so the content stream addresses glyphs by id and /ToUnicode serves extraction. A cmap disagreeing with the content stream is worse than none. CFF is refused by name rather than emitting a font with no glyphs. Nine real bugs, every one found by running the output through an independent tool rather than by reading the code: 1 page size discarded reading a generated file back 2 dict keys unescaped probing the writer 3 NaN written as a keyword probing the writer 4 subset zeroed the lsb fontTools outline compare 5 hmtx indexed by new gid fontTools outline compare 6 name table format read as count BaseFont came out "Embedded" 7 add_font shifted numbers already handed out 8 trees allocated over font numbers - object 29 written twice 9 widgets missing /F Print, /P and appearance /Resources 7 and 8 are the instructive pair: every reference resolved and every object existed, each simply named the wrong thing. pypdf reported correct field values from a file PDFium rendered blank. 9 is the one only a renderer could find - /F defaults to non-printable, and a form XObject naming a font its /Resources does not declare is discarded whole. Verified by three independent implementations: fontTools (0 outline mismatches of 12 against the source font), pypdf (metadata, page sizes, outline with resolved page numbers, all five fields, attachment byte-for-byte, labels ['i','1']) and PDFium, which renders both pages correctly. cargo run -p nigig-pdf-graphics --example generate_sample regenerates the sample. Fourteen mutations. Three survived and each exposed a weak test: the key test used an attachment name (written as a string, never a key), nothing read the outline open state, and /P could not be witnessed because page_index is supplied by the reader, which already knows the page. All three now killed. pdf: 789 passed (was 730). pdf-ui: 775. Coverage 85.17%.
202 lines
9.6 KiB
Markdown
202 lines
9.6 KiB
Markdown
# ADR 0019: document creation — a writer nobody had read back
|
||
|
||
- **Status:** Accepted
|
||
- **Date:** 2026-08-16
|
||
- **Review item:** `NIGIG_PDF_FEATURE_PARITY_PLAN.md` §1 Phase 4, "Document
|
||
creation / writing (pdf-document + pdf-cos)"
|
||
- **Supersedes:** nothing
|
||
- **Related:** ADR 0017 (declared-versus-delivered), ADR 0018 (destinations
|
||
— its reader is what the outline and named-destination tests assert
|
||
through)
|
||
|
||
## Context
|
||
|
||
Phase 4 asked for a full builder: outlines, page labels, named
|
||
destinations, metadata, viewer preferences, AcroForm creation, attachments
|
||
and TrueType subsetting. Almost none of it existed. `Outlines`,
|
||
`PageLabels`, `EmbeddedFiles` and `ViewerPreferences` appeared **nowhere in
|
||
the workspace**, in any crate, in any form.
|
||
|
||
What did exist was `PdfDocBuilder`, and its central method was this:
|
||
|
||
```rust
|
||
pub fn add_page_with_content(&mut self, _width: f64, _height: f64, content: &[u8]) {
|
||
self.content_streams.push(content.to_vec());
|
||
}
|
||
```
|
||
|
||
The page size is accepted and discarded — the parameters are named
|
||
`_width` and `_height` — and no `/MediaBox` was written at all. Asking for
|
||
a 200×400 page and a 300×500 page:
|
||
|
||
```
|
||
page 0: media_box=[0.0, 0.0, 612.0, 792.0]
|
||
page 1: media_box=[0.0, 0.0, 612.0, 792.0]
|
||
```
|
||
|
||
Both US Letter. The existing test asserted the output *contained the string*
|
||
`/Type /Page`, which it did.
|
||
|
||
Two more defects sat under that, in the object writer everything goes
|
||
through. Both produce files **our own parser rejects**:
|
||
|
||
| Input | Written | Reparsed |
|
||
|---|---|---|
|
||
| key `Weird Key` | `<</Weird Key 1/After 2>>` | `expected number` at byte 9 |
|
||
| `PdfObj::Real(f64::NAN)` | `<</W NaN>>` | `expected number` at byte 5 |
|
||
|
||
Dictionary *keys* are names and need the same escaping `PdfObj::Name` gets;
|
||
they had separate code and only one of them escaped. And PDF has no syntax
|
||
for a non-finite number — `format!("{}", f64::NAN)` emits `NaN`, which is a
|
||
keyword to a parser, so one such value anywhere made the whole document
|
||
unreadable.
|
||
|
||
## Decision
|
||
|
||
### Structure in `pdf-cos`, authoring in `pdf-graphics`
|
||
|
||
`PdfDocBuilder` grew the catalogue features: outlines (linked `/First`,
|
||
`/Last`, `/Next`, `/Prev`, `/Parent` with the sign of `/Count` carrying the
|
||
expanded state), `/PageLabels` as a number tree, `/Names /Dests`,
|
||
`/Names /EmbeddedFiles` with file specifications, `/Info`, XMP `/Metadata`,
|
||
`/ViewerPreferences`, `/PageMode` and `/PageLayout`.
|
||
|
||
`DocumentCreator` lives in `pdf-graphics`, not `pdf-document`, because the
|
||
dependency runs **cos ← document ← graphics** and font embedding needs the
|
||
subsetter. Putting it lower would have inverted that.
|
||
|
||
Reading these back needed an API too — a writer nothing can read is
|
||
untestable — so `catalog.rs` in `pdf-document` reads outlines, page labels,
|
||
attachments and viewer preferences. Page labels implement the real
|
||
numbering rules: roman numerals subtractively, and the `A`/`a` style as
|
||
A..Z, AA..ZZ, AAA..ZZZ rather than base-26, which would give `BA` for 27.
|
||
|
||
### TrueType subsetting
|
||
|
||
`subset.rs` rebuilds `glyf`, `loca`, `hmtx`, `hhea` and `maxp` for just the
|
||
glyphs used, follows composite-glyph references transitively and renumbers
|
||
them, and emits a deterministic six-letter subset tag. DejaVu Sans goes
|
||
from **759,720 bytes to 4,348** for twelve characters.
|
||
|
||
`cmap` is deliberately **not** rebuilt. The subset is embedded as a CID font
|
||
with `/Identity-H`, so the content stream addresses glyphs by id and no
|
||
character mapping is consulted at render time; text extraction is served by
|
||
a generated `/ToUnicode`. A `cmap` that disagreed with the content stream
|
||
would be worse than none. CFF outlines are refused by name (`SubsetError::
|
||
CffNotSupported`) rather than silently emitting a font with no glyphs.
|
||
|
||
### AcroForm creation
|
||
|
||
Text, checkbox, radio, choice and signature fields, with field flags,
|
||
`/MaxLen`, `/Opt`, and generated appearance streams. Radio groups are one
|
||
field with a widget kid per option, which is what makes them exclusive.
|
||
|
||
## Verification
|
||
|
||
### Independent, not self-referential
|
||
|
||
The plan's exit criterion is "generated PDFs open cleanly in external
|
||
viewers", which no test in this repository can assert. Three independent
|
||
tools were used:
|
||
|
||
- **fontTools** parsed every subset, decoded all 16 glyphs, resolved the
|
||
composites, and re-saved without error. Comparing **fully-resolved
|
||
outlines** against the source font: **0 mismatches of 12**.
|
||
- **pypdf** read back the metadata, both page sizes, the outline with
|
||
resolved page numbers, all five form fields with values, the attachment
|
||
byte-for-byte, and page labels `['i', '1']`.
|
||
- **PDFium** (the engine in Chrome and Edge) opened and rendered both pages.
|
||
|
||
### Nine real bugs, found by running it
|
||
|
||
| # | Bug | How it was found |
|
||
|---|---|---|
|
||
| 1 | page size discarded, no `/MediaBox` | reading a generated file back |
|
||
| 2 | dict keys unescaped → unparseable file | probing the writer |
|
||
| 3 | `NaN`/`inf` written as keywords | probing the writer |
|
||
| 4 | subset zeroed the left side bearing | fontTools outline compare |
|
||
| 5 | `hmtx` indexed by new gid on the old font | fontTools outline compare |
|
||
| 6 | `name` table: format read as count, so no family name ever found | `BaseFont` came out `Embedded` |
|
||
| 7 | fonts numbered before extras, so `add_font` retroactively shifted numbers already handed out | pypdf: `/ToUnicode` pointed at the descriptor, `/FontFile2` at the Type0 |
|
||
| 8 | trees allocated over font numbers — object 29 written twice | pypdf: `/F1` resolved to the `/Names` tree |
|
||
| 9 | widgets missing `/F` Print, `/P`, and appearance `/Resources` | PDFium rendered a blank form page |
|
||
|
||
Bugs 7 and 8 are the instructive pair. **Every reference resolved and every
|
||
object existed** — each simply named the wrong thing. No parse error, no
|
||
panic, and pypdf still reported correct field values while the page was
|
||
blank. This is the ADR 0017 pattern in the writer.
|
||
|
||
Bug 9 is the one only a renderer could find: `/F` defaults to *not
|
||
printable*, and a form XObject naming a font its `/Resources` does not
|
||
declare is discarded whole. pypdf read every value correctly from a file
|
||
that drew nothing.
|
||
|
||
### Mutation testing
|
||
|
||
Fourteen mutations. Eleven killed on the first attempt. **Three survived and
|
||
each one exposed a weak test**, which is the point of doing it:
|
||
|
||
| Mutation | First run | After strengthening |
|
||
|---|---|---|
|
||
| dict keys unescaped | survived — the test used an attachment *name*, which is written as a string, never as a key | killed, using a resource name |
|
||
| outline `/Count` sign dropped | survived — nothing read the open state | killed, `OutlineEntry::open` asserted |
|
||
| widgets get no `/P` | survived — `page_index` is supplied by the reader, which already knows the page | killed, asserting on `raw_dict` |
|
||
|
||
The `/P` case is worth keeping: an API that helpfully fills in a value
|
||
cannot witness that value missing from the file.
|
||
|
||
**Suite:** pdf **790 passed** (was 730), pdf-ui **775 passed**. Coverage
|
||
**85.17%** total; `create.rs` 94%, `writer.rs` 92%, `catalog.rs` 88%,
|
||
`subset.rs` 86%.
|
||
|
||
## Merge criteria
|
||
|
||
- [x] Page sizes honoured and asserted by reading them back
|
||
- [x] Dictionary keys escaped; a key with a delimiter round-trips
|
||
- [x] Non-finite numbers cannot corrupt a file
|
||
- [x] Outlines linked, nested, with the open/closed state preserved
|
||
- [x] Page labels: roman, letters, prefixes, start numbers
|
||
- [x] Named destinations resolve to their pages
|
||
- [x] Attachments round-trip byte-for-byte, including a zero byte
|
||
- [x] Viewer preferences, page mode and layout round-trip; explicit `false`
|
||
survives as `false`
|
||
- [x] Metadata round-trips, including a title containing `(`, `)` and `\`
|
||
- [x] TrueType subsetting verified against fontTools: 0 outline mismatches
|
||
- [x] Advances and bearings identical to the source for every glyph
|
||
- [x] Composite glyphs keep their components, renumbered
|
||
- [x] Subsets are byte-identical between runs
|
||
- [x] CFF refused by name
|
||
- [x] `/ToUnicode` emitted; bfchar sections capped at 100
|
||
- [x] All five field types created and read back with correct values
|
||
- [x] Radio groups are one field with per-option widgets
|
||
- [x] Every widget carries `/F` Print, `/P`, `/MK`, `/BS` and an appearance
|
||
- [x] Appearance streams declare the fonts they use
|
||
- [x] No object number is written twice
|
||
- [x] Verified by fontTools, pypdf and PDFium
|
||
- [x] 14 mutations; the 3 survivors each produced a stronger test
|
||
- [x] `TEST_TARGET=pdf` (790) and `pdf-ui` (775) green; fmt and clippy clean
|
||
|
||
## Consequences
|
||
|
||
**Positive.** nigig can now generate PDFs rather than only read them, and
|
||
the generated files are validated by three independent implementations.
|
||
Subsetting makes embedded-font output practical: 4 KB instead of 750 KB.
|
||
|
||
**Negative.** `PdfDocBuilder::finish` now carries a hand-managed object
|
||
numbering scheme, and two of the nine bugs were collisions in it. It is
|
||
documented in one block and every allocation goes through
|
||
`first_extra_object_number`, but it remains the fragile part of this work.
|
||
The `no_object_number_is_written_twice` test exists precisely because
|
||
review will not catch the next one.
|
||
|
||
**Risk.** The `/AP` appearances are generated by string formatting rather
|
||
than through `ContentWriter`, so they do not benefit from its escaping. They
|
||
are simple and fully covered, but a future field type should use the writer.
|
||
|
||
**Not done, deliberately:** CFF/Type1 embedding (refused by name), header
|
||
and footer helpers, image stamping (`add_page_resource` supports it, but
|
||
there is no convenience API), form field *reconciliation* against an
|
||
existing document, and encryption of generated files. The Phase 4 UI exit
|
||
criterion — creating a field in the Makepad UI and filling it via
|
||
`Selector::id(..).click()` — remains blocked on the headless backend that
|
||
has blocked every UI test since Phase 1.
|