Phase 8 #7 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md ("PDF/UA
structure tree"). Design and merge criteria in
REVIEWS/adr/0011-pdf-structure-tree.md.
Investigating the accessibility gap surfaced four defects in the
marked-content operators the structure tree depends on. The first is not
an accessibility problem at all.
1. BMC never parsed, and corrupted the colour of everything after it.
The dispatcher matched b'B'+b'M' only when the third byte was a
delimiter; BMC's third byte is 'C', so the arm never fired. Because
the operator was never recognised it never CLEARED ITS OPERAND, and
the leftover /Tag shifted the operands of whatever came next:
1 0 0 rg -> RgbFill(1.0, 0.0, 0.0) red
/Span BMC 1 0 0 rg -> RgbFill(0.0, 1.0, 0.0) green
Tagged documents are precisely the ones containing BMC, so the
documents that tried hardest to be accessible rendered wrong colours.
This is the fifth instance of the operator-shadowing family already
fixed for cm, rg, gs, b/b* and end-of-stream text operators. ADR
0009's every_multi_char_operator_parses_as_itself test exists to stop
exactly this - and would have caught it, except BMC was one of two
operators excluded from its table as "genuinely unimplemented".
Excluding a known-broken operator from the test whose job is finding
broken operators is how it survived. The exclusion list is gone.
2. BDC discarded its property list, which carries /MCID - the only link
between a run of page content and the structure element describing
it. Without it a tree can be parsed but never attached to anything.
3. The op produced by BDC was named MarkContentBmc, and BMC produced
nothing. The names were the wrong way round, which is how the missing
arm survived review: the enum looked like it had a BMC case.
4. Found while fixing 2: the content lexer had no dictionary support at
all. It read `<` as a hex string without checking for a second `<`,
so `<</MCID 0>>` parsed as the string "0C0D0". Content streams now
parse direct objects properly, bounded at 16 levels; a single `<` is
still a hex string and a test pins that.
On top of that, new pdf-document/src/structure.rs: /StructTreeRoot,
/StructElem trees, /RoleMap resolution, depth-first reading order,
/Alt, /ActualText, /E, /Lang, and MCID-to-element lookup. Cycles in /K
are cut at the first repeat and reported by object number rather than
expanded to the depth bound.
Accessibility findings are mechanical checks reported as findings, NOT a
conformance verdict: there is no is_pdf_ua and no ComplianceReport, and
a mutation-checked tripwire test fails if either appears. Real PDF/UA
conformance needs human judgement - whether /Alt text is accurate is not
mechanically decidable - so claiming it from six checks would be exactly
the overclaim this codebase keeps removing.
7 corpus fixtures, 14 acceptance tests, 22 unit tests, and a
parse_content_dict fuzz target for the new object parser.
TEST_TARGET=pdf 536 -> 575, TEST_TARGET=pdf-ui 580 -> 619.
rustfmt and clippy -D warnings clean.
Phase 8 #5 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md
("Advanced transparency"). Design and merge criteria in
REVIEWS/adr/0009-pdf-transparency.md.
The headline defect was not that transparency was missing. `gs` was
MIS-PARSED as CloseStroke:
/GS0 gs 1 0 0 rg 100 100 200 200 re f
-> [CloseStroke, RgbFill, Rectangle, FillWinding]
so every page setting a graphics state gained a stroked path the
document never asked for, and lost its alpha, blend mode and soft mask
silently. Downstream everything was dead: StrokeExtGState/FillExtGState
were never constructed, set_fill_opacity was never called and emitted no
command when it was, and PdfPage::ext_gstate was read by no code at all.
New pdf-graphics/src/transparency.rs: full /ExtGState (ca, CA, BM,
SMask, LW, LC, LJ, ML, D, AIS, TK), all sixteen blend modes including
the four non-separable ones, soft masks with /S, /G, /BC and a /TR
evaluated through ADR 0006's PdfFunction, and /Group parsing. Wired end
to end: parser -> PdfOp::SetExtGState -> device -> RenderCommand ->
Makepad renderer.
Compositing is NOT claimed. Backdrop blending needs render-to-texture,
which an engine-neutral crate has no framebuffer for. Constant alpha is
applied because it needs no backdrop; blend modes and soft masks are
reported through TransparencyError::Unsupported rather than dropped,
because a silently ignored /Multiply looks exactly like a correct
/Normal.
The ADR required a test enumerating every multi-character operator, on
the grounds that fixing the third instance of a shadowing bug (after cm
and rg) without preventing the fourth is not a fix. It immediately found
three more live bugs, none of them transparency-related:
- `b` and `b*` dropped their close-path, mapping to FillStroke* instead
of CloseFillStroke*, so every closed-and-stroked path drew with a gap.
- Text operators within three bytes of the end of a content stream were
mis-parsed: the b'T' arm guarded `*i + 3 < len` while reading only two
bytes, so a stream ending in `/F1 12 Tf` parsed as FillWinding.
And the transparency-group fixture found a fourth:
- PdfPage::xobjects was empty for every page of every document.
extract_xobjects called doc.resolve(), which follows the reference,
then asked the resolved object for as_ref() - always None - and only
accepted a bare dict when every XObject is a stream. No `Do` operator
could be resolved through the page model. Same defect as the one that
once destroyed annotation object references; it now has its own
regression test.
T* and BMC are genuinely unimplemented and are deliberately excluded
from the guard's table rather than papered over.
7 corpus fixtures, 12 acceptance tests, 32 unit tests asserting the
§11.3.5 formulas (not our own output), and a parse_ext_gstate fuzz
target.
TEST_TARGET=pdf 451 -> 497, TEST_TARGET=pdf-ui 495 -> 541.
rustfmt and clippy -D warnings clean.
Phase 8 #4 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md
("Advanced colour"). Design and merge criteria in
REVIEWS/adr/0006-pdf-advanced-color.md.
The interpreter tracked only the *name* of the active colour space and
then passed sc/scn operands to the device as if they were already RGBA.
Every non-device space therefore rendered a confident wrong colour with
no error:
/Spot cs 1.0 scn full tint of a spot ink -> pure red
/Idx cs 3 scn palette entry 3 -> near-black
/Lab cs 50 0 0 scn mid gray -> white (clamped)
/DevN cs (5 inks) five colorants -> inks 5+ discarded
/ICCBased profile-defined colour -> profile discarded
DeviceCMYK also used the additive 1-c-k conversion, which crushes any
colour printed over black.
Three new modules in pdf-graphics:
- function.rs PDF functions, all four types. Type 4 runs on a bounded
interpreter: depth 32, 32768 tokens, stack 100, 100000
steps, and an unknown operator is an error rather than a
no-op that would leave a plausible wrong colour.
- icc.rs ICC matrix/TRC and gray kTRC profiles, applied exactly.
LUT-class profiles are reported as such and the caller
falls back to /Alternate; they are never pretended to be
matrix profiles.
- colorspace.rs All eleven families, converting through XYZ with
Bradford adaptation and a real sRGB transfer function.
Wiring:
- PdfDevice gains set_stroke_components/set_fill_components, so SC/SCN
reach the device as components of the active space instead of being
read positionally as RGBA.
- cs/CS now resets to the space's initial colour (table 74), which is
why golden/colors.txt gains a line.
- PdfPage::color_spaces carries /Resources /ColorSpace fully
dereferenced with streams decoded; a half-resolved space would make
every ICC profile, palette and type 0/4 transform silently fall back.
- A space that cannot be resolved keeps the previous colour and records
a typed ColorError. No colour is invented, and no error is swallowed.
Tests: 12 corpus fixtures under tests/corpus/color/, 14 acceptance
tests in pdf-document/tests/color.rs asserting numeric RGB (the broken
code produced a colour for every one of these; only the value was
wrong), plus unit tests per function type and per curve type. Two fuzz
targets added: eval_function and parse_colorspace.
TEST_TARGET=pdf 387 -> 447 passing, TEST_TARGET=pdf-ui 431 -> 491.
rustfmt and clippy -D warnings clean.
Phase 5 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md.
Step 5.1, real metrics (new pdf-graphics/src/sfnt.rs):
- Reads head, hhea, OS/2 and hmtx from embedded TrueType/OpenType programs,
which the review asks for by name. sTypoAscender/Descender are preferred
over hhea when non-zero, since subset fonts often zero them; sCapHeight
and sxHeight are only read from OS/2 version 2 and later, because reading
them from a v1 table returns whatever bytes follow it.
- Every read is bounds-checked. A table pointing outside the file, a
truncated directory or a zero unitsPerEm yields None rather than a panic
or a divide by zero. These are untrusted embedded programs.
- resolve_font() now reads FontFile/FontFile2/FontFile3 and falls back to
the descriptors declared Ascent/Descent only when no program is embedded.
- renderer.rs looks up a per-font ascent instead of leaving ascent_em
permanently None, which had left the fallback ratio always in effect.
Step 5.2, accumulated advances (new pdf-graphics/src/advance.rs):
- measure_advance() implements PDF 32000-1 9.4.4 properly:
(w0/1000 * Tfs + Tc + Tw) * Th/100, with Tw restricted to single-byte
code 32. Applying Tw to a two-byte code whose low byte is 32 is a classic
composite-font bug and is now covered by a test.
- glyph_advances() gives per-glyph widths so a caret or a partial selection
rectangle no longer assumes even spacing.
- GlyphAdvanceCache is keyed by font name and size as the review specifies,
and carries a generation so it rebuilds when page data changes rather
than every frame. Redefining a font drops its stale measurements.
Step 5.3, composite fonts:
- The old build_cid_widths truncated every CID to u8, so any glyph above
255 silently took the default width. It is replaced by CidWidths keyed by
the real CID, with the CMap resolved from a predefined name or an
embedded stream.
- ResolvedFont::decode_text returns None for a composite font with no
ToUnicode map instead of guessing Latin-1 and producing plausible
nonsense.
Tests: new tests/phase5_exit_criterion.rs asserts the exit criterion.
Selection across a proportional and a monospaced run verifies the second
run starts at the first ones true end; per-glyph advances sum to the run
advance and H measures wider than l, which an even-spacing estimate would
not. Copy returns correct Unicode through ToUnicode including a non-ASCII
scalar, and through WinAnsi for simple fonts. Search finds both hits across
two lines with rectangles inside their own runs and the right vertical
order. Cache reuse is asserted through hit and miss counts.
Validation:
TEST_TARGET=pdf ./tools/test-rust-clean.sh (203 tests)
TEST_TARGET=pdf-ui ./tools/test-rust-clean.sh (238 tests)
Both rustfmt and clippy -D warnings clean.
Phase 2 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md. The exit
criterion is that a real content stream renders through the device path and
is verifiable; previously nothing could fail, so nothing was proven.
Step 2.3, the operations the review calls "the lies", all reached a dead end
in the interpreter. Each now reaches the device:
- Inline images: BI/ID/EI were emitted as two empty marker ops and the
payload was discarded, so an inline image could never be drawn. They are
now parsed into a single InlineImage op carrying the dictionary and bytes.
Abbreviated keys (/W /H /BPC /CS /F) and colour-space and filter
abbreviations are expanded. The EI scan requires delimiters on both sides
so binary data containing the bytes "EI" does not truncate the image, and
an unterminated image yields no image rather than invented pixels.
- Do (XObject) was an empty match arm. The interpreter cannot resolve a
resource name, so it now reports it through PdfDevice::paint_x_object and
the device performs the lookup.
- set_dash and set_miter_limit only mutated interpreter-local state and
emitted no command, so dashes never reached any renderer.
Three further defects surfaced while reviewing the generated goldens, all
of which silently corrupted output rather than failing:
- `cm` was never parsed at all. The single-character `m` arm matched first
and consumed it as a moveto, so every CTM change in every document was
lost and content drew at the wrong position. Two-character operators are
now tested before their one-character prefixes.
- `rg` and `RG` were shadowed by the `r` and `R` arms, so an RGB fill was
read as a single-component grey: `0.1 0.2 0.3 rg` produced 0.1 0.1 0.1.
- ImageInfo defaulted a missing /Filter to FlateDecode. An absent /Filter
means the data is stored raw, so every uncompressed image was undecodable.
Testing: adds format_commands(), a deterministic one-line-per-command text
form of a RenderCommand list, and eight golden files covering vector paths,
text, kerning and spacing, dash and stroke parameters, fill rules, inline
images, XObjects and colour operators. Floats are fixed-precision and
negative zero is normalised so no spurious diffs appear. UPDATE_GOLDEN=1
regenerates them for review.
Also fixes .gitignore: blanket *.txt and *.pdf rules were silently excluding
the golden expectations and the AcroForm fixture added in the previous
commit, which would have left both test suites unable to run on a fresh
clone.
Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh
132 tests pass; rustfmt and clippy -D warnings clean.
