Phase 8 #5 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md
("Advanced transparency"). Design and merge criteria in
REVIEWS/adr/0009-pdf-transparency.md.
The headline defect was not that transparency was missing. `gs` was
MIS-PARSED as CloseStroke:
/GS0 gs 1 0 0 rg 100 100 200 200 re f
-> [CloseStroke, RgbFill, Rectangle, FillWinding]
so every page setting a graphics state gained a stroked path the
document never asked for, and lost its alpha, blend mode and soft mask
silently. Downstream everything was dead: StrokeExtGState/FillExtGState
were never constructed, set_fill_opacity was never called and emitted no
command when it was, and PdfPage::ext_gstate was read by no code at all.
New pdf-graphics/src/transparency.rs: full /ExtGState (ca, CA, BM,
SMask, LW, LC, LJ, ML, D, AIS, TK), all sixteen blend modes including
the four non-separable ones, soft masks with /S, /G, /BC and a /TR
evaluated through ADR 0006's PdfFunction, and /Group parsing. Wired end
to end: parser -> PdfOp::SetExtGState -> device -> RenderCommand ->
Makepad renderer.
Compositing is NOT claimed. Backdrop blending needs render-to-texture,
which an engine-neutral crate has no framebuffer for. Constant alpha is
applied because it needs no backdrop; blend modes and soft masks are
reported through TransparencyError::Unsupported rather than dropped,
because a silently ignored /Multiply looks exactly like a correct
/Normal.
The ADR required a test enumerating every multi-character operator, on
the grounds that fixing the third instance of a shadowing bug (after cm
and rg) without preventing the fourth is not a fix. It immediately found
three more live bugs, none of them transparency-related:
- `b` and `b*` dropped their close-path, mapping to FillStroke* instead
of CloseFillStroke*, so every closed-and-stroked path drew with a gap.
- Text operators within three bytes of the end of a content stream were
mis-parsed: the b'T' arm guarded `*i + 3 < len` while reading only two
bytes, so a stream ending in `/F1 12 Tf` parsed as FillWinding.
And the transparency-group fixture found a fourth:
- PdfPage::xobjects was empty for every page of every document.
extract_xobjects called doc.resolve(), which follows the reference,
then asked the resolved object for as_ref() - always None - and only
accepted a bare dict when every XObject is a stream. No `Do` operator
could be resolved through the page model. Same defect as the one that
once destroyed annotation object references; it now has its own
regression test.
T* and BMC are genuinely unimplemented and are deliberately excluded
from the guard's table rather than papered over.
7 corpus fixtures, 12 acceptance tests, 32 unit tests asserting the
§11.3.5 formulas (not our own output), and a parse_ext_gstate fuzz
target.
TEST_TARGET=pdf 451 -> 497, TEST_TARGET=pdf-ui 495 -> 541.
rustfmt and clippy -D warnings clean.
Phase 8 #4 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md
("Advanced colour"). Design and merge criteria in
REVIEWS/adr/0006-pdf-advanced-color.md.
The interpreter tracked only the *name* of the active colour space and
then passed sc/scn operands to the device as if they were already RGBA.
Every non-device space therefore rendered a confident wrong colour with
no error:
/Spot cs 1.0 scn full tint of a spot ink -> pure red
/Idx cs 3 scn palette entry 3 -> near-black
/Lab cs 50 0 0 scn mid gray -> white (clamped)
/DevN cs (5 inks) five colorants -> inks 5+ discarded
/ICCBased profile-defined colour -> profile discarded
DeviceCMYK also used the additive 1-c-k conversion, which crushes any
colour printed over black.
Three new modules in pdf-graphics:
- function.rs PDF functions, all four types. Type 4 runs on a bounded
interpreter: depth 32, 32768 tokens, stack 100, 100000
steps, and an unknown operator is an error rather than a
no-op that would leave a plausible wrong colour.
- icc.rs ICC matrix/TRC and gray kTRC profiles, applied exactly.
LUT-class profiles are reported as such and the caller
falls back to /Alternate; they are never pretended to be
matrix profiles.
- colorspace.rs All eleven families, converting through XYZ with
Bradford adaptation and a real sRGB transfer function.
Wiring:
- PdfDevice gains set_stroke_components/set_fill_components, so SC/SCN
reach the device as components of the active space instead of being
read positionally as RGBA.
- cs/CS now resets to the space's initial colour (table 74), which is
why golden/colors.txt gains a line.
- PdfPage::color_spaces carries /Resources /ColorSpace fully
dereferenced with streams decoded; a half-resolved space would make
every ICC profile, palette and type 0/4 transform silently fall back.
- A space that cannot be resolved keeps the previous colour and records
a typed ColorError. No colour is invented, and no error is swallowed.
Tests: 12 corpus fixtures under tests/corpus/color/, 14 acceptance
tests in pdf-document/tests/color.rs asserting numeric RGB (the broken
code produced a colour for every one of these; only the value was
wrong), plus unit tests per function type and per curve type. Two fuzz
targets added: eval_function and parse_colorspace.
TEST_TARGET=pdf 387 -> 447 passing, TEST_TARGET=pdf-ui 431 -> 491.
rustfmt and clippy -D warnings clean.
Phase 7 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md, taken only now
because the review is explicit that it comes after correctness is proven.
The caching and threading live in pdf-graphics rather than the Makepad crate
because none of it needs a GPU. That is what lets the staleness, eviction and
cancellation rules be tested without a window; the widget keeps only the
parts that genuinely need a Cx.
Step 7.1, off-thread parsing (new pdf-graphics/src/worker.rs):
- RenderWorker interprets content streams on a background thread and returns
results tagged with the Generation they were requested for.
- PendingPages tracks in-flight pages so the widget can draw a placeholder
and never queues the same page twice.
- Drop joins the thread rather than detaching it: a detached thread writing
into a dropped channel is the kind of shutdown race that surfaces as a
flaky test months later.
- A content stream that fails to parse yields an empty page, so one broken
page cannot take down the document.
Step 7.2, texture and memory management (new pdf-graphics/src/cache.rs):
- PageCache is an LRU keyed by page index with a byte budget, not an entry
count: one image-heavy page can outweigh fifty text pages, so counting
entries would evict the wrong things.
- retain_around() releases pages that scrolled out of view, keeping a margin
so a small scroll does not immediately re-render.
- Decoding produces DecodedImage bytes off-thread; GPU upload stays on the
UI thread.
- A page larger than the whole budget is still stored, since refusing it
would mean re-rendering it every frame.
Step 7.3, render command cache:
- CachedPage holds the interpreted Vec<RenderCommand>, so replaying a page
skips re-parsing its content stream.
- invalidate_appearance() marks a form edit dirty without discarding the
commands, because a form edit changes what is drawn over the page, not the
page content stream.
Widget wiring: PdfPageWidget carries a generation, refuses PageContent from a
superseded document, and draws a placeholder while a page is still rendering.
Exit criterion (new pdf-document/tests/phase7_exit_criterion.rs, 9 tests)
against a new 60-page corpus fixture. Measured here: first page 3ms against
the 200ms budget, and a warm cache read 389x faster than re-parsing (1us vs
389us). The budget is deliberately loose because a shared CI runner is
unpredictable and a flaky performance test gets muted, and a muted test is
worse than none; it still catches the order-of-magnitude regression the
review is guarding against. Memory is asserted bounded across 30 document
switches, and every page of an abandoned document is refused.
Validation:
TEST_TARGET=pdf ./tools/test-rust-clean.sh (270 tests)
TEST_TARGET=pdf-ui ./tools/test-rust-clean.sh (307 tests)
Both rustfmt and clippy -D warnings clean.
