nigig-org/REVIEWS/adr/0009-pdf-transparency.md

ADR 0009: PDF transparency — ExtGState, blend modes and soft masks

• Status: Accepted
• Date: 2026-07-31
• Review item: Phase 8, DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md ("Advanced transparency (blend modes, soft masks, groups)")
• Supersedes: nothing
• Related: ADR 0006 (advanced colour) — its function evaluator is the prerequisite for soft-mask transfer functions, and its ColorSpace is what a luminosity mask's backdrop is measured in

Context

The gs operator is not merely unimplemented. It is mis-parsed into a different operator, and the one it produces draws.

The content-stream dispatcher matches b's' for the close-and-stroke operator before anything matches gs, and its b'g' arm requires a delimiter, so gs never reaches it. Probing /GS0 gs 1 0 0 rg 100 100 200 200 re f through the current parser:

ops parsed: 4
  CloseStroke                          <- /GS0 gs became this
  RgbFill(1.0, 0.0, 0.0)
  Rectangle(100.0, 100.0, 200.0, 200.0)
  FillWinding

So every page that sets a graphics state — which is most pages with any transparency, and a great many without — gets a spurious stroked path injected at that point. This is the third instance of the same defect class already fixed twice in this codebase: cm swallowed by m, and rg/RG shadowed by r/R. Single-character operators were matched before their two-character extensions.

Downstream of that, the rest of the transparency path is dead code:

• PdfOp::StrokeExtGState and PdfOp::FillExtGState exist in the enum and are never constructed and never interpreted.
• PdfDevice::set_stroke_opacity / set_fill_opacity exist and are never called by the interpreter. RecordingDevice's implementations mutate current_state but emit no RenderCommand, so even a direct call would be invisible to any renderer.
• PdfPage::ext_gstate is parsed out of /Resources /ExtGState into an ExtGStateResource { ca, ca_lower } that no code anywhere reads. It also captures only the two alpha constants, discarding /BM, /SMask, /LW, /LC, /LJ, /D, /Font and the rest.

The net effect: ca/CA are silently ignored, so a watermark at 10% alpha renders fully opaque; blend modes are silently ignored, so a /Multiply highlight paints over the text it should tint; and soft masks are silently ignored, so a gradient-masked image paints as a hard rectangle. Each is a confident wrong result with no error — the defect class rule 6 exists to prevent.

Decision

Implement the transparency model of PDF 32000-1 §11 that can be computed correctly on the CPU, in pdf-graphics, and be explicit about the boundary where it stops.

Fix the parser bug first, and generalise the guard

gs gets its own arm. More importantly, the recurring shadowing bug gets a test that enumerates every multi-character operator in the spec and asserts each parses to itself rather than to a prefix of itself. Fixing the third instance of a bug without preventing the fourth is not a fix.

ExtGState, in full

/ExtGState is parsed into a real structure carrying ca, CA, BM, SMask, LW, LC, LJ, ML, D, AIS and TK, and gs applies all of them to the graphics state. The entries that duplicate existing operators (LW, LC, LJ, ML, D) are applied through the same device calls those operators use, so there is one code path per effect.

Blend modes

All sixteen separable and non-separable modes of §11.3.5: Normal, Multiply, Screen, Overlay, Darken, Lighten, ColorDodge, ColorBurn, HardLight, SoftLight, Difference, Exclusion, Hue, Saturation, Color and Luminosity. These are pure functions on backdrop and source colour, fully specified and exactly testable, so they are implemented and unit-tested against the formulas rather than approximated.

An unrecognised blend-mode name falls back to Normal — which is what the specification itself requires (§11.3.5.2: "if the name is not recognised, it shall be treated as Normal") — and is recorded so the caller knows.

Soft masks

/SMask in an ExtGState is parsed into its /S subtype (/Alpha or /Luminosity), its /G transparency group, its /BC backdrop colour and its /TR transfer function. The transfer function is evaluated with ADR 0006's PdfFunction, which is why that ADR was the prerequisite for this one. /None clears the mask.

Transparency groups

/Group dictionaries on Form XObjects and pages are parsed: /S /Transparency, /CS (through ADR 0006's colour spaces), /I isolated and /K knockout.

What is emitted, and what is not

The interpreter emits typed render commands — SetBlendMode, SetSoftMask, ClearSoftMask, BeginTransparencyGroup, EndTransparencyGroup, SetFillAlpha, SetStrokeAlpha — carrying fully resolved parameters.

Actual GPU compositing is not implemented here and is not claimed. pdf-graphics is engine-neutral and has no framebuffer; performing real backdrop compositing requires render-to-texture in the Makepad layer. What this ADR delivers is that the information reaches the device correctly and completely instead of being dropped or mis-parsed, plus a CPU implementation of every blend function that a compositor can call. The Makepad renderer applies constant alpha immediately, because it can, and reports the rest as unsupported rather than pretending.

That boundary is stated in the code, not just here: RenderCommands a device cannot honour are surfaced through a typed TransparencyError::Unsupported list rather than silently ignored.

Non-negotiable rules

1. No operator may be parsed as a different operator. The multi-char operator test enumerates the spec's table and fails on any shadowing.
2. No dead integration points. Anything added here is constructed, interpreted, emitted and asserted end to end. This ADR exists partly because the previous transparency scaffolding was never wired.
3. Unsupported compositing is reported, never silently skipped. A blend mode the renderer cannot execute is named in a typed error.
4. Blend functions match the specification's formulas, tested against values computed from §11.3.5, not against our own output.
5. A soft mask that cannot be resolved does not become "no mask" silently; it is reported.

Merge criteria

• gs parses as gs and no longer produces a spurious CloseStroke.
• A regression test enumerates every multi-character operator and asserts none is shadowed by a shorter one.
• /ExtGState is parsed in full and gs applies it to the state.
• ca and CA reach the device and are observable in the command list.
• All sixteen blend modes are implemented and unit-tested against the specification's formulas, including the four non-separable ones.
• An unknown blend mode falls back to Normal and is reported.
• /SMask is parsed with subtype, group, backdrop and transfer function; /None clears it.
• A soft-mask transfer function is evaluated through ADR 0006's PdfFunction.
• /Group dictionaries are parsed with colour space, isolated and knockout flags.
• Corpus fixtures for constant alpha, each blend-mode family, a luminosity soft mask and a transparency group, generated by the checked-in script.
• Operations the renderer cannot composite are reported as typed unsupported entries, not dropped.
• TEST_TARGET=pdf ./tools/test-rust-clean.sh passes, rustfmt and clippy -D warnings clean.

All criteria met. TEST_TARGET=pdf went from 451 to 497 and TEST_TARGET=pdf-ui from 495 to 541.

What the operator-shadowing guard found

Writing the enumerating test was the highest-value part of this work. It immediately failed on operators unrelated to transparency, each a live bug:

• b and b* dropped their close-path. Both mapped to FillStroke* instead of CloseFillStroke*, so every closed-and-stroked path was drawn with a visible gap.
• Text operators within three bytes of the end of a content stream were mis-parsed. The b'T' arm guarded *i + 3 < data.len() when it reads only two bytes, so a stream ending in /F1 12 Tf parsed as FillWinding — a font selection silently became a fill. The inner arms also rejected end-of-stream as a delimiter.

A third bug was found by the transparency-group fixture rather than the guard:

• PdfPage::xobjects was empty for every page of every document. extract_xobjects called doc.resolve(), which follows the reference, then asked the resolved object for as_ref(), which is always None. It also only accepted a bare dict, while every XObject is a stream. So no Do operator could ever be resolved through the page model. This is the same defect that once destroyed annotation object references, and it now has its own regression test.

T* and BMC are genuinely unimplemented and are deliberately not in the guard's table; they are recorded here rather than papered over.

Consequences

Positive. The spurious-stroke bug disappears from every page using gs, which is a correctness fix well beyond transparency. Constant alpha works. The blend functions are exact and reusable by any future compositor. The soft-mask and group data is available for the GPU work rather than needing a second parsing pass.

Negative. RenderCommand grows, and devices must handle the new variants. The alternative — leaving them out — is what produced the dead hooks this ADR is cleaning up.

Risk. The temptation to claim "transparency is done" when compositing is not. Mitigated by the explicit boundary above, by the typed unsupported reporting, and by not ticking a criterion this ADR did not set.

Out of scope, deliberately: GPU backdrop compositing and render-to-texture; knockout and isolation semantics during compositing (the flags are parsed and reported, not enforced); spot-colour overprint and /OPM; /TK text knockout behaviour; and blend modes applied to images rather than to fills and strokes.