nigig-org/REVIEWS/adr/0009-pdf-transparency.md

# ADR 0009: PDF transparency — ExtGState, blend modes and soft masks

- **Status:** Accepted
- **Date:** 2026-07-31
- **Review item:** Phase 8, `DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md`
  ("Advanced transparency (blend modes, soft masks, groups)")
- **Supersedes:** nothing
- **Related:** ADR 0006 (advanced colour) — its function evaluator is the
  prerequisite for soft-mask transfer functions, and its `ColorSpace` is
  what a luminosity mask's backdrop is measured in

## Context

The `gs` operator is not merely unimplemented. **It is mis-parsed into a
different operator**, and the one it produces draws.

The content-stream dispatcher matches `b's'` for the close-and-stroke
operator before anything matches `gs`, and its `b'g'` arm requires a
delimiter, so `gs` never reaches it. Probing `/GS0 gs 1 0 0 rg 100 100 200
200 re f` through the current parser:

```
ops parsed: 4
  CloseStroke                          <- /GS0 gs became this
  RgbFill(1.0, 0.0, 0.0)
  Rectangle(100.0, 100.0, 200.0, 200.0)
  FillWinding
```

So every page that sets a graphics state — which is most pages with any
transparency, and a great many without — gets a **spurious stroked path**
injected at that point. This is the third instance of the same defect class
already fixed twice in this codebase: `cm` swallowed by `m`, and `rg`/`RG`
shadowed by `r`/`R`. Single-character operators were matched before their
two-character extensions.

Downstream of that, the rest of the transparency path is dead code:

- `PdfOp::StrokeExtGState` and `PdfOp::FillExtGState` exist in the enum and
  **are never constructed and never interpreted**.
- `PdfDevice::set_stroke_opacity` / `set_fill_opacity` exist and are **never
  called by the interpreter**. `RecordingDevice`'s implementations mutate
  `current_state` but emit no `RenderCommand`, so even a direct call would
  be invisible to any renderer.
- `PdfPage::ext_gstate` is parsed out of `/Resources /ExtGState` into an
  `ExtGStateResource { ca, ca_lower }` that **no code anywhere reads**. It
  also captures only the two alpha constants, discarding `/BM`, `/SMask`,
  `/LW`, `/LC`, `/LJ`, `/D`, `/Font` and the rest.

The net effect: `ca`/`CA` are silently ignored, so a watermark at 10% alpha
renders fully opaque; blend modes are silently ignored, so a `/Multiply`
highlight paints over the text it should tint; and soft masks are silently
ignored, so a gradient-masked image paints as a hard rectangle. Each is a
confident wrong result with no error — the defect class rule 6 exists to
prevent.

## Decision

Implement the transparency model of PDF 32000-1 §11 that can be computed
correctly on the CPU, in `pdf-graphics`, and be explicit about the boundary
where it stops.

### Fix the parser bug first, and generalise the guard

`gs` gets its own arm. More importantly, the recurring shadowing bug gets a
test that enumerates **every** multi-character operator in the spec and
asserts each parses to itself rather than to a prefix of itself. Fixing the
third instance of a bug without preventing the fourth is not a fix.

### `ExtGState`, in full

`/ExtGState` is parsed into a real structure carrying `ca`, `CA`, `BM`,
`SMask`, `LW`, `LC`, `LJ`, `ML`, `D`, `AIS` and `TK`, and `gs` applies all
of them to the graphics state. The entries that duplicate existing
operators (`LW`, `LC`, `LJ`, `ML`, `D`) are applied through the same device
calls those operators use, so there is one code path per effect.

### Blend modes

All sixteen separable and non-separable modes of §11.3.5: Normal,
Multiply, Screen, Overlay, Darken, Lighten, ColorDodge, ColorBurn,
HardLight, SoftLight, Difference, Exclusion, Hue, Saturation, Color and
Luminosity. These are pure functions on backdrop and source colour, fully
specified and exactly testable, so they are implemented and unit-tested
against the formulas rather than approximated.

An unrecognised blend-mode name falls back to Normal — which is what the
specification itself requires (§11.3.5.2: "if the name is not recognised, it
shall be treated as Normal") — and is recorded so the caller knows.

### Soft masks

`/SMask` in an ExtGState is parsed into its `/S` subtype (`/Alpha` or
`/Luminosity`), its `/G` transparency group, its `/BC` backdrop colour and
its `/TR` transfer function. The transfer function is evaluated with ADR
0006's `PdfFunction`, which is why that ADR was the prerequisite for this
one. `/None` clears the mask.

### Transparency groups

`/Group` dictionaries on Form XObjects and pages are parsed: `/S
/Transparency`, `/CS` (through ADR 0006's colour spaces), `/I` isolated and
`/K` knockout.

### What is emitted, and what is not

The interpreter emits typed render commands — `SetBlendMode`,
`SetSoftMask`, `ClearSoftMask`, `BeginTransparencyGroup`,
`EndTransparencyGroup`, `SetFillAlpha`, `SetStrokeAlpha` — carrying fully
resolved parameters.

**Actual GPU compositing is not implemented here and is not claimed.**
`pdf-graphics` is engine-neutral and has no framebuffer; performing real
backdrop compositing requires render-to-texture in the Makepad layer. What
this ADR delivers is that the information reaches the device *correctly and
completely* instead of being dropped or mis-parsed, plus a CPU
implementation of every blend function that a compositor can call. The
Makepad renderer applies constant alpha immediately, because it can, and
reports the rest as unsupported rather than pretending.

That boundary is stated in the code, not just here: `RenderCommand`s a
device cannot honour are surfaced through a typed
`TransparencyError::Unsupported` list rather than silently ignored.

## Non-negotiable rules

1. **No operator may be parsed as a different operator.** The multi-char
   operator test enumerates the spec's table and fails on any shadowing.
2. **No dead integration points.** Anything added here is constructed,
   interpreted, emitted and asserted end to end. This ADR exists partly
   because the previous transparency scaffolding was never wired.
3. **Unsupported compositing is reported, never silently skipped.** A blend
   mode the renderer cannot execute is named in a typed error.
4. **Blend functions match the specification's formulas**, tested against
   values computed from §11.3.5, not against our own output.
5. **A soft mask that cannot be resolved does not become "no mask"**
   silently; it is reported.

## Merge criteria

- [x] `gs` parses as `gs` and no longer produces a spurious `CloseStroke`.
- [x] A regression test enumerates every multi-character operator and
      asserts none is shadowed by a shorter one.
- [x] `/ExtGState` is parsed in full and `gs` applies it to the state.
- [x] `ca` and `CA` reach the device and are observable in the command list.
- [x] All sixteen blend modes are implemented and unit-tested against the
      specification's formulas, including the four non-separable ones.
- [x] An unknown blend mode falls back to Normal and is reported.
- [x] `/SMask` is parsed with subtype, group, backdrop and transfer
      function; `/None` clears it.
- [x] A soft-mask transfer function is evaluated through ADR 0006's
      `PdfFunction`.
- [x] `/Group` dictionaries are parsed with colour space, isolated and
      knockout flags.
- [x] Corpus fixtures for constant alpha, each blend-mode family, a
      luminosity soft mask and a transparency group, generated by the
      checked-in script.
- [x] Operations the renderer cannot composite are reported as typed
      unsupported entries, not dropped.
- [x] `TEST_TARGET=pdf ./tools/test-rust-clean.sh` passes, rustfmt and
      clippy `-D warnings` clean.

All criteria met. `TEST_TARGET=pdf` went from 451 to 497 and
`TEST_TARGET=pdf-ui` from 495 to 541.

### What the operator-shadowing guard found

Writing the enumerating test was the highest-value part of this work. It
immediately failed on operators unrelated to transparency, each a live bug:

- **`b` and `b*` dropped their close-path.** Both mapped to
  `FillStroke*` instead of `CloseFillStroke*`, so every closed-and-stroked
  path was drawn with a visible gap.
- **Text operators within three bytes of the end of a content stream were
  mis-parsed.** The `b'T'` arm guarded `*i + 3 < data.len()` when it reads
  only two bytes, so a stream ending in `/F1 12 Tf` parsed as
  `FillWinding` — a font selection silently became a fill. The inner arms
  also rejected end-of-stream as a delimiter.

A third bug was found by the transparency-group fixture rather than the
guard:

- **`PdfPage::xobjects` was empty for every page of every document.**
  `extract_xobjects` called `doc.resolve()`, which follows the reference,
  then asked the *resolved* object for `as_ref()`, which is always `None`.
  It also only accepted a bare dict, while every XObject is a stream. So
  no `Do` operator could ever be resolved through the page model. This is
  the same defect that once destroyed annotation object references, and it
  now has its own regression test.

`T*` and `BMC` are genuinely unimplemented and are deliberately not in the
guard's table; they are recorded here rather than papered over.

## Consequences

**Positive.** The spurious-stroke bug disappears from every page using
`gs`, which is a correctness fix well beyond transparency. Constant alpha
works. The blend functions are exact and reusable by any future
compositor. The soft-mask and group data is available for the GPU work
rather than needing a second parsing pass.

**Negative.** `RenderCommand` grows, and devices must handle the new
variants. The alternative — leaving them out — is what produced the dead
hooks this ADR is cleaning up.

**Risk.** The temptation to claim "transparency is done" when compositing
is not. Mitigated by the explicit boundary above, by the typed unsupported
reporting, and by not ticking a criterion this ADR did not set.

**Out of scope, deliberately:** GPU backdrop compositing and
render-to-texture; knockout and isolation *semantics* during compositing
(the flags are parsed and reported, not enforced); spot-colour overprint
and `/OPM`; `/TK` text knockout behaviour; and blend modes applied to
images rather than to fills and strokes.