nigig-org/crates/apps/pdf/pdf-graphics/tests/golden_render.rs
andodeki a994213e8b
Some checks failed
repo hygiene / hygiene (push) Has been cancelled
PDF engine / engine (push) Has been cancelled
PDF engine / makepad-integration (push) Has been cancelled
PDF engine / fuzz (push) Has been cancelled
feat(pdf): tagged structure tree, and the BMC bug that turned red green
Phase 8 #7 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md ("PDF/UA
structure tree"). Design and merge criteria in
REVIEWS/adr/0011-pdf-structure-tree.md.

Investigating the accessibility gap surfaced four defects in the
marked-content operators the structure tree depends on. The first is not
an accessibility problem at all.

1. BMC never parsed, and corrupted the colour of everything after it.
   The dispatcher matched b'B'+b'M' only when the third byte was a
   delimiter; BMC's third byte is 'C', so the arm never fired. Because
   the operator was never recognised it never CLEARED ITS OPERAND, and
   the leftover /Tag shifted the operands of whatever came next:

     1 0 0 rg            -> RgbFill(1.0, 0.0, 0.0)   red
     /Span BMC 1 0 0 rg  -> RgbFill(0.0, 1.0, 0.0)   green

   Tagged documents are precisely the ones containing BMC, so the
   documents that tried hardest to be accessible rendered wrong colours.

   This is the fifth instance of the operator-shadowing family already
   fixed for cm, rg, gs, b/b* and end-of-stream text operators. ADR
   0009's every_multi_char_operator_parses_as_itself test exists to stop
   exactly this - and would have caught it, except BMC was one of two
   operators excluded from its table as "genuinely unimplemented".
   Excluding a known-broken operator from the test whose job is finding
   broken operators is how it survived. The exclusion list is gone.

2. BDC discarded its property list, which carries /MCID - the only link
   between a run of page content and the structure element describing
   it. Without it a tree can be parsed but never attached to anything.

3. The op produced by BDC was named MarkContentBmc, and BMC produced
   nothing. The names were the wrong way round, which is how the missing
   arm survived review: the enum looked like it had a BMC case.

4. Found while fixing 2: the content lexer had no dictionary support at
   all. It read `<` as a hex string without checking for a second `<`,
   so `<</MCID 0>>` parsed as the string "0C0D0". Content streams now
   parse direct objects properly, bounded at 16 levels; a single `<` is
   still a hex string and a test pins that.

On top of that, new pdf-document/src/structure.rs: /StructTreeRoot,
/StructElem trees, /RoleMap resolution, depth-first reading order,
/Alt, /ActualText, /E, /Lang, and MCID-to-element lookup. Cycles in /K
are cut at the first repeat and reported by object number rather than
expanded to the depth bound.

Accessibility findings are mechanical checks reported as findings, NOT a
conformance verdict: there is no is_pdf_ua and no ComplianceReport, and
a mutation-checked tripwire test fails if either appears. Real PDF/UA
conformance needs human judgement - whether /Alt text is accurate is not
mechanically decidable - so claiming it from six checks would be exactly
the overclaim this codebase keeps removing.

7 corpus fixtures, 14 acceptance tests, 22 unit tests, and a
parse_content_dict fuzz target for the new object parser.

TEST_TARGET=pdf 536 -> 575, TEST_TARGET=pdf-ui 580 -> 619.
rustfmt and clippy -D warnings clean.
2026-07-31 22:19:45 +00:00

433 lines
15 KiB
Rust

//! Phase 2 golden tests: the interpreter's output for a given content stream
//! is captured as stable text and compared against a checked-in file.
//!
//! This is what makes "renders correctly" a claim that can fail. A change to
//! the interpreter that alters operator sequencing, advances or dropped
//! operations shows up as a diff instead of passing silently.
//!
//! Set `UPDATE_GOLDEN=1` to rewrite the expectations after an intentional
//! change, then review the diff before committing it.
use std::fs;
use std::path::PathBuf;
use nigig_pdf_graphics::content::{parse_content_stream, PdfOp};
use nigig_pdf_graphics::recording::{format_commands, RecordingDevice, RenderCommand};
fn golden_path(name: &str) -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/golden")
.join(format!("{name}.txt"))
}
/// Interpret `content` and compare the rendered command list against the
/// golden file for `name`.
fn assert_golden(name: &str, content: &[u8]) -> Vec<RenderCommand> {
let ops = parse_content_stream(content).expect("content stream should parse");
let commands = RecordingDevice::from_ops(&ops);
let actual = format_commands(&commands);
let path = golden_path(name);
if std::env::var("UPDATE_GOLDEN").is_ok() {
fs::create_dir_all(path.parent().expect("parent")).expect("create golden dir");
fs::write(&path, &actual).expect("write golden");
return commands;
}
let expected = fs::read_to_string(&path).unwrap_or_else(|_| {
panic!(
"missing golden file {}\nrerun with UPDATE_GOLDEN=1 to create it.\nactual output was:\n{actual}",
path.display()
)
});
assert_eq!(
actual,
expected,
"rendered commands for {name} do not match {}",
path.display()
);
commands
}
#[test]
fn golden_vector_path() {
assert_golden(
"vector_path",
b"q\n0.5 0 0 rg\n2 w\n100 200 m\n300 400 l\n350 450 400 500 450 550 c\nh\nB\nQ\n",
);
}
#[test]
fn golden_text_run() {
assert_golden(
"text_run",
b"BT\n/F1 12 Tf\n1 0 0 1 100 700 Tm\n(Hello World) Tj\nET\n",
);
}
#[test]
fn golden_text_with_kerning_and_spacing() {
// Exercises Tc/Tw/Tz and a TJ array with kerns both before and after a
// string. Every one of these was silently dropped before Phase 0.
assert_golden(
"text_kerning",
b"BT\n/F1 10 Tf\n0.5 Tc\n1.5 Tw\n90 Tz\n[(A) -120 (W) 50 (kern)] TJ\nET\n",
);
}
#[test]
fn golden_dash_and_stroke_params() {
assert_golden(
"dash_stroke",
b"[3 2] 1 d\n5 w\n1 J\n2 j\n8 M\n10 10 m\n200 10 l\nS\n",
);
}
#[test]
fn golden_rect_fill_rules() {
assert_golden(
"fill_rules",
b"10 20 100 50 re f\n10 20 100 50 re f*\n0 0 10 10 re W n\n",
);
}
#[test]
fn golden_inline_image() {
// BI/ID/EI with a tiny 2x2 grayscale payload.
assert_golden(
"inline_image",
b"q\nBI /W 2 /H 2 /BPC 8 /CS /G ID \x00\xFF\xFF\x00 EI\nQ\n",
);
}
#[test]
fn golden_xobject_and_graphics_state() {
assert_golden(
"xobject",
b"q\n1 0 0 1 50 60 cm\n/Im1 Do\nQ\nq\n/Fm0 Do\nQ\n",
);
}
#[test]
fn golden_cmyk_and_gray_colors() {
assert_golden(
"colors",
b"0.2 g\n0.8 G\n0.1 0.2 0.3 rg\n0.4 0.5 0.6 RG\n/DeviceCMYK cs\n0 0 0 1 sc\n",
);
}
/// The operations Step 2.3 lists as "the lies" must all reach the device.
#[test]
fn previously_dropped_operations_now_reach_the_device() {
let ops = parse_content_stream(b"[2 2] 0 d\nBI /W 1 /H 1 /BPC 8 /CS /G ID \xFF EI\n/Im1 Do\n")
.expect("parse");
let commands = RecordingDevice::from_ops(&ops);
assert!(
commands
.iter()
.any(|c| matches!(c, RenderCommand::SetDash(..))),
"set_dash was a TODO and never reached the device"
);
assert!(
commands
.iter()
.any(|c| matches!(c, RenderCommand::DrawInlineImage { .. })),
"inline images were silently dropped"
);
assert!(
commands
.iter()
.any(|c| matches!(c, RenderCommand::PaintXObject(_))),
"Do was a no-op"
);
}
#[test]
fn an_inline_image_carries_its_real_geometry_and_payload() {
let ops = parse_content_stream(b"BI /W 2 /H 3 /BPC 8 /CS /RGB ID abcdef EI\n").expect("parse");
let commands = RecordingDevice::from_ops(&ops);
let image = commands
.iter()
.find_map(|c| match c {
RenderCommand::DrawInlineImage {
width,
height,
color_space,
data,
..
} => Some((*width, *height, color_space.clone(), data.clone())),
_ => None,
})
.expect("an inline image command");
assert_eq!(image.0, 2, "abbreviated /W must be read");
assert_eq!(image.1, 3, "abbreviated /H must be read");
assert_eq!(image.2, "DeviceRGB", "/RGB must expand to DeviceRGB");
assert_eq!(image.3, b"abcdef", "the payload must survive parsing");
}
#[test]
fn binary_ei_bytes_inside_image_data_do_not_end_the_image_early() {
// "EI" appearing mid-payload without surrounding delimiters must not be
// mistaken for the terminator.
let ops = parse_content_stream(b"BI /W 4 /H 1 /BPC 8 /CS /G ID aEIb EI\nQ\n").expect("parse");
let commands = RecordingDevice::from_ops(&ops);
let data = commands
.iter()
.find_map(|c| match c {
RenderCommand::DrawInlineImage { data, .. } => Some(data.clone()),
_ => None,
})
.expect("inline image");
assert_eq!(data, b"aEIb", "payload truncated at an embedded EI");
}
#[test]
fn an_unterminated_inline_image_does_not_panic_or_emit_garbage() {
// Malformed input must degrade to "no image", never to invented pixels.
let ops = parse_content_stream(b"BI /W 2 /H 2 /BPC 8 /CS /G ID \x00\x01\x02").expect("parse");
let commands = RecordingDevice::from_ops(&ops);
assert!(!commands
.iter()
.any(|c| matches!(c, RenderCommand::DrawInlineImage { .. })));
}
#[test]
fn the_golden_format_is_deterministic_across_runs() {
let content = b"q\n1 0 0 rg\n10 20 30 40 re f\nBT /F1 9 Tf (x) Tj ET\nQ\n";
let render = || {
let ops = parse_content_stream(content).expect("parse");
format_commands(&RecordingDevice::from_ops(&ops))
};
assert_eq!(render(), render());
}
#[test]
fn negative_zero_does_not_produce_a_spurious_diff() {
let a = parse_content_stream(b"0 0 m\n").expect("parse");
let b = parse_content_stream(b"-0 -0 m\n").expect("parse");
assert_eq!(
format_commands(&RecordingDevice::from_ops(&a)),
format_commands(&RecordingDevice::from_ops(&b)),
"-0 and 0 must format identically"
);
}
/// Every multi-character operator must parse as *itself*, never as a
/// shorter operator that shares its prefix or suffix.
///
/// This bug has now been found three times in this codebase: `cm` swallowed
/// by `m`, `rg`/`RG` shadowed by `r`/`R`, and `gs` parsed as `CloseStroke`
/// (which silently injected a stroked path into every page using a graphics
/// state). Fixing the third without preventing the fourth is not a fix, so
/// this enumerates the operator table instead of spot-checking.
#[test]
fn every_multi_char_operator_parses_as_itself() {
// (operands + operator, a predicate identifying the correct PdfOp)
/// Content bytes, the operator's name, and a predicate identifying the
/// `PdfOp` it must produce.
type OperatorCase = (&'static [u8], &'static str, fn(&PdfOp) -> bool);
let cases: Vec<OperatorCase> = vec![
(b"1 0 0 1 0 0 cm\n", "cm", |o| {
matches!(o, PdfOp::ConcatTransform(..))
}),
(b"0.1 0.2 0.3 rg\n", "rg", |o| {
matches!(o, PdfOp::RgbFill(..))
}),
(b"0.1 0.2 0.3 RG\n", "RG", |o| {
matches!(o, PdfOp::RgbStroke(..))
}),
(b"0 0 0 1 k\n", "k", |o| matches!(o, PdfOp::CmykFill(..))),
(b"0 0 0 1 K\n", "K", |o| matches!(o, PdfOp::CmykStroke(..))),
(b"/GS0 gs\n", "gs", |o| matches!(o, PdfOp::SetExtGState(_))),
(b"BT\n", "BT", |o| matches!(o, PdfOp::BeginText)),
(b"ET\n", "ET", |o| matches!(o, PdfOp::EndText)),
(b"BT /F1 12 Tf ET\n", "Tf", |o| {
matches!(o, PdfOp::SetFont(..))
}),
(b"BT 10 20 Td ET\n", "Td", |o| {
matches!(o, PdfOp::NextLine(..))
}),
(b"BT 10 20 TD ET\n", "TD", |o| {
matches!(o, PdfOp::MoveTextSetLeading(..))
}),
(b"BT 1 0 0 1 0 0 Tm ET\n", "Tm", |o| {
matches!(o, PdfOp::TextMatrix(..))
}),
(b"BT (hi) Tj ET\n", "Tj", |o| {
matches!(o, PdfOp::ShowText(_))
}),
(b"BT [(hi)] TJ ET\n", "TJ", |o| {
matches!(o, PdfOp::ShowTextPositioned(_))
}),
(b"BT 14 TL ET\n", "TL", |o| {
matches!(o, PdfOp::SetLeading(_))
}),
(b"BT 2 Tc ET\n", "Tc", |o| {
matches!(o, PdfOp::SetCharSpacing(_))
}),
(b"BT 2 Tw ET\n", "Tw", |o| {
matches!(o, PdfOp::SetWordSpacing(_))
}),
(b"BT 90 Tz ET\n", "Tz", |o| {
matches!(o, PdfOp::SetHorizontalScale(_))
}),
(b"BT 3 Ts ET\n", "Ts", |o| {
matches!(o, PdfOp::SetTextRise(_))
}),
(b"BT 1 Tr ET\n", "Tr", |o| {
matches!(o, PdfOp::SetTextRendering(_))
}),
(b"/Im1 Do\n", "Do", |o| matches!(o, PdfOp::XObject(_))),
(b"/Sh0 sh\n", "sh", |o| matches!(o, PdfOp::Shading(_))),
(b"/DeviceRGB cs\n", "cs", |o| {
matches!(o, PdfOp::ColorSpaceFill(_))
}),
(b"/DeviceRGB CS\n", "CS", |o| {
matches!(o, PdfOp::ColorSpaceStroke(_))
}),
(b"0.5 sc\n", "sc", |o| matches!(o, PdfOp::FillColor(_))),
(b"0.5 SC\n", "SC", |o| matches!(o, PdfOp::StrokeColor(_))),
(b"0.5 scn\n", "scn", |o| matches!(o, PdfOp::FillColor(_))),
(b"0.5 SCN\n", "SCN", |o| matches!(o, PdfOp::StrokeColor(_))),
(b"W n\n", "W", |o| matches!(o, PdfOp::ClipWinding)),
(b"W* n\n", "W*", |o| matches!(o, PdfOp::ClipEvenOdd)),
(b"f*\n", "f*", |o| matches!(o, PdfOp::FillEvenOdd)),
(b"B*\n", "B*", |o| matches!(o, PdfOp::FillStrokeEvenOdd)),
(b"b*\n", "b*", |o| {
matches!(o, PdfOp::CloseFillStrokeEvenOdd)
}),
(b"b\n", "b", |o| matches!(o, PdfOp::CloseFillStrokeWinding)),
(b"/OC /MC0 BDC\n", "BDC", |o| {
matches!(o, PdfOp::BeginMarkedContentProps { .. })
}),
// BMC was excluded from this table as "genuinely unimplemented".
// It was not merely unimplemented: it failed to parse AND failed to
// clear its operand, so it shifted the operands of every following
// operator and made `1 0 0 rg` render green. Excluding a
// known-broken operator from the test whose purpose is to find
// broken operators is how it survived; the exclusion list is gone.
(b"/Span BMC\n", "BMC", |o| {
matches!(o, PdfOp::BeginMarkedContent(_))
}),
(b"EMC\n", "EMC", |o| matches!(o, PdfOp::EndMarkedContent)),
(b"100 200 d0\n", "d0", |o| {
matches!(o, PdfOp::Type3Width(..))
}),
(b"0 0 1 1 0 0 d1\n", "d1", |o| {
matches!(o, PdfOp::Type3BBox(..))
}),
];
for (content, name, matches_op) in cases {
let ops = parse_content_stream(content)
.unwrap_or_else(|e| panic!("{name}: content should parse: {e}"));
assert!(
ops.iter().any(matches_op),
"operator {name} did not parse as itself.\n input: {:?}\n got: {ops:?}\n\
This is the operator-shadowing bug that hit cm, rg and gs.",
String::from_utf8_lossy(content)
);
}
}
/// The specific regression: `gs` must not become a path-painting operator.
///
/// `/GS0 gs` previously produced `CloseStroke`, so every page that set a
/// graphics state gained a stroked path that the document never asked for.
#[test]
fn gs_does_not_inject_a_spurious_stroke() {
let ops = parse_content_stream(b"/GS0 gs\n1 0 0 rg\n100 100 200 200 re f\n").expect("parses");
assert!(
!ops.iter()
.any(|o| matches!(o, PdfOp::CloseStroke | PdfOp::Stroke)),
"gs was parsed as a stroke operator: {ops:?}"
);
assert!(
ops.iter()
.any(|o| matches!(o, PdfOp::SetExtGState(n) if n == "GS0")),
"gs should name its resource: {ops:?}"
);
// And it must not reach the device as a stroke either.
let cmds = RecordingDevice::from_ops(&ops);
assert!(
!cmds
.iter()
.any(|c| matches!(c, RenderCommand::Stroke | RenderCommand::ClosePath)),
"a spurious stroked path reached the device: {cmds:?}"
);
}
/// `BMC` did not merely fail to parse — it failed to *clear its operand*,
/// so the leftover `/Tag` shifted the operands of every operator after it.
/// A single `BMC` turned `1 0 0 rg` (red) into `RgbFill(0.0, 1.0, 0.0)`
/// (green), on exactly the tagged documents that contain `BMC`.
#[test]
fn bmc_does_not_shift_the_operands_of_following_operators() {
let baseline = parse_content_stream(b"1 0 0 rg\n").expect("parses");
let after_bmc = parse_content_stream(b"/Span BMC 1 0 0 rg\n").expect("parses");
let fill_of = |ops: &[PdfOp]| -> Option<(f64, f64, f64)> {
ops.iter().find_map(|o| match o {
PdfOp::RgbFill(r, g, b) => Some((*r, *g, *b)),
_ => None,
})
};
assert_eq!(fill_of(&baseline), Some((1.0, 0.0, 0.0)), "sanity");
assert_eq!(
fill_of(&after_bmc),
Some((1.0, 0.0, 0.0)),
"an unconsumed BMC operand shifted rg's operands: {after_bmc:?}"
);
}
/// `BDC` must keep its property list: `/MCID` is the only link between a
/// run of page content and the structure element describing it.
#[test]
fn bdc_retains_its_mcid() {
let ops = parse_content_stream(b"/P <</MCID 7>> BDC\n").expect("parses");
let mcid = ops.iter().find_map(|o| match o {
PdfOp::BeginMarkedContentProps { properties, .. } => {
properties.as_dict().and_then(|d| d.get_int("MCID"))
}
_ => None,
});
assert_eq!(mcid, Some(7), "the /MCID was discarded: {ops:?}");
// And it must reach the device, not just the op list.
let cmds = RecordingDevice::from_ops(&ops);
assert!(
cmds.iter()
.any(|c| matches!(c, RenderCommand::BeginMarkedContent { mcid: Some(7), .. })),
"the MCID never reached the device: {cmds:?}"
);
}
/// A content-stream dictionary operand must parse as a dictionary. The
/// lexer treated `<<` as the start of a hex string, so `<</MCID 0>>` became
/// the string "0C0D0" and every property list in every document was
/// unreadable.
#[test]
fn a_dictionary_operand_is_not_read_as_a_hex_string() {
let ops = parse_content_stream(b"/P <</MCID 1 /Lang (en)>> BDC\n").expect("parses");
let props = ops
.iter()
.find_map(|o| match o {
PdfOp::BeginMarkedContentProps { properties, .. } => Some(properties.clone()),
_ => None,
})
.expect("a BDC op");
let dict = props.as_dict().expect("properties must be a dictionary");
assert_eq!(dict.get_int("MCID"), Some(1));
// A single `<` must still be a hex string.
let hex = parse_content_stream(b"<48656C6C6F> Tj\n").expect("parses");
assert!(
hex.iter()
.any(|o| matches!(o, PdfOp::ShowText(t) if t == b"Hello")),
"hex strings regressed: {hex:?}"
);
}