nigig-org/crates/apps/pdf/pdf-document/tests/declared_resources.rs
andodeki cf73ef4c1d
Some checks failed
repo hygiene / hygiene (push) Has been cancelled
PDF engine / engine (push) Has been cancelled
PDF engine / makepad-integration (push) Has been cancelled
PDF engine / fuzz (push) Has been cancelled
test(pdf): assert what a file declares is delivered, and floor the coverage
Every serious bug in this stack has had one shape: a valid, well-typed,
empty-or-default value where the file plainly declared content. xobjects
empty for every document; acroform() dropping every field behind an
indirect reference; DCTDecode returning its own compressed bytes; a JPEG
decoder that was a stub returning black. None errored, none panicked, and
the tests asserted Ok, which they got.

Coverage would not have caught any of them. Measured when each shipped:
page.rs 92.4%, form.rs 93.6%, content.rs 89.2%, xref.rs 95.2%. The buggy
lines ran; nobody checked what they produced.

So: a property test that walks the raw object graph of every corpus
fixture, counts what the file declares, and requires the API to deliver
it - fonts, xobjects, graphics states, colour spaces, form fields,
filters, MediaBox. It reimplements the resolution rule independently of
page.rs on purpose; a test that asks the code under test what to expect
agrees with the bug.

It failed the day it was written, on a shape the corpus had never
contained. Every fixture wrote /Resources inline, and all six extractors
read it with dict.get_dict("Resources") - which returns None for an
indirect reference and never consulted /Parent. A page with
"/Resources 5 0 R", the commonest shape in real PDFs, reported no fonts,
no xobjects, no graphics states and no colour spaces. Same for a page
inheriting resources from its /Pages node. Empty, not wrong, so nothing
failed.

Fixed by resolving /Resources once in PdfPage::from_obj through a helper
implementing the full inheritance rule (32000-1 Table 30), and passing
the resolved dictionary down. Indirect /MediaBox entries resolve too.
Six resources/ fixtures cover the shapes that were missing.

Mutation-checked: reverting inheritance kills 5 tests, the sub-dict
reference 3, indirect MediaBox 2, and removing the depth bound hangs.
One mutation survived - a visited-set guarding a /Parent cycle, which
the depth bound already handles - so it was deleted rather than left as
untested defence with a reassuring comment.

tools/test-pdf-coverage.sh enforces a floor instead of printing a number,
with per-file floors as well as a total: image.rs could fall from 33% to
5% and move the total by under a point. All three failure modes verified
to fail. It caught a bug in itself first - its ignore regex matched its
own work directory and reported a confident TOTAL 0.00%.

.gitattributes marks *.pdf binary. An xref entry must be exactly 20 bytes
(7.5.4), so with a one-digit generation field it ends in a space, and
git diff --check was reporting unfixable "trailing whitespace" on every
fixture in the corpus.

TEST_TARGET=pdf: 695 passed, 0 failed (was 680). Coverage 83.42%.
ADR 0017 records the four mutations so they can be repeated by hand.
2026-08-16 19:04:57 +00:00

628 lines
22 KiB
Rust

//! What a document *declares*, it must *deliver*.
//!
//! Every "silently empty" bug this crate has shipped had the same shape: a
//! lookup returned an empty collection where the file plainly declared
//! entries, and because empty is a valid answer for a document that really
//! has none, nothing failed. `xobjects` was empty for every document,
//! `acroform()` dropped every field behind an indirect reference, and
//! `/Resources` reached by reference or inheritance produced no fonts at
//! all. Four separate bugs, one invariant.
//!
//! So this file does not test a function. It walks the raw object graph of
//! every corpus fixture, counts what the *file* declares, and requires the
//! API to deliver the same count. A future extractor that quietly returns
//! nothing fails here on the day it is written, without anybody thinking to
//! add a test for it.
//!
//! See `REVIEWS/adr/0017-pdf-declared-versus-delivered.md`.
use std::collections::HashSet;
use std::path::PathBuf;
use nigig_pdf_cos::filter::{decode_stream, REFUSED_CODECS};
use nigig_pdf_cos::{ObjRef, PdfDict, PdfObj};
use nigig_pdf_document::PdfDocument;
fn corpus_dir() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../tests/corpus")
}
/// Every fixture in the corpus, so a new one is covered automatically.
fn all_fixtures() -> Vec<PathBuf> {
let mut found = Vec::new();
let mut stack = vec![corpus_dir()];
while let Some(dir) = stack.pop() {
let Ok(entries) = std::fs::read_dir(&dir) else {
continue;
};
for entry in entries.flatten() {
let path = entry.path();
if path.is_dir() {
stack.push(path);
} else if path.extension().is_some_and(|e| e == "pdf") {
found.push(path);
}
}
}
found.sort();
found
}
/// Fixtures that are deliberately broken, where "declares X but delivers
/// nothing" is the correct behaviour rather than a bug.
///
/// Listed by directory, not by guesswork: everything under `malformed/` is
/// damaged on purpose. Nothing else is exempt, and adding an exemption is a
/// visible edit to this list.
fn is_deliberately_broken(path: &std::path::Path) -> bool {
path.components()
.any(|c| c.as_os_str() == "malformed" || c.as_os_str() == "encrypted")
}
/// Resolve `key` on a page dictionary the way the spec says to: follow an
/// indirect reference, and inherit from `/Parent` when absent.
///
/// Deliberately a *second, independent* implementation of the rule that
/// `page.rs` implements. A test that reuses the code under test to work out
/// the expected answer agrees with the bug.
fn declared_inherited(doc: &mut PdfDocument, start: &PdfDict, key: &str) -> Option<PdfObj> {
let mut current = start.clone();
let mut seen = HashSet::new();
for _ in 0..64 {
if let Some(v) = current.get(key) {
return match v {
PdfObj::Ref(r) => doc.resolve_ref(*r).ok(),
other => Some(other.clone()),
};
}
let parent = current.get_ref("Parent")?;
if !seen.insert(parent.num) {
return None;
}
current = doc.resolve_ref(parent).ok()?.as_dict()?.clone();
}
None
}
/// The names a page declares under `/Resources /<key>`.
fn declared_resource_names(doc: &mut PdfDocument, page: &PdfDict, key: &str) -> Vec<String> {
let Some(resources) = declared_inherited(doc, page, "Resources") else {
return Vec::new();
};
let Some(resources) = resources.as_dict() else {
return Vec::new();
};
let sub = match resources.get(key) {
Some(PdfObj::Dict(d)) => d.clone(),
Some(PdfObj::Ref(r)) => match doc.resolve_ref(*r).ok().and_then(|o| o.as_dict().cloned()) {
Some(d) => d,
None => return Vec::new(),
},
_ => return Vec::new(),
};
let mut names: Vec<String> = sub.map.keys().cloned().collect();
names.sort();
names
}
/// The raw page dictionary for `index`, read without going through
/// `PdfPage`.
fn page_dict(doc: &mut PdfDocument, index: usize) -> Option<PdfDict> {
let r = doc.page_object_ref(index)?;
doc.resolve_ref(r).ok()?.as_dict().cloned()
}
// ------------------------------------------------------- resource parity
#[test]
fn every_declared_font_is_delivered() {
let mut checked = 0usize;
for path in all_fixtures() {
if is_deliberately_broken(&path) {
continue;
}
let data = std::fs::read(&path).expect("readable");
let Ok(mut doc) = PdfDocument::parse(&data) else {
continue;
};
for index in 0..doc.page_count() {
let Some(pd) = page_dict(&mut doc, index) else {
continue;
};
let declared = declared_resource_names(&mut doc, &pd, "Font");
if declared.is_empty() {
continue;
}
let Ok(page) = doc.page(index) else { continue };
let mut delivered: Vec<String> = page
.fonts
.keys()
.cloned()
.chain(page.type3_fonts.keys().cloned())
.collect();
delivered.sort();
delivered.dedup();
assert_eq!(
declared,
delivered,
"{}: page {index} declares fonts {declared:?} but delivers {delivered:?}",
path.display()
);
checked += 1;
}
}
assert!(
checked >= 10,
"expected pages declaring fonts, checked {checked}"
);
}
#[test]
fn every_declared_xobject_is_delivered() {
let mut checked = 0usize;
for path in all_fixtures() {
if is_deliberately_broken(&path) {
continue;
}
let data = std::fs::read(&path).expect("readable");
let Ok(mut doc) = PdfDocument::parse(&data) else {
continue;
};
for index in 0..doc.page_count() {
let Some(pd) = page_dict(&mut doc, index) else {
continue;
};
let declared = declared_resource_names(&mut doc, &pd, "XObject");
if declared.is_empty() {
continue;
}
let Ok(page) = doc.page(index) else { continue };
let mut delivered: Vec<String> = page.xobjects.keys().cloned().collect();
delivered.sort();
assert_eq!(
declared,
delivered,
"{}: page {index} declares xobjects {declared:?} but delivers {delivered:?}",
path.display()
);
// An xobject that resolves to nothing is the same failure one
// level down.
for (name, xobj) in &page.xobjects {
assert!(
!xobj.subtype.is_empty() && xobj.subtype != "Unknown",
"{}: xobject /{name} has no usable /Subtype",
path.display()
);
}
checked += 1;
}
}
assert!(
checked >= 3,
"expected pages declaring xobjects, checked {checked}"
);
}
#[test]
fn every_declared_ext_gstate_is_delivered() {
let mut checked = 0usize;
for path in all_fixtures() {
if is_deliberately_broken(&path) {
continue;
}
let data = std::fs::read(&path).expect("readable");
let Ok(mut doc) = PdfDocument::parse(&data) else {
continue;
};
for index in 0..doc.page_count() {
let Some(pd) = page_dict(&mut doc, index) else {
continue;
};
let declared = declared_resource_names(&mut doc, &pd, "ExtGState");
if declared.is_empty() {
continue;
}
let Ok(page) = doc.page(index) else { continue };
let mut delivered: Vec<String> = page.ext_gstate.keys().cloned().collect();
delivered.sort();
assert_eq!(
declared,
delivered,
"{}: page {index} declares gstates {declared:?} but delivers {delivered:?}",
path.display()
);
checked += 1;
}
}
assert!(
checked >= 2,
"expected pages declaring graphics states, checked {checked}"
);
}
#[test]
fn every_declared_colour_space_is_delivered() {
let mut checked = 0usize;
for path in all_fixtures() {
if is_deliberately_broken(&path) {
continue;
}
let data = std::fs::read(&path).expect("readable");
let Ok(mut doc) = PdfDocument::parse(&data) else {
continue;
};
for index in 0..doc.page_count() {
let Some(pd) = page_dict(&mut doc, index) else {
continue;
};
let declared = declared_resource_names(&mut doc, &pd, "ColorSpace");
if declared.is_empty() {
continue;
}
let Ok(page) = doc.page(index) else { continue };
let mut delivered: Vec<String> = page.color_spaces.keys().cloned().collect();
delivered.sort();
assert_eq!(
declared,
delivered,
"{}: page {index} declares colour spaces {declared:?} but delivers {delivered:?}",
path.display()
);
checked += 1;
}
}
assert!(
checked >= 2,
"expected pages declaring colour spaces, checked {checked}"
);
}
// ----------------------------------------------------------- form parity
#[test]
fn every_declared_form_field_is_delivered() {
/// Count the leaves of an /AcroForm /Fields tree from the raw graph.
fn count_leaves(doc: &mut PdfDocument, node: &PdfObj, seen: &mut HashSet<u32>) -> usize {
let resolved = match node {
PdfObj::Ref(r) => {
if !seen.insert(r.num) {
return 0;
}
match doc.resolve_ref(*r) {
Ok(o) => o,
Err(_) => return 0,
}
}
other => other.clone(),
};
let Some(dict) = resolved.as_dict().cloned() else {
return 0;
};
match dict.get("Kids") {
Some(PdfObj::Array(kids)) => {
let kids = kids.clone();
// A node with /Kids is only an intermediate node when the
// kids are fields; kids that are widget annotations leave
// the parent itself a leaf field.
let mut total = 0;
let mut kid_fields = 0;
for kid in &kids {
let kd = match kid {
PdfObj::Ref(r) => {
doc.resolve_ref(*r).ok().and_then(|o| o.as_dict().cloned())
}
other => other.as_dict().cloned(),
};
let is_widget = kd.as_ref().is_some_and(|d| {
d.get_name("Subtype") == Some("Widget") && d.get("T").is_none()
});
if !is_widget {
kid_fields += 1;
total += count_leaves(doc, kid, seen);
}
}
if kid_fields == 0 {
1
} else {
total
}
}
_ => 1,
}
}
let mut checked = 0usize;
for path in all_fixtures() {
if is_deliberately_broken(&path) {
continue;
}
let data = std::fs::read(&path).expect("readable");
let Ok(mut doc) = PdfDocument::parse(&data) else {
continue;
};
let Some(root_ref) = doc.trailer().get_ref("Root") else {
continue;
};
let Ok(root) = doc.resolve_ref(root_ref) else {
continue;
};
let Some(acro) = root.as_dict().and_then(|d| d.get("AcroForm").cloned()) else {
continue;
};
let acro = match &acro {
PdfObj::Ref(r) => match doc.resolve_ref(*r) {
Ok(o) => o,
Err(_) => continue,
},
other => other.clone(),
};
let Some(fields) = acro.as_dict().and_then(|d| d.get("Fields").cloned()) else {
continue;
};
let PdfObj::Array(fields) = fields else {
continue;
};
let mut seen = HashSet::new();
let declared: usize = fields
.iter()
.map(|f| count_leaves(&mut doc, f, &mut seen))
.sum();
if declared == 0 {
continue;
}
let delivered = match doc.acroform() {
Ok(Some(form)) => form.field_count(),
_ => 0,
};
assert_eq!(
declared,
delivered,
"{}: /AcroForm declares {declared} fields but delivers {delivered}",
path.display()
);
checked += 1;
}
assert!(
checked >= 4,
"expected fixtures with form fields, checked {checked}"
);
}
// --------------------------------------------------------- filter parity
/// Every stream in every fixture, decoded through its declared `/Filter`.
///
/// A filter chain that silently returns its *input* is the failure mode
/// `DCTDecode` had: bytes came back, they were not decoded, and the caller
/// could not tell. So this asserts that a compressed stream either decodes
/// to something different from its raw bytes, or fails loudly.
#[test]
fn every_declared_filter_either_decodes_or_refuses_by_name() {
let refused: HashSet<&str> = REFUSED_CODECS.iter().map(|(n, _)| *n).collect();
let mut decoded_count = 0usize;
let mut refused_count = 0usize;
for path in all_fixtures() {
if is_deliberately_broken(&path) {
continue;
}
let data = std::fs::read(&path).expect("readable");
let Ok(mut doc) = PdfDocument::parse(&data) else {
continue;
};
let max = doc.max_object_number();
for num in 1..=max.min(400) {
let Ok(obj) = doc.resolve_obj_num(num) else {
continue;
};
let Some(stream) = obj.as_stream() else {
continue;
};
let names: Vec<String> = match stream.dict.get("Filter") {
Some(PdfObj::Name(n)) => vec![n.clone()],
Some(PdfObj::Array(a)) => a
.iter()
.filter_map(|o| o.as_name().map(String::from))
.collect(),
_ => continue,
};
if names.is_empty() {
continue;
}
let stream = stream.clone();
match decode_stream(&stream) {
Ok(out) => {
assert!(
names.iter().all(|n| !refused.contains(n.as_str())),
"{}: object {num} declares a refused codec {names:?} \
yet decode_stream returned {} bytes",
path.display(),
out.len()
);
// Returning the input unchanged is the "decoded" lie.
assert_ne!(
out,
stream.data,
"{}: object {num} filtered by {names:?} decoded to \
its own compressed bytes",
path.display()
);
decoded_count += 1;
}
Err(e) => {
let message = e.to_string();
assert!(
names.iter().any(|n| message.contains(n.as_str())),
"{}: object {num} with {names:?} failed with a message \
that does not name the filter: {message}",
path.display()
);
refused_count += 1;
}
}
}
}
assert!(
decoded_count >= 20,
"expected many filtered streams, decoded {decoded_count}"
);
assert!(
refused_count >= 1,
"expected at least one refused codec fixture, saw {refused_count}"
);
}
// -------------------------------------------------- page attribute parity
#[test]
fn an_inherited_media_box_is_not_silently_replaced_by_the_default() {
/// The default a page falls back to when it has no /MediaBox anywhere.
/// A page that *does* declare one, at any level, must never get this.
const DEFAULT: [f64; 4] = [0.0, 0.0, 612.0, 792.0];
let mut checked = 0usize;
for path in all_fixtures() {
if is_deliberately_broken(&path) {
continue;
}
let data = std::fs::read(&path).expect("readable");
let Ok(mut doc) = PdfDocument::parse(&data) else {
continue;
};
for index in 0..doc.page_count() {
let Some(pd) = page_dict(&mut doc, index) else {
continue;
};
let Some(declared) = declared_inherited(&mut doc, &pd, "MediaBox") else {
continue;
};
let PdfObj::Array(arr) = declared else {
continue;
};
let nums: Vec<f64> = arr
.iter()
.filter_map(|o| match o {
PdfObj::Ref(r) => doc.resolve_ref(*r).ok().and_then(|v| v.as_f64()),
other => other.as_f64(),
})
.collect();
if nums.len() < 4 {
continue;
}
let expected = [nums[0], nums[1], nums[2], nums[3]];
let Ok(page) = doc.page(index) else { continue };
assert_eq!(
expected,
page.media_box,
"{}: page {index} declares MediaBox {expected:?} but reports {:?}{}",
path.display(),
page.media_box,
if page.media_box == DEFAULT {
" (the fallback default, so the declaration was lost)"
} else {
""
}
);
checked += 1;
}
}
assert!(
checked >= 20,
"expected pages declaring a MediaBox, checked {checked}"
);
}
#[test]
fn resolving_every_object_of_every_fixture_terminates() {
// The cheapest possible guard against a resolution cycle: a fixture
// that hangs here fails the suite by timeout rather than in production.
for path in all_fixtures() {
let data = std::fs::read(&path).expect("readable");
let Ok(mut doc) = PdfDocument::parse(&data) else {
continue;
};
let max = doc.max_object_number();
for num in 1..=max.min(400) {
let _ = doc.resolve_obj_num(num);
}
}
}
// ------------------------------------------------- the new resource shapes
#[test]
fn an_indirect_resources_dictionary_yields_its_fonts() {
let data = std::fs::read(corpus_dir().join("resources/indirect.pdf")).expect("fixture");
let mut doc = PdfDocument::parse(&data).expect("parses");
let page = doc.page(0).expect("page 0");
assert_eq!(
page.fonts.keys().collect::<Vec<_>>(),
vec!["F1"],
"an indirect /Resources must still deliver its fonts"
);
assert_eq!(page.fonts["F1"].base_font, "Helvetica");
let gs = page.ext_gstate.get("GS1").expect("GS1 present");
assert_eq!(gs.ca, 0.5);
assert_eq!(gs.ca_lower, 0.25);
}
#[test]
fn resources_inherited_from_the_page_tree_are_found() {
let data = std::fs::read(corpus_dir().join("resources/inherited.pdf")).expect("fixture");
let mut doc = PdfDocument::parse(&data).expect("parses");
let page = doc.page(0).expect("page 0");
assert_eq!(page.fonts.keys().collect::<Vec<_>>(), vec!["F1"]);
}
#[test]
fn inheritance_climbs_past_an_intermediate_node_that_overrides() {
let data =
std::fs::read(corpus_dir().join("resources/inherited_two_levels.pdf")).expect("fixture");
let mut doc = PdfDocument::parse(&data).expect("parses");
let page = doc.page(0).expect("page 0");
// The font comes from the grandparent...
assert_eq!(page.fonts.keys().collect::<Vec<_>>(), vec!["F1"]);
// ...but the media box from the nearer node, which overrides it.
assert_eq!(page.media_box, [0.0, 0.0, 300.0, 300.0]);
}
#[test]
fn an_indirect_font_or_xobject_sub_dictionary_is_followed() {
let data =
std::fs::read(corpus_dir().join("resources/indirect_sub_dict.pdf")).expect("fixture");
let mut doc = PdfDocument::parse(&data).expect("parses");
let page = doc.page(0).expect("page 0");
assert_eq!(page.fonts.keys().collect::<Vec<_>>(), vec!["F1"]);
let im = page.xobjects.get("Im1").expect("Im1 present");
assert_eq!(im.subtype, "Image");
}
#[test]
fn indirect_media_box_entries_are_resolved_to_numbers() {
let data =
std::fs::read(corpus_dir().join("resources/indirect_media_box.pdf")).expect("fixture");
let mut doc = PdfDocument::parse(&data).expect("parses");
let page = doc.page(0).expect("page 0");
assert_eq!(
page.media_box,
[0.0, 0.0, 400.0, 500.0],
"indirect numbers in a /MediaBox must be resolved, not dropped"
);
}
#[test]
fn a_parent_cycle_does_not_hang_inheritance() {
let data = std::fs::read(corpus_dir().join("resources/parent_cycle.pdf")).expect("fixture");
let mut doc = PdfDocument::parse(&data).expect("parses");
// The assertion is that this returns at all.
let page = doc.page(0).expect("page 0");
assert_eq!(page.media_box, [0.0, 0.0, 612.0, 792.0]);
}
/// Guards the test above: `ObjRef` is used, so an unused-import warning
/// cannot quietly remove the type this file relies on.
#[test]
fn obj_ref_is_the_key_used_for_identity() {
let r = ObjRef { num: 7, gen: 0 };
assert_eq!(r.num, 7);
}