Phase 2 of NIGIG_PDF_FEATURE_PARITY_PLAN.md, the item it calls "the biggest parse-side gap". Design and merge criteria in REVIEWS/adr/0013-pdf-xref-streams.md. The parser could not open a PDF 1.5 file. Not render it wrong - not open it: PARSE FAILED: PDF error at byte 382: expected xref keyword XRefTable::parse_section required the literal bytes `xref` at the startxref offset. A PDF 1.5+ file has an indirect object there instead - the xref stream - so the parse aborted and the entire document was unreadable. Every feature built on top of the parser (encryption, signatures, forms, structure tree, transparency) was unreachable on any file produced in the last twenty years. ObjStm, XRefStm and /Type /XRef appeared nowhere in the crate. Implemented on the read side: - Xref streams: the packed binary table, /W field widths, /Index sparse subsections, and types 0/1/2. A zero-width /W column means "use the default" (type 1) - missing that rule yields a table of all-free entries and an apparently empty document rather than an error. - Object streams: type-2 entries resolve through /ObjStm, reading the header pairs and /First. The xref's index is used but verified against the object number it claims to be, because a wrong-but-in-range index would silently return a different object. - Hybrid files: a traditional table plus /XRefStm. Both are read, with the traditional table winning on conflict, which is the point of the layout. Bounds and refusals rather than silent degradation: /W widths are clamped and every field read is checked against the decoded buffer; a truncated table is flagged, not padded with free entries; an object claiming to live inside itself is refused; a /Type that is not /XRef is named in the error. Scope note: the writer is untouched. ADR 0003 keeps appending a traditional xref section, which remains correct - the appended trailer carries /Prev to the stream, so the chain stays readable by us and by conforming readers. Also verified against the rest of Phase 2: inline images, XObject Do, shading, and the full text state (Tc/Tw/TL/Tz/Ts/Td/TD/Tm/Tf) are already implemented and tested. Type 3 fonts and the streaming interpreter remain genuine gaps, but each degrades one feature rather than the whole file. 6 corpus fixtures, 9 acceptance tests asserting real page content rather than a successful parse, and a parse_xref_stream fuzz target because the table is attacker-controlled binary. Mutation-checked: restoring the old error fails 5 of the 9. TEST_TARGET=pdf 606 -> 615, TEST_TARGET=pdf-ui 651 -> 660. rustfmt and clippy -D warnings clean.
172 lines
6.2 KiB
Rust
172 lines
6.2 KiB
Rust
//! Xref-stream and object-stream acceptance tests.
|
|
//!
|
|
//! The merge criteria from `REVIEWS/adr/0013-pdf-xref-streams.md`.
|
|
//!
|
|
//! The defect these guard against was total: `XRefTable::parse_section`
|
|
//! required the literal bytes `xref` at the `startxref` offset, so a
|
|
//! PDF 1.5+ file — which has an indirect object there instead — failed with
|
|
//! "expected xref keyword" and **the entire document was unreadable**. Every
|
|
//! feature built on top of the parser was unreachable on any file produced
|
|
//! in the last twenty years.
|
|
//!
|
|
//! Each test asserts real content, not merely that parsing returned `Ok`: a
|
|
//! subtly wrong `/W` decode produces plausible offsets that resolve to the
|
|
//! wrong objects, which is worse than failing.
|
|
|
|
use std::path::PathBuf;
|
|
|
|
use nigig_pdf_document::PdfDocument;
|
|
use nigig_pdf_graphics::content::parse_content_stream;
|
|
|
|
fn corpus(relative: &str) -> Vec<u8> {
|
|
let path = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
|
|
.join("../tests/corpus")
|
|
.join(relative);
|
|
std::fs::read(&path).unwrap_or_else(|e| panic!("missing {}: {e}", path.display()))
|
|
}
|
|
|
|
/// Text drawn by the page, so a test can prove the *right* object resolved.
|
|
fn page_text(doc: &mut PdfDocument) -> String {
|
|
let page = doc.page(0).expect("page 0");
|
|
let ops = parse_content_stream(&page.content_data).expect("content parses");
|
|
ops.iter()
|
|
.filter_map(|op| match op {
|
|
nigig_pdf_graphics::content::PdfOp::ShowText(t) => {
|
|
Some(String::from_utf8_lossy(t).into_owned())
|
|
}
|
|
_ => None,
|
|
})
|
|
.collect()
|
|
}
|
|
|
|
#[test]
|
|
fn a_pdf_15_file_with_an_xref_stream_parses() {
|
|
let bytes = corpus("xref/stream_basic.pdf");
|
|
let mut doc =
|
|
PdfDocument::parse(&bytes).expect("a PDF 1.5 file must parse; this used to fail outright");
|
|
assert_eq!(doc.page_count(), 1);
|
|
assert!(
|
|
page_text(&mut doc).contains("xrefstream"),
|
|
"the page's own content must come back, not merely a successful parse"
|
|
);
|
|
}
|
|
|
|
/// The catalog and page tree of that fixture live *inside* an object stream,
|
|
/// so this proves type-2 entries resolve rather than the file merely opening.
|
|
#[test]
|
|
fn objects_inside_an_object_stream_resolve() {
|
|
let bytes = corpus("xref/stream_basic.pdf");
|
|
let mut doc = PdfDocument::parse(&bytes).expect("parses");
|
|
|
|
// Object 1 is the catalog; it is stored compressed in object stream 6.
|
|
let catalog = doc.resolve_obj_num(1).expect("catalog resolves");
|
|
let dict = catalog.as_dict().expect("catalog is a dict");
|
|
assert_eq!(dict.get_name("Type"), Some("Catalog"));
|
|
|
|
// Object 2 is the page tree, also compressed, at a different index.
|
|
let pages = doc.resolve_obj_num(2).expect("page tree resolves");
|
|
let dict = pages.as_dict().expect("page tree is a dict");
|
|
assert_eq!(dict.get_name("Type"), Some("Pages"));
|
|
assert_eq!(dict.get_int("Count"), Some(1));
|
|
}
|
|
|
|
/// `/W [0 4 2]`: a zero-width type column means the type defaults to 1.
|
|
/// Ignoring that rule reads every entry as free and yields an apparently
|
|
/// empty document instead of an error.
|
|
#[test]
|
|
fn a_zero_width_type_column_defaults_to_type_one() {
|
|
let bytes = corpus("xref/stream_default_width.pdf");
|
|
let mut doc = PdfDocument::parse(&bytes).expect("parses");
|
|
assert_eq!(doc.page_count(), 1);
|
|
assert!(page_text(&mut doc).contains("defaultwidth"));
|
|
}
|
|
|
|
#[test]
|
|
fn a_sparse_index_is_honoured() {
|
|
let bytes = corpus("xref/stream_indexed.pdf");
|
|
let mut doc = PdfDocument::parse(&bytes).expect("parses");
|
|
assert_eq!(doc.page_count(), 1);
|
|
assert!(page_text(&mut doc).contains("xrefstream"));
|
|
}
|
|
|
|
/// A hybrid-reference file carries both a traditional table and an
|
|
/// `/XRefStm`. Both must be read.
|
|
#[test]
|
|
fn a_hybrid_reference_file_reads_both_tables() {
|
|
let bytes = corpus("xref/hybrid.pdf");
|
|
let mut doc = PdfDocument::parse(&bytes).expect("parses");
|
|
assert_eq!(doc.page_count(), 1);
|
|
assert!(
|
|
page_text(&mut doc).contains("hybrid"),
|
|
"the content stream is described by the traditional table"
|
|
);
|
|
}
|
|
|
|
/// ADR 0013 rule 1: a malformed xref stream must not become an empty table,
|
|
/// which would look like a valid document containing nothing.
|
|
#[test]
|
|
fn a_truncated_xref_stream_is_an_error_not_an_empty_document() {
|
|
let bytes = corpus("xref/stream_truncated.pdf");
|
|
match PdfDocument::parse(&bytes) {
|
|
Err(_) => {}
|
|
Ok(mut doc) => {
|
|
// If it parses at all it must not silently claim to be a valid
|
|
// empty document; resolving the missing object must fail.
|
|
assert!(
|
|
doc.page(0).is_err() || doc.page_count() == 0,
|
|
"a truncated xref stream produced an apparently valid document"
|
|
);
|
|
}
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn a_wrong_type_at_startxref_is_named_not_guessed() {
|
|
let bytes = corpus("xref/stream_bad_type.pdf");
|
|
let err = PdfDocument::parse(&bytes)
|
|
.err()
|
|
.expect("a non-/XRef object at startxref must be refused");
|
|
let msg = err.to_string();
|
|
assert!(
|
|
msg.contains("XRef") || msg.contains("Frobnicate"),
|
|
"the error should name what was found: {msg}"
|
|
);
|
|
}
|
|
|
|
/// Traditional-xref documents must keep working: this is the whole existing
|
|
/// corpus, and the change touched the shared entry point.
|
|
#[test]
|
|
fn traditional_xref_documents_still_parse() {
|
|
for name in [
|
|
"basic/text.pdf",
|
|
"basic/multipage.pdf",
|
|
"forms/all_types.pdf",
|
|
"annotations/links.pdf",
|
|
] {
|
|
let bytes = corpus(name);
|
|
let mut doc =
|
|
PdfDocument::parse(&bytes).unwrap_or_else(|e| panic!("{name} regressed: {e}"));
|
|
assert!(doc.page_count() > 0, "{name} lost its pages");
|
|
assert!(doc.page(0).is_ok(), "{name} page 0 stopped resolving");
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn every_xref_fixture_parses_without_panicking() {
|
|
for name in [
|
|
"stream_basic.pdf",
|
|
"stream_default_width.pdf",
|
|
"stream_indexed.pdf",
|
|
"stream_truncated.pdf",
|
|
"stream_bad_type.pdf",
|
|
"hybrid.pdf",
|
|
] {
|
|
let bytes = corpus(&format!("xref/{name}"));
|
|
// Either a document or a typed error; never a panic and never a hang.
|
|
if let Ok(mut doc) = PdfDocument::parse(&bytes) {
|
|
let _ = doc.page_count();
|
|
let _ = doc.page(0);
|
|
let _ = doc.acroform();
|
|
}
|
|
}
|
|
}
|