nigig-org/crates/apps/pdf/pdf-graphics/tests/bitonal_images.rs
andodeki 674b2be66d
Some checks failed
PDF engine / engine (push) Has been cancelled
PDF engine / makepad-integration (push) Has been cancelled
PDF engine / fuzz (push) Has been cancelled
email.yml / feat(pdf): JPEG 2000 decoding — Phase 3 complete, all three codecs (push) Failing after 0s
repo hygiene / hygiene (push) Has been cancelled
feat(pdf): JPEG 2000 decoding — Phase 3 complete, all three codecs
The last codec ADR 0015 deferred. The plan recorded the blocker as a
dependency decision, not an algorithm: openjpeg would add a C dependency
that breaks the Android cross-compile. This is pure Rust and adds no
dependency at all.

It shares the MQ arithmetic decoder with JBIG2 — T.800 and T.88 specify
the same coder — so the previous tranche paid for most of this one.
Context::with_state moved onto the shared type because JPEG 2000 starts
three of its nineteen contexts away from state 0 and JBIG2 starts all of
them at 0.

Implemented: codestream and JP2 container parsing, packet headers with
tag trees and the bit-stuffing rule, EBCOT tier-1 (all three passes,
four zero-coding context tables, run-length mode), both 5/3 reversible
and 9/7 irreversible wavelets, RCT and ICT, arbitrary decomposition
levels, and multiple components.

Refused by name: multiple tiles, custom precinct partitions, code-block
style options, COC/QCC/RGN/POC overrides, subsampled components. Each
error says which feature the file needs. This matters more here than
anywhere else in the stack, because a JPEG 2000 decoder that quietly
skips something does not fail — it returns a slightly soft or banded
image that looks entirely fine.

That property also dictates how this is tested. Fixtures are produced by
OpenJPEG via Pillow and compared **exactly**, sample for sample: the
fixtures are lossless 5/3 so no tolerance is needed, and a tolerance is
where a subtly wrong decoder hides. Four images — grayscale raw
codestream, the same in a JP2 container, a larger one whose tag trees
actually branch, and RGB. A generator script is checked in beside them
so CI can prove the fixtures still match what produced them.

Verified by mutation. The first round was misleading and is worth
recording, because it is the same lesson as ADR 0017:

  DC level shift dropped        3 fail
  5/3 lifting rounding changed  2 fail
  RCT sign flipped              PASSED  <- survived
  RCT components swapped        PASSED  <- survived
  cleanup run-length disabled   PASSED  <- survived
  sign-context XOR dropped      PASSED  <- survived

Four mutations survived because Pillow writes MCT=0 by default, so the
RGB fixture coded its three components independently and never reached
the colour transform at all. The RCT branch was completely untested
while appearing covered — an untested branch that looks tested is worse
than one that looks missing. Added rgb8_mct.j2k with mct=1; all four
now fail. The header bit-stuffing mutation is caught by the unit test
rather than the round-trip.

Two real defects found while writing the tests:

- A corrupt marker length in a tile-part header walked the read cursor
  past the codestream and panicked on a slice. Found by the corruption
  sweep, not by review. The sweep now truncates at every length and
  flips every byte of a real file, and asserts only that nothing panics.
- The 9/7 flat-signal test initially asserted an amplitude I had derived
  from my own arithmetic. That is a test agreeing with the code by
  construction. It now asserts flatness — a ripple means the lifting or
  the edge extension is wrong — and the amplitude is pinned by the
  OpenJPEG round-trips instead, which use pixels this code did not
  produce.

Also removed two dead fields and an unused parameter that clippy found:
Subband::x0/y0 are always zero in the single-tile case this supports,
and dead state implying multi-tile support exists is worse than no
state.

JPX decodes on the image path, like JBIG2, because the codestream
carries its own geometry; it stays in REFUSED_CODECS with a reason
string saying where it is decoded rather than that it is missing.

Engine suite 866 -> 920. Coverage 85.66% -> 86.16%; jpx.rs at 93.72%
with a floor at 88.

Phase 3 is complete: CCITT, JBIG2 and JPX all land, and the plan is
updated to say so and to record how the two gating questions — JBIG2's
CVE record and JPX's C dependency — were actually answered.
2026-08-16 22:17:40 +00:00

380 lines
14 KiB
Rust

//! The bitonal image path: `CCITTFaxDecode` and `JBIG2Decode`.
//!
//! Both were refused by name until now. Both produce 1 bit per pixel, and
//! both are decoded on the *image* path rather than through the generic
//! filter facade — CCITT because its `/DecodeParms` defaults depend on the
//! image dimensions, JBIG2 because it needs `/Width` and `/Height` outright.
//!
//! Every assertion here checks *pixels*, not sizes or `Ok`-ness. ADR 0016
//! records a JPEG decoder that was a stub returning a correctly sized black
//! rectangle: it passed every test that asserted a length. The tests below
//! would have caught it, because black and white are different colours and
//! they say which they expect.
use nigig_pdf_cos::{PdfDict, PdfObj};
use nigig_pdf_graphics::image::ImageInfo;
/// Assemble a bit string, MSB first.
fn bits(s: &str) -> Vec<u8> {
let cleaned: String = s.chars().filter(|c| *c == '0' || *c == '1').collect();
let mut out = Vec::new();
let mut cur = 0u8;
let mut n = 0;
for c in cleaned.chars() {
cur = (cur << 1) | u8::from(c == '1');
n += 1;
if n == 8 {
out.push(cur);
cur = 0;
n = 0;
}
}
if n > 0 {
out.push(cur << (8 - n));
}
out
}
/// Read an RGBA buffer back as a picture, so a failure prints something a
/// human can look at rather than a hex dump.
fn render(rgba: &[u8], width: usize, height: usize) -> Vec<String> {
(0..height)
.map(|y| {
(0..width)
.map(|x| {
let i = (y * width + x) * 4;
if rgba[i] < 128 {
'#'
} else {
'.'
}
})
.collect()
})
.collect()
}
fn image_dict(filter: &str, width: i64, height: i64, parms: Option<PdfObj>) -> PdfDict {
let mut d = PdfDict::new();
d.set("Width", PdfObj::Int(width));
d.set("Height", PdfObj::Int(height));
d.set("BitsPerComponent", PdfObj::Int(1));
d.set("ColorSpace", PdfObj::Name("DeviceGray".into()));
d.set("Filter", PdfObj::Name(filter.into()));
if let Some(p) = parms {
d.set("DecodeParms", p);
}
d
}
fn ccitt_parms(k: i64, columns: i64, rows: i64) -> PdfObj {
let mut d = PdfDict::new();
d.set("K", PdfObj::Int(k));
d.set("Columns", PdfObj::Int(columns));
d.set("Rows", PdfObj::Int(rows));
PdfObj::Dict(d)
}
// ------------------------------------------------------------------ CCITT
#[test]
fn a_ccitt_image_decodes_to_the_right_pixels() {
// 4 white then 4 black, one row.
let dict = image_dict("CCITTFaxDecode", 8, 1, Some(ccitt_parms(0, 8, 1)));
let img = ImageInfo::from_dict(&dict, "Im0", bits("1011 011")).expect("builds");
let rgba = img
.decode_to_rgba()
.expect("CCITT decodes on the image path");
assert_eq!(rgba.len(), 8 * 4, "8 pixels, 4 bytes each");
assert_eq!(render(&rgba, 8, 1), vec!["....####"]);
}
/// Without `/DecodeParms` the image's own `/Width` and `/Height` must be
/// used, not the spec's 1728-column fax scan line. Falling back to 1728
/// would fail to decode every non-fax CCITT image in existence.
#[test]
fn a_ccitt_image_without_parms_uses_its_own_dimensions() {
let dict = image_dict("CCITTFaxDecode", 8, 1, None);
let img = ImageInfo::from_dict(&dict, "Im0", bits("1011 011")).expect("builds");
let rgba = img
.decode_to_rgba()
.expect("decodes using /Width and /Height");
assert_eq!(render(&rgba, 8, 1), vec!["....####"]);
}
/// `/DecodeParms` that omits `/Columns` must still fall back to `/Width`.
#[test]
fn partial_ccitt_parms_fall_back_to_the_image_dimensions() {
let mut p = PdfDict::new();
p.set("K", PdfObj::Int(0));
let dict = image_dict("CCITTFaxDecode", 8, 1, Some(PdfObj::Dict(p)));
let img = ImageInfo::from_dict(&dict, "Im0", bits("1011 011")).expect("builds");
let rgba = img
.decode_to_rgba()
.expect("an absent /Columns falls back to /Width");
assert_eq!(render(&rgba, 8, 1), vec!["....####"]);
}
/// The parallel-array case. `[/FlateDecode /CCITTFaxDecode]` has a matching
/// `/DecodeParms` array, and taking `arr[0]` hands the Flate parameters to
/// the fax decoder — the exact bug ADR 0015 records for the old chain code.
#[test]
fn ccitt_parms_are_taken_from_the_matching_array_slot() {
let mut dict = PdfDict::new();
dict.set("Width", PdfObj::Int(8));
dict.set("Height", PdfObj::Int(1));
dict.set("BitsPerComponent", PdfObj::Int(1));
dict.set("ColorSpace", PdfObj::Name("DeviceGray".into()));
dict.set(
"Filter",
PdfObj::Array(vec![
PdfObj::Name("ASCIIHexDecode".into()),
PdfObj::Name("CCITTFaxDecode".into()),
]),
);
dict.set(
"DecodeParms",
PdfObj::Array(vec![PdfObj::Null, ccitt_parms(0, 8, 1)]),
);
let img = ImageInfo::from_dict(&dict, "Im0", Vec::new()).expect("builds");
let parms = img.ccitt_parms.as_ref().expect("found the CCITT slot");
assert_eq!(
parms.get_int("Columns"),
Some(8),
"the parms must come from index 1, matching CCITT's place in /Filter"
);
}
#[test]
fn a_corrupt_ccitt_image_returns_none_rather_than_a_black_rectangle() {
let dict = image_dict("CCITTFaxDecode", 8, 4, Some(ccitt_parms(0, 8, 4)));
// One row of data where four are declared.
let img = ImageInfo::from_dict(&dict, "Im0", bits("1011 011")).expect("builds");
assert!(
img.decode_to_rgba().is_none(),
"a short image must fail visibly, not decode to plausible pixels"
);
}
// ------------------------------------------------------------------ JBIG2
/// Build a minimal embedded JBIG2 stream carrying one MMR generic region.
///
/// MMR is used for the round-trip because it is CCITT G4, so the expected
/// pixels can be stated by hand with confidence rather than being whatever
/// the arithmetic decoder happens to produce.
fn jbig2_mmr_stream(width: u32, height: u32, coded: &[u8]) -> Vec<u8> {
let mut body = Vec::new();
body.extend_from_slice(&width.to_be_bytes());
body.extend_from_slice(&height.to_be_bytes());
body.extend_from_slice(&0u32.to_be_bytes()); // x
body.extend_from_slice(&0u32.to_be_bytes()); // y
body.push(0); // external combination operator
body.push(1); // flags: MMR
body.extend_from_slice(coded);
let mut out = Vec::new();
out.extend_from_slice(&1u32.to_be_bytes()); // segment number
out.push(38); // immediate generic region, 1-byte page association
out.push(0x00); // no referred-to segments
out.push(0x01); // page 1
out.extend_from_slice(&(body.len() as u32).to_be_bytes());
out.extend_from_slice(&body);
out
}
#[test]
fn a_jbig2_image_decodes_to_the_right_pixels() {
// G4 horizontal mode: white 4, black 4.
let stream = jbig2_mmr_stream(8, 1, &bits("001 1011 011"));
let dict = image_dict("JBIG2Decode", 8, 1, None);
let img = ImageInfo::from_dict(&dict, "Im0", stream).expect("builds");
let rgba = img
.decode_to_rgba()
.expect("JBIG2 decodes on the image path");
assert_eq!(
render(&rgba, 8, 1),
vec!["....####"],
"JBIG2 is natively 1=black and must be inverted to PDF convention"
);
}
/// A JBIG2 image and the equivalent CCITT image must produce the *same*
/// picture. They are the same coding scheme underneath, so a disagreement
/// means one of the two conventions is inverted — the commonest bug in
/// this area and invisible unless the two are compared directly.
#[test]
fn jbig2_mmr_and_ccitt_agree_on_the_same_coded_bits() {
let coded = bits("001 1011 011");
let ccitt_dict = image_dict("CCITTFaxDecode", 8, 1, Some(ccitt_parms(-1, 8, 1)));
let ccitt = ImageInfo::from_dict(&ccitt_dict, "Im0", coded.clone())
.expect("builds")
.decode_to_rgba()
.expect("CCITT decodes");
let jbig2_dict = image_dict("JBIG2Decode", 8, 1, None);
let jbig2 = ImageInfo::from_dict(&jbig2_dict, "Im1", jbig2_mmr_stream(8, 1, &coded))
.expect("builds")
.decode_to_rgba()
.expect("JBIG2 decodes");
assert_eq!(
ccitt, jbig2,
"the same G4 bits must give the same pixels through either codec"
);
}
/// A declared `/JBIG2Globals` must refuse. Globals carry symbol
/// dictionaries, which are not decoded; decoding without them yields a
/// blank or partial image that every caller would treat as a success.
#[test]
fn a_declared_jbig2_globals_refuses_rather_than_decoding_without_it() {
let mut parms = PdfDict::new();
// In a real file this is an indirect reference this layer cannot
// resolve; what matters is that its presence is noticed.
parms.set(
"JBIG2Globals",
PdfObj::Ref(nigig_pdf_cos::ObjRef { num: 9, gen: 0 }),
);
let dict = image_dict("JBIG2Decode", 8, 1, Some(PdfObj::Dict(parms)));
let img = ImageInfo::from_dict(&dict, "Im0", jbig2_mmr_stream(8, 1, &bits("001 1011 011")))
.expect("builds");
assert!(
img.jbig2_globals_declared,
"the declaration must be noticed even when it cannot be resolved"
);
assert!(
img.decode_to_rgba().is_none(),
"decoding without the declared symbol dictionary would be a \
silently wrong image"
);
}
#[test]
fn an_undeclared_globals_does_not_block_decoding() {
let dict = image_dict("JBIG2Decode", 8, 1, None);
let img = ImageInfo::from_dict(&dict, "Im0", jbig2_mmr_stream(8, 1, &bits("001 1011 011")))
.expect("builds");
assert!(!img.jbig2_globals_declared);
assert!(img.decode_to_rgba().is_some());
}
#[test]
fn a_jbig2_stream_needing_a_text_region_returns_none() {
// Segment type 6: immediate text region. Not decoded, by decision.
let mut stream = Vec::new();
stream.extend_from_slice(&1u32.to_be_bytes());
stream.push(6);
stream.push(0x00);
stream.push(0x01);
stream.extend_from_slice(&2u32.to_be_bytes());
stream.extend_from_slice(&[0, 0]);
let dict = image_dict("JBIG2Decode", 8, 1, None);
let img = ImageInfo::from_dict(&dict, "Im0", stream).expect("builds");
assert!(
img.decode_to_rgba().is_none(),
"a text region must not decode to a blank page"
);
}
#[test]
fn a_zero_sized_bitonal_image_returns_none() {
for (w, h) in [(0i64, 4i64), (4, 0)] {
let dict = image_dict("JBIG2Decode", w, h, None);
let img = ImageInfo::from_dict(&dict, "Im0", vec![0; 16]).expect("builds");
assert!(img.decode_to_rgba().is_none(), "{w}x{h} has no pixels");
}
}
#[test]
fn both_codecs_report_as_bitonal_and_others_do_not() {
for filter in ["CCITTFaxDecode", "JBIG2Decode"] {
let dict = image_dict(filter, 8, 1, None);
let img = ImageInfo::from_dict(&dict, "Im0", Vec::new()).expect("builds");
assert!(img.is_bitonal_fax(), "{filter} is a bitonal codec");
}
for filter in ["DCTDecode", "FlateDecode", "JPXDecode"] {
let dict = image_dict(filter, 8, 1, None);
let img = ImageInfo::from_dict(&dict, "Im0", Vec::new()).expect("builds");
assert!(!img.is_bitonal_fax(), "{filter} is not a bitonal codec");
}
}
// ------------------------------------------------------------- JPEG 2000
fn jpx_corpus(name: &str) -> Vec<u8> {
let path = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("../tests/corpus/jpx")
.join(name);
std::fs::read(&path).unwrap_or_else(|e| panic!("missing {}: {e}", path.display()))
}
/// A JPX image decodes through the ordinary image path, to real pixels.
///
/// The gradient fixture is a horizontal ramp, so the assertion is that the
/// left edge is dark and the right edge is light. That is deliberately a
/// statement about the *content*: a decoder returning a uniform grey, or a
/// transposed image, passes any check on the buffer length.
#[test]
fn a_jpx_image_decodes_through_the_image_path() {
let mut dict = PdfDict::new();
dict.set("Width", PdfObj::Int(8));
dict.set("Height", PdfObj::Int(8));
dict.set("BitsPerComponent", PdfObj::Int(8));
dict.set("ColorSpace", PdfObj::Name("DeviceGray".into()));
dict.set("Filter", PdfObj::Name("JPXDecode".into()));
let img = ImageInfo::from_dict(&dict, "Im0", jpx_corpus("gray8_lossless.j2k")).expect("builds");
let rgba = img.decode_to_rgba().expect("JPX decodes on the image path");
assert_eq!(rgba.len(), 8 * 8 * 4);
let px = |x: usize, y: usize| rgba[(y * 8 + x) * 4];
assert!(
px(0, 0) < px(7, 0),
"the gradient must run left-to-right: got {} at x=0 and {} at x=7",
px(0, 0),
px(7, 0)
);
assert!(
rgba.chunks(4).any(|p| p[0] != rgba[0]),
"a uniform image means the codestream was not really decoded"
);
}
/// Three components must come out as three different channels. A decoder
/// that dropped the colour transform would return a grey image here and
/// pass every check that only looks at sizes.
#[test]
fn a_jpx_rgb_image_decodes_with_distinct_channels() {
let mut dict = PdfDict::new();
dict.set("Width", PdfObj::Int(8));
dict.set("Height", PdfObj::Int(8));
dict.set("BitsPerComponent", PdfObj::Int(8));
dict.set("ColorSpace", PdfObj::Name("DeviceRGB".into()));
dict.set("Filter", PdfObj::Name("JPXDecode".into()));
let img = ImageInfo::from_dict(&dict, "Im0", jpx_corpus("rgb8_lossless.j2k")).expect("builds");
let rgba = img.decode_to_rgba().expect("JPX RGB decodes");
assert_eq!(rgba.len(), 8 * 8 * 4);
let differs = rgba.chunks(4).any(|p| p[0] != p[1] || p[1] != p[2]);
assert!(
differs,
"every pixel came out grey, so the colour transform was skipped"
);
}
#[test]
fn a_corrupt_jpx_image_returns_none() {
let mut dict = PdfDict::new();
dict.set("Width", PdfObj::Int(8));
dict.set("Height", PdfObj::Int(8));
dict.set("Filter", PdfObj::Name("JPXDecode".into()));
let img = ImageInfo::from_dict(&dict, "Im0", vec![0xFF, 0x4F, 0x00, 0x01]).expect("builds");
assert!(
img.decode_to_rgba().is_none(),
"a malformed codestream must fail visibly"
);
}