42 commits

Author	SHA1	Message	Date
andodeki	0bef30a6d5	feat(pdf): compositing and overprint — the blend maths had no backdrop ... `transparency.rs` implemented all sixteen blend modes and unit-tested them against the specification's formulas. Nothing ever called them with a backdrop. The Makepad renderer's `SetBlendMode` pushed a `TransparencyError::Unsupported` and then painted the source colour, so a /Multiply highlight and a /Normal one produced byte-identical output and every test passed — because every test asked "was the right command issued", not "does the page look right". Overprint had no code at all. /OP, /op and /OPM were not parsed, so an overprinting object knocked out the inks under it. That is not a missing feature, it is the inverse of the instruction: on a press it is the difference between a colour and a hole. - `composite.rs`: a straight-alpha RGBA `Canvas` implementing §11.3.6's union formula, weighted by backdrop alpha so a Multiply over transparency is the source rather than black. Constant alpha and per-pixel soft masks. Transparency groups composite as a unit; knockout groups are refused by name rather than silently treated as non-knockout. - Overprint as `composite_cmyk`, separate from the RGB path rather than a flag on it: overprint is a statement about inks and RGB has none. /op defaults to /OP per table 58 — defaulting it to false makes the common `<< /OP true >>` knock out every fill. §10.7.5's "no effect on an RGB device" is asserted, so our doing nothing there is the spec rather than an omission. - `raster.rs`: a CPU rasteriser that replays a command list onto a canvas. Not on the display path, no anti-aliasing, no fonts; it exists so compositing has a verifiable output. In pdf-graphics and not pdf-makepad because a test that needs a GPU is a test that does not run. - Golden **pixels** for shading, mesh, blend and overprint pages — Phase 7's exit criterion, which the Phase 2 command-text goldens cannot meet. ASCII grids with a colour legend, quantised to quarter steps; each test asserts its exact colours before comparing, so a wrong-but-stable render cannot be blessed by an UPDATE_GOLDEN run. Six mutations, all killed, including the two that describe the old behaviour: discarding the blend result, and ignoring the overprint flag. 1364 tests pass. ADR 0030.	2026-08-18 19:25:15 +00:00
andodeki	c1d1e67f3a	feat(pdf): shadings — the sh operator was parsed and thrown away ... ADR 0028, the first of Phase 7's eight bullets. content.rs contained `PdfOp::Shading(_name) => {}`. The operator was lexed, given its own variant, matched during interpretation, and discarded. A page whose background is a gradient rendered as nothing. Nothing caught it for the usual reason: a blank region is a legal thing for a page to contain, so "drew nothing" and "drew what was asked" are indistinguishable without an assertion naming the expected colour. The golden corpus had no shading page, so there was nothing to be wrong. Two of the three pieces already existed — function.rs evaluates the colour function and colorspace.rs converts it to RGB. What was missing was the geometry between them. Sampling rather than a gradient primitive: a PDF shading is defined by an arbitrary function, possibly a sampled table or a PostScript program, and neither reduces to a stop list without loss. A device with a native gradient can still recognise the two-stop case from the samples. "No colour here" is None, not black. Black is a colour a shading can legitimately produce, so returning it for "outside an unextended shading" would paint a rectangle the author never asked for and the caller could not tell the two apart. Types 1-5 exact. Coons and tensor patches are flattened to their corners, which loses the curvature, and is_approximate says so rather than leaving a caller to assume fidelity. An unknown type is refused by number: a mesh drawn as a flat fill is a plausible-looking wrong answer. paint_shading is a new trait method, so the compiler found every implementor. The Makepad renderer records the request in pending_shadings, mirroring pending_xobjects — it cannot resolve a /Shading resource because it does not own the page dictionary, and recording the request is what stops the operator vanishing a second time. That holds even for types we refuse, so a host can warn the user. Four mutations, all killed. The first — discarding sh again — fails three tests. Stated plainly and left unticked: the mesh path is written but NOT exercised by any real stream. shading.rs is at 68% and the uncovered part is exactly parse_mesh and triangulate. Mesh support should be treated as unproven, not working: the code runs and produces triangles, and nothing yet demonstrates they are the right triangles. That is the position image.rs was in before ADR 0016 found the JPEG decoder was a stub. The Phase 7 status line is a table from the start this time — one row per spec bullet, seven of them saying "not started". Per ADR 0021, written before the work rather than after it. pdf: 1321 passed (was 1291). pdf-ui: 1366. Coverage 87.60%, floors met.	2026-08-18 17:30:39 +00:00
andodeki	cb8912f762	test(pdf): close the two real coverage gaps in the signing module ... Asked to verify Phase 6 was complete *with test coverage*, I measured sign.rs per function rather than trusting the file-level 82%. Most of the apparent gap is error arms inside covered functions — llvm-cov attributes each `map_err` closure separately — but two things were genuinely untested, and one of them was not code that should exist. algorithm_name() was dead. It returned a &'static str describing the algorithm and nothing called it: `algorithm()` supersedes it, returns a type rather than a string, and is what the CMS writer actually uses. Deleted rather than tested, because a test would have preserved code whose only caller was the test. SigningError's Display impl was never exercised. These strings reach a user through a host application. ContentsTooSmall in particular must carry both numbers — a caller cannot raise the reservation without knowing by how much — and that is now driven through the real signing path with a 32-byte reservation rather than by constructing the error. sign.rs 82.07% -> 83.79%. pdf: 1291 passed. Coverage 88.03%, floors met.	2026-08-18 17:08:11 +00:00
andodeki	374af5ccad	feat(pdf): the five Phase 6 bullets the status line omitted ... ADR 0027. Asked whether Phase 6 was 100% complete, I checked the plan's bullets against the code instead of answering from the status line. Five were not implemented and the status line named none of them: detached/ATTACHED signatures /SubFilter hardcoded to adbe.pkcs7.detached PAdES basics ETSI.CAdES.detached was a string in a match external_signing_test.dart absent; SigningIdentity needs an in-memory key OCSP/CRL lookup CRL only; OCSP counted, never parsed Fulcio identity absent (optional in the plan) This is the second time. ADR 0021 recorded the same failure in Phase 4 and wrote the rule meant to prevent it — enumerate criteria from the plan text first, then mark each done or explicitly deferred. I wrote that rule and then produced another prose summary of what I had built. A summary written from the work cannot show what the work omitted. PAdES is a real profile, not a label. CAdES signs a set of signed attributes, one carrying the document digest, and the signature is over those attributes re-tagged as a SET (RFC 5652 5.4) rather than over the [0] IMPLICIT SEQUENCE they are carried in. Verification checks the messageDigest attribute against the document as well as verifying the attribute signature; without that, a signature over somebody else's digest would be accepted. /SubFilter now comes from the profile, so a document cannot claim CAdES while carrying plain PKCS#7. ExternalSigner is a trait: bytes in, signature out. A smartcard or KMS never hands out its key, so SigningIdentity could not represent one. SigningIdentity implements the trait rather than sitting beside it, so there is one signing path — a second path for hardware keys would be a second place the byte range could be computed differently. OCSP is decoded with the der crate already present rather than adding the ocsp crate for two fields. Revoked from any response beats Good from any other. Attached signatures are REFUSED, not deferred. Both attached profiles (adbe.pkcs7.sha1, adbe.x509.rsa_sha1) are SHA-1 based, and SHA-1 is broken for signatures. They are parsed so such documents can be read; they cannot be written, enforced by the absence of a SignatureProfile variant. Same decision as RC4 in ADR 0024. Recorded as refused rather than not-done, because "not done" invites someone to finish it. Four mutations, all killed first attempt: messageDigest not compared, CAdES verified against the wrong bytes, /SubFilter hardcoded again, OCSP revoked read as good. The status line is now the plan's own bullets in a table, one row per spec item, not prose. Two wrong status lines in the same direction is a pattern, and the fix is structural: a missing row is visible, a missing sentence is not. Four rows are left unticked — Fulcio, independent review, Acrobat interoperability, and signing a document that already has an AcroForm. qpdf accepts documents under both profiles. pdf: 1289 passed (was 1276). Coverage 87.96%.	2026-08-18 12:06:59 +00:00
andodeki	99aebc202a	fix(pdf): security review of the signing code — a forgery verified as valid ... ADR 0026. Both ADR 0024 and ADR 0025 said this code needed a security review before shipping. This is that review, done adversarially: for each way a signature could be defeated, a test that attempts it. It found a critical vulnerability in the code as shipped last turn. FINDING 1, critical, exploitable with no special access. Verification recovered the certificate and the signature by *scanning* the blob for DER-shaped bytes rather than decoding it. The signature was checked against certificates[0]; trust was checked against ANY certificate present. Two questions, two different certificates. So: the attacker signs a forgery with their own key the attacker appends the victim's trusted certificate to the blob signature_valid = true (their signature over their own content is real) chain_trusted = true (the victim's certificate is present) is_valid() = true Demonstrated before the fix, with the message "I hereby transfer everything to the attacker" verifying as valid. Fixed by decoding the ContentInfo/SignedData structure and finding the certificate the SignerInfo actually names, by issuer AND serial, then evaluating both the signature and the trust path against that one certificate. Trailing data now fails the decode instead of being ignored. The scanning functions are deleted, not left unused: dead code that once returned the wrong answer is an invitation to call it again. FINDING 2, moderate. signer_certificate() returned chain[0] unconditionally, so a chain whose first entry was not the signing key's certificate made the SignerInfo name the wrong one. Not a forgery route — the signature fails — but a UI showing "signed by <somebody trustworthy>" beside a failed check is its own kind of dangerous. Now it finds the entry whose public key matches the key doing the signing. FINDING 3, informational. digest_matches was hardcoded true under a comment claiming it was computed. Not exploitable, because is_valid() also requires signature_valid and the signature covers the bytes — but a field asserting an unperformed check is ADR 0017's pattern exactly. The four items ADR 0025 left unticked are closed: PKIX chain building, with each link's issuer signature verified. A name match alone is not a chain; anyone can put any name in a certificate. Pinning still short-circuits first. Stapled revocation from /DSS, offline only. Unknown is the default and a first-class answer: treating "no information" as "not revoked" is a claim a verifier cannot support. Signature appearances, with the claimed time labelled "Time claimed" because a self-declared /M carries no authority. One-call sign_document. Three things were wrong first: the /ByteRange placeholder was too narrow for real offsets so patching them moved every later byte; /Contents must be a hex string because a literal full of NULs needs escaping and changes length; and a signature dictionary nothing points at is invisible — the first version wrote one and the reader reported zero signatures over a correctly signed document. Four mutations, all killed — two only after strengthening the tests. My first smuggling test put the attacker's certificate first, where certificates[0] finds it anyway, so it passed with or without the issuer/serial match. Putting the TRUSTED certificate first is what distinguishes them, and writing that test is what exposed Finding 2. qpdf --check accepts the signed documents. pdf: 1276 passed. Coverage 87.98%. Left unticked, deliberately: an independent review by someone who did not write the code. This is a self-review; it found two real vulnerabilities, which is evidence the method works and not evidence that nothing remains. Also untested against Acrobat, which is stricter than the spec, and sign_document replaces rather than merges an existing AcroForm.	2026-08-18 10:39:20 +00:00
andodeki	9a5ce9c0e6	feat(pdf): signing and verification — Valid becomes reachable, with a policy ... ADR 0025, completing Phase 6's functional core. This partially reverses ADR 0010, which refused signing and cryptographic verification outright, and it reverses only the half whose justification expired. ADR 0010 gave two reasons. The first — a parsing library has no business signing — stopped being true when Phase 4 began creating documents and Phase 5 editing them. The second is still true and is preserved intact: deciding which certificate authorities to trust is a policy decision that belongs to the host, not to a parsing library So VerificationStatus::Valid is still not reachable by default. Verification returns three independent booleans and is_valid() needs all three; the third, chain_trusted, can only become true through a caller-supplied TrustAnchors. There is no TrustAnchors::system(), no bundled root store, no Default that trusts anything. A caller with no policy is told "cryptographically intact, signed by somebody you have not said you trust" — a different fact from "forged", and a host that cannot tell them apart shows the wrong thing to a user. RSA PKCS#1 v1.5, ECDSA P-256 and Ed25519, all with SHA-256. PSS is stronger and not universally accepted by PDF verifiers, so v1.5 is what is written. Ed25519 carries an interoperability caveat in the doc comment on the variant itself, because that is where someone choosing it will read it: ISO 32000-2 does not list it and most desktop viewers will reject it. No network. Revocation is not implemented rather than smuggled in: the engine crates are CI-gated against reaching outward, and that gate is a rule about layering, not an obstacle to work around. Every test generates a real key and a real certificate at run time. Nothing asserts against a checked-in blob — a fixed expectation only proves the code still does what it did, which is the wrong question for a signature. The tampering tests assert the signature verifies FIRST, then flip a bit; without that half they could pass by never verifying anything. Four mutations, all killed. The one that matters is the first: making an empty anchor set confer trust is exactly the regression that would turn this back into the thing ADR 0010 refused, and it fails immediately. Two bugs the tests found: UTCTime cannot encode a year past 2049 (RFC 5280 4.1.2.5.1). The first fixture used a 2096 expiry and every certificate failed to encode. The certificate scanner assumed a two-byte DER length. RSA certificates are large enough to use that form, so RSA and P-256 passed while Ed25519 found no certificate at all — its certificate is small enough for the short form. A scanner tested only against the largest input fails silently on the smallest. 72 dependency packages pulled in, zero non-compliant licences, no C. Stated plainly and left unticked in the ADR: chain_trusted is anchor identity matching, not PKIX path building. Correct for certificate pinning, a false negative for a real CA hierarchy. Also outstanding: revocation, signature appearance generation, and one-call incremental signing. pdf: 1247 passed. Coverage 88.08%, floors met.	2026-08-18 09:45:01 +00:00
andodeki	d4e3e9a443	feat(pdf): encryption on save — AES-128 and AES-256 (Phase 6, part one) ... ADR 0024. This reverses ADR 0005's "never write encryption", and the reason it is safe to reverse is that the facts changed underneath it. A crate that only reads cannot produce weak ciphertext, so refusing to write any was free. Now that Phase 4 creates documents and Phase 5 edits them, the refusal does something worse than protect nobody: open a password-protected file, change one annotation, save, and the output is plaintext. No error, no warning — the protection is silently dropped. That is this project's recurring failure mode in the one place where the consequence is a breach. The principle survives in a narrower form: no hand-rolled crypto, and no weak cipher offered as an option. RC4 stays readable because files use it and is not writable — EncryptionAlgorithm has no RC4 variant, so the refusal is a type, not a runtime check someone can route around. The encryptor is the literal inverse of the decryptor and imports its primitives rather than restating them; two implementations of one algorithm drift, and here they drift towards "decrypts to garbage". Every unit test round-trips through the existing Decryptor. Encryption sits at one choke point: PdfWriter holds the Encryptor and write_object_at encrypts everything passing through. Not per call site — there are twenty-two of those in PdfDocBuilder, and one stream written in the clear inside an encrypted document is not a partial failure, it is a leak that no reader will report because the file is otherwise valid. The /Encrypt dictionary is the single deliberate exemption: it holds the salts a reader needs before it has a key, so encrypting it bricks the file. Verified against implementations we share no code with, now gated in CI: ok qpdf opens it with the password ok it really is AES-256 ok the wrong password is refused ok poppler decrypts the content ok no plaintext in the encrypted file Four mutations, all killed — two only after the tests were strengthened, and both misses are the interesting part: A fixed IV survived two_saves_of_one_document_are_not_byte_identical, because the AES-256 file key is fresh per save and that alone makes the output differ. The property actually needed is narrower: one encryptor, identical plaintext, different bytes. In CBC a repeated IV under one key leaks that two plaintexts are equal. A wrong /Length survived because our own reader recovers by scanning for endstream — a robustness fix from ADR 0023. An independent reader that trusts /Length reads a truncated stream and decrypts garbage. A lenient reader hides a broken writer, which is why the external gate exists. The /Length test itself had a bug first: it searched a from_utf8_lossy view and reported a stream declaring 80 bytes holding 156. Ciphertext is not UTF-8; the replacement characters shifted every offset. Unencrypted output stays byte-reproducible; encrypted output cannot be, and a test asserts that loss rather than leaving it implicit. pdf: 1220 passed (was 1187). pdf-ui: green. Coverage 88.21%, encrypt_write.rs at 96.5%. Signing is NOT started. It needs the trust-anchor decision ADR 0010 deferred: VerificationStatus::Valid is unreachable by construction, and making sign -> verify pass is a policy change, not an implementation detail. The plan's Phase 6 status now says so.	2026-08-18 07:27:45 +00:00
andodeki	2ba1837055	fix(pdf): main was red — three clippy errors broke the engine build ... The PDF engine did not compile under `-D warnings`, which is what CI's engine job runs, so that job could not have passed on any of the last eleven PDF commits. The tests themselves were fine (1187 passing); the build was not. Two lints in the new jpx.rs, one in tests/filters.rs. One root cause each: jpx.rs:1279,1290 needless_range_loop on the inverse component transform. The lint's suggestion does not work here: each pass reads and writes three component planes at the same index, and `components.iter_mut()` cannot express three simultaneous mutable borrows of one Vec. Allowed locally with the reason written down, rather than restructuring correct code to satisfy a lint that has misread it. filters.rs:383 vec_init_then_push, where the lint is simply right. No behaviour change. Verified after the fix, on a clean checkout of df3c650: TEST_TARGET=pdf 1187 passed TEST_TARGET=pdf-ui 1232 passed coverage 87.98%, all floors met corpus matches generate.py; golden expectations unchanged fuzz manifest gate, no-makepad and no-process gates all pass	2026-08-18 05:41:22 +00:00
andodeki	b870dc4c69	feat(pdf): outline, page label and struct-tree editing — Phase 5 complete ... The last functional item. `catalog.rs` read all three; nothing could write them into an existing document. `PdfDocBuilder` can emit an outline when *creating* a file, but a document already on disk could not have its bookmarks changed. An outline is a doubly-linked tree — `/First`, `/Last`, `/Next`, `/Prev`, `/Parent` and a signed `/Count` — and every pointer has to agree. A viewer walking `/Next` and one walking `/First`..`/Last` must see the same list, or bookmarks vanish in one reader and not another with no error anywhere. Object numbers are reserved before any dictionary is built, because each item names its parent, its siblings and its children. `/Count` is signed and that matters: positive means open and counts *visible* descendants, negative means closed. A closed child contributes itself but hides its own children. Writing the total unconditionally makes every node render expanded. Page labels are a number tree, so the keys are sorted before writing and two rules starting on the same page are refused — that page's label would be undefined, and picking one arbitrarily is worse than saying so. Struct-tree editing is deliberately **removal only**. Editing the tree in place means rewriting `/K` arrays whose entries are marked-content ids inside page content streams; the tree and the content must stay in step, and changing one without the other produces a document whose accessibility information describes content that is no longer there. Removal is honest — the document stops claiming to be tagged — and `/MarkInfo` goes with it, because `/Marked true` with no tree tells a screen reader there is structure to find. Verified by mutation, seven defects, all caught: /Prev never written 1 fail /Next never written 5 fail /Count always positive 1 fail page validation skipped 2 fail /MarkInfo left behind 1 fail children not linked via /First 2 fail label rules not sorted 1 fail The last one needed a new test. Our reader walks `/Nums` linearly, so it tolerates any order and the round-trip passed unsorted — but a conforming reader binary-searches it and would label pages arbitrarily. Only reading the raw array catches that, which is the same lesson as the stale `/Count` in the page-ops tranche: our parser's tolerance hides defects that harm other readers. Engine suite 1158 -> 1187. External readers still pass. **Phase 5 is complete** but for the `ui.rs` interaction tests, blocked on the same missing Makepad headless backend as Phase 4's. The plan records the item-by-item status and, separately, the six defects the round-trip tests found in code that already existed — an unordered dictionary writer that made every generated PDF differ run to run, a short /Length that silently truncated streams, two readers disagreeing by a byte, a nested paren that truncated a string and desynchronised the stream, undecoded # escapes in names, and unknown operators being dropped outright.	2026-08-17 12:26:29 +00:00
andodeki	41c43df0e5	feat(pdf): redaction and object compaction — and three reader defects ... Phase 5's last two functional items. Together they are what makes a redaction real, which is why they are one commit: redaction removes the content, compaction removes the revision that still holds it. **Redaction removes operators; it does not draw rectangles.** The famous failure is painting black over text and shipping it — the text is still there, and `pdftotext` prints it. This module draws nothing. It removes the text-showing operators whose position falls inside a rectangle, and the test that matters is that the text can no longer be extracted. Positioning needs the text matrix, so the module tracks `Tm`/`Td`/`TD`/`T*` and the CTM through `q`/`Q`/`cm`. It cannot reach the graphics layer — the crate boundary again — so it treats a showing operator's origin as its position and removes the whole run. That is coarse in the *safe* direction: removing more than asked loses content the user can see is missing; removing less leaves the secret in the file. What it refuses to claim is as important. Images are removed entirely rather than cropped. Metadata and attachments are untouched. And an incremental redaction leaves the original text in the earlier revision — the report says so via `earlier_revisions_retain_content` rather than implying the job is done. **Compaction finishes it.** The output is built from the object graph reachable from `/Root`, so dead objects, superseded revisions and the bytes behind a redaction are not copied — they are simply never written. A signed document is refused unless `allow_signed` is set, because compaction destroys the revision a signature covers and would leave every signature unverifiable with no warning. The end-to-end test is the point: redact, compact, then search the output bytes for the secret. It is gone. **Three reader defects, all found by writing the tests.** - **`PdfWriter` wrote dictionary keys unordered.** `PdfDict` is a HashMap and Rust seeds its hasher per process, so *every generated PDF differed run to run*. Found by compaction's idempotence test — compacting an already-compact file produced the same objects at the same offsets with their keys shuffled. Verified fixed by running four separate processes and getting a byte-identical file. Same defect as the one fixed in `content_edit::write_dict`; this one affected every file this codebase has ever written. - **A short `/Length` silently truncated a stream.** The reader guarded against a `/Length` running past the buffer but trusted one that was too small, cutting the stream at the wrong place and losing the rest with no error. Short lengths are common in hand-edited files. `endstream` is now the authority when the two disagree — but only when it is *further* on, so binary data containing the word `endstream` is still bounded by its declared length. - **Two stream readers disagreed by one byte.** `read_object_at` did not trim the EOL before `endstream` while `find_endstream` did, so a write-read-write cycle grew every stream by a newline. A test fixture had encoded the bug: it declared `/Length 9` for eight bytes of content and asserted the newline came back as data. Both corrected — the newline is syntax (§7.3.8.1), not content. Verified by mutation. Ten defects across the two modules, all caught: redaction covers instead of removes 16 fail CTM ignored 1 fail Q does not restore the CTM 1 fail operands kept when operator removed 12 fail revision warning always false 1 fail signature guard removed 1 fail reachability keeps everything 2 fail dropped reference left dangling 1 fail unresolvable object kept as reachable 1 fail writer dictionary order unsorted 1 fail short-/Length fix reverted 1 fail /Length not rewritten on compaction 1 fail One mutation survived and deleted code rather than adding a test: a `continue` skipping `/Length` in the compaction loop was dead, because the `set` after the loop overwrites it either way. Removed rather than left as untested defence with a reassuring comment — the same call ADR 0017 made about the visited-set guard. A second mutation moved a test rather than a fixture: a stale `/Length` can no longer reach `renumber` through a file, because the reader now repairs it first, so that branch is tested directly instead. Engine suite 1108 -> 1158. External readers still pass. Phase 5 remaining: outline, page label and struct-tree editing.	2026-08-17 12:20:25 +00:00
andodeki	e0d86274d8	feat(pdf): flatten annotations and form fields into page content ... Phase 5's `flatten_test.dart`. An annotation draws from its /AP /N stream, which lives beside the page rather than in it; flattening moves that appearance into the page's own content and removes the annotation, so what is drawn is part of the page and cannot be turned off, edited or extracted as a field. That is what "finalise this form" and "make these comments permanent" mean, and it is irreversible by design. The placement transform is the whole problem, and it is §12.5.5: transform the /BBox by /Matrix, take the bounding box of the *result*, then fit that onto /Rect. Skip a step and the stamp lands at the origin, or in the right place at the wrong size, and the page still renders. A rotated appearance is the case that exposes it — rotation swaps the transformed box's width and height, so fitting the untransformed box squashes it. What is refused matters as much as what is done: - **No appearance stream**: left in place and reported. Dropping it loses it; inventing an appearance draws something the producer never specified. - **Hidden or /NoView**: not drawn on screen, so burning it in would *add* ink the user never saw. - **A /Popup**: the pop-up window of another annotation, never drawn on the page itself. Appearances are painted as XObjects rather than having their operators spliced in. Splicing needs the stream's resources merged into the page's with every name collision renamed, and it loses the /BBox clip an XObject applies for free. `pdf-document` cannot depend on `pdf-graphics` — the crate boundary is cos -> document -> graphics and inverting it to reuse `write_ops` would be a far worse trade than emitting the four operators (`q`, `cm`, `Do`, `Q`) directly. The number formatting follows the same shortest-exact rule as `content_edit::write_real`, and for the same reason. Verified by mutation, five defects, each confirmed red: placement matrix ignored 1 fail /BBox not transformed first 2 fail hidden check removed 1 fail annotation kept after flattening 7 fail existing page content dropped 1 fail **Externally verified, and it found a real gap.** Flattening the sample's seven annotations passed `qpdf --check` and kept every text run — but poppler still reported `Form: AcroForm` on a document with no fields left, because the catalogue entry survived. A viewer may still offer to fill in a form that no longer exists. `remove_acroform_if_empty` drops it, but only when *no* widget survives anywhere: flattening one page of a three-page form must not strip the fields still live on the others. before: Form: AcroForm after: Form: none `flatten_document` is the whole-document entry point — every page, then the form entry — and re-parses between pages because each flatten appends a revision the next must read. 22 round-trip tests through the saved file, including that flattening twice is idempotent (a Do count that grows on every save is how a "flatten" button pressed twice doubles every stamp), that existing page resources survive, and that a /AP /N state dictionary resolves through /AS. Engine suite 1075 -> 1108. Remaining in Phase 5: object compaction, redaction, and outline, page label and struct-tree editing.	2026-08-17 12:03:59 +00:00
andodeki	e0aab74452	feat(pdf): page operations — insert, reorder, duplicate, delete, import ... Phase 5's page management, matching dart-pdf's `page_ops_test.dart`, `page_index_map_test.dart` and `import_source_test.dart`. A reader sees "page 3"; the file holds a tree of /Pages nodes with /Kids and /Count and /Parent back-pointers, any of which can be left stale. So the writer **flattens to a single level**: a one-level /Pages node with every page as a direct kid is valid, is what most producers emit, and removes the entire class of bug where an intermediate node's /Count no longer matches what is under it. Preserving an arbitrary tree shape through arbitrary reordering is far more code for nothing a reader can see. `PagePlan` accumulates operations and applies them together, so intermediate states never have to be valid — delete page 0 and insert a new one at 0 without the document momentarily having no first page. `PageIndexMap` reports where every page went, which is the only way to fix an outline entry, named destination or link annotation afterwards. What each operation carries matters and differs: - Reorder and delete rewrite only the kid array, so page objects and their resources are untouched. - Duplicate writes a new page dictionary that **shares** the original's resource references. Two pages naming one font object is normal; deep-copying would double the file and change nothing visible. - Import must deep-copy the page and everything it reaches, renumbered, because source object numbers mean nothing in the destination. /Parent is deliberately not followed — it leads back to the source's page tree and from there to every other page in that file. Inheritable attributes are resolved *before* a page is imported. /Resources, /MediaBox, /CropBox and /Rotate may live on an ancestor (Table 30) that is not coming with it, so a page imported without them renders at the wrong size with no fonts, and nothing reports an error. **Round-trip tested through the saved file**, which is Phase 5's exit criterion: 20 tests that save, re-parse, and assert on what a reader actually gets. Pages are identified by /MediaBox width rather than object number, because object numbers are exactly what a page-tree bug scrambles. Mutation testing changed two things. Seven defects injected: /Count left stale 1 fail /Count omitted entirely 1 fail imported /Parent not rewritten 1 fail inherited attributes not resolved 1 fail import does not deep-copy 3 fail duplicate loses /Contents 1 fail re-parenting skipped 1 fail The last two only fail because of tests the mutations forced: - **A stale /Count passed everything.** Our own parser walks /Kids and never reads /Count, so it cannot see the disagreement — but other readers trust /Count, and a document where the two differ opens with a different page count in different viewers. The test now reads the raw page-tree node instead of asking the document. - **Re-parenting could be deleted with every test still green**, because the flat fixture's pages already parent to the root. Added a nested fixture with an intermediate /Pages node supplying an inherited /MediaBox — the case where leaving /Parent stale means a page keeps inheriting from a node it is no longer under. Externally verified: a generated sample with pages swapped and duplicated passes `qpdf --check` with no warnings, and poppler reads 4 pages with the reordering visible in extracted text. Engine suite 1039 -> 1075. Remaining in Phase 5: flatten, object compaction, redaction, and outline and struct-tree editing.	2026-08-17 10:38:55 +00:00
andodeki	b8bd71852f	feat(pdf): content-stream serialiser and editor — the Phase 5 foundation ... Phase 5 needs to write operators back, and `content.rs` has only ever parsed them. Every editing feature the phase asks for — insert, delete, replace, rewrite a text run, flatten an annotation — rests on that, and a serialiser that is subtly wrong does not throw: it writes a valid content stream that draws something else. So this is the serialiser plus the gate, and nothing built on top yet. The contract is a property, run over the whole corpus: parse(write(parse(bytes))) == parse(bytes) Operators, not bytes. Byte equality would be the wrong test — `1.0` may legally be written `1`, whitespace is free, and a writer that reproduced its input byte for byte would only prove it had copied it. It passes: **16,466 operators across 154 streams in 109 files**, plus stability, idempotence, and the same property after an edit. **Then mutation testing showed the corpus gate was not enough.** Six injected defects, and *five passed*: dropping name escaping, unescaping string parens, un-sorting dictionary keys, discarding unknown operators, and a fixed six-decimal number format. Real files are written by well-behaved producers, so 16,000 corpus operators contain no name with a space, no nested parenthesis, no seven-key inline dictionary and no vendor operator. A gate that only sees well-formed input cannot catch a writer that mishandles the rest. The adversarial set fixes that — eighteen streams, each a legal shape the corpus lacks, each chosen because a specific defect survives without it. Writing it found **three live bugs in the parser**, none of which the round trip could see on its own: - **Nested parentheses truncated a string to nothing.** `((nested))` parsed as the empty string, and worse, left the reader mid-string so every operator after it was parsed from the wrong offset. §7.3.4.2 says balanced parens nest and need no escaping. - **`#` escapes in names were never decoded.** `/My#20Font` — how every producer writes a font whose name contains a space — parsed as the literal `My#20Font` and never matched the page's resource. - **`PdfOp::Unknown` was declared and never constructed.** An operator the parser did not recognise vanished. Survivable for a renderer, fatal for an editor: parse, change one operator, write back, and every vendor extension in the page is silently gone from the saved file. And two in my own serialiser, both found the same way: - A fixed `{:.6}` flushed 1e-7 to zero — a scale factor silently becoming zero collapses whatever it transforms — and rounded `1.234567891` to a different number. Precision is now the shortest that parses back to the identical f64, exact by construction rather than by choosing a number. - Sorted dictionary keys turned out to be load-bearing. `PdfDict` is a HashMap and Rust seeds its hasher per process, so an unsorted writer is stable within a run and different on every new one: rebuild the same document twice, get two different files. Neither the round trip nor a within-process stability check can see it — both sides are equally unordered. Verified by running five separate processes and getting five different key orders. Two of those needed tests the round trip structurally cannot provide, so they assert on the parser directly: what `((nested))` must produce, and that operators after it are still read at the right offset. Final mutation run, eight defects, all caught: fixed 6-decimal precision 1 fail name escaping dropped (writer) 1 fail name unescaping dropped (parser) 2 fail nested-paren fix reverted 1 fail unknown operators discarded 1 fail string parens unescaped 1 fail close-paren unescaped 1 fail dictionary keys unsorted 2 fail `ContentEditor` sits on top: insert, append, prepend, delete, replace, isolate, and text-run rewriting that preserves the operator *kind* — a `'` stays a `'` and keeps its line advance, a `TJ` keeps its kerning numbers while its strings change. Every mutation is balance-checked, so an edit that would leave `q` without `Q`, or `BT` without `ET`, is refused at the edit rather than discovered at save time. `PdfOp` gained `PartialEq`, which is what makes the property expressible at all. Engine suite 1025 -> 1039. Phase 5's remaining items — page ops, import/merge, flatten, compaction, redaction — build on this and are not started.	2026-08-17 10:29:39 +00:00
andodeki	1220f89fc6	feat(pdf): close the last three Phase 4 items — reconciliation, CFF, cmap ... The three items the previous commit's audit found unimplemented while the status line said "complete". All three are done and externally verified. **1. Field-value reconciliation** (`reconcile.rs`). A field carries its value in /V and its rendered look in /AP, and nothing in the format keeps them in step. Files arrive with them disagreeing all the time: a producer writes /V and leaves appearances to the viewer, or something edits /V without touching /AP. Until now this crate simply believed /V and regenerated appearances only for fields it had itself edited — right for a field we changed, wrong for a field that arrived inconsistent. The module deliberately does **not** pick a winner. PDF 32000-1 §12.7.3.3 settles exactly one case — /NeedAppearances true means /V is authoritative — and is silent on the other, where a conforming viewer renders /AP and never looks at /V. So it classifies the disagreement and resolves it against a caller-declared `Intent`, because the right answer genuinely differs: a viewer must show /AP to match other viewers, an extractor must read /V, an editor must regenerate so the saved file agrees with itself. Silently choosing one would be ADR 0017's failure in a new place — every answer plausible, none checkable, the caller unaware a decision was made for it. Two cases are not judgement calls and are handled outright. A missing or dangling appearance renders *blank*, and blank is never what the producer meant, so even Display regenerates. An unselected radio member showing /Off while the group's /V names another member is correct, not a conflict — reporting it would flag every well-built radio group there is. **2. Type1/CFF embedding** (`embed_opentype_whole`). The spec says "if feasible". Subsetting CFF is not — it means rebuilding the CFF INDEX, charset and charstrings, a second font format inside the first — and `subset_truetype` rightly keeps refusing it by name. Embedding the program *whole* is feasible, and that is what this does: /FontFile3 with /Subtype /OpenType under a CIDFontType0 descendant, per Table 126. Each of those keys matters and none is guessable from the others. A CFF program in /FontFile2, or under a CIDFontType2 descendant, still produces a file qpdf accepts and a font that loads as the wrong type or not at all. /CIDToGIDMap is omitted because it is defined for CIDFontType2 only. The trade is made visible rather than buried: `EmbeddedFont::is_subsetted` is false here, so a caller with a size budget — or a licence that forbids shipping a whole face — can refuse instead of discovering it from the output size. **3. `repair-cmap`** (`glyph_index`). A symbol font declares no Unicode subtable: it maps glyphs into the private-use area at 0xF000 + the low byte under platform 3, encoding 0. Asking it for 'A' found nothing and the character silently vanished from the output — the font "missing" a glyph it plainly has. Now the (3,0) subtable is kept as a fallback and retried at 0xF000 + low byte, after the proper lookup fails so a font with both subtables is still read through the Unicode one. Format 0 is read too; omitting it left legacy and symbol fonts mapping nothing while appearing to have a usable cmap. The repair must not manufacture glyphs, which is its own test: a character the font genuinely lacks still returns None, because turning a missing character into a wrong one is worse. **Fixtures.** No CFF or symbol font ships on the CI image, and neither can be tested honestly against a hand-built stub — the point is that the bytes are a font program a third-party reader accepts. Both are generated from DejaVu by checked-in fontTools scripts: `cff_sample.otf` (1.6 KB, real OTTO/CFF outlines) and `symbol_sample.ttf` (664 B, a single (3,0) subtable so the repair path is the only route to its glyphs). Both generators pin `head.created`/`head.modified` to zero. fontTools stamps the current time, so the output differed on every run and CI's "fixtures match their generator" check failed against a file nobody had edited. Caught by running that check rather than assuming it passed. A fixture that cannot be regenerated byte-for-byte is not reviewable: you cannot tell a deliberate change from a rebuild. **Verified by mutation**, seven injected defects, each confirmed red: NeedAppearances ignored 1 fail dangling /AS not detected 1 fail blank rendering shown faithfully 1 fail CFF written to /FontFile2 1 fail CFF given a CIDFontType2 descendant 1 fail whole font claims to be subset 1 fail cmap 0xF000 retry removed 3 fail **Verified externally.** The sample now carries a third page set in the whole-embedded CFF font, and `check-pdf-external-readers.sh` gained `pdffonts` — the only check that inspects a font *program* rather than the file structure, which is exactly where a wrong /FontFile key shows up. poppler reports both fonts embedded and distinguishes them correctly: ETXLDI+DejaVuSans CID TrueType Identity-H emb yes sub yes NigigTestCFF CID Type 0C (OT) Identity-H emb yes sub no and extracts "Hello CFF 123", which only works if the CFF program loaded, /Identity-H addressed its glyphs and /ToUnicode mapped them back. That check also caught its own page-count assertion going stale when the third page landed — a gate that notices its own fixture changing is working. Engine suite 953 -> 985. Coverage 87.27%, all floors met. Phase 4 is complete but for the ui.rs interaction tests, which are written and blocked on the Makepad fork's missing headless backend.	2026-08-17 09:31:41 +00:00
andodeki	b26e6a1f14	feat(pdf): CCITT G3/G4 decoding — the codec Phase 3 deferred ... ADR 0015 refused CCITTFaxDecode by name and recorded it as the recommended next codec: well specified, no arithmetic coding, no C dependency. This implements it. T.4 and T.6, all three schemes selected by /K: G3 1D modified Huffman, G4 two-dimensional, and G3 mixed with a tag bit after each EOL. Both run-length code books, makeup and extended makeup codes, and the pass/horizontal/vertical mode codes. /Columns, /Rows, /BlackIs1 and /EncodedByteAlign are honoured; /Columns and /Rows are bounds-checked before anything is sized from them, because both are attacker-controlled in a hostile file. It decodes to real pixels, so unlike DCTDecode it belongs in the filter facade rather than the image path: the generic filter contract promises decoded bytes and this can honestly keep that promise. Removed from REFUSED_CODECS, added to SUPPORTED_FILTERS — the registry now describes what the crate actually does. Both existing data-driven registry tests pick this up without editing. Three defects were found by writing the tests rather than by reading the code: - A zero-length run recorded no transition. That is exactly how a row beginning with black is coded — a white run of zero, then the black run — so every such row came out with its colours shifted by one run: "####...." decoded as "....####". - Decoding stopped at bits_left() == 0, but encoders pad the final row to a byte boundary. The padding was fed to the decoder as though it were a code, failed to match, and lost the whole image. Now a trailing all-zero tail is recognised as padding, which is unambiguous because every code book needs a 1 bit. - A row of zero-length runs did not advance the pixel position and looped forever. Found by mutation, not by review. Bounded by the column count: a hang is a worse failure than an error. Verified by mutation, five injected defects, each confirmed to turn the suite red: a0 starts at 0 not -1 1 fails pack_row fills black 13 fails find_b1 parity dropped 1 fails short-/Rows check removed 1 fails read_run returns 0 2 fails Two of those did not fail on the first attempt and changed the tests: - a0 = 0 survived, because no fixture placed a colour change at column 0 — the one position where the off-by-one is visible. Added group4_codes_a_change_at_column_zero. - read_run returning 0 survived because the new run bound also errors, so an assertion of merely "some CCITT error" could not tell the two mechanisms apart. The assertions now name the specific failure. 30 unit tests in the codec, asserting decoded pictures rather than byte counts, plus 6 integration tests through the filter facade covering the chain case, truncation and the spec defaults. The facade test asserts output != input: ADR 0015 records DCTDecode "succeeding" by returning its own compressed input, and a test that only asserted Ok passed against that bug. Engine suite 796 -> 825. Coverage 84.82% -> 85.15%; ccitt.rs at 92.57% with a floor at 88. JBIG2 and JPX remain refused and are the next two tranches.	2026-08-16 21:50:46 +00:00
andodeki	7d6fc4cbbe	feat(pdf): document creation — outlines, forms, attachments, font subsetting ... Phase 4 of NIGIG_PDF_FEATURE_PARITY_PLAN.md. ADR 0019. Almost none of it existed: Outlines, PageLabels, EmbeddedFiles and ViewerPreferences appeared nowhere in the workspace, in any crate. What did exist was a builder whose central method was pub fn add_page_with_content(&mut self, _width: f64, _height: f64, ...) which accepted a page size and discarded it. Asking for 200x400 and 300x500 gave two US Letter pages, because no /MediaBox was written at all. The test asserted the output contained the string "/Type /Page", which it did. Two more defects sat in the object writer, both producing files our own parser rejects: dictionary keys were written unescaped (a key with a space reparses as "expected number"), and f64::NAN was emitted as the literal token NaN, so one non-finite value anywhere made the document unreadable. Added: outline trees with the open/closed state in the sign of /Count, /PageLabels as a number tree with real roman and A..Z/AA..ZZ numbering, named destinations, attachments with file specs, /Info, XMP, viewer preferences, page mode and layout; AcroForm creation for text, checkbox, radio, choice and signature fields with generated appearances; and TrueType subsetting - DejaVu Sans goes from 759,720 bytes to 4,348 for twelve characters. cmap is deliberately not rebuilt: the subset is embedded as a CID font with Identity-H, so the content stream addresses glyphs by id and /ToUnicode serves extraction. A cmap disagreeing with the content stream is worse than none. CFF is refused by name rather than emitting a font with no glyphs. Nine real bugs, every one found by running the output through an independent tool rather than by reading the code: 1 page size discarded reading a generated file back 2 dict keys unescaped probing the writer 3 NaN written as a keyword probing the writer 4 subset zeroed the lsb fontTools outline compare 5 hmtx indexed by new gid fontTools outline compare 6 name table format read as count BaseFont came out "Embedded" 7 add_font shifted numbers already handed out 8 trees allocated over font numbers - object 29 written twice 9 widgets missing /F Print, /P and appearance /Resources 7 and 8 are the instructive pair: every reference resolved and every object existed, each simply named the wrong thing. pypdf reported correct field values from a file PDFium rendered blank. 9 is the one only a renderer could find - /F defaults to non-printable, and a form XObject naming a font its /Resources does not declare is discarded whole. Verified by three independent implementations: fontTools (0 outline mismatches of 12 against the source font), pypdf (metadata, page sizes, outline with resolved page numbers, all five fields, attachment byte-for-byte, labels ['i','1']) and PDFium, which renders both pages correctly. cargo run -p nigig-pdf-graphics --example generate_sample regenerates the sample. Fourteen mutations. Three survived and each exposed a weak test: the key test used an attachment name (written as a string, never a key), nothing read the outline open state, and /P could not be witnessed because page_index is supplied by the reader, which already knows the page. All three now killed. pdf: 789 passed (was 730). pdf-ui: 775. Coverage 85.17%.	2026-08-16 21:02:19 +00:00
andodeki	6d2e3fb696	fix(pdf): repair the tree a hand-resolved merge left red ... Commit 258fa32 ("Merge origin/main: resolve xref/document conflicts") is not a merge - it has one parent - and it landed a PDF tree that does not pass its own gates. Found by running the suite, not by reading the diff. Four separate breakages: 1. xref_streams::a_wrong_type_at_startxref_is_named_not_guessed FAILED. The conflict resolution replaced the diagnostic error with a generic "expected xref or cross-reference stream". The fixture writes /Type /Frobnicate and the parser has that name in hand; throwing it away leaves the reader knowing only that this is not what we wanted, which is the least useful half of the story. Restored, and extended to name a stream with no /Type at all. This is an ADR 0013 merge criterion. 2. golden_render::golden_text_line_ops FAILED - the merge added the test but not its golden file. Verified the output by hand against the spec before blessing it: leading 14, lines at 700/684/670/656/642, which is correct for Td/TD/T*/'/". A golden file blessed without reading it locks in whatever bug exists. That golden cannot witness the " operator's two spacing operands: word and character spacing live in the graphics state and emit no render command, so dropping both would leave the file byte-identical. Added a test that asserts the state directly. Mutation-checked: removing the two set_*_spacing calls fails it, and leaves the golden untouched. 3. clippy -D warnings failed with 6 errors on pdf-cos and pdf-document, so CI's engine job could not have passed. Two unused imports, a dead read_be (superseded by the `field` closure in parse_xref_stream), a collapsible if-let, and - committed into the source - // ... keep the entire top of the file unchanged until the resolve_num function ... an editing instruction left in document.rs as a doc comment. 4. cargo fmt --check failed on document.rs and xref.rs. Mutation-checked both fixes: reverting the xref message fails 1 test, dropping the " spacing operands fails 1. pdf: 730 passed. pdf-ui: 775 passed. Coverage 84.15%, floors met; destinations.rs holds at 98.65%.	2026-08-16 20:09:48 +00:00
andodeki	258fa3259e	Merge origin/main: resolve xref/document conflicts, add makepad_table	2026-08-16 22:53:23 +03:00
andodeki	82eb6b9c73	feat(pdf): internal links that actually go somewhere ... ADR 0017 left destinations.rs at 0% coverage as an open item. The obvious reading is "an untested module". The real one is worse: nothing called it. It was pub use'd from lib.rs and referenced from nowhere else in the workspace. 0% was not a gap in the tests, it was the symptom of dead code, and nothing else was doing the job. Meanwhile PdfAnnotation read a link's target as dict.get_name("Dest") - a *name* /Dest and nothing else. Not /Dest [4 0 R /Fit], and not /A << /S /GoTo /D ... >>, which is how internal links are written in practically every real document. The corpus has had one since Phase 6, in annotations/links.pdf, and no test asserted where it went: Link { uri: None, dest: None } -> action=None Clicking it did nothing. No error, no warning - the viewer got no action and correctly performed none. A link to nowhere and a link the reader cannot parse look identical from outside. The viewer was already wired for this: PdfAction::GoToPage exists, is matched in test_host.rs, and was never constructed by anything. A complete delivery path with nothing at the source. Now: all three legal spellings parse, named destinations resolve through the /Names /Dests tree *and* the pre-1.2 /Root /Dests dictionary, and resolution happens in page_annotations where the catalogue is in reach. XYZ keeps Option per component because null is meaningful there and only there - it means "leave unchanged". Reading it as 0.0 scrolls to the origin at 0% magnification. Zoom 0 means the same as null and is normalised. Lookup uses a deliberate shallow resolve. Deep-resolving a destination array replaces [4 0 R /Fit] with the page dictionary and destroys the only thing identifying the target - the defect that once emptied every AcroForm (ADR 0006) and every annotation reference (ADR 0004). GoToAction now requires /S to be GoTo. The old code ignored /S and took /D from whatever it was handed, so a /GoToR (another file), /Launch (a program) or /JavaScript carrying a /D was reported as a local page jump. Refuse by verb, same policy as ADR 0012. An unresolvable destination is left unresolved, never defaulted to page 0: silently landing on page one is the worst outcome because it looks like the link worked. Seven mutations, all killed. M1 - removing the /S check - reported as surviving on the first attempt. It had not survived: the patch string omitted an interleaved comment so the mutation never applied and I measured the unmutated build. A harness that does not verify its own mutation says "weak test" when the truth is "never ran", and the conclusion would have been to delete a real security check. Every mutation now asserts it applied. destinations.rs 0% -> 98.65%; total 83.42% -> 83.86%. Floors added for destinations.rs and annotations.rs, verified to fail when breached. AnnotationType::Link changes shape (dest: Option<String> -> destination: Option<Destination>) and AnnotationAction gains GoToDestination; the old field could not express an explicit destination, so keeping it meant keeping the bug. AnnotationAction loses Eq because a destination carries f64 coordinates. pdf: 724 passed (was 695). pdf-ui: 769 passed (was 725). ADR 0018.	2026-08-16 19:34:01 +00:00
andodeki	cf73ef4c1d	test(pdf): assert what a file declares is delivered, and floor the coverage ... Every serious bug in this stack has had one shape: a valid, well-typed, empty-or-default value where the file plainly declared content. xobjects empty for every document; acroform() dropping every field behind an indirect reference; DCTDecode returning its own compressed bytes; a JPEG decoder that was a stub returning black. None errored, none panicked, and the tests asserted Ok, which they got. Coverage would not have caught any of them. Measured when each shipped: page.rs 92.4%, form.rs 93.6%, content.rs 89.2%, xref.rs 95.2%. The buggy lines ran; nobody checked what they produced. So: a property test that walks the raw object graph of every corpus fixture, counts what the file declares, and requires the API to deliver it - fonts, xobjects, graphics states, colour spaces, form fields, filters, MediaBox. It reimplements the resolution rule independently of page.rs on purpose; a test that asks the code under test what to expect agrees with the bug. It failed the day it was written, on a shape the corpus had never contained. Every fixture wrote /Resources inline, and all six extractors read it with dict.get_dict("Resources") - which returns None for an indirect reference and never consulted /Parent. A page with "/Resources 5 0 R", the commonest shape in real PDFs, reported no fonts, no xobjects, no graphics states and no colour spaces. Same for a page inheriting resources from its /Pages node. Empty, not wrong, so nothing failed. Fixed by resolving /Resources once in PdfPage::from_obj through a helper implementing the full inheritance rule (32000-1 Table 30), and passing the resolved dictionary down. Indirect /MediaBox entries resolve too. Six resources/ fixtures cover the shapes that were missing. Mutation-checked: reverting inheritance kills 5 tests, the sub-dict reference 3, indirect MediaBox 2, and removing the depth bound hangs. One mutation survived - a visited-set guarding a /Parent cycle, which the depth bound already handles - so it was deleted rather than left as untested defence with a reassuring comment. tools/test-pdf-coverage.sh enforces a floor instead of printing a number, with per-file floors as well as a total: image.rs could fall from 33% to 5% and move the total by under a point. All three failure modes verified to fail. It caught a bug in itself first - its ignore regex matched its own work directory and reported a confident TOTAL 0.00%. .gitattributes marks *.pdf binary. An xref entry must be exactly 20 bytes (7.5.4), so with a one-digit generation field it ends in a space, and git diff --check was reporting unfixable "trailing whitespace" on every fixture in the corpus. TEST_TARGET=pdf: 695 passed, 0 failed (was 680). Coverage 83.42%. ADR 0017 records the four mutations so they can be repeated by hand.	2026-08-16 19:04:57 +00:00
andodeki	63ff45149a	feat(pdf): a real JPEG decoder — the old one was a stub returning black ... Completes Phase 3 of NIGIG_PDF_FEATURE_PARITY_PLAN.md. Design and merge criteria in REVIEWS/adr/0016-pdf-image-decode-surface.md. ADR 0015 refused DCTDecode at the generic filter boundary and left the image path alone, noting JPEG "is decoded on the image path". That claim did not hold: fn decode_jpeg_data(_data, _pixels, _width, _height, _components) -> Option<()> { Some(()) } Every argument discarded. It wrote nothing and returned success. The caller allocated a zero-filled buffer, passed it in, and returned it as decoded pixels. Probing a real 8x8 JPEG through ImageInfo: decode_to_rgba -> 256 bytes, first 12: [0,0,0,255, 0,0,0,255, 0,0,0,255] Pure black at full alpha. Not an error, not None - a correctly sized, entirely fabricated image. EVERY JPEG IN EVERY PDF rendered as a black rectangle and nothing reported it. The underscore-prefixed parameters are the tell: the signature was written to silence the unused warnings that would otherwise have announced the stub. image.rs was at 14.2% line coverage, the lowest in the crate. Replaced with a real baseline decoder in pdf-graphics/src/jpeg.rs: huffman, dequantisation, IDCT, chroma upsampling, YCbCr/YCCK conversion including the Adobe APP14 transform flag. No new dependency - adding `image` or `jpeg-decoder` would pull a tree into a crate that has one, on a target the team is already fighting to cross-compile. Progressive JPEG is refused BY NAME rather than approximated; a partial implementation would reproduce exactly the defect being fixed. decode_to_rgba's Option is why the stub survived - "could not decode" and "decoded to nothing" were the same value. The decoder returns a typed JpegError so a caller learns why an image is missing. Also in this tranche, from the same plan bullets: - ImageInfo::downsample, integer-factor box filter. Refuses factor 0, and refuses data that is not raw samples rather than averaging compressed bytes as though they were pixels. - Round-trip tests for encode_flate and encode_ascii_hex over adversarial inputs: empty, single byte, all-zero, all-0xFF, random binary. THE IDCT TOOK THREE ATTEMPTS AND THE FAILURES WERE INFORMATIVE The first version, adapted from a hand-tuned integer kernel, decoded greyscale exactly (128 -> 128) while colour came out a UNIFORM 64 levels off. A constant offset across every channel is a scaling-factor mistake, not a coefficient one - guessing at coefficients would never have found it. Two rounds of guess-and-check made it worse. The fix was to stop guessing: derive ground truth from the float reference in T.81 A.3.3, then transcribe the separable form directly with a documented fixed-point scale. The cosine table is a const fn so it cannot drift from the formula beside it, and tests assert against the reference rather than our output. 4 corpus fixtures with real JPEGs (Pillow at generate time only; the .pdf files are committed so CI never needs it), 16 acceptance tests asserting PIXEL VALUES rather than buffer lengths - a length assertion would have passed against the stub. Mutation-checked: reinstating the zero buffer fails four tests. Coverage on image.rs 14.2% -> 32.9%, new jpeg.rs 82.8%, crate 83.65% -> 84.22%. TEST_TARGET=pdf 651 -> 680, TEST_TARGET=pdf-ui 696 -> 725. rustfmt and clippy -D warnings clean.	2026-08-16 18:33:56 +00:00
andodeki	fb95b25a67	fix(pdf): LZW was broken outright; refuse image codecs instead of faking them ... Phase 3 of NIGIG_PDF_FEATURE_PARITY_PLAN.md, lossless half. Design and merge criteria in REVIEWS/adr/0015-pdf-filters-and-codecs.md. LZW DID NOT WORK Fed the worked example from PDF 32000-1 section 7.4.4.2: LZW default : Err("LZW previous code out of range") decode_lzw seeded a 256-entry dictionary but set next_code = 258, because 256 and 257 are the clear and EOI codes. New entries were appended with table.push, landing at index 256 - so the counter and the real index were permanently two apart and every dictionary reference resolved to the wrong entry. Any PDF using LZW was affected, which is a whole class of older files. Also in the same area: - /EarlyChange was ignored. It selects when the code width grows; a file setting 0 decoded to GARBAGE rather than failing, which is worse. - Predictors were applied to Flate only, though /Predictor is equally legal on LZWDecode. TWO MORE BUGS FOUND WHILE IMPLEMENTING decode_stream read /Filter as a single NAME and fell through to "unsupported filter" for an array. The document layer calls decode_stream, so every chained stream in every document failed to decode - including the common [/ASCII85Decode /FlateDecode]. It now delegates to decode_stream_with_params, leaving one decoding path. decode_flate_with_predictor inflated its own input, so calling it from a chain decompressed already-decompressed bytes. Split into apply_predictor, which works on decoded data. IMAGE CODECS: REFUSED, NOT FAKED DCTDecode and JPXDecode previously returned their COMPRESSED bytes as though decoded: "DCTDecode" | "JPXDecode" | "Crypt" => data, A caller received a Vec<u8> that looked like image data, was not, and produced garbage pixels rather than an error. CCITTFaxDecode, JBIG2Decode, JPXDecode and DCTDecode now return a typed error naming the filter. image.rs still sniffs and decodes JPEG on the image path, so that route is unaffected; what stops is the generic filter claiming a success it did not achieve. /Crypt stays a pass-through, correctly - decryption already ran. Not implementing CCITT/JBIG2/JPX is a decision, not an omission: JBIG2's CVE record is why browsers sandbox it, and JPX via openjpeg would add a C dependency that breaks the Android cross-compile the team is already fighting. CCITT is the tractable one and is the recommended next step. 4 corpus fixtures, 14 acceptance tests. Mutation-checked - and one check initially misled me: removing the reserved-slot seeding did not fail the tests, because the clear-code branch re-seeds independently and every real LZW stream opens with a clear code. Removing both fails all three LZW tests. Recorded in the ADR. One pre-existing defect deliberately left: the PNG predictors do not consume the per-row filter-type byte. Fixing it risks every Flate-with-predictor document in the corpus and is not what this ADR set out to do, so it is documented rather than quietly half-fixed. TEST_TARGET=pdf 637 -> 651, TEST_TARGET=pdf-ui 682 -> 696. rustfmt and clippy -D warnings clean.	2026-08-16 18:04:07 +00:00
andodeki	6a18886185	feat(pdf): Type 3 fonts and streaming interpretation — Phase 2 complete ... The last two items of NIGIG_PDF_FEATURE_PARITY_PLAN.md Phase 2. Design and merge criteria in REVIEWS/adr/0014-pdf-type3-fonts-and-streaming.md. TYPE 3 FONTS DREW NOTHING A Type 3 font's glyphs are not outlines - they are content streams, listed in /CharProcs and mapped to text space by /FontMatrix. Probing a document with one: fonts on page: ["T3"] T3: subtype=Type3 base=Unknown -> are the glyph procedures reachable? no CharProcs field exists -> is /FontMatrix exposed? no field exists The font was detected and then nothing could be done with it. /CharProcs and /FontMatrix appeared nowhere in the crate, so the procedures were unreachable and the text was silently invisible - a page that renders, reports no error, and is missing content. New pdf-document/src/type3.rs parses /FontMatrix, /CharProcs, /Differences, /Widths, /FontBBox and the font's own /Resources, and resolves a character code to its glyph procedure's decoded bytes. /FontMatrix is applied as written rather than assumed to be the common 0.001 scale - Type 3 fonts routinely use other matrices, which is the point of the entry. A missing /CharProcs entry is a typed error naming the glyph, not a blank. A THIRD BUG, FOUND WHILE WIRING d0/d1 The interpreter parsed both operators and discarded them: PdfOp::Type3Width(_wx, _wy) => {} PdfOp::Type3BBox(_x1, _y1, _x2, _y2) => {} They are how a Type 3 glyph declares its advance, so even a renderer that could draw the glyphs would stack them all at one point. Wiring them to the device exposed that `d1` takes SIX operands - wx wy llx lly urx ury - and the parser read four, so the "bounding box" was really the advance and the advance was lost entirely. Now `Type3BBox { wx, wy, bbox }`, reading all six. STREAMING INTERPRETATION parse_content_stream materialised every operator into a Vec before interpreting any of them: peak memory proportional to the whole content stream, on a stream walked once and discarded. Adds ContentStreamIter and interpret_streaming, with parse_content_stream reimplemented on top of the iterator so there is ONE tokeniser rather than two that can drift. Equivalence is proven, not asserted: a test compares both paths across every corpus fixture, and a streaming_interpreter fuzz target compares them over arbitrary bytes, which is where a divergence would actually hide. 4 corpus fixtures, 13 acceptance tests, 9 unit tests. Mutation-checked: reverting d0 to a no-op fails glyph_advances_reach_the_device. Phase 2 is now complete; the plan is updated with an item-by-item audit. Several entries were already done (inline images, Do, text state, shading); the plan's "biggest gap" was xref streams, closed in ADR 0013. TEST_TARGET=pdf 615 -> 637, TEST_TARGET=pdf-ui 660 -> 682. rustfmt and clippy -D warnings clean.	2026-08-16 17:28:44 +00:00
andodeki	2faadb777f	feat(pdf): read xref streams and object streams (PDF 1.5+) ... Phase 2 of NIGIG_PDF_FEATURE_PARITY_PLAN.md, the item it calls "the biggest parse-side gap". Design and merge criteria in REVIEWS/adr/0013-pdf-xref-streams.md. The parser could not open a PDF 1.5 file. Not render it wrong - not open it: PARSE FAILED: PDF error at byte 382: expected xref keyword XRefTable::parse_section required the literal bytes `xref` at the startxref offset. A PDF 1.5+ file has an indirect object there instead - the xref stream - so the parse aborted and the entire document was unreadable. Every feature built on top of the parser (encryption, signatures, forms, structure tree, transparency) was unreachable on any file produced in the last twenty years. ObjStm, XRefStm and /Type /XRef appeared nowhere in the crate. Implemented on the read side: - Xref streams: the packed binary table, /W field widths, /Index sparse subsections, and types 0/1/2. A zero-width /W column means "use the default" (type 1) - missing that rule yields a table of all-free entries and an apparently empty document rather than an error. - Object streams: type-2 entries resolve through /ObjStm, reading the header pairs and /First. The xref's index is used but verified against the object number it claims to be, because a wrong-but-in-range index would silently return a different object. - Hybrid files: a traditional table plus /XRefStm. Both are read, with the traditional table winning on conflict, which is the point of the layout. Bounds and refusals rather than silent degradation: /W widths are clamped and every field read is checked against the decoded buffer; a truncated table is flagged, not padded with free entries; an object claiming to live inside itself is refused; a /Type that is not /XRef is named in the error. Scope note: the writer is untouched. ADR 0003 keeps appending a traditional xref section, which remains correct - the appended trailer carries /Prev to the stream, so the chain stays readable by us and by conforming readers. Also verified against the rest of Phase 2: inline images, XObject Do, shading, and the full text state (Tc/Tw/TL/Tz/Ts/Td/TD/Tm/Tf) are already implemented and tested. Type 3 fonts and the streaming interpreter remain genuine gaps, but each degrades one feature rather than the whole file. 6 corpus fixtures, 9 acceptance tests asserting real page content rather than a successful parse, and a parse_xref_stream fuzz target because the table is attacker-controlled binary. Mutation-checked: restoring the old error fails 5 of the 9. TEST_TARGET=pdf 606 -> 615, TEST_TARGET=pdf-ui 651 -> 660. rustfmt and clippy -D warnings clean.	2026-08-16 17:10:42 +00:00
andodeki	b720a166ec	feat(pdf): AcroForm actions and validation; JavaScript refused, not run ... Phase 8 #8 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md ("AcroForm full support"). Design and merge criteria in REVIEWS/adr/0012-pdf-acroform-full.md. Form editing already worked. What was missing is everything that makes a real form behave like one. Probing an invoice-shaped document: "qty" required=true /AA present? false "price" required=false /AA present? true "total" required=false /AA present? true -> is any /AA action exposed? no accessor exists -> is /CO exposed? no accessor exists The /AA dictionaries sat in the raw field dictionary, reachable only by a caller who knew to go digging. Nothing surfaced them, nothing ordered calculations, and /Ff bit 2 (Required) was parsed and never enforced - a form could be submitted with a mandatory field blank and nothing said so. THE JAVASCRIPT DECISION The review names JavaScript actions as a prerequisite. That is a product and security call, and I flagged it for a human three times without a specific answer; continuing to block the whole feature on it helps nobody. This takes the reversible option and documents it loudly enough to overrule: JavaScript is parsed, exposed, and NEVER EXECUTED. Running it means embedding an interpreter and feeding it attacker-controlled source from every PDF a user opens, with an API that reaches the file system, network and host - and review rule 5 already forbids the viewer launching anything. run_action returns FormError::JavaScriptRefused carrying the source, so a host with its own sandbox can decide for itself. A later ADR turns a refusal into an execution; nothing has to be un-built. Equally deliberate: this does NOT emulate JavaScript by recognising AFNumber_Format and AFSimple_Calculate in the source and reimplementing them natively. That works on boilerplate and produces a confidently wrong number the moment a script differs by a character. WHAT IS IMPLEMENTED - /AA parsed into typed actions across all 14 triggers, on fields and widgets, with indirect action dictionaries resolved. - /CO calculation order exposed in document order. - Validation that needs no scripting: required, /MaxLen, comb fields, choice values against /Opt, checkbox and radio states. - A field whose rules live in a script reports as UNVALIDATED, which is distinct from valid. Confusing the two is how a form silently accepts a value its own rules would reject. - /SubmitForm parsed into URL, flags and fields and returned as data. This crate opens no sockets. Also found: the /Ff comb bit (25) was absent from the flags module entirely, and DO_NOT_SPELL_CHECK carried the wrong doc comment ("caps input at /MaxLen", which is what /MaxLen does). 4 corpus fixtures, 13 acceptance tests, 20 unit tests. Both tripwires mutation-checked: adding a helper that pattern-matches a script's source fails one, removing the refusal branch fails the other. TEST_TARGET=pdf 575 -> 607, TEST_TARGET=pdf-ui 619 -> 651. rustfmt and clippy -D warnings clean.	2026-07-31 22:41:50 +00:00
andodeki	a994213e8b	feat(pdf): tagged structure tree, and the BMC bug that turned red green ... Phase 8 #7 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md ("PDF/UA structure tree"). Design and merge criteria in REVIEWS/adr/0011-pdf-structure-tree.md. Investigating the accessibility gap surfaced four defects in the marked-content operators the structure tree depends on. The first is not an accessibility problem at all. 1. BMC never parsed, and corrupted the colour of everything after it. The dispatcher matched b'B'+b'M' only when the third byte was a delimiter; BMC's third byte is 'C', so the arm never fired. Because the operator was never recognised it never CLEARED ITS OPERAND, and the leftover /Tag shifted the operands of whatever came next: 1 0 0 rg -> RgbFill(1.0, 0.0, 0.0) red /Span BMC 1 0 0 rg -> RgbFill(0.0, 1.0, 0.0) green Tagged documents are precisely the ones containing BMC, so the documents that tried hardest to be accessible rendered wrong colours. This is the fifth instance of the operator-shadowing family already fixed for cm, rg, gs, b/b* and end-of-stream text operators. ADR 0009's every_multi_char_operator_parses_as_itself test exists to stop exactly this - and would have caught it, except BMC was one of two operators excluded from its table as "genuinely unimplemented". Excluding a known-broken operator from the test whose job is finding broken operators is how it survived. The exclusion list is gone. 2. BDC discarded its property list, which carries /MCID - the only link between a run of page content and the structure element describing it. Without it a tree can be parsed but never attached to anything. 3. The op produced by BDC was named MarkContentBmc, and BMC produced nothing. The names were the wrong way round, which is how the missing arm survived review: the enum looked like it had a BMC case. 4. Found while fixing 2: the content lexer had no dictionary support at all. It read `<` as a hex string without checking for a second `<`, so `<</MCID 0>>` parsed as the string "0C0D0". Content streams now parse direct objects properly, bounded at 16 levels; a single `<` is still a hex string and a test pins that. On top of that, new pdf-document/src/structure.rs: /StructTreeRoot, /StructElem trees, /RoleMap resolution, depth-first reading order, /Alt, /ActualText, /E, /Lang, and MCID-to-element lookup. Cycles in /K are cut at the first repeat and reported by object number rather than expanded to the depth bound. Accessibility findings are mechanical checks reported as findings, NOT a conformance verdict: there is no is_pdf_ua and no ComplianceReport, and a mutation-checked tripwire test fails if either appears. Real PDF/UA conformance needs human judgement - whether /Alt text is accurate is not mechanically decidable - so claiming it from six checks would be exactly the overclaim this codebase keeps removing. 7 corpus fixtures, 14 acceptance tests, 22 unit tests, and a parse_content_dict fuzz target for the new object parser. TEST_TARGET=pdf 536 -> 575, TEST_TARGET=pdf-ui 580 -> 619. rustfmt and clippy -D warnings clean.	2026-07-31 22:19:45 +00:00
andodeki	e4a0c0a79e	feat(pdf): read signatures, and fix a third ObjRef-destroying resolve ... Phase 8 #6 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md ("Signatures"). Design and merge criteria in REVIEWS/adr/0010-pdf-signatures.md. Two defects, and the first is not about signatures at all. 1. acroform() dereferenced /AcroForm with self.resolve(), which recurses. That replaced every /Fields [5 0 R] entry with an inline dictionary, so AcroForm::walk saw node.as_ref() == None, decided the field had no identity to key an edit on, and dropped it. The form came back EMPTY rather than wrong, which is indistinguishable from a document with no fields, so nothing announced the loss. This is the third appearance of one mistake: page_annotations once destroyed every annotation's obj_ref the same way, and extract_xobjects resolved a reference then asked the resolved object for as_ref(), leaving every page's XObject map empty. It was invisible because both existing fixtures declare /AcroForm as an INDIRECT reference, where only one level is dereferenced and the refs survive. A direct /AcroForm dictionary - equally legal - hits it. The new fixture uses one deliberately; reverting the one-line fix fails 9 of the 12 new acceptance tests. 2. Nothing read signatures. FieldType::Signature was classified and then ignored; /ByteRange, /Contents, /SubFilter and /DocMDP appear nowhere in the codebase. A signed contract was presented exactly like an unsigned one. New pdf-document/src/signature.rs reads the signature dictionary and checks BYTE-RANGE INTEGRITY, which needs no cryptography and catches the common real-world tampering: whether the signed range reaches the end of the file. A signature that stops short leaves appended bytes uncovered, which is exactly how an incremental-update attack hides content behind a signature that still verifies. Cryptographic verification is NOT implemented and cannot be faked: VerificationStatus has no Valid variant. That is enforced by the type, not by convention, because the failure mode for a signature feature is not "it doesn't work" - it is a green tick beside a document nobody checked. A test asserts the capability's absence so adding Valid without the cryptography breaks the build rather than shipping a false tick. Signing is out of scope entirely: no private keys in this crate. Also verified ADR 0003's claim that an append-only save preserves a signed byte range, byte for byte, rather than leaving it asserted. Implementation bug worth recording: /Contents was initially hex-decoded, but the COS lexer already decodes <...>. Running it twice on the common 128-zero-byte placeholder - which contains no hex digits - produced an EMPTY vector, silently discarding the signature blob while every other field looked right. Caught by asserting the blob is non-empty rather than asserting the parse returned Ok. 6 corpus fixtures, 12 acceptance tests, 28 unit tests, and a parse_signature fuzz target because /ByteRange is four attacker-controlled integers used to index the file. TEST_TARGET=pdf 497 -> 536 passing. rustfmt and clippy -D warnings clean.	2026-07-31 20:02:29 +00:00
andodeki	d8d29c226d	feat(pdf): transparency, and four operator-parsing bugs it exposed ... Phase 8 #5 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md ("Advanced transparency"). Design and merge criteria in REVIEWS/adr/0009-pdf-transparency.md. The headline defect was not that transparency was missing. `gs` was MIS-PARSED as CloseStroke: /GS0 gs 1 0 0 rg 100 100 200 200 re f -> [CloseStroke, RgbFill, Rectangle, FillWinding] so every page setting a graphics state gained a stroked path the document never asked for, and lost its alpha, blend mode and soft mask silently. Downstream everything was dead: StrokeExtGState/FillExtGState were never constructed, set_fill_opacity was never called and emitted no command when it was, and PdfPage::ext_gstate was read by no code at all. New pdf-graphics/src/transparency.rs: full /ExtGState (ca, CA, BM, SMask, LW, LC, LJ, ML, D, AIS, TK), all sixteen blend modes including the four non-separable ones, soft masks with /S, /G, /BC and a /TR evaluated through ADR 0006's PdfFunction, and /Group parsing. Wired end to end: parser -> PdfOp::SetExtGState -> device -> RenderCommand -> Makepad renderer. Compositing is NOT claimed. Backdrop blending needs render-to-texture, which an engine-neutral crate has no framebuffer for. Constant alpha is applied because it needs no backdrop; blend modes and soft masks are reported through TransparencyError::Unsupported rather than dropped, because a silently ignored /Multiply looks exactly like a correct /Normal. The ADR required a test enumerating every multi-character operator, on the grounds that fixing the third instance of a shadowing bug (after cm and rg) without preventing the fourth is not a fix. It immediately found three more live bugs, none of them transparency-related: - `b` and `b*` dropped their close-path, mapping to FillStroke* instead of CloseFillStroke*, so every closed-and-stroked path drew with a gap. - Text operators within three bytes of the end of a content stream were mis-parsed: the b'T' arm guarded `*i + 3 < len` while reading only two bytes, so a stream ending in `/F1 12 Tf` parsed as FillWinding. And the transparency-group fixture found a fourth: - PdfPage::xobjects was empty for every page of every document. extract_xobjects called doc.resolve(), which follows the reference, then asked the resolved object for as_ref() - always None - and only accepted a bare dict when every XObject is a stream. No `Do` operator could be resolved through the page model. Same defect as the one that once destroyed annotation object references; it now has its own regression test. T* and BMC are genuinely unimplemented and are deliberately excluded from the guard's table rather than papered over. 7 corpus fixtures, 12 acceptance tests, 32 unit tests asserting the §11.3.5 formulas (not our own output), and a parse_ext_gstate fuzz target. TEST_TARGET=pdf 451 -> 497, TEST_TARGET=pdf-ui 495 -> 541. rustfmt and clippy -D warnings clean.	2026-07-31 18:58:31 +00:00
andodeki	00f1dfbc12	fix(pdf): fuzz every target, and close two ADR 0004 gaps ... Three defects found by auditing the ADR merge criteria against the code rather than against memory. 1. The scheduled fuzz job ran five hardcoded targets. Four had been added since and were never fuzzed: parse_revision_chain, decrypt, eval_function and parse_colorspace. eval_function is the sharpest of those - it executes PostScript taken verbatim from an untrusted file. The list now comes from `cargo fuzz list` and the job fails rather than passing vacuously if it comes back empty. `cargo fuzz list` reads the manifest, so a target file added without its [[bin]] entry would still be skipped silently. The engine job, which runs on every push, now checks the two agree. Both directions of that guard were exercised before committing. 2. ADR 0004 rule 3 promises an annotation whose appearance cannot be generated "keeps its original /AP and is reported as skipped". The keeping worked - to_dict clones the source dictionary - but nothing reported it: SaveReport only tracked skipped appearances for form fields. A caller who moved a stamp was never told its artwork still showed the old position. Adds SaveReport::annotation_appearances_skipped and appearance_is_generated, which enumerates the out-of-scope types explicitly so a new AnnotationType fails to compile until classified. 3. SetContents and SetFlags had no round-trip test. Both were implemented and unit-tested against the in-memory model, but neither was ever reparsed from written bytes - the assertion ADR 0004 calls central. New fixture annotations/stamp.pdf carries real /AP artwork for a Stamp (undrawable: must be preserved and reported) beside a Square (drawable: must not be reported), so the reporting cannot pass by reporting everything. The stamp test was mutation-checked: it fails when the reporting line is removed. ADR 0003 and 0004 merge criteria are now ticked. 0003's were genuine paperwork - every box traced to an existing named test. 0004's were not, and its ADR now records what was missing rather than implying it always worked. TEST_TARGET=pdf 447 -> 451 passing. rustfmt and clippy -D warnings clean.	2026-07-31 18:34:25 +00:00
andodeki	7d21532ebf	feat(pdf): real colour spaces, ICC profiles and PDF functions ... Phase 8 #4 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md ("Advanced colour"). Design and merge criteria in REVIEWS/adr/0006-pdf-advanced-color.md. The interpreter tracked only the *name* of the active colour space and then passed sc/scn operands to the device as if they were already RGBA. Every non-device space therefore rendered a confident wrong colour with no error: /Spot cs 1.0 scn full tint of a spot ink -> pure red /Idx cs 3 scn palette entry 3 -> near-black /Lab cs 50 0 0 scn mid gray -> white (clamped) /DevN cs (5 inks) five colorants -> inks 5+ discarded /ICCBased profile-defined colour -> profile discarded DeviceCMYK also used the additive 1-c-k conversion, which crushes any colour printed over black. Three new modules in pdf-graphics: - function.rs PDF functions, all four types. Type 4 runs on a bounded interpreter: depth 32, 32768 tokens, stack 100, 100000 steps, and an unknown operator is an error rather than a no-op that would leave a plausible wrong colour. - icc.rs ICC matrix/TRC and gray kTRC profiles, applied exactly. LUT-class profiles are reported as such and the caller falls back to /Alternate; they are never pretended to be matrix profiles. - colorspace.rs All eleven families, converting through XYZ with Bradford adaptation and a real sRGB transfer function. Wiring: - PdfDevice gains set_stroke_components/set_fill_components, so SC/SCN reach the device as components of the active space instead of being read positionally as RGBA. - cs/CS now resets to the space's initial colour (table 74), which is why golden/colors.txt gains a line. - PdfPage::color_spaces carries /Resources /ColorSpace fully dereferenced with streams decoded; a half-resolved space would make every ICC profile, palette and type 0/4 transform silently fall back. - A space that cannot be resolved keeps the previous colour and records a typed ColorError. No colour is invented, and no error is swallowed. Tests: 12 corpus fixtures under tests/corpus/color/, 14 acceptance tests in pdf-document/tests/color.rs asserting numeric RGB (the broken code produced a colour for every one of these; only the value was wrong), plus unit tests per function type and per curve type. Two fuzz targets added: eval_function and parse_colorspace. TEST_TARGET=pdf 387 -> 447 passing, TEST_TARGET=pdf-ui 431 -> 491. rustfmt and clippy -D warnings clean.	2026-07-28 20:48:55 +00:00
andodeki	147ca7de23	test(pdf): prove the AES-256 path and harden malformed encryption ... Closes the gaps in ADR 0005's own merge criteria. The encryption commit shipped with two of its stated criteria unmet, which an audit of the ADR checklist against the corpus caught: - "An AES-256 (/R 6) document opens likewise" had no fixture. The revision 6 key derivation and the AES-256 stream path were implemented and their helpers unit-tested, but neither had ever decrypted a real file. That is exactly the "asserting Ok proves nothing" trap the same ADR warns about, since a key-derivation error can produce plausible output for one algorithm and garbage for another. - "Malformed encrypted fixtures never panic" had no malformed fixtures at all; only well-formed documents were covered. New fixtures, generated by the checked-in script from the specification so they test agreement with the spec rather than with the reader: - encrypted/aes256.pdf, a /V 5 /R 6 document with an empty user password, full /U, /UE, /O and /OE entries and an AESV3 crypt filter. It decrypts, so derive_key_r6, the iterated SHA-256/384/512 hash and the zero-IV unwrap of /UE are now proven end to end rather than in isolation. - encrypted/truncated_u.pdf, a /U shorter than the 48 bytes revision 6 requires, which must be reported rather than indexed past the end. - encrypted/missing_o.pdf, an /Encrypt dictionary with no /O. - encrypted/absurd_length.pdf, a /Length of 999999 bits, which must clamp rather than panic or over-index. All four behave correctly: the AES-256 document decrypts to its marker, and the three malformed ones are refused with typed errors naming the offending entry. Encryption tests go from 13 to 17. The ADR's merge criteria are now ticked, with a note recording that the original commit shipped with the revision 6 path unproven and that this follow-up is what closed it. Not done here: the decrypt fuzz target could not be re-run. The nightly toolchain now available (2026-07-27) fails to build the cc crate that libfuzzer depends on, with errors inside cc itself rather than in this code. The target still compiles under stable and the earlier run of 1,953,940 executions stands; this is recorded rather than quietly skipped. Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh (387 tests) Both rustfmt and clippy -D warnings clean.	2026-07-28 17:50:44 +00:00
andodeki	01e16b6383	feat(pdf): implement encryption (Phase 8) ... Phase 8 feature 3 of 10, designed in REVIEWS/adr/0005-pdf-encryption.md. An encrypted PDF did something worse than fail: it succeeded. Probing a structurally valid RC4-encrypted file through the parser gave parsed OK: pages=1 page 0 content bytes=44 content parsed into 0 ops No error and no warning. The document reported a page, the page reported content, and the content interpreted to nothing because it was ciphertext. The user saw a blank page and was told the file was fine. That is the defect class Phase 0 existed to remove, and it was the worst one left in the PDF stack because it was silent. New pdf-cos/src/encrypt.rs implements the standard security handler for reading: - V1/R2 RC4 40-bit, V2/R3 RC4 40 to 128-bit, V4/R4 crypt filters selecting RC4 or AES-128, and V5/R6 AES-256 with the SHA-256 based revision 6 hash. - The empty user password, which is the common case for a document encrypted only to set permissions, and explicit user or owner passwords. The owner path recovers the user password from /O and re-derives. - Per-object keys, as the spec requires. Reusing one keystream across objects would be a real cryptographic break, so the object and generation numbers are mixed in by construction and a test asserts the keys differ. Every primitive comes from audited RustCrypto crates: aes, cbc, rc4, md-5 and sha2, all MIT OR Apache-2.0, which deny.toml already permits. Phase 0 deleted a hand-rolled MD5/SHA/AES/RC4 implementation from this codebase and called it a CVE factory; ADR 0005 keeps that rule. Refusals rather than half-open documents: a public-key or otherwise unsupported handler is refused and named, an unsupported V/R combination is refused, and a wrong password returns a distinct error so a caller can prompt again rather than reporting a damaged file. Permissions are parsed and exposed but deliberately not enforced, and the code says why: once content is decrypted a caller can read it regardless, so enforcing here would imply a guarantee that does not exist. Saving an encrypted document stays refused, as ADR 0003 established. Decrypting and then writing plaintext would silently strip the protection the author applied, which is not a decision a library should make. Fixtures: tests/corpus/encrypted/ gains RC4 40-bit, RC4 128-bit, AES-128 and an unsupported-handler document. The generator implements the handler's algorithms independently from the specification, so a fixture that decrypts shows the reader agrees with the spec rather than merely with itself. Each plaintext contains a marker the tests assert on, and one test additionally asserts the decrypted content interprets to real render commands, because asserting Ok from the parser is exactly what the old broken behaviour did. Fuzzing: adds a decrypt target covering key derivation, which consumes attacker-controlled /O, /U, /P, /Length, filter names and file id. Run for real rather than compile-checked: 1,953,940 executions, no crashes. Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh (383 tests) TEST_TARGET=pdf-ui ./tools/test-rust-clean.sh (427 tests, 6 ignored) Both rustfmt and clippy -D warnings clean.	2026-07-28 17:39:38 +00:00
andodeki	f04f8ba34e	feat(pdf): implement annotation editing (Phase 8) ... Phase 8 feature 2 of 10, designed in REVIEWS/adr/0004-pdf-annotation-editing.md. Its prerequisites are "document model, appearance generation, incremental save"; the first two landed in Phase 3 and the third in ADR 0003, so this was the ready one. Chosen ahead of the other unblocked feature, AcroForm full support, because the review lists that one as needing JavaScript actions. Running document-supplied code is a large new dependency and a security surface that deserves its own ADR and threat review rather than arriving as a side effect of finishing a form feature. Annotations were strictly read-only: the module had public fields and from_dict, and not one mutator or &mut self method. The viewer could report a click on a link but could not move a highlight, restyle a square or delete a stamp. New pdf-document/src/annotation_edit.rs, deliberately the same shape as DocumentFormEditor so a caller wiring a drag gesture does not have to learn a second contract: - AnnotationEdit covers Move, Resize, SetColor, SetInteriorColor, SetBorderWidth, SetOpacity, SetContents, SetFlags and Delete. - Every edit is validated before anything changes, so a rejected edit leaves the annotation untouched. Degenerate and non-finite rectangles, colours outside 0..1, negative border widths and out-of-range opacities are all refused with typed errors. - A degenerate rectangle is refused rather than silently normalised: it usually means a bug in the UI upstream, and quietly fixing it hides that. An inverted but valid rectangle is normalised on store, so hit testing and appearance sizing never see one upside down. - Read-only annotations refuse edits unless the caller opts in through an explicit allowing_read_only(), with one exception: clearing the read-only flag itself is permitted, or a locked annotation could never be unlocked. - Colour reading converts the grey and CMYK forms of /C to RGB, since the array length selects the space. Identity: PdfAnnotation gains an obj_ref, because an index into /Annots is not stable across a save. Populating it exposed a real bug in page_annotations: it called self.resolve() on the /Annots array, which recurses and replaced every entry with its dictionary, destroying the references. It now resolves only the array itself. Saving: save_annotation_edits appends a revision through the ADR 0003 writer. Deleting rewrites the page dictionary so the reference leaves /Annots, because an object that stops existing while the page still points at it produces a file other readers reject. Out of scope and recorded in the ADR rather than implied: creating new annotations, appearance generation for types this crate cannot draw (a Stamp keeps its existing /AP rather than being blanked), applying redactions, and rich text. Tests: 23 unit tests plus 13 corpus acceptance tests covering the ADR merge criteria. The round trips reparse from the written bytes rather than reusing in-memory state, assert a deleted annotation is gone from the reparsed page's /Annots and not merely from the model, that unrelated annotations survive a deletion, that two saves chain, and that editing then saving never panics on the malformed corpus. Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh (354 tests) TEST_TARGET=pdf-ui ./tools/test-rust-clean.sh (398 tests, 6 ignored) Both rustfmt and clippy -D warnings clean.	2026-07-28 17:12:40 +00:00
andodeki	dc2bf234c6	test(pdf): harden the xref revision chain ... Incremental save made /Prev chain walking load-bearing for every document, not only saved ones: XRefTable::parse now follows offsets taken straight from the file on every parse. Phase 6 established that code consuming untrusted input needs corpus and fuzz coverage. That path had neither, so this adds it and fixes what it found. Corpus (7 new fixtures, generated by the checked-in script as usual): - revisions/two.pdf, three.pdf: chained revisions that override a form value. These are direct regression tests for the two bugs the previous commit fixed. Before it, two.pdf read back as "first" rather than "second", because find_xref_start never matched its own keyword and fell through to the oldest section in the file. - revisions/added_page.pdf: a revision that rewrites /Pages, so the newer definition must win for structure as well as for values. - malformed/prev_loop.pdf, prev_out_of_range.pdf, prev_negative.pdf and prev_chain_bomb.pdf: the hostile shapes. Two robustness defects found by those fixtures: - A broken /Prev orphaned every object the unreachable sections defined, even though the bytes were still in the file, so a document with one bad offset failed to open at all. The chain now sets a recovered flag and sweeps the file for object headers, filling only genuine gaps: entries a parsed section supplied always win, because those reflect the document's own view of which revision is current, and scanning cannot tell newer from older. - A negative /Prev was filtered to None, which silently ended the chain as though the file had no history. It is now treated as a broken link and triggers the same recovery. Also caps the chain at 64 revisions. A legitimate document has a handful; a file with thousands is an attack, not a history. prev_chain_bomb.pdf asserts the cap holds and that parsing stays fast. The recovered flag is public so a caller can distinguish a cleanly parsed document from a salvaged one rather than being handed a guess silently. A test asserts it stays false for healthy files, or it would mean nothing. Fuzzing: adds parse_revision_chain, which splices fuzzer input onto a valid base document so the fuzzer spends its time on chain shapes rather than on rediscovering PDF syntax. Run for real, not merely compile-checked: parse_revision_chain 1,926,164 runs parse_xref 1,387,713 runs parse_document 1,279,328 runs No crashes. The two re-run targets cover the file this commit changes. One fixture-generator bug fixed on the way: the helper that reads a file's startxref took the first token after rfind without skipping the keyword, producing a startxref that pointed at its own text. Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh (318 tests) TEST_TARGET=pdf-ui ./tools/test-rust-clean.sh (362 tests, 6 ignored) Both rustfmt and clippy -D warnings clean.	2026-07-28 16:58:16 +00:00
andodeki	d59bed5868	feat(pdf): implement incremental save (Phase 8) ... Phase 8 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md, designed in REVIEWS/adr/0003-pdf-incremental-save.md. The review treats Phase 8 as ten independent projects, each needing its own design doc and merge criteria. Incremental save is taken first because it is the only one whose prerequisites are already met, it is listed as a prerequisite by two others (annotation editing and full AcroForm support), and it closes a real credibility gap: DocumentFormEditor has been able to edit form fields since Phase 3, and there was no way to save the result. A grep for a public save API across all four crates returned nothing. Design decision: append a revision, never rewrite. The original bytes are copied verbatim and changed objects are appended with a new xref chained through /Prev. A full rewrite would be easier and wrong: it would silently discard everything this parser does not yet model (structure trees, optional content, embedded files), and it would invalidate any signature, foreclosing a feature listed later in the same phase. ADR 0003 records this in full. Two latent bugs surfaced while building it, both pre-existing: - find_xref_start searched with windows(10) for the 9-byte keyword "startxref", so it never matched. Every parse silently fell through to a forward scan for the first "xref" in the file. On a single-revision document that happens to be correct; on an incrementally saved one it is the *oldest* revision, so a saved edit read back as its pre-edit value. This had no visible effect before because nothing produced multi-revision files. - XRefTable::parse read one section and ignored /Prev entirely, so a multi-revision document lost every object the earlier revisions defined. It now walks the chain newest-first, keeping the first definition of each object, with a visited set against /Prev loops and bounds checks on the offsets, which come from the file and cannot be trusted. A bad link ends the chain instead of indexing out of bounds. The xref unit fixture claimed startxref 408 in a 191-byte file and only ever passed because of the windows(10) defect; it is corrected rather than adjusted to keep passing. Implementation: - pdf-cos/src/incremental.rs: IncrementalUpdate builds one revision. Recomputes stream /Length so a caller cannot write an inconsistent one, emits xref subsections for contiguous runs, sizes /Size over the whole chain, and is byte-reproducible for a given set of edits. - pdf-document/src/save.rs: turns dirty AcroForm fields into a revision, writing the new /V and a regenerated appearance stream referenced from /AP, keyed by state name for checkboxes and radios. Refusals rather than partial saves: an encrypted document returns SaveError::Encrypted, because writing plaintext objects into it would corrupt the file; a source with no startxref or no /Root is refused; and a save with no pending edits returns the input unchanged rather than growing the file and churning its timestamp. Tests: 10 acceptance tests in tests/save_roundtrip.rs covering the ADR merge criteria. The central one reparses from the written bytes rather than reusing in-memory state, so it tests the file rather than the writer against itself. Also asserts three chained revisions still reparse with the newest value winning, that pages and annotations survive a save, and that saving never panics on the malformed corpus. Known limitations, recorded in the ADR rather than glossed: cross-reference streams and object streams are not written, and superseded objects are not compacted. Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh (307 tests) TEST_TARGET=pdf-ui ./tools/test-rust-clean.sh (351 tests, 6 ignored) Both rustfmt and clippy -D warnings clean.	2026-07-28 16:35:22 +00:00
andodeki	1c7dc9e9c0	feat(pdf): complete Phase 7 performance work ... Phase 7 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md, taken only now because the review is explicit that it comes after correctness is proven. The caching and threading live in pdf-graphics rather than the Makepad crate because none of it needs a GPU. That is what lets the staleness, eviction and cancellation rules be tested without a window; the widget keeps only the parts that genuinely need a Cx. Step 7.1, off-thread parsing (new pdf-graphics/src/worker.rs): - RenderWorker interprets content streams on a background thread and returns results tagged with the Generation they were requested for. - PendingPages tracks in-flight pages so the widget can draw a placeholder and never queues the same page twice. - Drop joins the thread rather than detaching it: a detached thread writing into a dropped channel is the kind of shutdown race that surfaces as a flaky test months later. - A content stream that fails to parse yields an empty page, so one broken page cannot take down the document. Step 7.2, texture and memory management (new pdf-graphics/src/cache.rs): - PageCache is an LRU keyed by page index with a byte budget, not an entry count: one image-heavy page can outweigh fifty text pages, so counting entries would evict the wrong things. - retain_around() releases pages that scrolled out of view, keeping a margin so a small scroll does not immediately re-render. - Decoding produces DecodedImage bytes off-thread; GPU upload stays on the UI thread. - A page larger than the whole budget is still stored, since refusing it would mean re-rendering it every frame. Step 7.3, render command cache: - CachedPage holds the interpreted Vec<RenderCommand>, so replaying a page skips re-parsing its content stream. - invalidate_appearance() marks a form edit dirty without discarding the commands, because a form edit changes what is drawn over the page, not the page content stream. Widget wiring: PdfPageWidget carries a generation, refuses PageContent from a superseded document, and draws a placeholder while a page is still rendering. Exit criterion (new pdf-document/tests/phase7_exit_criterion.rs, 9 tests) against a new 60-page corpus fixture. Measured here: first page 3ms against the 200ms budget, and a warm cache read 389x faster than re-parsing (1us vs 389us). The budget is deliberately loose because a shared CI runner is unpredictable and a flaky performance test gets muted, and a muted test is worse than none; it still catches the order-of-magnitude regression the review is guarding against. Memory is asserted bounded across 30 document switches, and every page of an abandoned document is refused. Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh (270 tests) TEST_TARGET=pdf-ui ./tools/test-rust-clean.sh (307 tests) Both rustfmt and clippy -D warnings clean.	2026-07-27 17:30:50 +00:00
andodeki	b23df6a5cb	feat(pdf): complete Phase 6 testing infrastructure ... Phase 6 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md: "Real PDFs. Real regressions. No test theater." Step 6.1, corpus. 27 fixtures across basic, fonts, forms, annotations, images, edge and malformed, in the layout the review specifies. They are produced by tests/corpus/generate.py rather than committed as opaque blobs, because a corpus you cannot read is a corpus you cannot trust; CI regenerates them and fails if they differ. Hand-rolled rather than library-produced, since fixtures for a parser must contain constructs a library refuses to emit. Step 6.2, corpus tests (pdf-document/tests/corpus.rs, 27 tests). Text, vectors, Flate, multipage, CID fonts, every form field type, inherited field keys, link actions, hidden annotations, XObjects, inline images with embedded EI bytes, rotation, crop boxes, nested CTMs and content arrays. Step 6.3, robustness (pdf-document/tests/robustness.rs, 3 tests) plus five cargo-fuzz targets. cargo-fuzz needs nightly and libFuzzer so it cannot gate a stable CI run; the harness covers the same ground deterministically by mutating the real corpus with a fixed-seed PRNG, so a failure is reproducible from the seed rather than only from a saved artefact. The fuzz targets remain the deeper coverage-guided search and run on a schedule. Three crashes on untrusted input, all found by this work and all previously reachable from a malformed file: - collect_pages_ref recursed forever on a /Kids cycle. Stack overflow aborts the process; it cannot be caught. Now tracks visited nodes and bounds depth. - PdfDocument::resolve and the COS lexer recursed once per nesting level, so a file of 5000 open brackets overflowed the stack. Both are now bounded. - decode_85_group multiplied an accumulator that a malformed group can overflow, and subtracted below zero on a digit outside the valid range. Both panic in a debug build. Now saturating. Step 6.4, CI (.forgejo/workflows/pdf.yml). An engine job that runs the corpus and robustness suites under rustfmt and clippy -D warnings; a separate makepad-integration job so a missing system library is not reported as a PDF regression; and a scheduled fuzz job. The engine job also enforces the two architectural rules mechanically rather than in prose: no Makepad dependency or import in the engine crates, and no process or URL launching anywhere in them. Also fixes .gitignore: the blanket *.pdf rule silently excluded all 27 fixtures, which would have left CI unable to run them on a fresh clone. Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh (233 tests) TEST_TARGET=pdf-ui ./tools/test-rust-clean.sh (268 tests) Both rustfmt and clippy -D warnings clean; all five fuzz targets compile.	2026-07-27 17:08:23 +00:00
andodeki	b7d23f26bc	feat(pdf): complete Phase 2 render pipeline with golden tests ... Phase 2 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md. The exit criterion is that a real content stream renders through the device path and is verifiable; previously nothing could fail, so nothing was proven. Step 2.3, the operations the review calls "the lies", all reached a dead end in the interpreter. Each now reaches the device: - Inline images: BI/ID/EI were emitted as two empty marker ops and the payload was discarded, so an inline image could never be drawn. They are now parsed into a single InlineImage op carrying the dictionary and bytes. Abbreviated keys (/W /H /BPC /CS /F) and colour-space and filter abbreviations are expanded. The EI scan requires delimiters on both sides so binary data containing the bytes "EI" does not truncate the image, and an unterminated image yields no image rather than invented pixels. - Do (XObject) was an empty match arm. The interpreter cannot resolve a resource name, so it now reports it through PdfDevice::paint_x_object and the device performs the lookup. - set_dash and set_miter_limit only mutated interpreter-local state and emitted no command, so dashes never reached any renderer. Three further defects surfaced while reviewing the generated goldens, all of which silently corrupted output rather than failing: - `cm` was never parsed at all. The single-character `m` arm matched first and consumed it as a moveto, so every CTM change in every document was lost and content drew at the wrong position. Two-character operators are now tested before their one-character prefixes. - `rg` and `RG` were shadowed by the `r` and `R` arms, so an RGB fill was read as a single-component grey: `0.1 0.2 0.3 rg` produced 0.1 0.1 0.1. - ImageInfo defaulted a missing /Filter to FlateDecode. An absent /Filter means the data is stored raw, so every uncompressed image was undecodable. Testing: adds format_commands(), a deterministic one-line-per-command text form of a RenderCommand list, and eight golden files covering vector paths, text, kerning and spacing, dash and stroke parameters, fill rules, inline images, XObjects and colour operators. Floats are fixed-precision and negative zero is normalised so no spurious diffs appear. UPDATE_GOLDEN=1 regenerates them for review. Also fixes .gitignore: blanket *.txt and *.pdf rules were silently excluding the golden expectations and the AcroForm fixture added in the previous commit, which would have left both test suites unable to run on a fresh clone. Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh 132 tests pass; rustfmt and clippy -D warnings clean.	2026-07-27 15:55:53 +00:00
andodeki	3f107bf2a5	feat(pdf): complete Phase 3 document model with real form state ... Phase 3 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md. The previous form.rs was keyed by name strings, had no inheritance and no way to edit anything, so the review item "delete PdfFormFilling and replace it with a DocumentFormEditor that returns real errors" had no implementation. Step 3.1 object identity: - Fields are keyed by ObjRef, not by name. Widget-to-field identity is preserved across parse, and a field defined as a direct object is skipped rather than given an invented identity. - PdfDocument::page_index_of() maps a page reference back to its index and returns None for a stranger. Step 3.2 annotations: - PdfDocument::page_annotations() reads /Annots and records the real page on every annotation; the hardcoded page_index: None is gone. - Annotations expose a typed action() (OpenUri / GoToNamed / GoToPage). The document reports intent only and never opens anything itself (rule 5). - contains_point() normalises the rectangle: a PDF /Rect is any two opposite corners, so an inverted one previously never hit-tested. Step 3.3 form model: - /Parent chain inheritance for FT, Ff, V, DV, DA, MaxLen and Opt, with the child key overriding the ancestor. - /Ff resolved into concrete types: checkbox vs radio vs pushbutton, combo vs list box. - DocumentFormEditor takes typed edits and returns FormError. Read-only, type-mismatched, over-MaxLen, non-option and unknown-state edits are all refused, and a refused edit leaves the field untouched and not dirty. - MaxLen counts characters, not bytes. - Checking a box uses the on state the widget declares, not an assumed /Yes. - The field tree walk is depth-bounded so a cyclic /Kids cannot recurse until the stack dies. Step 3.4 appearance generation (new appearance.rs): - Document-level, not widget code. Generates text, multiline, choice and checkbox appearances as Form XObjects with correct /BBox and /Length. - /DA parsing resolves font, size and colour, converting gray and CMYK. - Auto-size (0 Tf) resolves to a size that fits the widget. - Values are escaped, so a parenthesis in a value cannot terminate the string and corrupt the stream; a single-line value cannot break out via newlines; content is clipped to the widget box. - Unsupported kinds (pushbutton, signature) and degenerate rectangles return AppearanceError instead of a blank stream that would erase the field. Wiring: PdfDocument::acroform() parses the form against the real page tree, which is what makes widget page resolution meaningful. Tests: adds tests/acroform.pdf, a hand-written two-page AcroForm fixture with an inherited field, a checkbox with /On and /Off states and a URI link, all on page index 1 so any code defaulting to page 0 fails. 12 integration tests assert the Phase 3 exit criterion against that real parsed file. Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh 118 tests pass; rustfmt and clippy -D warnings clean.	2026-07-27 15:49:46 +00:00
andodeki	af4ef6f170	fix(pdf): restore a compiling, warning-free PDF baseline ... Phase 0 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md requires a clean baseline before any feature work. The four pdf crates did not compile at all, so every test claim about them was unverified. Compile fixes: - decode_lzw/decode_run_length returned Vec<u8> where callers expected PdfResult, so decode_stream_with_params did not type-check. - interpret_ops called a current_state_mut() method that PdfDevice does not have; colour-space tracking now goes through explicit device hooks. - image.rs used miniz_oxide without depending on it; PNG inflate now reuses the COS crate through a new pdf_cos::filter::inflate_zlib. Correctness fixes found while making the code build: - LZW and RunLength silently truncated malformed input and indexed unchecked; both now return typed errors (rule 6: no silent degradation). - Tw/Tc/Tz were parsed and thrown away, and the " operator dropped its word and character spacing, so every advance after them drifted. - TJ attached each kern to the preceding string instead of the following one and discarded a trailing kern entirely. - GlyphWidths::width() fell back to default_width for out-of-range codes; PDF 32000-1 9.6.2.1 requires /MissingWidth. - Text advances silently substituted a guessed font_size * 0.6 when no width table was present; ShowTextWithMetrics now carries advance_is_measured so callers can distinguish a measurement from an unknown. Tests: two tests had never compiled and were wrong once they ran (WinAnsi 0x99 is U+2122 not U+2019; q/cm/l/Q records four commands not three). The document test asserted only that the writer emits a %PDF header; replaced with real page-tree, out-of-range and malformed-input coverage. Adds a pdf target to tools/test-rust-clean.sh that tests the three UI-independent crates bottom-up under rustfmt and clippy -D warnings. Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh 72 tests pass; rustfmt and clippy -D warnings clean.	2026-07-27 15:33:15 +00:00
andodeki	a2ea0ffc7c	updated map	2026-07-26 18:00:48 +03:00
andodeki	cc05abdc71	Initial commit	2026-07-26 19:38:26 +03:00