nigig-org/.forgejo/workflows/pdf.yml
andodeki 89ca5186c6
Some checks failed
email.yml / docs(pdf): Phase 4 is not 100% — audit it, and verify the half that is (push) Failing after 0s
repo hygiene / hygiene (push) Has been cancelled
PDF engine / engine (push) Has been cancelled
PDF engine / makepad-integration (push) Has been cancelled
PDF engine / fuzz (push) Has been cancelled
docs(pdf): Phase 4 is not 100% — audit it, and verify the half that is
Asked whether Phase 4 was complete, I checked the tree instead of my own
commit message, and the commit message was wrong.

Three items named in the Phase 4 spec are **not** implemented, and the
status line said "complete" over them:

- **Field-value reconciliation** (`form_reconcile_test.dart`). Setting a
  value writes /V, marks the field dirty and regenerates /AP — that all
  works. What is missing is the reconciliation case: a file opened with
  /V and /AP *already disagreeing*, where the right answer depends on
  /NeedAppearances. Nothing decides that today.
- **Type1/CFF embedding.** The spec hedges with "if feasible", so this is
  a legitimate deferral rather than an oversight — but "complete" did not
  say so. `sfnt.rs` detects CFF outlines and `font.rs` reads an existing
  /FontFile3; nothing writes one. Creation is TrueType-only.
- **`repair-cmap`.** No equivalent exists.

`text_box_appearance_test.dart` *is* covered, by appearance.rs:235 — it
just does not carry that filename, which is why a grep for the dart test
names is a starting point and not an answer.

The other half of the exit criterion — "generated PDFs open cleanly in
external viewers" — had never been checked at all. The sample generator's
own doc comment admits no test in this repository can assert it. So I
ran it through implementations we share no code with, and **it passes**:

  qpdf --check           no syntax or stream encoding errors
  pdfinfo                title, author, subject, keywords, 2 pages,
                         Form: AcroForm
  pdftotext              all text, including the embedded DejaVu subset
                         and its em-dash
  qpdf --list-attachments  readme.txt, extracted by name with description
  catalogue              /Outlines /Names /EmbeddedFiles /PageLabels
                         /Dests /PageMode /ViewerPreferences /AcroForm

`tools/check-pdf-external-readers.sh` makes that repeatable, and pdf.yml
runs it. It treats a qpdf *warning* as failure, not just an error: qpdf
warns where it had to reconstruct, and reconstructing is exactly what a
stricter viewer will refuse to do. Negative-tested twice — removing the
attachment fails 3 checks, and corrupting the startxref offset makes
qpdf report "file is damaged".

Two defects that audit found:

- **The sample never exercised XMP**, so the Phase 4 feature most likely
  to be silently missing was also the one nothing looked at. Probed
  separately: `set_xmp_metadata` works, pdfinfo reports
  `Metadata Stream: yes`.
- **A `Banner` naming an unregistered font produces a structurally valid
  PDF that renders no text.** qpdf --check passes; poppler says
  `Unknown font tag 'F1'` and draws nothing. `stamp.rs` cannot register
  the font itself — fonts belong to the document, and a banner does not
  know which document it will be drawn into — so this is now documented
  on `Banner` with a worked example, and pinned by
  `a_banner_font_must_be_registered_or_the_page_lacks_the_resource`,
  which asserts on the page's /Font resources because that is the thing
  actually missing and the thing a caller can check.

The plan now records that it was wrong once, rather than quietly
correcting itself. A status line that has been overstated should show its
working.

Engine suite 952 -> 953. Phase 4's engine half is verified end to end
against third-party readers; the ui.rs interaction half is written and
still blocked on the Makepad headless backend.
2026-08-17 09:11:03 +00:00

195 lines
8.1 KiB
YAML

name: PDF engine
on:
push:
paths:
- 'crates/apps/pdf/**'
- 'tools/test-rust-clean.sh'
- 'rust-toolchain.toml'
- '.forgejo/workflows/pdf.yml'
pull_request:
paths:
- 'crates/apps/pdf/**'
- 'tools/test-rust-clean.sh'
- 'rust-toolchain.toml'
- '.forgejo/workflows/pdf.yml'
jobs:
# The engine crates parse untrusted input and carry no UI dependency, so
# they run everywhere and are the gate that must never be skipped.
engine:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
# Runs pdf-cos, pdf-document and pdf-graphics bottom-up under the
# toolchain declared in rust-toolchain.toml, then rustfmt --check and
# clippy -D warnings. Includes the corpus and robustness suites.
- name: Engine tests, fmt and clippy
run: TEST_TARGET=pdf ./tools/test-rust-clean.sh
- name: Reject whitespace errors
run: git diff --check
# The corpus is generated by a checked-in script so every fixture is
# reviewable. If the generator and the fixtures disagree, one of them
# was edited by hand and the corpus no longer means what it says.
- name: Corpus fixtures match their generator
run: |
set -e
python3 crates/apps/pdf/tests/corpus/generate.py
git diff --exit-code -- crates/apps/pdf/tests/corpus \
|| { echo 'corpus fixtures differ from generate.py output'; exit 1; }
# Golden render expectations must be regenerated deliberately, never
# silently by a test run.
- name: Golden render expectations are unchanged
run: git diff --exit-code -- crates/apps/pdf/pdf-graphics/tests/golden
# The scheduled fuzz job enumerates targets with `cargo fuzz list`,
# which reads the manifest. A target file added without its [[bin]]
# entry would therefore never be fuzzed while looking present in the
# tree. This runs per-push so the omission is caught immediately
# rather than by a silent no-op months later.
- name: Every fuzz target file is declared in the manifest
run: |
set -e
cd crates/apps/pdf/pdf-cos/fuzz
files="$(ls fuzz_targets/*.rs | xargs -n1 basename | sed 's/\.rs$//' | sort)"
declared="$(grep -A1 '^\[\[bin\]\]' Cargo.toml \
| grep 'name = ' | sed 's/name = //; s/"//g' | sort)"
if [ "$files" != "$declared" ]; then
echo 'fuzz_targets/*.rs and the [[bin]] entries disagree:' >&2
diff <(echo "$declared") <(echo "$files") >&2 || true
echo 'an undeclared target is never fuzzed' >&2
exit 1
fi
echo "all $(echo "$files" | wc -l) fuzz targets are declared"
# A coverage number that is only printed drifts downwards. This
# enforces a whole-stack floor plus per-file floors on the files that
# have actually harboured bugs, so a decoder cannot quietly return to
# being a stub the way the JPEG one did (ADR 0016, ADR 0017).
- name: Coverage floors
run: ./tools/test-pdf-coverage.sh
# Phase 4's exit criterion: "generated PDFs open cleanly in external
# viewers". No test in this repository can assert that — our reader
# and our writer can share a bug and agree perfectly. qpdf and
# poppler are implementations we share no code with.
- name: Generated PDFs satisfy external readers
run: |
sudo apt-get update -qq
sudo apt-get install -y -qq qpdf poppler-utils
./tools/check-pdf-external-readers.sh
# Boundary enforcement: the engine crates must never reach for the UI
# framework. This checks real manifests and imports, not doc comments.
- name: Engine crates must not depend on Makepad
run: |
set -e
for crate in pdf-cos pdf-document pdf-graphics; do
dir="crates/apps/pdf/$crate"
if grep -n -i 'makepad' "$dir/Cargo.toml"; then
echo "Makepad dependency declared in $crate"
exit 1
fi
if grep -rn --include='*.rs' \
-E '^[[:space:]]*(use|extern crate)[[:space:]]+makepad' "$dir"; then
echo "Makepad imported in $crate"
exit 1
fi
done
echo 'no Makepad dependency in the PDF engine crates'
# Rule 5: the viewer emits actions, the host acts on them. A direct
# process launch or URL open from the engine is a policy violation.
- name: Engine must not launch processes or open URLs
run: |
set -e
if grep -rn --include='*.rs' \
-E 'std::process::Command|open::that|webbrowser::' \
crates/apps/pdf/pdf-cos crates/apps/pdf/pdf-document \
crates/apps/pdf/pdf-graphics; then
echo 'the PDF engine must not launch anything'
exit 1
fi
echo 'no process or URL launching in the engine'
# The Makepad integration needs system GUI libraries, so it is a separate
# job: a missing X11 package must not be reported as a PDF regression.
makepad-integration:
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
- name: Install Makepad's system dependencies
run: |
sudo apt-get update -qq
sudo apt-get install -y -qq \
libwayland-dev libxkbcommon-dev libxcursor-dev \
libpulse-dev libx11-dev libssl-dev libasound2-dev pkg-config
# Adds pdf-makepad: the interactive widget, its event routing and the
# Phase 4 exit-criterion tests.
- name: Widget tests, fmt and clippy
run: TEST_TARGET=pdf-ui ./tools/test-rust-clean.sh
# The Studio-driven UI tests are #[ignore] by default because they need
# a Studio hub. Where one is available they are the only coverage of
# real Makepad event delivery, so run them rather than let them rot.
# Not allowed to fail silently: if the hub is present they must pass.
- name: Studio UI tests
continue-on-error: true
run: |
set -e
sudo apt-get install -y -qq xvfb
cd crates/apps/pdf/pdf-makepad
xvfb-run -a cargo test --test ui -- --ignored --test-threads=1
# Coverage-guided fuzzing needs nightly and libFuzzer. It runs on a
# schedule rather than per-push so it cannot block a merge, but the
# targets are compile-checked on every push by the engine job's clippy.
fuzz:
runs-on: ubuntu-latest
# Sized for every target in fuzz_targets/ at 180s each, plus build time.
timeout-minutes: 75
# Only on demand or on a schedule: a five-minute run per target is too
# slow to gate every push, and a fuzzer finding nothing proves little.
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
steps:
- uses: actions/checkout@v4
- name: Install nightly and cargo-fuzz
run: |
set -e
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
| sh -s -- -y --profile minimal --default-toolchain nightly
. "$HOME/.cargo/env"
cargo install cargo-fuzz --locked
# Phase 6 step 6.3: zero panics on arbitrary input.
#
# The target list is derived from `cargo fuzz list`, never written out
# here. A hardcoded list silently stopped covering four targets
# (parse_revision_chain, decrypt, eval_function, parse_colorspace) as
# they were added, which is the failure this loop must not repeat:
# a fuzz job that skips a target reports success for code it never ran.
- name: Fuzz the parsers
run: |
set -e
. "$HOME/.cargo/env"
cd crates/apps/pdf/pdf-cos/fuzz
targets="$(cargo +nightly fuzz list)"
if [ -z "$targets" ]; then
echo "no fuzz targets found - the job would pass vacuously" >&2
exit 1
fi
echo "fuzzing $(echo "$targets" | wc -l) targets:"
echo "$targets"
for target in $targets; do
echo "== fuzzing $target =="
cargo +nightly fuzz run "$target" -- -max_total_time=180
done