Some checks failed
email.yml / docs(pdf): Phase 4 is not 100% — audit it, and verify the half that is (push) Failing after 0s
repo hygiene / hygiene (push) Has been cancelled
PDF engine / engine (push) Has been cancelled
PDF engine / makepad-integration (push) Has been cancelled
PDF engine / fuzz (push) Has been cancelled
Asked whether Phase 4 was complete, I checked the tree instead of my own
commit message, and the commit message was wrong.
Three items named in the Phase 4 spec are **not** implemented, and the
status line said "complete" over them:
- **Field-value reconciliation** (`form_reconcile_test.dart`). Setting a
value writes /V, marks the field dirty and regenerates /AP — that all
works. What is missing is the reconciliation case: a file opened with
/V and /AP *already disagreeing*, where the right answer depends on
/NeedAppearances. Nothing decides that today.
- **Type1/CFF embedding.** The spec hedges with "if feasible", so this is
a legitimate deferral rather than an oversight — but "complete" did not
say so. `sfnt.rs` detects CFF outlines and `font.rs` reads an existing
/FontFile3; nothing writes one. Creation is TrueType-only.
- **`repair-cmap`.** No equivalent exists.
`text_box_appearance_test.dart` *is* covered, by appearance.rs:235 — it
just does not carry that filename, which is why a grep for the dart test
names is a starting point and not an answer.
The other half of the exit criterion — "generated PDFs open cleanly in
external viewers" — had never been checked at all. The sample generator's
own doc comment admits no test in this repository can assert it. So I
ran it through implementations we share no code with, and **it passes**:
qpdf --check no syntax or stream encoding errors
pdfinfo title, author, subject, keywords, 2 pages,
Form: AcroForm
pdftotext all text, including the embedded DejaVu subset
and its em-dash
qpdf --list-attachments readme.txt, extracted by name with description
catalogue /Outlines /Names /EmbeddedFiles /PageLabels
/Dests /PageMode /ViewerPreferences /AcroForm
`tools/check-pdf-external-readers.sh` makes that repeatable, and pdf.yml
runs it. It treats a qpdf *warning* as failure, not just an error: qpdf
warns where it had to reconstruct, and reconstructing is exactly what a
stricter viewer will refuse to do. Negative-tested twice — removing the
attachment fails 3 checks, and corrupting the startxref offset makes
qpdf report "file is damaged".
Two defects that audit found:
- **The sample never exercised XMP**, so the Phase 4 feature most likely
to be silently missing was also the one nothing looked at. Probed
separately: `set_xmp_metadata` works, pdfinfo reports
`Metadata Stream: yes`.
- **A `Banner` naming an unregistered font produces a structurally valid
PDF that renders no text.** qpdf --check passes; poppler says
`Unknown font tag 'F1'` and draws nothing. `stamp.rs` cannot register
the font itself — fonts belong to the document, and a banner does not
know which document it will be drawn into — so this is now documented
on `Banner` with a worked example, and pinned by
`a_banner_font_must_be_registered_or_the_page_lacks_the_resource`,
which asserts on the page's /Font resources because that is the thing
actually missing and the thing a caller can check.
The plan now records that it was wrong once, rather than quietly
correcting itself. A status line that has been overstated should show its
working.
Engine suite 952 -> 953. Phase 4's engine half is verified end to end
against third-party readers; the ui.rs interaction half is written and
still blocked on the Makepad headless backend.
195 lines
8.1 KiB
YAML
195 lines
8.1 KiB
YAML
name: PDF engine
|
|
|
|
on:
|
|
push:
|
|
paths:
|
|
- 'crates/apps/pdf/**'
|
|
- 'tools/test-rust-clean.sh'
|
|
- 'rust-toolchain.toml'
|
|
- '.forgejo/workflows/pdf.yml'
|
|
pull_request:
|
|
paths:
|
|
- 'crates/apps/pdf/**'
|
|
- 'tools/test-rust-clean.sh'
|
|
- 'rust-toolchain.toml'
|
|
- '.forgejo/workflows/pdf.yml'
|
|
|
|
jobs:
|
|
# The engine crates parse untrusted input and carry no UI dependency, so
|
|
# they run everywhere and are the gate that must never be skipped.
|
|
engine:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
# Runs pdf-cos, pdf-document and pdf-graphics bottom-up under the
|
|
# toolchain declared in rust-toolchain.toml, then rustfmt --check and
|
|
# clippy -D warnings. Includes the corpus and robustness suites.
|
|
- name: Engine tests, fmt and clippy
|
|
run: TEST_TARGET=pdf ./tools/test-rust-clean.sh
|
|
|
|
- name: Reject whitespace errors
|
|
run: git diff --check
|
|
|
|
# The corpus is generated by a checked-in script so every fixture is
|
|
# reviewable. If the generator and the fixtures disagree, one of them
|
|
# was edited by hand and the corpus no longer means what it says.
|
|
- name: Corpus fixtures match their generator
|
|
run: |
|
|
set -e
|
|
python3 crates/apps/pdf/tests/corpus/generate.py
|
|
git diff --exit-code -- crates/apps/pdf/tests/corpus \
|
|
|| { echo 'corpus fixtures differ from generate.py output'; exit 1; }
|
|
|
|
# Golden render expectations must be regenerated deliberately, never
|
|
# silently by a test run.
|
|
- name: Golden render expectations are unchanged
|
|
run: git diff --exit-code -- crates/apps/pdf/pdf-graphics/tests/golden
|
|
|
|
# The scheduled fuzz job enumerates targets with `cargo fuzz list`,
|
|
# which reads the manifest. A target file added without its [[bin]]
|
|
# entry would therefore never be fuzzed while looking present in the
|
|
# tree. This runs per-push so the omission is caught immediately
|
|
# rather than by a silent no-op months later.
|
|
- name: Every fuzz target file is declared in the manifest
|
|
run: |
|
|
set -e
|
|
cd crates/apps/pdf/pdf-cos/fuzz
|
|
files="$(ls fuzz_targets/*.rs | xargs -n1 basename | sed 's/\.rs$//' | sort)"
|
|
declared="$(grep -A1 '^\[\[bin\]\]' Cargo.toml \
|
|
| grep 'name = ' | sed 's/name = //; s/"//g' | sort)"
|
|
if [ "$files" != "$declared" ]; then
|
|
echo 'fuzz_targets/*.rs and the [[bin]] entries disagree:' >&2
|
|
diff <(echo "$declared") <(echo "$files") >&2 || true
|
|
echo 'an undeclared target is never fuzzed' >&2
|
|
exit 1
|
|
fi
|
|
echo "all $(echo "$files" | wc -l) fuzz targets are declared"
|
|
|
|
# A coverage number that is only printed drifts downwards. This
|
|
# enforces a whole-stack floor plus per-file floors on the files that
|
|
# have actually harboured bugs, so a decoder cannot quietly return to
|
|
# being a stub the way the JPEG one did (ADR 0016, ADR 0017).
|
|
- name: Coverage floors
|
|
run: ./tools/test-pdf-coverage.sh
|
|
|
|
# Phase 4's exit criterion: "generated PDFs open cleanly in external
|
|
# viewers". No test in this repository can assert that — our reader
|
|
# and our writer can share a bug and agree perfectly. qpdf and
|
|
# poppler are implementations we share no code with.
|
|
- name: Generated PDFs satisfy external readers
|
|
run: |
|
|
sudo apt-get update -qq
|
|
sudo apt-get install -y -qq qpdf poppler-utils
|
|
./tools/check-pdf-external-readers.sh
|
|
|
|
# Boundary enforcement: the engine crates must never reach for the UI
|
|
# framework. This checks real manifests and imports, not doc comments.
|
|
- name: Engine crates must not depend on Makepad
|
|
run: |
|
|
set -e
|
|
for crate in pdf-cos pdf-document pdf-graphics; do
|
|
dir="crates/apps/pdf/$crate"
|
|
if grep -n -i 'makepad' "$dir/Cargo.toml"; then
|
|
echo "Makepad dependency declared in $crate"
|
|
exit 1
|
|
fi
|
|
if grep -rn --include='*.rs' \
|
|
-E '^[[:space:]]*(use|extern crate)[[:space:]]+makepad' "$dir"; then
|
|
echo "Makepad imported in $crate"
|
|
exit 1
|
|
fi
|
|
done
|
|
echo 'no Makepad dependency in the PDF engine crates'
|
|
|
|
# Rule 5: the viewer emits actions, the host acts on them. A direct
|
|
# process launch or URL open from the engine is a policy violation.
|
|
- name: Engine must not launch processes or open URLs
|
|
run: |
|
|
set -e
|
|
if grep -rn --include='*.rs' \
|
|
-E 'std::process::Command|open::that|webbrowser::' \
|
|
crates/apps/pdf/pdf-cos crates/apps/pdf/pdf-document \
|
|
crates/apps/pdf/pdf-graphics; then
|
|
echo 'the PDF engine must not launch anything'
|
|
exit 1
|
|
fi
|
|
echo 'no process or URL launching in the engine'
|
|
|
|
# The Makepad integration needs system GUI libraries, so it is a separate
|
|
# job: a missing X11 package must not be reported as a PDF regression.
|
|
makepad-integration:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 45
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Install Makepad's system dependencies
|
|
run: |
|
|
sudo apt-get update -qq
|
|
sudo apt-get install -y -qq \
|
|
libwayland-dev libxkbcommon-dev libxcursor-dev \
|
|
libpulse-dev libx11-dev libssl-dev libasound2-dev pkg-config
|
|
|
|
# Adds pdf-makepad: the interactive widget, its event routing and the
|
|
# Phase 4 exit-criterion tests.
|
|
- name: Widget tests, fmt and clippy
|
|
run: TEST_TARGET=pdf-ui ./tools/test-rust-clean.sh
|
|
|
|
# The Studio-driven UI tests are #[ignore] by default because they need
|
|
# a Studio hub. Where one is available they are the only coverage of
|
|
# real Makepad event delivery, so run them rather than let them rot.
|
|
# Not allowed to fail silently: if the hub is present they must pass.
|
|
- name: Studio UI tests
|
|
continue-on-error: true
|
|
run: |
|
|
set -e
|
|
sudo apt-get install -y -qq xvfb
|
|
cd crates/apps/pdf/pdf-makepad
|
|
xvfb-run -a cargo test --test ui -- --ignored --test-threads=1
|
|
|
|
# Coverage-guided fuzzing needs nightly and libFuzzer. It runs on a
|
|
# schedule rather than per-push so it cannot block a merge, but the
|
|
# targets are compile-checked on every push by the engine job's clippy.
|
|
fuzz:
|
|
runs-on: ubuntu-latest
|
|
# Sized for every target in fuzz_targets/ at 180s each, plus build time.
|
|
timeout-minutes: 75
|
|
# Only on demand or on a schedule: a five-minute run per target is too
|
|
# slow to gate every push, and a fuzzer finding nothing proves little.
|
|
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Install nightly and cargo-fuzz
|
|
run: |
|
|
set -e
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
|
|
| sh -s -- -y --profile minimal --default-toolchain nightly
|
|
. "$HOME/.cargo/env"
|
|
cargo install cargo-fuzz --locked
|
|
|
|
# Phase 6 step 6.3: zero panics on arbitrary input.
|
|
#
|
|
# The target list is derived from `cargo fuzz list`, never written out
|
|
# here. A hardcoded list silently stopped covering four targets
|
|
# (parse_revision_chain, decrypt, eval_function, parse_colorspace) as
|
|
# they were added, which is the failure this loop must not repeat:
|
|
# a fuzz job that skips a target reports success for code it never ran.
|
|
- name: Fuzz the parsers
|
|
run: |
|
|
set -e
|
|
. "$HOME/.cargo/env"
|
|
cd crates/apps/pdf/pdf-cos/fuzz
|
|
targets="$(cargo +nightly fuzz list)"
|
|
if [ -z "$targets" ]; then
|
|
echo "no fuzz targets found - the job would pass vacuously" >&2
|
|
exit 1
|
|
fi
|
|
echo "fuzzing $(echo "$targets" | wc -l) targets:"
|
|
echo "$targets"
|
|
for target in $targets; do
|
|
echo "== fuzzing $target =="
|
|
cargo +nightly fuzz run "$target" -- -max_total_time=180
|
|
done
|