109 commits

Author	SHA1	Message	Date
Arena Agent	0a8d5b5abb	test(cad): move the integration suite out of src/ (Phase 4.7) ... src/.../cad/tests.rs -> tests/cad_integration.rs. It was 2,840 lines of integration tests living inside the library, compiled into every `--lib` build and able to reach anything in the crate. The move is worth more than the line count suggests, because a test target compiles as a separate crate and so sees only the public API. That turned an invisible question into a compile error: six items would have needed `pub` for the file to build in its new home — cad_mesh_data_from_solid, part_mesh_buffers, CommandBorrows, CadRenderMode, DrawingState, SnapSettings, plus the private fields of the last two. Visibility was not widened. Promoting editor internals to a public contract so a file can sit in a different directory is the wrong trade, and `pub` is far harder to take back than to grant. Instead the 16 tests that reach those items moved to where the items live: - viewport.rs gains `mod viewport_helper_tests` (10 tests: the mesh producers and the plain-data helpers). - mod.rs gains `mod editor_state_tests` (6 tests: SnapSettings polar defaults, DrawingState beam sections, CadEditorActivePane). Two of the relocated tests are worthless and are now visible as such: cad_render_mode_variants and command_borrows_fields_accessible assert that a type exists and derives Debug, which the compiler already guarantees. Left in place rather than deleted in the same commit as a move; they are for the Phase 6 sweep. The rule and its rationale are now written down in ARCHITECTURE.md under "Where a test goes", alongside corrected LOC figures for the six files that changed size since the table was written. CI gained a step. The existing job ran only `--lib`, which does not build a test target, so all 154 relocated tests would have run in no pipeline. The step names cad_integration explicitly instead of testing the whole crate, because tests/cost_estimator.rs and tests/cost_estimator_ui.rs do not compile — pre-existing upstream breakage, verified against a clean checkout, documented in a comment with instructions to fold them in once fixed. Test names diffed before and after: 0 lost, 12 gained (the Phase 4.4 characterization tests). 608 lib + 154 integration = 762.	2026-07-28 18:29:26 +00:00
andodeki	01e16b6383	feat(pdf): implement encryption (Phase 8) ... Phase 8 feature 3 of 10, designed in REVIEWS/adr/0005-pdf-encryption.md. An encrypted PDF did something worse than fail: it succeeded. Probing a structurally valid RC4-encrypted file through the parser gave parsed OK: pages=1 page 0 content bytes=44 content parsed into 0 ops No error and no warning. The document reported a page, the page reported content, and the content interpreted to nothing because it was ciphertext. The user saw a blank page and was told the file was fine. That is the defect class Phase 0 existed to remove, and it was the worst one left in the PDF stack because it was silent. New pdf-cos/src/encrypt.rs implements the standard security handler for reading: - V1/R2 RC4 40-bit, V2/R3 RC4 40 to 128-bit, V4/R4 crypt filters selecting RC4 or AES-128, and V5/R6 AES-256 with the SHA-256 based revision 6 hash. - The empty user password, which is the common case for a document encrypted only to set permissions, and explicit user or owner passwords. The owner path recovers the user password from /O and re-derives. - Per-object keys, as the spec requires. Reusing one keystream across objects would be a real cryptographic break, so the object and generation numbers are mixed in by construction and a test asserts the keys differ. Every primitive comes from audited RustCrypto crates: aes, cbc, rc4, md-5 and sha2, all MIT OR Apache-2.0, which deny.toml already permits. Phase 0 deleted a hand-rolled MD5/SHA/AES/RC4 implementation from this codebase and called it a CVE factory; ADR 0005 keeps that rule. Refusals rather than half-open documents: a public-key or otherwise unsupported handler is refused and named, an unsupported V/R combination is refused, and a wrong password returns a distinct error so a caller can prompt again rather than reporting a damaged file. Permissions are parsed and exposed but deliberately not enforced, and the code says why: once content is decrypted a caller can read it regardless, so enforcing here would imply a guarantee that does not exist. Saving an encrypted document stays refused, as ADR 0003 established. Decrypting and then writing plaintext would silently strip the protection the author applied, which is not a decision a library should make. Fixtures: tests/corpus/encrypted/ gains RC4 40-bit, RC4 128-bit, AES-128 and an unsupported-handler document. The generator implements the handler's algorithms independently from the specification, so a fixture that decrypts shows the reader agrees with the spec rather than merely with itself. Each plaintext contains a marker the tests assert on, and one test additionally asserts the decrypted content interprets to real render commands, because asserting Ok from the parser is exactly what the old broken behaviour did. Fuzzing: adds a decrypt target covering key derivation, which consumes attacker-controlled /O, /U, /P, /Length, filter names and file id. Run for real rather than compile-checked: 1,953,940 executions, no crashes. Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh (383 tests) TEST_TARGET=pdf-ui ./tools/test-rust-clean.sh (427 tests, 6 ignored) Both rustfmt and clippy -D warnings clean.	2026-07-28 17:39:38 +00:00
andodeki	dc2bf234c6	test(pdf): harden the xref revision chain ... Incremental save made /Prev chain walking load-bearing for every document, not only saved ones: XRefTable::parse now follows offsets taken straight from the file on every parse. Phase 6 established that code consuming untrusted input needs corpus and fuzz coverage. That path had neither, so this adds it and fixes what it found. Corpus (7 new fixtures, generated by the checked-in script as usual): - revisions/two.pdf, three.pdf: chained revisions that override a form value. These are direct regression tests for the two bugs the previous commit fixed. Before it, two.pdf read back as "first" rather than "second", because find_xref_start never matched its own keyword and fell through to the oldest section in the file. - revisions/added_page.pdf: a revision that rewrites /Pages, so the newer definition must win for structure as well as for values. - malformed/prev_loop.pdf, prev_out_of_range.pdf, prev_negative.pdf and prev_chain_bomb.pdf: the hostile shapes. Two robustness defects found by those fixtures: - A broken /Prev orphaned every object the unreachable sections defined, even though the bytes were still in the file, so a document with one bad offset failed to open at all. The chain now sets a recovered flag and sweeps the file for object headers, filling only genuine gaps: entries a parsed section supplied always win, because those reflect the document's own view of which revision is current, and scanning cannot tell newer from older. - A negative /Prev was filtered to None, which silently ended the chain as though the file had no history. It is now treated as a broken link and triggers the same recovery. Also caps the chain at 64 revisions. A legitimate document has a handful; a file with thousands is an attack, not a history. prev_chain_bomb.pdf asserts the cap holds and that parsing stays fast. The recovered flag is public so a caller can distinguish a cleanly parsed document from a salvaged one rather than being handed a guess silently. A test asserts it stays false for healthy files, or it would mean nothing. Fuzzing: adds parse_revision_chain, which splices fuzzer input onto a valid base document so the fuzzer spends its time on chain shapes rather than on rediscovering PDF syntax. Run for real, not merely compile-checked: parse_revision_chain 1,926,164 runs parse_xref 1,387,713 runs parse_document 1,279,328 runs No crashes. The two re-run targets cover the file this commit changes. One fixture-generator bug fixed on the way: the helper that reads a file's startxref took the first token after rfind without skipping the keyword, producing a startxref that pointed at its own text. Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh (318 tests) TEST_TARGET=pdf-ui ./tools/test-rust-clean.sh (362 tests, 6 ignored) Both rustfmt and clippy -D warnings clean.	2026-07-28 16:58:16 +00:00
Arena Agent	b5471e32e3	fix(cad): make the crate buildable, testable and safe to ship ... Phases 0-2 of CAD_ASSESSMENT_AND_PLAN.md. The crate did not compile and no test had ever run; it now builds clean with a green suite. Build and CI (Phase 0) - Pin all 33 git dependency manifests to an explicit rev. A branch dependency re-resolves on every build and is a code-execution path into CI if force-pushed. - Commit Cargo.lock (540 packages). Producing it required fixing three resolution failures the workspace had always had: a non-existent makepad-widgets feature, two rusqlite versions both linking sqlite3, and four missed CellId call sites in spreadsheet-ui. - Add .forgejo/workflows/nigig-build.yml. - Replace five stale CAD docs that contradicted the code with one ARCHITECTURE.md; add PHASE0/1/2_STATUS.md and TEST_BASELINE.md. Correctness (Phase 1) - Rotation units: transform_point bound sin_cos() backwards, transposed X and Z, and applied axes in reverse order, so every exported STL was wrong even at zero rotation. It now shares the renderer's matrix helpers. - GLB quaternions had norm 0.125 (half-angle applied to cos/sin, degrees read as radians) - invalid per the glTF spec. - PDF wall/door/window yaw fed degrees to cos/sin. - Fix a TOCTOU unwrap in touch picking; viewport.rs now has no unwrap(). - CommandContext gains update_node/insert_node_at/node_index: resize and modify were delete+create, silently moving nodes to the end of the scene. - Wire MAX_UNDO_LEVELS (defined, exported, never read) and switch the undo stack to VecDeque; this also made the existing drag-merge logic reachable. - CadNode::size() returned a fake 1x1x1 for CSG and extruded solids, making them unpickable outside a 1x1x1 box at their origin. - Reject non-finite script input; makepad_csg clamps NaN rather than propagating it, so bad input produced silently wrong geometry. Test baseline: 0 -> 722 passing, 0 failing - 17 pre-existing failures fixed: 10 real defects (dependency-cycle detection, over-allocation of unassigned tasks, quote/backslash corruption on save, default rooms lost for all but the first region, RGA text ordering) and 7 tests that were themselves wrong, each checked against its production caller first. Security (Phase 2) - env!("CARGO_MANIFEST_DIR") was used as a runtime path in three places, including as the AI agent's working directory. All runtime data now goes under app_data_dir(). - Remove the hardcoded LAN LLM endpoint. It is now opt-in via NIGIG_CAD_LOCAL_OPENAI_URL/_MODEL and refuses plaintext HTTP to anything but loopback. - Bound and content-sniff AI image attachments (8 MB cap, magic bytes); the MIME type came from the filename extension. - Escape SVG/HTML output, and add SRI to the exported viewer's script tag. The pinned model-viewer@3.5.1 does not exist, so every exported viewer was silently broken; now 4.0.0 with a verified hash. - Stop embedding $USER in exported PDFs and logging document content in release builds. - CI now rejects reintroducing the runtime-path and hardcoded-endpoint classes; both gates were verified to fail on a reintroduced defect. Add system_prompt.md and embed it with include_str!. The file was missing from the repository, so the agent silently used a one-line fallback.	2026-07-28 16:49:30 +00:00
andodeki	d3ccc2e00f	test(pdf): close the three gaps carried from Phases 4 to 7 ... Three items were carried forward as known gaps rather than quietly dropped. This addresses all three; two are closed outright and one is bounded by an environment limit that is now documented rather than implied. 1. Fuzzing had never actually run (Phase 6 step 6.3). The five cargo-fuzz targets were only compile-checked, so "zero panics on arbitrary input" was an aspiration. They have now been run under nightly libFuzzer: parse_object 1,970,750 runs parse_xref 2,471,345 runs decode_stream 1,120,019 runs parse_content_stream 2,655,663 runs parse_document 2,381,367 runs About 10.6 million executions in total, no crashes and no new findings. That is a real result rather than a green checkmark: the three crashes the corpus found in Phase 6 were the ones worth finding, and the fuzzer confirms the fixes hold under adversarial input. 2. Combo dropdown overlay (Phase 4 step 4.3). A combo box that cannot be opened is a text field with extra steps, so the open list is real state, not a rendering detail. Clicking a combo box opens its options; the dropdown takes a click before any field underneath it, matching the draw order; choosing a row sets the value through DocumentFormEditor; clicking elsewhere dismisses it without changing the value. render_open_combo() returns placement data so the drawing code stays trivial and the geometry is testable without a renderer. 3. Makepad event delivery. Upstream added a makepad_test framework, so this is now testable in principle. Adds a test host binary and six UI tests that drive the widget through the Studio protocol: a real click on the fixture link must surface OpenUri on the host, typing must reach the field, and a click on empty space must emit nothing so the positive assertions are not vacuous. They are #[ignore] by default because the Studio hub cannot start an app in this sandbox: the harness launches with --stdin-loop, which Makepad refuses without a Studio websocket, and the build exits 101 before startup. Upstream own spreadsheet-ui and map UI suites fail identically here with the same error, so this is the environment rather than this code. The tests are checked in and compiled by cargo test so they cannot rot, CI runs them where a hub exists, and the module documents how to run them by hand. Getting there also fixed a real defect in the test host: it copied ui.main_view.render() from the spreadsheet app startup hook, but a plain View has no render method, so the app errored at startup. Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh (270 tests) TEST_TARGET=pdf-ui ./tools/test-rust-clean.sh (321 tests, 6 ignored) cargo +nightly fuzz run <target> -- -max_total_time=60 (5 targets) Both rustfmt and clippy -D warnings clean.	2026-07-27 18:09:57 +00:00
andodeki	b23df6a5cb	feat(pdf): complete Phase 6 testing infrastructure ... Phase 6 of REVIEWS/DART_PDF_VS_MAKEPAD_PDF_GAP_ANALYSIS.md: "Real PDFs. Real regressions. No test theater." Step 6.1, corpus. 27 fixtures across basic, fonts, forms, annotations, images, edge and malformed, in the layout the review specifies. They are produced by tests/corpus/generate.py rather than committed as opaque blobs, because a corpus you cannot read is a corpus you cannot trust; CI regenerates them and fails if they differ. Hand-rolled rather than library-produced, since fixtures for a parser must contain constructs a library refuses to emit. Step 6.2, corpus tests (pdf-document/tests/corpus.rs, 27 tests). Text, vectors, Flate, multipage, CID fonts, every form field type, inherited field keys, link actions, hidden annotations, XObjects, inline images with embedded EI bytes, rotation, crop boxes, nested CTMs and content arrays. Step 6.3, robustness (pdf-document/tests/robustness.rs, 3 tests) plus five cargo-fuzz targets. cargo-fuzz needs nightly and libFuzzer so it cannot gate a stable CI run; the harness covers the same ground deterministically by mutating the real corpus with a fixed-seed PRNG, so a failure is reproducible from the seed rather than only from a saved artefact. The fuzz targets remain the deeper coverage-guided search and run on a schedule. Three crashes on untrusted input, all found by this work and all previously reachable from a malformed file: - collect_pages_ref recursed forever on a /Kids cycle. Stack overflow aborts the process; it cannot be caught. Now tracks visited nodes and bounds depth. - PdfDocument::resolve and the COS lexer recursed once per nesting level, so a file of 5000 open brackets overflowed the stack. Both are now bounded. - decode_85_group multiplied an accumulator that a malformed group can overflow, and subtracted below zero on a digit outside the valid range. Both panic in a debug build. Now saturating. Step 6.4, CI (.forgejo/workflows/pdf.yml). An engine job that runs the corpus and robustness suites under rustfmt and clippy -D warnings; a separate makepad-integration job so a missing system library is not reported as a PDF regression; and a scheduled fuzz job. The engine job also enforces the two architectural rules mechanically rather than in prose: no Makepad dependency or import in the engine crates, and no process or URL launching anywhere in them. Also fixes .gitignore: the blanket *.pdf rule silently excluded all 27 fixtures, which would have left CI unable to run them on a fresh clone. Validation: TEST_TARGET=pdf ./tools/test-rust-clean.sh (233 tests) TEST_TARGET=pdf-ui ./tools/test-rust-clean.sh (268 tests) Both rustfmt and clippy -D warnings clean; all five fuzz targets compile.	2026-07-27 17:08:23 +00:00
andodeki	c6e122c3fc	feat(pay): complete Phase 1 and fix M-Pesa store defects B1/B4 ... Phase 1 of NIGIG_PAY_CONSOLIDATED_REVIEW.md is now closed. 1.1 build governance: - Declare license = "MIT" on both payment crates. cargo-deny correctly reported them as unlicensed, which would block any distribution review. - Version-pin the nigig-pay-domain path dependency; a bare path dependency is a wildcard requirement. 1.2 quality gates: - Add deny.toml and a CI job running cargo-deny over both payment crates. Advisories, bans, licences and sources all pass. The config bans the makepad-* crates outright and restricts sources to crates.io. 1.3 canonical ownership (ADR 0002): - Record the domain/storage/platform/UI layering and its one-way deps. - The review's A5 "fork farm" table is stale: one copy each of parser.rs, store.rs, pending_store.rs and pay_flow_handler.rs, not three. - Fix B1: store.rs parsed category, sub_category, status and confidence from disk then overwrote them with Default::default(), losing every user categorisation on reload. Persistence also wrote display names, which are not reversible, so this adds stable storage tokens with a legacy-display fallback so existing rows still load. - Fix B4: clean/restore mapped '|' to '~' and reversed every '~', so "JOHN~DOE" loaded as "JOHN|DOE". Replaced with bijective backslash escaping covering the separator, newlines and carriage returns. - B6 (f64 money) deliberately deferred to Phase 6: it is a type change that ripples into UI consumers. These were previously recorded as untestable because nigig-core is not a workspace member. That was wrong: the three files involved need only serde, chrono, one log! macro and one app_data_dir() helper. The new tools/test-mpesa-store-clean.sh supplies those shims in a throwaway crate and runs 9 tests, two of which reproduced the defects before the fix. 1.4 boundary: enforced twice, by a CI manifest/import check and by the deny.toml ban list. 1.5 shims: three re-exports in nigig-pay/src/lib.rs had zero callers and are deleted. The remaining four carry a caller count and a named migration target so they have a deletion plan rather than an open-ended lifetime. 1.6 scope (ADR 0001): accepted that Nigig Pay is a read-only tracker and launcher, not a payment processor, until an authorised provider integration exists. This is the decision the review required before further UI work. SECURITY: a live Cloudflare API token was found committed in README.md, present since the initial commit and pushed to a public remote. Removed and recorded as R-SEC-001 in PAYMENT_RISK_REGISTER.md. Redaction does not revoke it; it remains in history and must be rotated by the owner. Validated on rustc 1.97.1: domain and storage each pass test --locked, fmt, clippy -D warnings and cargo-deny; storage also passes --features sqlcipher; domain benches run; 9 M-Pesa store tests pass. 49 manifests parse.	2026-07-27 15:23:20 +00:00
andodeki	01bc055d1e	feat(pay): complete the partially-done review phases ... No new phase was started. This finishes every item tranches 1-2 left partial, so Phases 1, 3, 7 and 8 are complete for the two pure crates or explicitly blocked on crates that cannot be built here. Phase 1 (build governance): - Check in Cargo.lock for both payment crates and build with --locked. .gitignore excluded them, which would have made item 1.1 a false claim. - tools/test-rust-clean.sh now reads the channel from rust-toolchain.toml instead of a hardcoded default that had already drifted, and gates on fmt, clippy -D warnings, the sqlcipher feature and the benchmarks. - CI enforces lockfile presence/freshness and a Makepad-boundary check that inspects manifests and use/extern lines rather than prose (1.4). Building on the declared 1.97.1 toolchain surfaced five lints 1.85 missed; all are fixed. The two Default impls are annotated rather than derived because each encodes a security or state-machine decision. Phase 3 (secure repository): - 3.1 encryption at rest: new encryption.rs and an opt-in sqlcipher feature. PRAGMA key is applied first and verified by a forced read, so a wrong key fails as KeyRejected rather than as corruption. DatabaseKey redacts its Debug and zeroes on drop. No key is ever derived or persisted here, and there is no unencrypted fallback. A test asserts the recipient MSISDN is absent from the raw database bytes. - 3.5: preferences.rs replaces the ANDROID_DATA marker file whose existence was the value; every field fails safe. - 3.6: redact.rs. PaymentIntent's derived Debug leaked a customer MSISDN into any log line; it now masks phone, name and ids, with a regression test. StorageError no longer prints a full intent id. Phase 7: - 7.7 benchmarks over money, validation, fees, the intent lifecycle and evidence handling, using the stable harness so they run on the pinned toolchain. BASELINE.md records measured output. - 7.1/7.2/7.3 verified already satisfied; the review text is stale. Phase 8: - 8.1 simulator.rs with a scripted gateway and biometric, no clock or I/O. - 8.2 property tests over every permutation of a representative event set: no ordering dispatches twice, untrusted events never settle a payment, ambiguity never becomes failure, Confirmed is terminal, duplicate confirmations are idempotent, and replayed SMS cannot fake a conflict. Validated on rustc 1.97.1 outside the incomplete workspace graph: domain 66 tests, storage 20 tests, storage+sqlcipher 25 tests, fmt clean, clippy -D warnings clean on both crates and both feature sets, benches run. 49 manifests parse; git diff --check clean. Still unclaimed and blocked on uncompilable crates: UI wiring, Keystore key provisioning, Phase 4 draw_walk I/O, Phase 5 adapters and legal review, the Java PIN scrub, and B1/B4/B6 in nigig-core.	2026-07-27 15:07:28 +00:00
andodeki	4efd7b3780	ci(pay): test isolated domain and storage crates	2026-07-26 19:22:18 +00:00