792 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
andodeki	a099ab9980	test(spreadsheet): cover public workbook lifecycle API	2026-08-18 08:02:17 +00:00
andodeki	2824b49f0e	test(spreadsheet): cover public workbook deserialization API	2026-08-18 07:44:23 +00:00
andodeki	1629994de7	test(spreadsheet): add public persistence integration test	2026-08-18 07:30:43 +00:00
andodeki	d4e3e9a443	feat(pdf): encryption on save — AES-128 and AES-256 (Phase 6, part one) ... ADR 0024. This reverses ADR 0005's "never write encryption", and the reason it is safe to reverse is that the facts changed underneath it. A crate that only reads cannot produce weak ciphertext, so refusing to write any was free. Now that Phase 4 creates documents and Phase 5 edits them, the refusal does something worse than protect nobody: open a password-protected file, change one annotation, save, and the output is plaintext. No error, no warning — the protection is silently dropped. That is this project's recurring failure mode in the one place where the consequence is a breach. The principle survives in a narrower form: no hand-rolled crypto, and no weak cipher offered as an option. RC4 stays readable because files use it and is not writable — EncryptionAlgorithm has no RC4 variant, so the refusal is a type, not a runtime check someone can route around. The encryptor is the literal inverse of the decryptor and imports its primitives rather than restating them; two implementations of one algorithm drift, and here they drift towards "decrypts to garbage". Every unit test round-trips through the existing Decryptor. Encryption sits at one choke point: PdfWriter holds the Encryptor and write_object_at encrypts everything passing through. Not per call site — there are twenty-two of those in PdfDocBuilder, and one stream written in the clear inside an encrypted document is not a partial failure, it is a leak that no reader will report because the file is otherwise valid. The /Encrypt dictionary is the single deliberate exemption: it holds the salts a reader needs before it has a key, so encrypting it bricks the file. Verified against implementations we share no code with, now gated in CI: ok qpdf opens it with the password ok it really is AES-256 ok the wrong password is refused ok poppler decrypts the content ok no plaintext in the encrypted file Four mutations, all killed — two only after the tests were strengthened, and both misses are the interesting part: A fixed IV survived two_saves_of_one_document_are_not_byte_identical, because the AES-256 file key is fresh per save and that alone makes the output differ. The property actually needed is narrower: one encryptor, identical plaintext, different bytes. In CBC a repeated IV under one key leaks that two plaintexts are equal. A wrong /Length survived because our own reader recovers by scanning for endstream — a robustness fix from ADR 0023. An independent reader that trusts /Length reads a truncated stream and decrypts garbage. A lenient reader hides a broken writer, which is why the external gate exists. The /Length test itself had a bug first: it searched a from_utf8_lossy view and reported a stream declaring 80 bytes holding 156. Ciphertext is not UTF-8; the replacement characters shifted every offset. Unencrypted output stays byte-reproducible; encrypted output cannot be, and a test asserts that loss rather than leaving it implicit. pdf: 1220 passed (was 1187). pdf-ui: green. Coverage 88.21%, encrypt_write.rs at 96.5%. Signing is NOT started. It needs the trust-anchor decision ADR 0010 deferred: VerificationStatus::Valid is unreachable by construction, and making sign -> verify pass is a policy change, not an implementation detail. The plan's Phase 6 status now says so.	2026-08-18 07:27:45 +00:00
andodeki	118fbefe9a	test(spreadsheet): move format detection test to integration suite	2026-08-18 07:07:12 +00:00
nigig-ci	ee8d10d978	fix(pay): the M-Pesa PIN field has been visible by default since 69f6fb2 ... The first thing the newly registered runner found. tools/check-no-pin-capture.sh fails on main: FAIL: pin_input is not hidden by default in the DSL (review item 0.3: hiding must survive a DSL reload) Reproduced locally, so this is a real defect and not runner flakiness. 69f6fb2 -- "fix(map): NigigMapView packed vertex shader for new DrawVector; fix pay sheet visible" -- deleted TWO lines from shared_pay_sheet.rs. Its message says it removed `visible: false` from `pin_eye_btn := Button` because "Button does not have DSL property visible". Whatever the merit of that for the Button, the same commit silently took the line off `pin_input` as well, and pin_input is a RobrixTextInput, which does support the property -- 6f05c47 had added it there deliberately eight days earlier for exactly this reason. Consequence: both M-Pesa PIN controls have rendered by default in every non-demo build since. The runtime code only ever calls set_visible(cx, true), and only under #[cfg(feature = "demo")] -- nothing hides them -- so the DSL was the entire mechanism. The comment three lines above still read "The DSL hides the PIN controls by default so a reload cannot surface them", which had become false. That is review item 0.3 / defect B11: the hiding must survive a DSL reload, because a hot-reloaded or re-instantiated sheet re-applies the definition and a runtime-only call does not run again. Restored on both controls. On the Button: the stated reason for removing it does not hold in this tree -- nigig-build's cad/mod.rs has three Buttons carrying `visible: false` in the DSL (workspace_split_axis_btn, desktop_workspace_sync_toggle_btn, ai_cancel_button) and they do not warn. Verified `cargo check -p nigig-pay-ui --lib` is clean with both lines back. Also hid the parent `pin_visibility_row`, so the container and its children agree and a future edit to one child cannot re-expose the field on its own. tools/check-no-pin-capture.sh now passes end to end: PIN controls are hidden by default in the UI definition no PIN capture in a packaging build; the default retains it deliberately Worth stating plainly: this shipped because a map shader commit touched a payments file and no CI ran. The gate that catches it has existed the whole time.	2026-08-18 07:05:04 +00:00
andodeki	ad3fe19b90	docs(pdf): the four missing ADRs — codecs, Phase 4 completion, editing, redaction ... Ten PDF feature commits landed without an ADR, covering three whole phases. Every feature gets one; these are the four that were owed. Written against the code as it stands and re-verified by running it, not transcribed from the commit messages. 0020 CCITT, JBIG2 and JPEG 2000 — the codecs ADR 0015 refused by name 0021 Phase 4 completion — stamping, reconciliation, CFF, cmap, and the audit that corrected a false "complete" in ADR 0019 0022 Editing — content_edit, page_ops, flatten, catalog_edit 0023 Redaction and compaction, and the three reader defects they found Verified rather than assumed, on the tree at f75c1cc: ccitt 30, jbig2 29, jpx 44 unit tests; jpx_roundtrip's 7 compare against OpenJPEG-generated pixels, exactly, because a JPEG 2000 bug produces a plausible image rather than an error. Redaction re-checked with a test written from outside the module. The middle line is the one worth keeping: after redact, secret present = true <- earlier revision after compact, secret present = false public text retained = true Redaction alone leaves the secret in the bytes. That is why the two shipped together, and why the report carries earlier_revisions_retain_content. Determinism re-checked by generating the same document from four separate processes: byte-identical. The HashMap key-order defect really is fixed. This ADR contains a correction to its own first draft. 0022 initially listed text-run rewriting and annotation sub-types as deferred, on the strength of a grep for "rewrite" finding nothing. Both are implemented — the rewriter is ContentEditor::set_text, which preserves the operator kind so a ' keeps its line advance and a TJ keeps its kerning; and "callout" and "comment" are dart-pdf's names for a FreeText with a /CL and a Text annotation, not distinct /Subtype values. Checking the plan's claim against the code was right; concluding from a failed grep was not. One criterion across the four is left unticked, and it is real: the ui.rs interaction tests, blocked on the Makepad headless backend since Phase 1. 0021 also records a process note. ADR 0019's merge criteria were derived from what had been built, so all of them ticked while three unbuilt items stayed invisible. Criteria should come from the plan text first, then be marked done or explicitly deferred — a deferral stated is fine, a deferral unstated is a false claim. pdf: 1187 passed. No code changed.	2026-08-18 05:53:36 +00:00
andodeki	676b47087a	refactor(spreadsheet-ui): centralize dirty region state	2026-08-18 05:49:55 +00:00
andodeki	f75c1cc966	feat(spreadsheet-ui): model dirty render regions	2026-08-18 05:41:42 +00:00
andodeki	2ba1837055	fix(pdf): main was red — three clippy errors broke the engine build ... The PDF engine did not compile under `-D warnings`, which is what CI's engine job runs, so that job could not have passed on any of the last eleven PDF commits. The tests themselves were fine (1187 passing); the build was not. Two lints in the new jpx.rs, one in tests/filters.rs. One root cause each: jpx.rs:1279,1290 needless_range_loop on the inverse component transform. The lint's suggestion does not work here: each pass reads and writes three component planes at the same index, and `components.iter_mut()` cannot express three simultaneous mutable borrows of one Vec. Allowed locally with the reason written down, rather than restructuring correct code to satisfy a lint that has misread it. filters.rs:383 vec_init_then_push, where the lint is simply right. No behaviour change. Verified after the fix, on a clean checkout of df3c650: TEST_TARGET=pdf 1187 passed TEST_TARGET=pdf-ui 1232 passed coverage 87.98%, all floors met corpus matches generate.py; golden expectations unchanged fuzz manifest gate, no-makepad and no-process gates all pass	2026-08-18 05:41:22 +00:00
nigig-ci	632479c964	fix(ci): email.yml has been invalid YAML for six commits ... `python3 -c "yaml.safe_load(open('.forgejo/workflows/email.yml'))"` fails: mapping values are not allowed here in ".forgejo/workflows/email.yml", line 455, column 35 A workflow that does not parse does not fail -- it does not RUN. So every gate in this file has been silently absent: the S2 password checks, the multi-recipient regression check, the TLS check, the coverage floors. All of them. The file has looked like protection while providing none. Cause: the "Coverage floors" step was rewritten to call tools/test-email-coverage.sh, and ten lines of the previous inline implementation were left behind underneath the new `run:` scalar. YAML reads the first `echo "$out" | grep -E '^test result:'` as a new mapping key and gives up. Broken by 3dab4a1 and still broken at 6bf138d -- six commits, every one of which believed it was adding or tightening a gate. Verified the removal is safe: tools/test-email-coverage.sh exists, is executable, and enforces the floors itself (TOTAL_FLOOR=88 plus per-file floors), so the orphaned lines were duplicating work the script already does. Nothing was lost. The repo's own repo-hygiene.yml WOULD have caught this -- it has an "Every workflow file must be valid YAML" step, added precisely because commit 8c9ccb9 once broke nigig-build.yml the same way and silently disabled the CAD gates. I ran that step by hand against this tree and it fails, correctly. It did not catch it because no runner is registered, so repo-hygiene has never executed on these commits. That is the actual lesson here and it is not about YAML: a gate that has never run is indistinguishable from a gate that does not exist. This is the third time in this crate's history that a check existed, looked right, and was doing nothing. After the fix: YAML valid, 4 jobs / 32 steps, all 12 source gates pass, cargo check --all-targets clean on both crates, 213 email domain tests pass, fmt clean.	2026-08-18 05:33:18 +00:00
andodeki	77255965fa	test(spreadsheet-ui): measure headless controller coverage	2026-08-18 05:30:48 +00:00
andodeki	df3c650c3e	fix(coverage): invoke native preflight portably	2026-08-18 05:25:29 +00:00
andodeki	ee81d292ed	test(spreadsheet): cover literal and comparison formula edges	2026-08-18 05:22:48 +00:00
andodeki	0fcb2d3fae	test(spreadsheet): cover sum boolean text error branches	2026-08-17 19:44:27 +00:00
andodeki	4a955c5c89	docs: repo-wide coverage plan — census first, then phases ... The CAD plan covered one module. This is the level above it: all 58 workspace members, 310,287 lines. The census is the point of the document: A gated 5 crates 45,190 lines B measured, not gated 2 crates 15,373 C tested, unmeasured 27 crates 225,149 D zero tests 5 crates 4,160 E templated shells 19 crates 20,415 Three things the census turned up that were not visible from inside any one crate. Tier B is free money. spreadsheet and the doc workspace both have coverage scripts with per-file floors already written, and no CI job runs either. Same for the CAD widget layer. Fifteen thousand measured lines with nothing stopping them decaying. pageflipnav is 24,117 lines with 20 tests, no coverage tooling, and no mention in any review document in this repository. It is the worst ratio here by a distance and nobody has looked at it. The nineteen "app" crates are one program. Diff any two main.rs files after normalising the name and you get a background colour and a root screen identifier. Covering them as nineteen crates is nineteen times the work for one crate of risk; the plan asks for a decision rather than quietly doing it. On the goal itself: the only honest cost estimate available is the measured one. The CAD engine's 13,752 lines took about six sessions to reach 97%, on the easiest half of one module. Tier C is 225,149 lines, so straight-line extrapolation is ~98 sessions and the extrapolation is optimistic. That is not an argument against 100% — it is an argument that sequence matters more than destination, because the first fifth of the effort can cover most of the risk if pointed at the right code. So the phases are ordered by risk, not size: gate what is measured, measure everything else, then payments and SMS before anything larger. Phase 4 (GUI) is explicitly gated on the CAD makepad-test spike, so nobody commits three months to an approach that may not work here. Also recorded: #[coverage(off)] is unstable on the pinned 1.97.1 toolchain, verified with E0658, so unreachable code cannot be annotated away. It has to be covered, moved to an excluded file, or subtracted in the open.	2026-08-17 12:44:56 +00:00
andodeki	211ce31e9f	docs(cad): a phased plan to 100% coverage, written after measuring ... Asked for a plan to 100%. Four things had to be established first, because each one changes the plan's shape, and three of them contradict what I would have assumed. **There are already 148 widget tests and they have never run.** tests/ui.rs is 1,889 lines of #[makepad_test] tests driving the real app through TestApp/Selector. Line 1 imports a crate that is not a dependency, so the file has never compiled, and no CI job names it. Same defect the spreadsheet-ui suite documents about itself; same class as the nigig-email binary that had never been built. **They compile with a one-line manifest change, and they run.** Adding makepad-test as a dev-dependency produces a binary; under xvfb-run all 148 execute in 59 seconds without hanging. **All 148 fail at a known point.** The harness's child build of the app exits non-zero before startup: code 127 with no cargo on the child PATH, code 101 after fixing that — while `cargo check -p nigig-build --bins` passes. So the blocker is in how the harness invokes the child, not in the app, and spreadsheet-ui already documents the workaround. **#[coverage(off)] is unstable on 1.97.1.** There is no way to annotate a line as legitimately unreachable, so anything genuinely uncoverable has to be covered, moved to an excluded file, or subtracted openly. The plan puts unblocking those 148 tests first, because it is the cheapest large prize and because its outcome resizes everything after it. Engine cleanup runs in parallel since it is independent. Extraction work is explicitly held until Phase 0 reports, so nobody extracts logic the app-level tests already cover. It also argues against 100% as a target for the widget half. The 148 tests are mostly wait_visible(); they will move the number a long way while proving that widgets exist. Of the four real defects this work has found, three came from reading uncovered regions and asking why they were unreachable, not from driving a percentage. The plan targets 100% of what is worth executing and names the ~48 subtracted lines.	2026-08-17 12:40:35 +00:00
andodeki	dffa140123	fix(makepad-table): measure text with the layouter instead of estimating it ... Right-aligned text still overflowed its column after the previous fix, because that fix kept the 7px-per-character estimate and only clamped the result. Real glyphs at this size average wider than 7px, so the estimate came out short, `pos.x + width - pad - estimated_width` placed the run too far right, and it ran past the cell edge. The clamp only guards the left side. No fixed per-character figure can work here: under-measuring overflows and over-measuring leaves a visible gap. `draw_cells` now measures with the same layouter that draws the run — `DrawText::layout(..).size_in_lpxs.width`, the pattern `glass_panel.rs` uses — and both the truncation and the alignment use that measurement, so the two cannot disagree. `align_text_x` takes a width rather than a string. `fit_text_to_cell` becomes `fit_text_measured`, taking a measuring closure and bisecting for the longest prefix that fits; laying out a string is not free, and a long value in a wide column would otherwise be measured once per character every frame. Tests 59 -> 62, and the existing ones were rewritten around an injected measurer. `varied()` gives different characters different widths, as a real font does, so the suite now fails on anything that assumes a constant width. Verified against four defects. Three of the guards did not work on the first attempt, and all three failures were mine: - **Estimating the alignment width — the actual reported bug — passed.** The source check asserted `size_in_lpxs` appeared *somewhere* in `draw_cells`, and the fitting call still mentioned it after the aligning call had been replaced with an estimate. It now counts both measurements and rejects any `chars().count() as f64 *` in the draw path. - **Removing the clamp passed.** Every test reached `align_text_x` through `fit_text_measured`, and once text has been shortened to fit, the clamp never fires. `alignment_clamps_a_run_wider_than_its_cell` calls it directly with an over-wide value. The clamp still earns its place: the fitted width and the aligned width are separate measurements of separate strings, and any disagreement is what it catches. - **An off-by-one in the bisection hung instead of failing.** `lo = mid - 1` stops the interval shrinking and the loop spins forever — a frozen frame, not a wrong pixel, and no assertion catches it without a timeout. The loop is now a bounded `for` capped at the number of steps a correct bisection can need, so the same mistake produces a wrong answer that a test can see. `the_fit_search_terminates_on_hostile_input` covers degenerate measurers.	2026-08-17 12:31:43 +00:00
andodeki	b870dc4c69	feat(pdf): outline, page label and struct-tree editing — Phase 5 complete ... The last functional item. `catalog.rs` read all three; nothing could write them into an existing document. `PdfDocBuilder` can emit an outline when *creating* a file, but a document already on disk could not have its bookmarks changed. An outline is a doubly-linked tree — `/First`, `/Last`, `/Next`, `/Prev`, `/Parent` and a signed `/Count` — and every pointer has to agree. A viewer walking `/Next` and one walking `/First`..`/Last` must see the same list, or bookmarks vanish in one reader and not another with no error anywhere. Object numbers are reserved before any dictionary is built, because each item names its parent, its siblings and its children. `/Count` is signed and that matters: positive means open and counts *visible* descendants, negative means closed. A closed child contributes itself but hides its own children. Writing the total unconditionally makes every node render expanded. Page labels are a number tree, so the keys are sorted before writing and two rules starting on the same page are refused — that page's label would be undefined, and picking one arbitrarily is worse than saying so. Struct-tree editing is deliberately **removal only**. Editing the tree in place means rewriting `/K` arrays whose entries are marked-content ids inside page content streams; the tree and the content must stay in step, and changing one without the other produces a document whose accessibility information describes content that is no longer there. Removal is honest — the document stops claiming to be tagged — and `/MarkInfo` goes with it, because `/Marked true` with no tree tells a screen reader there is structure to find. Verified by mutation, seven defects, all caught: /Prev never written 1 fail /Next never written 5 fail /Count always positive 1 fail page validation skipped 2 fail /MarkInfo left behind 1 fail children not linked via /First 2 fail label rules not sorted 1 fail The last one needed a new test. Our reader walks `/Nums` linearly, so it tolerates any order and the round-trip passed unsorted — but a conforming reader binary-searches it and would label pages arbitrarily. Only reading the raw array catches that, which is the same lesson as the stale `/Count` in the page-ops tranche: our parser's tolerance hides defects that harm other readers. Engine suite 1158 -> 1187. External readers still pass. **Phase 5 is complete** but for the `ui.rs` interaction tests, blocked on the same missing Makepad headless backend as Phase 4's. The plan records the item-by-item status and, separately, the six defects the round-trip tests found in code that already existed — an unordered dictionary writer that made every generated PDF differ run to run, a short /Length that silently truncated streams, two readers disagreeing by a byte, a nested paren that truncated a string and desynchronised the stream, undecoded # escapes in names, and unknown operators being dropped outright.	2026-08-17 12:26:29 +00:00
andodeki	41c43df0e5	feat(pdf): redaction and object compaction — and three reader defects ... Phase 5's last two functional items. Together they are what makes a redaction real, which is why they are one commit: redaction removes the content, compaction removes the revision that still holds it. **Redaction removes operators; it does not draw rectangles.** The famous failure is painting black over text and shipping it — the text is still there, and `pdftotext` prints it. This module draws nothing. It removes the text-showing operators whose position falls inside a rectangle, and the test that matters is that the text can no longer be extracted. Positioning needs the text matrix, so the module tracks `Tm`/`Td`/`TD`/`T*` and the CTM through `q`/`Q`/`cm`. It cannot reach the graphics layer — the crate boundary again — so it treats a showing operator's origin as its position and removes the whole run. That is coarse in the *safe* direction: removing more than asked loses content the user can see is missing; removing less leaves the secret in the file. What it refuses to claim is as important. Images are removed entirely rather than cropped. Metadata and attachments are untouched. And an incremental redaction leaves the original text in the earlier revision — the report says so via `earlier_revisions_retain_content` rather than implying the job is done. **Compaction finishes it.** The output is built from the object graph reachable from `/Root`, so dead objects, superseded revisions and the bytes behind a redaction are not copied — they are simply never written. A signed document is refused unless `allow_signed` is set, because compaction destroys the revision a signature covers and would leave every signature unverifiable with no warning. The end-to-end test is the point: redact, compact, then search the output bytes for the secret. It is gone. **Three reader defects, all found by writing the tests.** - **`PdfWriter` wrote dictionary keys unordered.** `PdfDict` is a HashMap and Rust seeds its hasher per process, so *every generated PDF differed run to run*. Found by compaction's idempotence test — compacting an already-compact file produced the same objects at the same offsets with their keys shuffled. Verified fixed by running four separate processes and getting a byte-identical file. Same defect as the one fixed in `content_edit::write_dict`; this one affected every file this codebase has ever written. - **A short `/Length` silently truncated a stream.** The reader guarded against a `/Length` running past the buffer but trusted one that was too small, cutting the stream at the wrong place and losing the rest with no error. Short lengths are common in hand-edited files. `endstream` is now the authority when the two disagree — but only when it is *further* on, so binary data containing the word `endstream` is still bounded by its declared length. - **Two stream readers disagreed by one byte.** `read_object_at` did not trim the EOL before `endstream` while `find_endstream` did, so a write-read-write cycle grew every stream by a newline. A test fixture had encoded the bug: it declared `/Length 9` for eight bytes of content and asserted the newline came back as data. Both corrected — the newline is syntax (§7.3.8.1), not content. Verified by mutation. Ten defects across the two modules, all caught: redaction covers instead of removes 16 fail CTM ignored 1 fail Q does not restore the CTM 1 fail operands kept when operator removed 12 fail revision warning always false 1 fail signature guard removed 1 fail reachability keeps everything 2 fail dropped reference left dangling 1 fail unresolvable object kept as reachable 1 fail writer dictionary order unsorted 1 fail short-/Length fix reverted 1 fail /Length not rewritten on compaction 1 fail One mutation survived and deleted code rather than adding a test: a `continue` skipping `/Length` in the compaction loop was dead, because the `set` after the loop overwrites it either way. Removed rather than left as untested defence with a reassuring comment — the same call ADR 0017 made about the visited-set guard. A second mutation moved a test rather than a fixture: a stale `/Length` can no longer reach `renumber` through a file, because the reader now repairs it first, so that branch is tested directly instead. Engine suite 1108 -> 1158. External readers still pass. Phase 5 remaining: outline, page label and struct-tree editing.	2026-08-17 12:20:25 +00:00
andodeki	01d889c90b	docs(cad): what the first widget extraction actually bought ... 13.15% -> 13.25%, and viewport_input.rs is still at 0.00%. Worth writing down rather than quietly celebrating a fix. Extraction makes logic testable by moving it out of the handler, so the handler shrinks instead of getting covered: nav_pad.rs took 64 lines to 100%, and viewport_input.rs went from 736 lines uncovered to 672 lines uncovered. At ~60 lines an extraction the remaining input handler is ten more of these. The pixel bug found on the first one suggests the yield is real, so it is a reasonable way to spend effort — but anyone expecting the widget number to climb fast should know it will not, and that actually covering the handlers means driving them with an event loop through makepad-test, which tests/cad_ui.rs and spreadsheet-ui already use.	2026-08-17 12:20:21 +00:00
andodeki	b7eb271a0d	feat(spreadsheet): logical and conditional formula functions ... The formula engine could sum, average and do single-cell math, but had no way to combine conditions or aggregate selectively — no `=IF(AND(...))`, no `=SUMIF(...)`. This closes that gap with seven Excel-compatible functions. Logical (evaluated lazily, like IF, so they short-circuit): - `AND(a, b, ...)` — TRUE iff every argument coerces to TRUE; ranges flatten to their cells. Stops at the first FALSE, so `AND(FALSE, 1/0)` is FALSE rather than `#DIV/0!`. - `OR(a, b, ...)` — TRUE iff any argument is TRUE, short-circuiting on the first TRUE. - `NOT(x)` — logical negation of the single argument. - `IFERROR(value, fallback)` — `value` unless it errors, in which case the fallback is evaluated and returned (lazily). Conditional aggregates (criteria-matched by position): - `COUNTIF(range, criteria)` — count of matching cells. - `SUMIF(range, criteria[, sum_range])` — sum of `sum_range` (or `range`) cells whose position matches; text in the summed region is skipped, errors propagate. - `AVERAGEIF(range, criteria[, average_range])` — mean of the matched cells, `#DIV/0!` when nothing matches. Criteria accept numbers, comparison operators (`>5`, `>=5`, `<5`, `<=5`, `<>5`, `=5`), case-insensitive text (`"apple"`, `=apple`, `<>apple`), and cell references holding any of those. Wildcards are not supported. Dependencies flow through the existing AST walk, so a SUMIF's range and criteria cell are tracked by the dependency graph and the formula recalculates when an input is edited — pinned by an end-to-end test through SpreadsheetData. Engine unit tests 331 -> 343.	2026-08-17 12:19:57 +00:00
andodeki	e1d1346b27	fix(mpesa): parse SMS from in-memory messages, not serde-skipped cache ... OfflineSmsMessage.body has #[serde(skip)] so it round-trips as empty. scan_sms was upserting to the offline store then loading back and parsing from the empty body, which always fails. Parse directly from the in-memory list_messages() result instead.	2026-08-17 15:13:33 +03:00
andodeki	70bbff7ecb	fix(cad): the nav pad's zoom buttons were a pixel from their own hit zone ... First test written against the widget layer, and it found something on the way in. The viewport's navigation pad — fit, four pans, zoom in, zoom out — had its layout written twice. `viewport_render.rs` drew each button at `col * (BTN_W + GAP)`. `viewport_input.rs` decided what a click hit with hand-written arithmetic inline in a 630-line event handler: `BTN_W * 3.5 + GAP * 3`. Those agree for whole-numbered columns and disagree at the half column the zoom pair sits in: 3.5 * 24 = 84 drawn, 22 * 3.5 + 2 * 3 = 83 hit-tested. The zoom buttons' clickable area sat one pixel left of the buttons, so their right-hand pixel column did nothing and a pixel of empty space beside them zoomed. `PanRight` was 2px short at its right edge for the same reason. One pixel is not much on its own. The mechanism is what matters, and it is the third instance of it in this module: two copies of one piece of geometry, free to drift, with nothing able to notice. The camera-to-world pair was the first, the scene-cache benchmarks the second. Both callers now read `nav_pad::LAYOUT`. The renderer iterates it; the hit test tests against it; the offsets come from one function. That also turns 88 lines of inline conditionals in the event handler into 32 lines of match, which is a readability win I would not have bothered with on its own. The bounds are now the drawn rectangle exactly — half-open, BTN_W by BTN_H, gaps dead. The old hit zones were 2px larger than the buttons in several places. Being strict is deliberate: a hit area larger than its button is indistinguishable from a misaligned one the next time something looks wrong. 7 tests, 100% of the new module. The one that matters is `drawn_and_hit_zones_agree`: every drawn rectangle must hit-test to its own button at all four corners and the centre. That test fails on the old code, which is the only reason to trust it. Verified with a real compiler, which this environment turns out to have: 1051 lib tests pass (7 new), cad_integration 154 pass, cargo fmt clean, engine coverage 97.16% with every floor met including nav_pad at 100%.	2026-08-17 12:12:34 +00:00
andodeki	6bf138d027	ci(email): cover the trip-report modules ... The domain test filter and floor (225) now include finance_report and email_receipts, and test-email-coverage.sh instruments both new files. Domain tests 216 -> 234; coverage 90.6% over 15 files. The review doc records the new feature.	2026-08-17 12:08:19 +00:00
andodeki	074066a382	feat(email): export the trip report from the inbox ... spawn_export_trip_report fetches the inbox (shared fetch_inbox_messages helper), extracts trip receipts, builds the report, and writes trip-expense-report.pdf into app-data; it posts TripReportExported with the path and a summary. The More page gains a Finance card with an 'Export trip report (PDF)' button and a status line.	2026-08-17 12:08:19 +00:00
andodeki	ee61546c2c	feat(email): trip-expense report PDF for the finance department ... finance_report.rs renders TripReceipts into a self-contained PDF with the nigig PDF stack (nigig-pdf-graphics, base-14 Helvetica): a summary table of dates and amounts with a total row, then one receipt block per trip, A4 with pagination and a bookmark outline. build_trip_report is pure and its tests read the output back through PdfDocument, asserting page sizes, dates, amounts and the total landed. Native-only (gated behind not(wasm32)), so the wasm build stays free of the PDF dependency. 6 tests, plus a runnable example.	2026-08-17 12:08:19 +00:00
andodeki	a935133bb4	feat(email): trip-receipt extraction (Bolt ride receipts) ... email_receipts.rs turns inbox messages into a TripReceipt — date, amount, currency, route, receipt id — for the finance department's expense report. Sender detection (bolt/uber/taxify), a tolerant currency+amount finder (total > fare > amount priority, comma/space grouping, KSh→KES), a date extractor (ISO, d/m/y, '17 Aug 2026', 'Aug 17, 2026', with the email timestamp as fallback), and label-prefix value extraction for the route and receipt id. A message without an amount is not a receipt. 12 host tests over realistic fixtures.	2026-08-17 12:08:19 +00:00
andodeki	e0d86274d8	feat(pdf): flatten annotations and form fields into page content ... Phase 5's `flatten_test.dart`. An annotation draws from its /AP /N stream, which lives beside the page rather than in it; flattening moves that appearance into the page's own content and removes the annotation, so what is drawn is part of the page and cannot be turned off, edited or extracted as a field. That is what "finalise this form" and "make these comments permanent" mean, and it is irreversible by design. The placement transform is the whole problem, and it is §12.5.5: transform the /BBox by /Matrix, take the bounding box of the *result*, then fit that onto /Rect. Skip a step and the stamp lands at the origin, or in the right place at the wrong size, and the page still renders. A rotated appearance is the case that exposes it — rotation swaps the transformed box's width and height, so fitting the untransformed box squashes it. What is refused matters as much as what is done: - **No appearance stream**: left in place and reported. Dropping it loses it; inventing an appearance draws something the producer never specified. - **Hidden or /NoView**: not drawn on screen, so burning it in would *add* ink the user never saw. - **A /Popup**: the pop-up window of another annotation, never drawn on the page itself. Appearances are painted as XObjects rather than having their operators spliced in. Splicing needs the stream's resources merged into the page's with every name collision renamed, and it loses the /BBox clip an XObject applies for free. `pdf-document` cannot depend on `pdf-graphics` — the crate boundary is cos -> document -> graphics and inverting it to reuse `write_ops` would be a far worse trade than emitting the four operators (`q`, `cm`, `Do`, `Q`) directly. The number formatting follows the same shortest-exact rule as `content_edit::write_real`, and for the same reason. Verified by mutation, five defects, each confirmed red: placement matrix ignored 1 fail /BBox not transformed first 2 fail hidden check removed 1 fail annotation kept after flattening 7 fail existing page content dropped 1 fail **Externally verified, and it found a real gap.** Flattening the sample's seven annotations passed `qpdf --check` and kept every text run — but poppler still reported `Form: AcroForm` on a document with no fields left, because the catalogue entry survived. A viewer may still offer to fill in a form that no longer exists. `remove_acroform_if_empty` drops it, but only when *no* widget survives anywhere: flattening one page of a three-page form must not strip the fields still live on the others. before: Form: AcroForm after: Form: none `flatten_document` is the whole-document entry point — every page, then the form entry — and re-parses between pages because each flatten appends a revision the next must read. 22 round-trip tests through the saved file, including that flattening twice is idempotent (a Do count that grows on every save is how a "flatten" button pressed twice doubles every stamp), that existing page resources survive, and that a /AP /N state dictionary resolves through /AS. Engine suite 1075 -> 1108. Remaining in Phase 5: object compaction, redaction, and outline, page label and struct-tree editing.	2026-08-17 12:03:59 +00:00
andodeki	0596fc66ed	test(cad): measure the widget layer — 13.15%, and six files at zero ... The number nobody had. The engine harness is structurally blind to viewport*.rs, workspace*.rs and friends, so "97.14% covered" has always been a statement about the smaller half. With the Makepad Linux packages installed, the real crate builds and its tests run under -C instrument-coverage, which makes the other half measurable in about six minutes. 10,714 lines. 13.15% covered. 9,305 never executed by any test. viewport_input.rs 736 lines 0.00% viewport_render.rs 2,083 lines 0.00% workspace_actions.rs 690 lines 0.00% cad_editor_sheet.rs 224 lines 0.00% viewport_2d.rs 138 lines 0.00% code_editor.rs 67 lines 0.00% viewport.rs 3,548 lines 14.37% workspace.rs 2,462 lines 14.18% script_bindings.rs 724 lines 71.27% viewport_input.rs is every click, drag, modifier and keystroke the editor handles, and not one line of it has ever run in a test. viewport_render.rs is every draw call. This reframes the six sessions of engine work above it. 1044 green tests and a 97% engine coexist with an input layer nothing has touched. Those facts were never in tension — they were just never on the same page, because the tool that produced the good number could not see the bad one. A coverage figure that excludes the risky half is not a summary, it is an average with the interesting term deleted. The new script reports the widget files ONLY, on purpose. Folding them into one number would let a 97% engine hide a 0% input layer, which is the arithmetic this exists to prevent. No floors yet, deliberately: a floor at 13% reads as a blessing rather than a debt. The first real input test should set one behind it. Verified: two runs, cold and warm, same numbers; script cleans its profraw data on exit and refuses with a useful message when the native packages or llvm-tools are missing.	2026-08-17 11:59:06 +00:00
andodeki	f1c3c18374	docs(cad): the widget layer was never unbuildable — I never tried ... Every claim I have made about this environment's limits was wrong, and it cost six sessions of work routed around an obstacle that was not there. `cargo test -p nigig-build` needs wayland, X11, GL, alsa and polkit. I turned that into "does not build here" early on, wrote it into commit messages, wrote it into TEST_BASELINE.md, wrote it into the review, and never retested it. In a plain container: bash tools/makepad-native-libs.sh --install # ~8s cargo check --locked -p nigig-build --lib # 2m52s, clean cargo test --locked -p nigig-build --lib # 5m22s, 1044 passed cargo test --locked -p nigig-build --test cad_integration # 154 passed cargo fmt -p nigig-build -- --check # clean The install script existed the whole time, written for exactly this, with a comment explaining that libasound and libpulse are needed to *link* a test binary rather than merely to check it. This file's own "Reproducing" section pointed at it. So did the review's "Required next commands". Two things follow. First, every commit I have pushed to this crate is now verified rather than CI-gated-and-hoped-for, including `refactor(cad): one screen-to-world path, not two`, whose message says plainly that its compile could not be checked locally. It compiles; the suite passes; the gates pass. Second, the honest reading of the last six sessions: I asked once whether to attempt the heavy build, got a reasonable "ship what you can actually test", and then treated that as settled fact rather than a decision worth revisiting when the cost of being wrong kept growing. An assumption made once and never retested is indistinguishable from a fact. That is precisely the criticism this work levelled at the CAD review entry, and I earned it too. The 750-test figure in this file was also stale; it is 1044 now. The host-only harness stays. It needs no apt, no root and no desktop packages, runs in ~20s warm against five minutes for the full crate, and is what the CI coverage gate uses. The full build is what to reach for when a change touches the widget layer, which the harness cannot see.	2026-08-17 11:55:25 +00:00
andodeki	8ca5070b26	test(spreadsheet): cover the remaining serialization and border branches ... The last reachable lines in data.rs, all in the persistence layer and one style command: - serialize/deserialize round-trip for named ranges (the NAME block line), text colour, and an unknown block tag (ignored for forward compatibility), plus a malformed NAME line inserting nothing. - The legacy CSV deserializer's `|B` bold and `|F` formula metadata segments. - apply_command with BorderTarget::None clearing all four edges (the "remove borders" action), the complement of the per-edge arm the existing test covers. - A large-range self-reference reporting CycleDetected (pins the fix in the previous commit). data.rs 97.91% -> 99.49%; engine total 98.50% -> 99.13%. Unit tests 325 -> 331. Deliberately uncovered, as documented: the B17 invariant's debug_assert/#ERROR! patch (only reachable on a dep-graph bug), the topological-sort underflow guard, the let-else continue after a cell disappears mid-iteration, and the `}` attribution regions after unconditional returns.	2026-08-17 11:49:00 +00:00
andodeki	e1dcbefda3	fix(spreadsheet): large-range cycle detection must report, not zero ... The large-range fast path in DataEvalContext::get_range_values (used for ranges over 64 cells) handled a re-entrant formula cell — a reference cycle — by silently pushing 0.0. The small-range path in get_cell_value reports FormulaError::CycleDetected for the same situation. So a cyclic formula inside a large range contributed zero to the sum instead of surfacing #CYCLE!. The inconsistency is invisible in normal recalculation: the topological sort in recalculate_incremental detects cycles before any evaluation, so the branch only fires through the legacy recursive get_cell_value API. The fix makes the two range paths agree.	2026-08-17 11:49:00 +00:00
andodeki	2ba06f34c5	fix(makepad-table): stop overlong cell text spilling into the next column ... A right-aligned cell whose text is wider than its column drew over the left-aligned text in the column beside it. Two separate causes, one on each side of the cell. **Spilling left.** `align_text_x` computed `pos.x + width - pad.right - text_width` for right alignment and `pos.x + (width - text_width) * 0.5` for centre. Once `text_width` exceeds the column width both go *negative relative to the cell*, so the run started outside its own cell and reached backwards over the previous column. That is the reported collision: the right-aligned column's text sitting on top of its left-aligned neighbour. The result is now clamped to the padded left edge, so an overlong string starts where a left-aligned one would and runs forwards. **Spilling right.** Clamping alone only fixes the near side — the run would still continue past the cell's right edge into the following column, which is the same overlap seen from the other direction. `fit_text_to_cell` shortens anything wider than the padded width and appends an ellipsis. `DrawText` can do this itself via `text_overflow: Ellipsis`, but only through `draw_walk`, which needs a turtle; these cells are drawn absolutely with `draw_abs`. The truncation therefore uses the same 7px-per-character estimate the positioning already used, so one approximation is applied consistently rather than two that can disagree. When a real text measurer replaces it, both callers change together. Order matters in `draw_cells`: the text is truncated first and the *truncated* string is aligned. Aligning the original and drawing a shorter one positions the run by a width it no longer has, which puts the overlap back. Tests 51 -> 59. Verified by reintroducing four defects: removing the clamp fails the left-spill test, removing truncation fails three, aligning the full text fails the ordering test, and measuring bytes fails the multi-byte test. Two of those needed the tests fixed first, and both were mine: - The draw-path ordering had no coverage at all, because `draw_cells` needs a live `Cx`. It now reads the source for the order of the two calls. Blunt, but an accidental reordering is exactly the regression this invites. - `truncation_counts_characters_not_bytes` passed with `len()` substituted for `chars().count()`. The byte count only gates *whether* to truncate, and the case I wrote was one that should be truncated either way. The test now uses a multi-byte string that comfortably fits — which `len()` would wrongly shorten — so the assertion turns on the difference rather than merely being near it.	2026-08-17 11:38:55 +00:00
andodeki	306504bc0b	fix(makepad-table): pin the pressed-state fill, and put the caret after the text ... Two bugs in the table demo's cell editor, both reported after the previous colour pass. **White on click.** `TextInput`'s `draw_bg` blends through six fill states, and the previous change pinned five of them. It missed `color_down`. The widget's animator holds `down: 1.0` for as long as the pointer is pressed on it, so clicking into a cell blended toward the theme's light fill and stayed there until the pointer moved off and the state decayed — which is exactly the reported "goes white when I click, correct once I move the mouse away". It read like a hover state and was the press state. `color_down` is now pinned, along with the six `color_2*` gradient partners. The shader only mixes those when `color_2.x > -0.5` and the default is a -1.0 sentinel, so they were inert — but the theme sets them, and anything that later turned the gradient on would have pulled theme colours back in. **Caret at the start.** `begin_edit` called `set_text` and nothing else. `set_text` loads the value and leaves the cursor at index 0, so typing into a cell that already had content inserted at the front. `move_cursor_text_end` after the load puts it where every spreadsheet puts it, and where `commit_edit` reading the whole buffer back already assumed the user was working. Tests 49 -> 51, and the existing state test was rewritten. It had been checking a hand-written subset of states and passed the whole time `color_down` was missing — a test that only covers the states someone remembered is a test that misses the one they forgot. The fill states are now their own exhaustive check. Verified by reintroducing each defect: removing `color_down` fails the fill test, removing the caret call fails the caret test, and swapping the caret call before `set_text` fails it too. That last case needed the test fixed first. The ordering guard compared `body.find("set_text")` against `body.find("move_cursor_text_end")`, and `find` returns the first match anywhere — including inside the doc comment above the code, which mentions `set_text`. Swapping the two statements left the comment in place, so the naive check still passed. It now compares the first non-comment line containing each call.	2026-08-17 11:33:20 +00:00
andodeki	691b868264	fix(spreadsheet): make an editing cell readable — no doubled border, cell's own ink and fill ... The same three symptoms were reported twice. The first fix went to `makepad_table`, which is not in the APK — `pageflipnav` does not depend on it. The editor actually being clicked is `SpreadsheetGrid::draw_edit_overlay`, reached through `nigig-build -> spreadsheet-ui`, and it is a custom-drawn overlay rather than a `TextInput`. This fixes that one. 1. **No border of its own.** `draw_walk` calls `draw_selection_overlay` immediately before `draw_edit_overlay`, and the selection rectangle is already stroked around this exact cell in `selected_border_color`. The overlay then drew four more 2px rects inside it, which is the doubled frame: an outer selection edge and an inner editor edge a pixel apart. The four strokes are gone. Checked before removing them that this cannot leave a cell unframed. All three paths into `request_begin_edit` are on an already-selected cell: the keystroke path uses `self.selected_cell()` by definition, and both double-tap paths record `InputIntent::Select` for the same cell on the first tap. The caret remains the signal that the cell is in edit mode. 2. **The cell's own ink, not a forced one.** The overlay did `self.draw_text.color = self.text_color;` with no branching, so a cell with a user text colour, a bold cell, or a formula all changed colour the moment the caret landed in them. It now resolves the same way the resting draw path does: per-cell colour, then formula green, then bold, then the default. 3. **The surface it rests on.** The fill was always `edit_bg_color`, which equals `cell_bg_color` — so an odd row, which rests on `cell_alt_bg_color`, visibly changed shade when editing began, and a cell the user had given a background lost it entirely. That last case is the "background goes a different colour and the text disappears" report: the fill came from `edit_bg_color` while the ink came from the cell's own style, and nothing kept the two in agreement. It also covers the selection fill. A cell being edited is by definition selected, so `draw_cell_bg` had already painted `selected_bg_color` (#x2d4a63, a blue-grey) underneath — text was sitting on a wash it was never coloured for. That is the grey. `grid.rs` had no tests and is excluded from the coverage report, because the widget needs a live `Cx`. The background choice does not, so it moved into `editing_bg_source` and is tested there; the border and ink changes are pinned by reading this file, which is blunt but is the only thing short of a GPU that can catch "compiles, runs, wrong on screen". 6 tests, verified against all three defects: restoring the border stroke fails 1, forcing `text_color` again fails 1, and ignoring row striping and cell backgrounds fails 3. The 12 clippy warnings in this crate tree are pre-existing and unchanged — checked by running the same command on a stashed tree. None are in the hunks here.	2026-08-17 11:07:39 +00:00
andodeki	60117099ae	fix(makepad-table): make an editing cell readable — white fill, dark ink, one border ... Three reported problems, one cause. `TextInput` does not have a colour, it has a set of them, and `get_color` blends between them: `color_focus` on focus, `color_empty*` when the field is empty, `color_hover`, `color_down`, plus `border_color_2*` gradient partners for every border state. The cell editor set four properties — `color`, `color_focus`, `border_color`, `border_color_focus` — and left the rest to fall back to the active theme. The active theme is the dark one: `theme_desktop_dark::script_mod` runs after the light one in `widgets/src/lib.rs` and wins. So the moment a cell took focus the editor drew `theme.color_text` (near white) on `theme.color_inset_hover` (dark grey), and an emptied cell hit the `color_empty` path, which was also unset. Fixed by pinning every state rather than the ones that happened to be visible in one configuration: 1. **No border of its own.** `border_size: 0.0` and every border state transparent, including the six `border_color_2*` partners, which render as a faint grey edge even at zero width on some backends. The 2px green frame is `draw_select`, drawn over the same rect in `draw_walk` — the editor's border sat inside it and read as a doubled frame. The green selection is unchanged. 2. **Editing ink equals resting ink.** `#x1f2937` in all eight text states, the same colour a non-editing cell uses. An editing cell should look like a resting cell with a caret in it; the caret is the affordance and a colour change only costs contrast. `draw_cursor` is pinned too — `theme.color_text_cursor` is chosen for a dark inset and nearly vanishes on white, which would have left no cue at all. 3. **Plain white fill**, in all five fill states rather than just two. This is also the "background goes white and the text disappears" case: that was the fill switching to the pinned white while the text switched to the unpinned theme colour, so the two moved independently. `draw_selection` is pinned to a pale green that matches the frame, instead of the theme's blue, so selected text stays dark-on-light. The same latent bug was in all six of the invoicer's inputs — the five header fields and the search box — which set the same four properties. They are pinned the same way. Their placeholders stay grey deliberately: a placeholder is a prompt, not content. Tests 44 -> 49. The colours live in `script_mod!`, which is data this crate does not parse, and no test can ask a widget what it drew without a GPU — so these read the DSL source and assert the states are present. That is blunt, and worth having anyway, because the failure mode is precisely "compiles, runs, looks wrong only on screen". Verified by restoring the original four-property block: all five fail.	2026-08-17 10:49:44 +00:00
andodeki	e0aab74452	feat(pdf): page operations — insert, reorder, duplicate, delete, import ... Phase 5's page management, matching dart-pdf's `page_ops_test.dart`, `page_index_map_test.dart` and `import_source_test.dart`. A reader sees "page 3"; the file holds a tree of /Pages nodes with /Kids and /Count and /Parent back-pointers, any of which can be left stale. So the writer **flattens to a single level**: a one-level /Pages node with every page as a direct kid is valid, is what most producers emit, and removes the entire class of bug where an intermediate node's /Count no longer matches what is under it. Preserving an arbitrary tree shape through arbitrary reordering is far more code for nothing a reader can see. `PagePlan` accumulates operations and applies them together, so intermediate states never have to be valid — delete page 0 and insert a new one at 0 without the document momentarily having no first page. `PageIndexMap` reports where every page went, which is the only way to fix an outline entry, named destination or link annotation afterwards. What each operation carries matters and differs: - Reorder and delete rewrite only the kid array, so page objects and their resources are untouched. - Duplicate writes a new page dictionary that **shares** the original's resource references. Two pages naming one font object is normal; deep-copying would double the file and change nothing visible. - Import must deep-copy the page and everything it reaches, renumbered, because source object numbers mean nothing in the destination. /Parent is deliberately not followed — it leads back to the source's page tree and from there to every other page in that file. Inheritable attributes are resolved *before* a page is imported. /Resources, /MediaBox, /CropBox and /Rotate may live on an ancestor (Table 30) that is not coming with it, so a page imported without them renders at the wrong size with no fonts, and nothing reports an error. **Round-trip tested through the saved file**, which is Phase 5's exit criterion: 20 tests that save, re-parse, and assert on what a reader actually gets. Pages are identified by /MediaBox width rather than object number, because object numbers are exactly what a page-tree bug scrambles. Mutation testing changed two things. Seven defects injected: /Count left stale 1 fail /Count omitted entirely 1 fail imported /Parent not rewritten 1 fail inherited attributes not resolved 1 fail import does not deep-copy 3 fail duplicate loses /Contents 1 fail re-parenting skipped 1 fail The last two only fail because of tests the mutations forced: - **A stale /Count passed everything.** Our own parser walks /Kids and never reads /Count, so it cannot see the disagreement — but other readers trust /Count, and a document where the two differ opens with a different page count in different viewers. The test now reads the raw page-tree node instead of asking the document. - **Re-parenting could be deleted with every test still green**, because the flat fixture's pages already parent to the root. Added a nested fixture with an intermediate /Pages node supplying an inherited /MediaBox — the case where leaving /Parent stale means a page keeps inheriting from a node it is no longer under. Externally verified: a generated sample with pages swapped and duplicated passes `qpdf --check` with no warnings, and poppler reads 4 pages with the reordering visible in extracted text. Engine suite 1039 -> 1075. Remaining in Phase 5: flatten, object compaction, redaction, and outline and struct-tree editing.	2026-08-17 10:38:55 +00:00
andodeki	b8bd71852f	feat(pdf): content-stream serialiser and editor — the Phase 5 foundation ... Phase 5 needs to write operators back, and `content.rs` has only ever parsed them. Every editing feature the phase asks for — insert, delete, replace, rewrite a text run, flatten an annotation — rests on that, and a serialiser that is subtly wrong does not throw: it writes a valid content stream that draws something else. So this is the serialiser plus the gate, and nothing built on top yet. The contract is a property, run over the whole corpus: parse(write(parse(bytes))) == parse(bytes) Operators, not bytes. Byte equality would be the wrong test — `1.0` may legally be written `1`, whitespace is free, and a writer that reproduced its input byte for byte would only prove it had copied it. It passes: **16,466 operators across 154 streams in 109 files**, plus stability, idempotence, and the same property after an edit. **Then mutation testing showed the corpus gate was not enough.** Six injected defects, and *five passed*: dropping name escaping, unescaping string parens, un-sorting dictionary keys, discarding unknown operators, and a fixed six-decimal number format. Real files are written by well-behaved producers, so 16,000 corpus operators contain no name with a space, no nested parenthesis, no seven-key inline dictionary and no vendor operator. A gate that only sees well-formed input cannot catch a writer that mishandles the rest. The adversarial set fixes that — eighteen streams, each a legal shape the corpus lacks, each chosen because a specific defect survives without it. Writing it found **three live bugs in the parser**, none of which the round trip could see on its own: - **Nested parentheses truncated a string to nothing.** `((nested))` parsed as the empty string, and worse, left the reader mid-string so every operator after it was parsed from the wrong offset. §7.3.4.2 says balanced parens nest and need no escaping. - **`#` escapes in names were never decoded.** `/My#20Font` — how every producer writes a font whose name contains a space — parsed as the literal `My#20Font` and never matched the page's resource. - **`PdfOp::Unknown` was declared and never constructed.** An operator the parser did not recognise vanished. Survivable for a renderer, fatal for an editor: parse, change one operator, write back, and every vendor extension in the page is silently gone from the saved file. And two in my own serialiser, both found the same way: - A fixed `{:.6}` flushed 1e-7 to zero — a scale factor silently becoming zero collapses whatever it transforms — and rounded `1.234567891` to a different number. Precision is now the shortest that parses back to the identical f64, exact by construction rather than by choosing a number. - Sorted dictionary keys turned out to be load-bearing. `PdfDict` is a HashMap and Rust seeds its hasher per process, so an unsorted writer is stable within a run and different on every new one: rebuild the same document twice, get two different files. Neither the round trip nor a within-process stability check can see it — both sides are equally unordered. Verified by running five separate processes and getting five different key orders. Two of those needed tests the round trip structurally cannot provide, so they assert on the parser directly: what `((nested))` must produce, and that operators after it are still read at the right offset. Final mutation run, eight defects, all caught: fixed 6-decimal precision 1 fail name escaping dropped (writer) 1 fail name unescaping dropped (parser) 2 fail nested-paren fix reverted 1 fail unknown operators discarded 1 fail string parens unescaped 1 fail close-paren unescaped 1 fail dictionary keys unsorted 2 fail `ContentEditor` sits on top: insert, append, prepend, delete, replace, isolate, and text-run rewriting that preserves the operator *kind* — a `'` stays a `'` and keeps its line advance, a `TJ` keeps its kerning numbers while its strings change. Every mutation is balance-checked, so an edit that would leave `q` without `Q`, or `BT` without `ET`, is refused at the edit rather than discovered at save time. `PdfOp` gained `PartialEq`, which is what makes the property expressible at all. Engine suite 1025 -> 1039. Phase 5's remaining items — page ops, import/merge, flatten, compaction, redaction — build on this and are not started.	2026-08-17 10:29:39 +00:00
arena-agent	b478945c34	ci(doc): gate the doc-workspace coverage floor on every push ... Adds the doc-workspace-coverage job to the nigig-build workflow, mirroring the CAD gate: checkout, then ./tools/test-doc-workspace-coverage.sh, which installs its own instrumented toolchain into a shell-trap-cleaned temp dir and fails if the total floor (92%) or any per-file floor is not met. The script joins the workflow's push/PR path filters next to tools/test-cad-coverage.sh so edits to the harness itself re-run the gate. The doc README gains the milestone section recording the 28.55% -> 96.76% line measurement, the honest exclusions (widget layer, persistence write-path wrappers, defensive traversal guards) and the behavior pins and defect fixes the drive surfaced.	2026-08-17 10:25:27 +00:00
arena-agent	949cf24189	test(doc): doc-workspace coverage harness with enforced floors ... tools/test-doc-workspace-coverage.sh measures line+region coverage of the doc module's pure layer the same way the CAD gate does: it copies the dependency-free sources (advanced_json, crdt_bridge, mobile_gesture, persistence, projection_layout, projection_session, and the collaboration/, editing/, layout/, model/, plugins/ trees) plus tests_pure.rs into a temporary host-only crate with the real module path, satisfies the five makepad-math symbols the pure layer uses through a 30-line makepad-widgets shim, runs the suite under -C instrument-coverage with a toolchain it installs itself, and enforces a total floor plus a per-file floor for every instrumented file. The per-file floors are the point: a lone total waves through the silent loss of one whole file's tests. Measurement moved from a 28.55% line baseline to 96.76% (6170 lines) with the tranche in the parent commit; floors sit a few points under per file, except persistence.rs (55%), whose three write-path entry points write into the host's real application-data directory and are covered through their path-injected seams instead -- the honest exclusions, the exact table, and the two defect fixes this drive surfaced (ReplaceBlockRange validation order, dead RgaText::visit_children) are written down in the module's new COVERAGE.md. Everything the script touches -- pinned toolchain, cargo home, target dir, fetched Makepad tree, profraw data -- lives under one mktemp dir removed by a shell trap on every exit path; nothing lands in the repo or $HOME unless KEEP_COVERAGE=1 is set for a debugging run. DOC_WS_COVERAGE_REPORT_ONLY=1 measures without gating.	2026-08-17 10:25:27 +00:00
arena-agent	71c31cd19f	test(doc): host-only suite split + pure-layer coverage tranche ... Split the doc module's tests in two so the dependency-free majority can also run under coverage instrumentation on a host-only crate: * tests_pure.rs (new) holds every test that needs no Cx -- model, layout, editing, collaboration, advanced JSON, projection layout/session, CRDT bridge, persistence seams, mobile gestures -- and is the file tools/test-doc-workspace-coverage.sh copies byte-for-byte into its harness. * tests.rs keeps the widget-runtime and boot tests; shared helpers (projection_table_engine, only_table) live in tests_pure so both suites use them. On top of the split, this tranche adds ~80 tests covering the pure layer's real gaps: every Command apply arm and its inverse (text, atoms, blocks by stable id, image properties, block ranges, table cells/rows/columns, merges/splits/restores, node insert/delete/ replace), the DocumentController's CRDT typing lifecycle (insert/replace/delete ranges, backspace/forward delete, position sync), remote operation classification (Applied/Duplicate/Deferred/Rejected), tombstone compaction at a peer-acknowledged frontier, a two-peer MemoryTransport conversation, typing coalescing and history limits, the cell-text editing and multi-line geometry helpers in projection_layout, table cell cursor/range/merge queries, and the small model/session/selection behaviour surface. Two defects found while writing the tests, fixed with pins: * Command::ReplaceBlockRange validated a caller-supplied block_ids length AFTER draining blocks and legacy ids out of the document, so a malformed (remote) command destroyed content before reporting failure. Validation now runs before any mutation; a test proves a rejected replace leaves blocks, ids and order untouched. * model::crdt::RgaText::visit_children was dead code -- a String-collecting duplicate of visit_atoms with no callers. Removed. Two semantics that were folklore are now pinned with inline reasoning: a multi-peer conversation only converges when each peer owns a distinct document.crdt.local_actor (the two-controller sync test assigns alice/bob), and a mid-range replace_range_crdt renders its replacement after the tombstoned subtree it replaced, because RGA sibling order walks by atom id ("hello" -> "hloY" is asserted, not assumed). cargo test -p nigig-build --lib: 1031 passed, 0 failed.	2026-08-17 10:25:27 +00:00
andodeki	179fe0533d	docs(cad): replace the guess about the widget layer with the compiler's answer ... This file said ~18,000 lines "needing live_design!, Cx and an event loop". That was a grep-shaped guess, and it was wrong twice: it swept in tools.rs, which needed no such thing, and profile_benchmarks.rs, whose only Makepad imports are the math types. Replaced with a per-file table produced by adding each of the eleven files to the harness on its own and reading what the compiler asked for. The answer is unglamorous but now trustworthy: every remaining file is blocked by a widget or platform type — Cx, Cx2d, Event, Widget, LiveId, ScriptVmBase — and there is no cheap seam left. Measuring any of them means a real Cx, or lifting logic out of `impl CadViewport` into free functions first, which needs a compiler for the widget layer. Recording the sizes alongside the blockers so the next person can pick the cheapest one deliberately instead of by feel.	2026-08-17 10:17:46 +00:00
andodeki	ac145bfaab	test(cad): measure tools.rs, which a false comment had ruled out — 0% to 97.07% ... tools.rs opens with: Struct definitions stay in mod.rs (they use #[derive] macros that need the script_mod! context). Only the impl blocks are here. That is not true, and it is the reason 250 lines of pure tool-state logic — tool cycling, work-plane mapping, the inclined-UCS maths — had never been measured or tested. `CadTool`, `WorkPlane`, `AxisLock`, `DrawingState`, `SnapSettings` and `InclinedPlane` each derive some combination of Clone/Copy/Debug/PartialEq/Eq and nothing else, and none is inside the `script_mod!` block. I found it by asking the compiler instead of grepping. Adding each of the eleven unmeasured CAD files to the harness one at a time and reading the errors gives the real dependency, not a guess: tools.rs needed six plain types; viewport_2d.rs needs `CadViewport` and `Cx`; code_editor.rs needs `Cx2d` and `DrawStep`. Only the first of those is a documentation problem rather than a real one. My previous two triage passes used a grep heuristic and were wrong twice, including about this file. The harness now mirrors those six declarations, extracted from mod.rs at run time so they cannot drift, exactly as it already did for ViewMode and SelectionMode. **No production code moved.** Moving the declarations for real is a smaller job than the comment implies but not a free one: DrawingState, SnapSettings and InclinedPlane have private fields that mod.rs and viewport.rs read directly, so their fields need widening first. CadTool and WorkPlane are fieldless and could move today. That is now written in the file for whoever has a compiler for the widget layer. 13 tests, on the properties that break quietly: - Cycling forward visits all 17 tools exactly once and closes the ring. `cycle_next` is a hand-written 17-arm match; a duplicated or skipped arm makes a tool unreachable from the keyboard and nothing else would notice. - Backwards is asserted to be the exact inverse, per tool. Shift-Tab that does not undo Tab reads as "the tool picker jumps". - Labels must be unique — two buttons reading the same is a UI bug with no test otherwise — and every tool needs a description. - `to_kind` maps only the drawing tools; Select, Delete and Measure must return None or they would create geometry on click. - `InclinedPlane::from_3_points` produces a unit normal perpendicular to both edges, and rejects collinear or coincident picks rather than returning a NaN basis from a zero-length cross product. - The plane basis is orthonormal in both branches, including the vertical-normal case that exists because the usual "up" reference is parallel to the normal there. Also fixes a real bug in this script's own drift detector: it tested membership with `case " ${ENGINE_FILES[*]} "`, and `[*]` joins on the first character of IFS, which this script sets to a newline. The pattern could never match, so the note fired for every non-widget file. It was right about tools.rs by accident. Floor: tools.rs 95. Total unchanged at 97.14% over a larger denominator. Verified: 568 tests green, every floor met, hermetic run clean.	2026-08-17 10:16:52 +00:00
andodeki	b83e7122c4	feat(makepad-table): file picker, search, recents, New/Delete (Invoicer UI Phase 3) ... The last open phase. The sidebar gains a search box, a filtered document list and a recents list; the toolbar gains New Invoice/Quote/Receipt, Open, Save As and Delete. **The file picker is robius, not makepad's.** Makepad has an `open_system_openfile_dialog`, and it is implemented on macOS only — the Linux and Android backends never handle `CxOsOp::SelectFileDialog`, so the op is queued and dropped. It compiles, it runs, the dialog never appears. That is the worst kind of broken, so this uses `robius-file-picker`, the same crate `nigig-build`, `nigig-pay-ui` and `nigig-sms` already depend on at the same pinned revision, which goes through `rfd` on desktop and the platform picker on Android. CI gates against the macOS-only call returning. The picker's callback runs off the UI thread with no `Cx`, so it parks its outcome in a mutex and signals; `drain_file_picker` applies it on the next `Event::Signal`. Same shape as the SMS bulk CSV import. Model additions, in `makepad-doc-model` so they are testable without a window: `DocKind` with `blank()` constructors, `DocumentLibrary::create`, `remove`, and `selection_after_remove`. Decisions worth naming, because each has a wrong answer that looks fine: - **A new document is empty**, not seeded from the samples. A blank invoice arriving with "Acme Studio LLC" on it invites someone to export it without noticing whose name is there. `issue_date` is blank too — there is no clock in that crate and a guessed date is worse than none. - **Generated numbers cannot collide**, including with documents loaded from disk, and they reuse gaps left by deletions. The number becomes the filename: two documents called INV-1 save over each other and one is lost silently. - **Delete removes the row, not the file.** Removing an entry from a list is not consent to delete a document off disk, and there is no undo here. The status line says the file is untouched. - **Save reports "Choose where to save…", not "Saved."** The dialog being open is not the file being written. - **Search filters on every keystroke**, unlike the header fields, which commit on Return. Every prefix of a query is a valid narrower search; there is no such thing as a half-typed one. - **Searching does not move the selection.** Filtering is a view change, and switching the open document because a letter was typed loses the user's place. - **`selection_after_remove` is separate and exhaustively tested.** Deleting before the selection shifts it, deleting the selection keeps the index unless it was last, deleting after it changes nothing, and emptying the library selects nothing. Every wrong answer silently shows a different document; one of them indexes out of range. The document list is a fixed pool of 12 button slots rather than a `PortalList`, because this app opens documents one at a time. The pool is honest about its limit: anything past it renders as "+n more — narrow the search to reach them" rather than being dropped. Tests 79 -> 90. Six of them are the invoicer's first: `App` derives `Script` and cannot be built outside a live `Cx`, so the sidebar's presentation logic was extracted into four pure functions and tested there. Verified by reintroducing six defects across the two crates — silent overflow, a selection marker that shifts the indent, whitespace counting as a search, colliding numbers, a selection that ignores the shift, and a `blank()` that pre-fills. Also fixed, all pre-existing and all now blocking the `-D warnings` gate that has been running on these crates since the workflow was added: `std::io::Error::new(ErrorKind::Other, _)` in two crates, a manual `RangeInclusive::contains`, a manual `is_multiple_of`, a single-arm `match`, and a duplicated `#[test]` attribute that was annotating one function twice — which is why the count reads 36 rather than 37 here; no test was lost. The sample data keeps its `12_000_00` money literals, where the last group is the minor units and the number reads as "12,000.00" at a glance. `inconsistent_digit_grouping` is allowed at the crate root with that reasoning, rather than regrouping every amount into thousands and making each one need arithmetic to check against its comment.	2026-08-17 10:01:43 +00:00
andodeki	5f999b7e9f	fix(tools): make test scripts executable for CI runners	2026-08-17 09:57:13 +00:00
andodeki	189377a3a3	ci(email): build the wasm path; document the closed §8 gaps ... email.yml: install the wasm32-unknown-unknown target and check the email domain's credential-bearing wasm half (call_email_api, WasmFetchTransport, set_email_api_url) so a browser-only breakage cannot reach main unseen. The domain test floor ratchets 205 -> 210. The review doc's §8 is rewritten: the TLS handshake, the wasm build, B1 and the test/clippy baselines are now executed/measured; the only entries left are the ones that genuinely cannot run in CI (a live relay's cert, a browser's fetch), stated with their exact reasons.	2026-08-17 09:39:04 +00:00
andodeki	216202a90a	test(email): execute the TLS handshake, not just read it (§8) ... The assessment's 'What I have not verified' listed the TLS handshake and a MITM test as gaps — the S1/S3 analysis of relay()/TlsParameters::new was a read of lettre's source, never observed. Two tests now execute the PRODUCTION build_transport path over a real TCP + TLS socket: - a_starttls_downgrade_is_refused_without_sending_credentials: a server that cannot STARTTLS receives no AUTH/MAIL FROM/RCPT TO/DATA — credentials and the message never cross a cleartext link (S3/T-E1 downgrade protection). - a_self_signed_certificate_is_rejected: a server presenting a self-signed cert (minted with rcgen, served by tokio-rustls/rustls) is rejected by the transport, whose accept_invalid_certs is false — the active-MITM scenario (S1), observed rather than assumed. The SMTP sink now records every command line so a test can assert a command was never sent. Domain tests 214 -> 216.	2026-08-17 09:39:04 +00:00
andodeki	1220f89fc6	feat(pdf): close the last three Phase 4 items — reconciliation, CFF, cmap ... The three items the previous commit's audit found unimplemented while the status line said "complete". All three are done and externally verified. **1. Field-value reconciliation** (`reconcile.rs`). A field carries its value in /V and its rendered look in /AP, and nothing in the format keeps them in step. Files arrive with them disagreeing all the time: a producer writes /V and leaves appearances to the viewer, or something edits /V without touching /AP. Until now this crate simply believed /V and regenerated appearances only for fields it had itself edited — right for a field we changed, wrong for a field that arrived inconsistent. The module deliberately does **not** pick a winner. PDF 32000-1 §12.7.3.3 settles exactly one case — /NeedAppearances true means /V is authoritative — and is silent on the other, where a conforming viewer renders /AP and never looks at /V. So it classifies the disagreement and resolves it against a caller-declared `Intent`, because the right answer genuinely differs: a viewer must show /AP to match other viewers, an extractor must read /V, an editor must regenerate so the saved file agrees with itself. Silently choosing one would be ADR 0017's failure in a new place — every answer plausible, none checkable, the caller unaware a decision was made for it. Two cases are not judgement calls and are handled outright. A missing or dangling appearance renders *blank*, and blank is never what the producer meant, so even Display regenerates. An unselected radio member showing /Off while the group's /V names another member is correct, not a conflict — reporting it would flag every well-built radio group there is. **2. Type1/CFF embedding** (`embed_opentype_whole`). The spec says "if feasible". Subsetting CFF is not — it means rebuilding the CFF INDEX, charset and charstrings, a second font format inside the first — and `subset_truetype` rightly keeps refusing it by name. Embedding the program *whole* is feasible, and that is what this does: /FontFile3 with /Subtype /OpenType under a CIDFontType0 descendant, per Table 126. Each of those keys matters and none is guessable from the others. A CFF program in /FontFile2, or under a CIDFontType2 descendant, still produces a file qpdf accepts and a font that loads as the wrong type or not at all. /CIDToGIDMap is omitted because it is defined for CIDFontType2 only. The trade is made visible rather than buried: `EmbeddedFont::is_subsetted` is false here, so a caller with a size budget — or a licence that forbids shipping a whole face — can refuse instead of discovering it from the output size. **3. `repair-cmap`** (`glyph_index`). A symbol font declares no Unicode subtable: it maps glyphs into the private-use area at 0xF000 + the low byte under platform 3, encoding 0. Asking it for 'A' found nothing and the character silently vanished from the output — the font "missing" a glyph it plainly has. Now the (3,0) subtable is kept as a fallback and retried at 0xF000 + low byte, after the proper lookup fails so a font with both subtables is still read through the Unicode one. Format 0 is read too; omitting it left legacy and symbol fonts mapping nothing while appearing to have a usable cmap. The repair must not manufacture glyphs, which is its own test: a character the font genuinely lacks still returns None, because turning a missing character into a wrong one is worse. **Fixtures.** No CFF or symbol font ships on the CI image, and neither can be tested honestly against a hand-built stub — the point is that the bytes are a font program a third-party reader accepts. Both are generated from DejaVu by checked-in fontTools scripts: `cff_sample.otf` (1.6 KB, real OTTO/CFF outlines) and `symbol_sample.ttf` (664 B, a single (3,0) subtable so the repair path is the only route to its glyphs). Both generators pin `head.created`/`head.modified` to zero. fontTools stamps the current time, so the output differed on every run and CI's "fixtures match their generator" check failed against a file nobody had edited. Caught by running that check rather than assuming it passed. A fixture that cannot be regenerated byte-for-byte is not reviewable: you cannot tell a deliberate change from a rebuild. **Verified by mutation**, seven injected defects, each confirmed red: NeedAppearances ignored 1 fail dangling /AS not detected 1 fail blank rendering shown faithfully 1 fail CFF written to /FontFile2 1 fail CFF given a CIDFontType2 descendant 1 fail whole font claims to be subset 1 fail cmap 0xF000 retry removed 3 fail **Verified externally.** The sample now carries a third page set in the whole-embedded CFF font, and `check-pdf-external-readers.sh` gained `pdffonts` — the only check that inspects a font *program* rather than the file structure, which is exactly where a wrong /FontFile key shows up. poppler reports both fonts embedded and distinguishes them correctly: ETXLDI+DejaVuSans CID TrueType Identity-H emb yes sub yes NigigTestCFF CID Type 0C (OT) Identity-H emb yes sub no and extracts "Hello CFF 123", which only works if the CFF program loaded, /Identity-H addressed its glyphs and /ToUnicode mapped them back. That check also caught its own page-count assertion going stale when the third page landed — a gate that notices its own fixture changing is working. Engine suite 953 -> 985. Coverage 87.27%, all floors met. Phase 4 is complete but for the ui.rs interaction tests, which are written and blocked on the Makepad fork's missing headless backend.	2026-08-17 09:31:41 +00:00
andodeki	587a43864f	test(spreadsheet-ui): cover the remaining controller branches ... The UI controller modules are the one part of the spreadsheet stack the coverage report could not reach before the memory-conscious build (-j1); measured now, their last reachable gaps are closed. geometry.rs 98.77% -> 99.51%: - row_resize_at: a point just inside a row's bottom edge resizes that row, and a point in the middle of a row is no resize target. The exact boundary between two rows resolves to the lower one, so the bottom-edge arm had no other path to it. render_cache.rs 98.21% -> 98.68%: - get_or_insert_with on a missing cell runs the builder (the entry API's insert arm, the complement of the hit arm), and a later read sees the cached state without rebuilding. model.rs 90.99% -> 98.40%: - load_saved() and save() were only exercised by a #[ignore]d test. The test now runs by default: it serialises on a module-local lock and restores whatever was in the shared generated/ directory before it ran, the same save/restore convention the engine's own persistence tests use. ui-controllers total 96.99% -> 99.11% (floor 96). UI lib tests 52 -> 55, all pass, none ignored. Deliberately uncovered, as before: the panic-arm canaries in event_router's positive tests, the frozen-pane `return None` guards in col_at_x/row_at_y (the frozen loop spans exactly the frozen interval, so they cannot fire), the existing unreachable-builder closure in render_cache's hit test, and the environment-dependent save/restore cleanup branch in model's disk test.	2026-08-17 09:26:55 +00:00