573 commits

Author	SHA1	Message	Date
andodeki	4cbb155cb7	perf(cad): stroke per group, not per item -- Phase 4 of the render plan ... A 2,000-part plan view with everything on screen cost 2,270 tessellation calls a frame. It now costs four: two for the grid, two for the parts. `stroke()` tessellates the whole accumulated path and clears it -- `tessellate_path_stroke` ends in `path.clear()` -- so queueing many subpaths and stroking once is one tessellation instead of N. The idiom was already in this file: `queue_dashed_line` has done it for the axis grid since Phase 3.9, guarded by a test. Phase 4 applies it to the two loops that never adopted it. **Base grid: two passes, two strokes.** Minors queued and stroked at 0.55, majors at 1.6 -- the stroke width is the one thing that genuinely needs its own call. `GridRange::has_minor_lines`/`has_major_lines` decide whether a pass runs at all and `frame_budget` counts strokes with the same two predicates, because an empty `stroke()` still enters the tessellator and a budget that assumed two when the renderer made one would be wrong in the direction that hides work. Minors stroke first so majors land on top where they cross; same colour either way, so the only visible difference is that the thicker line wins a crossing, which is the right answer. **Parts grouped by colour.** New `batching::ColorKey` -- the bit pattern, because `f32` is not `Hash` and two colours whose bits differ are two colours -- feeding the same `group_in_first_appearance_order` that Phase 3 groups shapes with. The colour policy moved out of the two draw loops into `constants::part_outline_color`, so the renderer and `frame_budget` cannot disagree about how many groups a frame has; the 2D loop had `vec4(1.0, 0.82, 0.40, 1.0)` written out where `PART_SELECT_COLOR` already existed. **Selected and hovered parts stroke last**, in their own groups, so a highlight is never hidden under a neighbour's outline. They were interleaved in document order before and could be. `FrameBudget` gained `grid_lines` and `part_outlines` beside the call counts. Geometry volume and call count are different numbers now and both are worth reading -- `VectorSubmission { outlines, stroke_calls }` mirrors Phase 3's `MeshSubmission` for the same reason. Measured (bench_frame_submission_budget, 1920x1080, 200 m site): zoom 5 m, 2000 parts: 12 visible outlines -> 4 tessellations (was 2170) zoom 200 m, 2000 parts: 2000 visible outlines -> 4 tessellations (was 2270) The second row is the point, and it is the row Phase 1 could not move: everything is on screen, culling removes nothing, and the frame still costs four calls. WHAT THIS DOES NOT DO: vertex volume is unchanged. The same 2,000 rectangles are tessellated -- in two calls rather than 2,000. What is saved is per-call overhead: tessellator setup, two `std::mem::take`s and an `append_geometry` each time. If a 2,000-part plan view is still slow after this, the remaining cost is triangles, which is Phase 5 and should only happen if a measurement asks for it. One visible-behaviour caveat, stated rather than buried: parts of the same colour are now drawn together, so where two outlines of *different* colours overlap, which is on top can change. They are 1.8 px outlines and the highlight ordering got strictly better, but it is a change to what is drawn, not only to how. Two tests were wrong before the code was, which is becoming this plan's pattern. `constants.rs` fell to 81.82% and the coverage floor caught it -- `part_outline_color` had no tests, and it now has five. And the guard test's first draft looked for a closing brace at a fixed indentation, matched the wrong one, and failed on correct code; it matches braces properly now. Verified: tools/test-cad-coverage.sh green -- total 97.33%, batching.rs 100%, cull.rs 100%, render_budget.rs 99.68%, constants.rs 98.55%, all floors met; cargo check --locked -p nigig-build --lib clean; cargo test --lib 1114 passed (1100 + 14 new); --test cad_integration 154 passed; CAD_BENCH=1 harness green; cargo fmt --check and git diff --check clean.	2026-08-21 05:08:14 +00:00
andodeki	06a3de6126	feat(spreadsheet): dropdown cells and button cells (#9 remainder, complete) ... Finish the widget-in-cell catalog with the two deferred controls, each a plain value plus a render flag (same convention as checkbox/slider/Markdown). - Dropdown: CellStyle.choices (a serialized, pipe-escaped list) makes a cell cycle through its choices on click. The toolbar "Drop" button parses the selected cell's value as a "Low|Med|High" list into choices (undoable SetChoices) and selects the first entry; a dropdown renders its value with a trailing ▾ affordance. dropdown.rs holds next_choice / parse_choice_list. - Button: CellStyle.button makes a cell a button whose value is a TARGET[+N] action spec (A1 ref + optional signed step, default +1). Clicking increments the target cell's numeric value through set_cell (undoable, recalculates dependents) — the spreadsheet-native counterpart of the reference's "+10" boost. button.rs holds parse_button_spec / button_step / format_step_value; the cell renders as a raised box with the spec centred. Toolbar "Btn" toggles the flag. Both flags ride in the render cache so cached cells stay styled, and both commands route through WorkbookCommand (apply + apply_command + dirty marking). Choices serialize after the markdown/slider/button flags; older files default to off/empty. Engine: 491 lib tests (+4) + integration. UI controllers: 142 tests (+8, button 5 + dropdown 4). Coverage: engine 96.59%, ui-controllers 99.45% (floors 96); button.rs and dropdown.rs at 100%.	2026-08-21 04:52:00 +00:00
andodeki	328690dfbf	feat(spreadsheet): 3-state header sort (asc/desc/off) with a timed report ... Close the last sort gap: the spreadsheet grid's header sort now cycles ascending → descending → off like the datagrid reference, and reports how long each sort took. Engine: SpreadsheetData::sort_rows remembers a RowSort { ascending, key_col, restore } where restore[view_row] is the row the data originally lived on, composed across sort chains. New unsort_rows applies that map to restore the pre-sort order (no-op without a sort, clears undo, recalculates moved formulas); sort_state() reports the active (column, direction). Workbook gains unsort_active_sheet and active_sort_state. The sort order is transient — reset by deserialize, never serialized. UI: sort_state::next_sort_state replaces the 2-state next_sort_direction with the asc→desc→off cycle, and the workspace HeaderClicked handler times the sort with std::time::Instant, drives unsort_active_sheet on "off", and appends the report ("sorted 1000 rows by B descending in N ms") to the status bar via a cached sort_status field. Engine: 487 lib tests (+6) + integration. UI controllers: 134 tests. Coverage: engine 96.66%, ui-controllers 99.42% (floors 96).	2026-08-21 04:52:00 +00:00
andodeki	dc1defd7e8	perf(cad): one draw call per shape, not per part -- Phase 3 of the render plan ... A 2,000-part model of six repeated shapes now draws in six calls. It drew in 2,000 before, and in 733 after Phase 1 culling at a working camera distance -- but at 400 m, with the whole site on screen and nothing to cull, it still drew in 2,000. Culling decides which parts are submitted; instancing decides how many calls carry them. They are different axes and this is the one that does not care where the camera is. No shader change was needed and that was the surprise of the phase. `DrawCadMesh`'s `transform`, `color`, `depth_clip` and `display_mode` are `#[live]` fields after `#[deref] draw_vars` in a `#[repr(C)]` struct, which is precisely the per-instance row `DrawVars::as_slice` packs and sends. The loop was already sending instance rows -- one per call. Three methods (`begin_instances`, `push_instance`, `end_instances`), ported from `DrawPbr::begin_many_instances_for_mesh` in the pinned fork, keep the call open across a group instead. New `batching.rs` (pure, 100% covered, floored): group_in_first_appearance_order(items) -> Vec<(K, Vec<T>)> batch_count(keys) -> usize First-appearance order rather than iterating a `HashMap`, deliberately. `HashMap` order is randomised per process, so batch order -- and the order parts reach the GPU -- would differ between runs and after a rehash. For opaque depth-tested geometry that is invisible, which is exactly what makes it a bad thing to depend on: the day someone adds a translucent material it becomes a flicker that reproduces on one machine in five. `FrameBudget` now carries `MeshSubmission { instances, batches }`. Named fields rather than a second positional `usize` because the entire point of the phase is that the two numbers now differ, and a caller that swapped them would report the win backwards. Measured (`bench_frame_submission_budget`, counts): camera 20 m, 2000 parts: 733 visible -> 6 draw calls (was 2000) camera 80 m, 2000 parts: 1459 visible -> 6 draw calls (was 2000) camera 400 m, 2000 parts: 2000 visible -> 6 draw calls (was 2000) ...and with every part a different size, 733/1459/2000 -- the ceiling, which is in the table for the same reason Phase 2's is. WHAT IS NOT VERIFIED, plainly. The instanced submission has never run. There is no GPU, no window and no `Cx` here, and tests/ui.rs still fails at child-build exit 101. What is verified: it compiles against the real Makepad API; the grouping is right (seven tests, including "no item is lost or duplicated", which is the failure mode hardest to see in a screenshot); the budget arithmetic is right; and the batch cannot be left open on any path -- `every_instanced_batch_is_closed_before_the_loop_turns` is a source check in the same style as the redraw_all guard, because a batch left open drops its rows and the parts simply vanish with no error anywhere. Someone with a window needs to open a 3D model and confirm the picture is unchanged. Two things limit the damage if it is not: `begin_instances` returning false falls back to the old one-call-per-part loop (it returns false while the draw shader is still compiling, which happens on the first frames of every window), and batch order is deterministic, so a defect reproduces instead of flickering. Verified: tools/test-cad-coverage.sh green -- batching.rs 100%, cull.rs 100%, total 97.30%, all floors met; cargo check --locked -p nigig-build --lib clean at the 127-warning baseline; cargo test --lib 1100 passed (1091 + 9 new); --test cad_integration 154 passed; CAD_BENCH=1 harness green; cargo fmt --check and git diff --check clean.	2026-08-21 04:41:48 +00:00
andodeki	1e5d014198	perf(cad): one GPU buffer per shape, not per part -- Phase 2 of the render plan ... Two hundred identical columns held two hundred GPU vertex buffers. They build byte-identical triangles -- `CadSolid::build_mesh` is a pure function of the solid payload and returns local-space geometry, and the upload path (`part_mesh_buffers_from_mesh` -> `MeshSpace::Model`) carries no transform and no colour -- so the duplication bought nothing but memory and upload time. `part_geoms` is now `HashMap<ShapeHash, Geometry>`. **ShapeHash**, new in cad_scene.rs: the solid payload and nothing else. No node id, no transform, no material. `ParamHash` is now *defined in terms of it* -- `id` then `ShapeHash` -- rather than repeating the match arm by arm. That is deliberate: last commit corrected a plan written on the assumption that equal `ParamHash` meant "same shape", which the leading `node.id.hash()` quietly made false. With one shared body the two hashes cannot drift apart again, and `param_hash_moves_whenever_the_shape_hash_moves` pins the join. Three consequences worth naming: - **Staleness became structural.** The key IS the content hash, so the draw loop's `.filter(|(hash, _)| *hash == ParamHash::from_node(part))` is gone: an edited part looks up a key that does not exist yet and `ensure_part_geometry` uploads it on the same frame. There is no "entry uploaded from different parameters" state left to guard against, which is the class of bug `subdivide_selected` shipped. - **Eviction is by live shape, not live id.** This is the hazard the change introduces and it is not obvious: deleting one of two hundred identical columns must NOT drop the buffer the other 199 draw from. An eviction written as "remove the deleted node's entry" would blank most of the model. `geometry_is_retained_by_live_shape_not_by_live_id` pins it. - **Uploads deduplicate within the frame.** Without the `queued` set, the first frame of a 200-column scene would call `get_or_build` two hundred times before the map had anything in it. Measured (`bench_geometry_buffers_shared_by_shape`, counts not timings): 200 identical walls 200 parts -> 1 buffer (200x) 420-part repetitive model 420 parts -> 6 buffers (70x) 420 all-distinct parts 420 parts -> 420 buffers (1x) The last row is in the table on purpose. Sharing is a property of the model, not of the code; a scene where every part differs gets nothing from this phase. And draw calls are unchanged -- still one per visible part -- exactly as the plan predicted. Collapsing those is Phase 3, which needed these shared buffers to be possible at all. Estimated at a week, took under a day. Two things the estimate did not know: `MeshCache::get_or_build` was already a pure function of `node.solid`, and the upload path already produced transform-free, colour-free buffers. It assumed both would need untangling; they were built right the first time. Verified: tools/test-cad-coverage.sh green -- total 97.27%, cad_scene.rs 98.56%, cull.rs 100%, all floors met; cargo check --locked -p nigig-build --lib clean (and one warning fewer: the `ParamHash` re-export in mod.rs is no longer needed by the widget layer); cargo test --lib 1091 passed (1085 + 6 new); --test cad_integration 154 passed; CAD_BENCH=1 harness green; cargo fmt --check and git diff --check clean.	2026-08-21 04:18:33 +00:00
andodeki	4aaebafe4e	perf(cad): cull parts against the viewport -- Phase 1 of the render plan ... Before this, `grep -niE "cull|frustum|offscreen|in_view"` over the 2,438 line renderer returned nothing. Every part in the document was tessellated in 2D and issued its own draw call in 3D, on screen or not. At a 5 m drafting zoom over a 200 m site, a 2,000-part model spent 2,000 tessellations and 2,001 draw calls per frame to show twelve parts. New `cull.rs` -- pure, host-testable, 100% covered with a floor in tools/test-cad-coverage.sh: draw_part_2d(centre, half_w, half_h, view, decorated) -> bool Frustum::from_view_projection(view, projection) -> Frustum Frustum::draw_part_3d(aabb, decorated) -> bool world_aabb_from_local_bounds(min, max, model) -> WorldAabb It lives in its own module for the same reason nav_pad.rs and render_budget.rs do: viewport_render.rs is 2,438 lines at 0.00% coverage and needs a Cx, so a predicate written inline there could not be tested at all. Both draw loops AND CadViewport::frame_budget call these same functions, which is what stops the reported budget from drifting away from what is drawn -- the discipline render_budget.rs already applies to the grid loops. Measured (bench_frame_submission_budget, 1920x1080, parts on a 200 m site; full tables in BENCH_BASELINE.md): 2D, 5 m zoom, 2,000 parts: 12 visible. 2,170 -> 182 tessellations, 2,001 -> 13 draw calls. 3D, 20 m camera, 2,000: 733 draw calls, 63% culled. 3D, 80 m camera, 2,000: 1,459 draw calls, 27% culled. Whole site on screen: nothing culled, and nothing should be. That last row is the honest half and it is in the doc too: when the view holds the whole model, all 2,000 parts are genuinely visible and culling cannot help. Phases 2 and 3 (shape-shared geometry, then instancing) are what address that case. Three things came out different from the plan I wrote last commit, and the plan was wrong about each: - **AABB, not bounding sphere.** A sphere around an AABB is looser at the same cost -- a 6 m wall gets a 3 m radius ball. The p-vertex AABB-versus-plane test is strictly tighter. So there is no bounding_sphere helper, and in particular no bounding_sphere FIELD on CadNode: the AABB is already cached by PlacedHash in SceneCache, which invalidates itself on a move or a resize, where a stored field would have to be maintained at 77 construction sites. - **world_aabb_from_local_bounds is shared with pick_part**, replacing the eight-corner transform that was inline there. Two copies would be two chances to disagree about a part's bounds, and picking a part the renderer culled is precisely the bug that disagreement produces. - **The "drawn extent" hazard split into two concrete rules.** The 2D test takes part_to_plane_2d and part_size_on_plane -- the very values the draw uses -- so it tests the drawn rect, not the model extent. And no selected or hovered part is ever culled: the highlight and the tooltip are drawn at the cursor, arbitrarily far from the part. The frustum comes from Gribb-Hartmann row arithmetic on projection * view, deliberately not from mat4_inverse -- that function had a mistyped index in all sixteen cofactors until 2ea5a74, and the terms that were wrong are exactly the ones a perspective matrix makes non-zero. Makepad's Mat4f::perspective is the OpenGL convention (xr/src/scene/xr_root.rs builds the camera with it), so -w <= x,y,z <= w is right; a 0..w depth projection would only make the near plane more permissive, which is the safe direction. Both predicates are conservative by construction: NaN coordinates, NaN AABBs, negative extents and an identity camera all fall through to "draw it". A part kept but invisible costs one submission; a part culled but visible is a bug the user sees. One test of mine was wrong before the code was: the first draft asserted a part 40 m off axis was invisible from a 50 m camera at 60 deg on 16:9. tan(30 deg) * 16/9 is 1.03, so the horizontal half-angle reaches past 45 deg and the part is on screen. The frustum was right; the test is now narrow-fov and says so in its docstring. Verified: tools/test-cad-coverage.sh green -- cull.rs 100.00%, total 97.26% (was 97.20%), all floors met; cargo check --locked -p nigig-build --lib clean; cargo test --locked -p nigig-build --lib 1085 passed (1062 + 23 new); --test cad_integration 154 passed; cargo fmt --check clean; git diff --check clean. Not done here: the two hard-coded 1.2 margins in viewport_render.rs (397-400, 1374-1377) still do not read render_budget::VIEW_MARGIN. cull.rs does, so cull and grid agree by the constant rather than by the code. Folded into Phase 4, which rewrites those loops anyway.	2026-08-20 21:45:45 +00:00
andodeki	4d681dfdbe	feat(spreadsheet): slider cells and inline-Markdown cells (#9 remainder) ... Complete the widget-in-cell gap the same value/style-convention way as the checkbox: two more cell kinds, both plain data with a render flag. Engine: - CellStyle gains `markdown` and `slider` bool flags, serialized as two trailing columns on the CELL line (older files default them off), plus WorkbookCommand::{SetMarkdown, SetSlider} routed through apply and apply_command via mutate_cell — undoable like every style mutation. UI: - markdown.rs (measured): a flat inline parser splitting a cell value into Regular/Bold/Italic/Code runs (`**bold**`, `*italic*`, `` `code` ``); unclosed markers and empty spans stay literal/dropped. The grid draws each run with the bold or regular resource (code tinted like formulas). - slider.rs (measured): 0-100 fraction/value mapping (rounded to whole steps) and track/fill/handle geometry. A slider cell draws the control instead of text, and a press/drag sets the value through set_cell — one undo step per drag (reverse-order ChangeSet application restores the pre-drag value). - Toolbar "Md" and "Slider" buttons toggle the flags on the selection; the render cache carries the two flags so cached cells stay styled. Dropdown and button cells are intentionally not included: a dropdown needs a per-cell choices list and a button needs an action semantic that a spreadsheet does not have — both would require a CellKind in the data model rather than a render flag. Engine: 481 lib tests (+2) + integration. UI controllers: 133 tests (+13, markdown 10 + slider 3). Coverage: engine 96.62%, ui-controllers 99.42% (floors 96); markdown.rs and slider.rs at 100%.	2026-08-20 21:36:21 +00:00
andodeki	42b1e7152a	docs(cad): merge a second render review into the plan, and correct the hash both reviews got wrong ... A second optimisation plan arrived (frustum culling, geometry merging, GPU instancing, octree/LOD). Rather than run two plans, every claim in it was checked against the tree at 14aa0d5 and merged into REVIEWS/CAD_RENDER_OPTIMISATION_PLAN.md with a verdict and the evidence behind it. Three of its ideas change what the phases will do; two would have made things worse; one of them found nothing, but reading it found something in mine. The correction that matters, and it lands on my own plan: ParamHash::from_node hashes node.id FIRST (cad_scene.rs:1871), before a single geometric parameter. My Phase 2 said "parts with equal ParamHash are geometrically identical by definition -- that is what the hash means". True and vacuous: equal ParamHash also means the same node. Two identical columns hash differently, so re-keying part_geoms by ParamHash -- the central move of that phase -- would have shared exactly nothing. The incoming plan inherited the same assumption from the same type name, which is the kind of coincidence that argues for a test rather than a paragraph, so there is now one: param_hash_is_not_a_shape_key_it_includes_the_node_id. The id is not a bug. MeshCache and part_geoms are both keyed by NodeId and only ask "is this entry still valid for this node", where the id is a constant. Sharing geometry needs a second hash, ShapeHash: the same body without the id. Phase 2 now carries that instead. Adopted from the incoming plan: - Frustum culling in 3D. SceneState3D { view, projection } is already captured into self.last_view/last_proj every frame (viewport.rs:3781) immediately before draw_scene, so the planes are a row add/subtract away -- and deliberately not via mat4_inverse, which had a mistyped index in all sixteen cofactors until 2ea5a74. My Phase 1 had only an AABB-versus-rect test, which is right for 2D and wrong for perspective. - Instancing, upgraded from "confirm the machinery exists" to "here it is". DrawPbr::begin_many_instances_for_mesh / push_many_instance_with_transform / end_many_instances (draw/src/shader/draw_pbr.rs:2696-2745 in the pinned fork) do exactly this for a shared mesh geometry, and per-instance data is just the #[live] fields after #[deref] draw_vars -- the layout DrawCadMesh already has (color, transform, depth_clip, display_mode). No shader rewrite; ~30 lines ported. The incoming plan rated it 7 days and high risk with "if supported"; it is neither. DrawCadMesh is also declared alpha_blend: false, so regrouping draw order is safe here. - Its dependency graph: instancing batches per geometry, so shared geometry genuinely gates it. Rejected, with reasons in the doc: - bounding_sphere as a stored field on CadNode. 77 construction sites, and this codebase has already paid for hand-maintained derived state once -- the part_geoms staleness bug (viewport.rs:4629) drew the pre-edit shape because one edit site forgot to invalidate. The sphere is two lines from SceneCache::world_aabb_for, which is keyed by PlacedHash so it invalidates itself on a move or a resize, and is benchmarked at 4.72x. - Merged vertex buffers per PartKind with transforms baked in. MeshCache stores local-space meshes on purpose ("the transform and the material are NOT hashed"); baking transforms inverts that and turns dragging one column into a full re-concatenate and re-upload of every column, every frame of the drag. It also cannot express per-part selection colour. That phase's own draw loop still sets a transform per part, which is still N draw calls -- the two halves contradict each other. - Octree above 500 parts. The cached world-AABB pass is 20 us at 500 parts, 0.12% of a 16.7 ms frame. A tree that replaces 20 us cannot pay for itself. Revisit on a measurement, not a part count. - "Log skipped-part count per frame" as verification. Phase 0 already built the counting seam; a log line nobody reads is a step that cannot fail. Also corrected REVIEWS/CAD_DRAWCALL_STRATEGY_ANALYSIS.md, which is where the ParamHash claim originated: section 0 is now two corrections rather than one. Verified: tools/test-cad-coverage.sh green, total 97.20%, all floors met, cad_scene.rs 98.53%; rustfmt clean; git diff --check clean.	2026-08-20 21:17:07 +00:00
andodeki	14aa0d5017	perf(cad): measure what a frame submits — Phase 0 of the render plan ... Phase 0 of REVIEWS/CAD_RENDER_OPTIMISATION_PLAN.md. No optimisation here; this is the measurement everything after it depends on. 0a first: Makepad does not count draw calls. `Cx::performance_stats` is `PerformanceStats { last_frame_time, max_frame_times }` — frame times only. So the seam had to be built. `render_budget.rs` reports two costs, deliberately not summed: tessellations CPU calls into tessellate_path_stroke draw calls GPU submissions Measured at 1920x1080, now in BENCH_BASELINE.md: zoomed in (0.2 m grid, 170 lines) 2000 parts -> 2170 tess, 2001 draw calls zoomed out (5 m grid, 270 lines) 2000 parts -> 2270 tess, 2001 draw calls The grid column barely moves across a 40x zoom range because the step adapts — that half is already virtualised. The parts column is every part, every frame, on screen or not. **This is not a model of the renderer.** The obvious way to count submissions is to write a second copy of the loop structure and count what it would do, which is exactly how the scene-cache benchmarks ended up timing a function that cannot cache. So `grid_range` owns the decision and `draw_2d_vector_scene` now drives its loops from it: the count and the drawing come from one function and cannot disagree. The ~25 lines of nice-number step arithmetic that were inline in the renderer now live in the pure module, with tests. 11 tests, 100% of the new module, and two of them are there to pin things people get wrong: - the 2D scene is ONE draw call regardless of part count, because DrawVector::end() submits the whole accumulation; - 3D is one draw call per uploaded part, which is where they multiply. One test — culling_shows_up_as_fewer_part_tessellations — asserts the *shape* of Phase 1's improvement before the work starts: parts fall proportionally, the grid column does not move. Verified: 1062 lib tests pass, cad_integration 154, cargo fmt clean, engine coverage 97.20% with every floor met including render_budget at 100%. The benchmark runs host-only under CAD_BENCH=1.	2026-08-20 21:01:52 +00:00
andodeki	8c1a4ad446	feat(spreadsheet-ui): Big Data virtual tab — 1B cells, procedural, timed sort ... Close the UI half of the virtualization gap. A new VirtualGrid widget renders one billion virtual cells (1,000,000 rows × 1,000 columns) from the engine's VirtualSheet: only the visible range is drawn, values derive from a hashed (row, col), and a column-header click sorts the full million rows by permuting an index — with the elapsed time reported in a status strip, as the reference does. - virtual_grid.rs (DSL widget, excluded from coverage like grid.rs): zebra-striped cells, row/column headers with the sort glyph, gridlines, drag-to-pan and wheel scroll, and a 3-state header sort (asc → desc → off) delegated to VirtualSheet::sort_rows/reset_sort. All placement and hit-testing reuses the measured GridMetrics; text widths reuse TextMeasureCache. - workspace.rs: a "BigData" toolbar button overlays the virtual grid over the spreadsheet grid via two child Views toggled with View::set_visible, so the existing spreadsheet path is untouched. Engine: 479 lib tests + integration (grid data-provider commit included). UI controllers: 119 tests. Coverage: engine 96.67% (data_source.rs 100%), ui-controllers 99.35% (floors 96).	2026-08-20 20:48:35 +00:00
andodeki	f14823a6da	feat(spreadsheet): grid data-provider abstraction and a virtual 1B-cell source ... Close the engine half of the virtualization gap (report section A). Add a GridDataSource trait — row/col counts, per-cell display text, sort, and a column label — implemented by SpreadsheetData (the real store, delegating to get_display_value/sort_rows) and by a new VirtualSheet. VirtualSheet derives every cell procedurally from a hashed (row, col), so 1,000,000 rows × 1,000 columns (one billion cells) cost one struct and no stored data. Its rendered text and numeric sort key come from the same hash, so sorting by the key sorts what the user sees; sort_rows permutes a Vec<u32> row index (O(n log n) over the index, not the data) with a deterministic tie-break. Column labels mirror the reference (# / Name / City / Balance / Score / Active, then ·N repeats). Engine: 478 lib tests (+12) + integration. Coverage: data_source.rs 100%, engine total 96.67% (floor 96).	2026-08-20 20:48:35 +00:00
andodeki	7946aa889f	feat(spreadsheet-ui): OS-clipboard TSV copy and toolbar zoom (#13, #7) ... Close the two remaining small UI gaps from the datagrid analysis: - TSV copy to the OS clipboard (#13): clipboard.rs gains a tested tsv_escape/rows_to_tsv pair (tab/newline/quote escaping, Excel-style), and the grid answers Hit::TextCopy with the selection as TSV while Ctrl+C also writes it straight to the OS clipboard via Cx::copy_to_clipboard. The in-app clipboard still feeds Ctrl+V paste. - Zoom (#7): a new measured zoom module holds the step factor (×1.15), 50-400% clamps and the minimum/maximum cell-size clamp. The grid's apply_zoom/reset_zoom rescale the default cell size deterministically from a captured base (no floating-point drift), and the cell text's font_scale follows the zoom — with text widths scaled at draw time so right/centre alignment stays true. The toolbar gains - / 100% / + buttons. Engine untouched. UI controllers: 119 tests (+7). Coverage: ui-controllers 99.35% (floor 96), clipboard.rs and zoom.rs at 100%.	2026-08-20 20:14:51 +00:00
andodeki	2adec957e9	feat(spreadsheet-ui): TrendChart line + candlesticks wired to the selected row ... Close gap #11 (chart integration) from the datagrid gap analysis, split as usual into testable headless logic plus a thin DSL widget: - chart.rs (measured): candle bucketing (OHLC from a price series), series bounds, line-point mapping with the reference's 8% padding, the vertical strips that trace a polyline with axis-aligned quads, candle body/wick geometry with up/down classification, and the 1/2/5×10^k axis tick step. - market.rs (measured): a deterministic live market — the reference's splitmix-style `mix64`, a HISTORY-capped random walk per symbol, lazy per-row symbol growth (so any selected row charts), tick() with roll-off, and the derived stats (last/change/pct_change/day_range/candles). - trend_chart.rs: a `TrendChart` widget (excluded from coverage like grid.rs) that colours and draws chart.rs output — gridlines, a polyline for a series, or candle bodies + wicks — via `set_series`/`set_candles`. - workspace.rs glue: a 220px chart panel under the grid (line + candlesticks side by side) fed from a 0.25s `Timer` ticker; selecting a grid row switches the charted symbol and updates the title label. Engine untouched. UI controllers: 112 tests (+16 chart/market). Coverage: ui-controllers 99.32% (floor 96), chart.rs 100%, market.rs 99.36%.	2026-08-20 19:02:02 +00:00
andodeki	1a36ca5538	feat(spreadsheet-ui): interactive checkbox cells (widget-in-cell) ... Close gap #9 (cells hosting widgets) the spreadsheet-native way. The datagrid reference hosts a live CheckBox widget per visible cell and recycles instances from a per-template pool; our grid instead draws the control from batched quads, so there is no per-cell widget object to instantiate or recycle — the render cache and reused draw buffers already fill the role the pool does. - A plain value cell holding TRUE/FALSE now renders as a checkbox: a centred square box with a checkmark when TRUE, and its TRUE/FALSE label to the right. A single click toggles the value (through the normal set_cell path, so it is undoable and recalculates dependents), selects the cell, and does not open the editor — a checkbox is a button, not a text surface. Formula cells are never checkboxes, so a click can't clobber a formula. - The predicate (is_checkbox), the value flip (toggled) and the box geometry (checkbox_layout) live in a new measured checkbox module with unit tests; grid.rs stays thin glue (draw the box/tick/label, reposition the label, and the click handler's toggle). UI controllers: 96 tests (+6). Coverage: ui-controllers 99.20% (floor 96), checkbox.rs at 100%. No engine changes.	2026-08-20 18:28:46 +00:00
andodeki	1a4b479013	feat(spreadsheet): SPARKLINE(range) formula rendered as in-cell sparkline bars ... Close gap #10 (sparkline cells) from the datagrid gap analysis, split as usual into testable headless logic plus thin grid glue: - Engine: a `SPARKLINE(range)` function resolves its single range argument to a flat numeric series and returns a new `Value::Sparkline(Vec<f64>)`. The variant is not a scalar — arithmetic on it is a `#VALUE:` error, and the aggregation/lookup helpers skip it — so no code path invents a number from a chart. `SpreadsheetData::apply_formula_result` stores the series on `CellData.sparkline` (derived state, like spills: never serialized, re-derived on recalculation) and clears it when the formula stops returning a sparkline. Dependency tracking is inherited from the range reference, so editing a cell in the source range re-derives the bars. - UI: a new measured `sparkline` module computes the bar rectangles (bars rise from the series minimum, tinted up/down by last-vs-first trend, gap-shrunk for narrow cells) — the same geometry as the reference `Sparkline` widget, but testable headlessly. The grid draws the bars with a dedicated `draw_spark` resource (depth 0.4) when a cell carries a sparkline and skips the text path; the series is cached in `CellRenderState` alongside the display text. The in-cell editor keeps its text on top by suppressing the bars while editing. Engine: 467 lib tests + integration (483 total, +6 sparkline tests). UI controllers: 90 tests (+5). Coverage: engine 96.60%, ui-controllers 99.15% (floors 96); sparkline.rs at 100%.	2026-08-20 18:17:27 +00:00
andodeki	e13aa03dca	feat(spreadsheet-ui): header sort, column reorder and row/col/all selection ... Wire up the three missing header interactions from the datagrid gap analysis, split as usual into testable headless logic plus thin grid glue: - Selection kinds: SelectionController gains GridSelectKind (Cells/Rows/Cols/All). Header clicks select a whole column / row, the corner (or Ctrl+A) selects the sheet, and bounds() expands the row/column/all kinds to the sheet's full extent so the fill, copy, delete, format and autofill paths handle every kind without branching. - Header-click sort: clicking a column header toggles the sort direction (same column flips asc/desc, a new column starts ascending) and re-sorts the active sheet; the sorted header shows an ▲/▼ glyph. The toggle lives in the new measured sort_state module; the grid holds the (col, asc) state and the workspace applies Workbook::sort_active_sheet. - Column reorder: dragging a header past the tap threshold moves the column, with a drop-indicator line at the insertion index (computed by the new, tested GridMetrics::col_insert_at). Engine addition SpreadsheetData::move_column(from, to) remaps cells and column-width overrides, rebuilds the dependency graph, recalculates, and clears undo history; Workbook::move_active_column propagates to cross-sheet readers via the new WorkbookCommand::MoveColumn. Selection overlay drawing is clipped to the cell area so a full row/column/sheet selection border no longer paints over the headers and the surrounding workspace. Engine: 461 lib tests + integration (477 total). UI controllers: 85 tests. Coverage: engine 96.66% and ui-controllers 99.11% (floors 96).	2026-08-20 17:54:54 +00:00
andodeki	61d666699b	feat(spreadsheet): sort rows — the engine half of the datagrid sort ... Tranche 3 of the gap analysis: the datagrid example sorts a million rows on a header click. This is the engine-side sort that makes that possible (the UI header wiring and the procedural data provider are later tranches). `SpreadsheetData::sort_rows(ascending, key_col)`: - Sorts every row by the values in one column. Numbers sort before text (case-insensitively), and empty cells always sort last — in both directions, unlike a naive `reverse` which would float blanks to the top on a descending sort. - Stable: equal keys keep their original order via a row-index tie-break. - Formulas move with their rows and recalculate against the sorted positions, matching Excel's reference-by-position semantics. - Undo history is cleared (a sort is a destructive bulk reorder); row heights, column widths and named ranges stay positional, as in Excel. `Workbook::sort_active_sheet(ascending, key_col)`: - Sorts the active sheet and recalculates cross-sheet dependents so readers on other sheets see the sorted values. Tests: 6 (ascending/descending, numbers-vs-text-vs-empty ordering, stable ties, formula recalculation, non-undoability, workbook-level sort with cross-sheet propagation). Engine unit tests 447 -> 453. Engine coverage 96.61% (floor 96).	2026-08-20 17:18:33 +00:00
andodeki	481a0e133c	feat(spreadsheet-ui): red error cells and a visible-cell status bar ... Tranche 2 of the datagrid gap analysis — the small UI polish. (Zebra stripes were already present as `cell_alt_bg_color` alternating rows.) Error cells render red: - `render_cache::is_error_text` matches a display value against the engine's own `FormulaError::from_display` surface (plus `#SPILL!`), so real errors are red while a user value like `#hashtag` stays plain text. - The grid's text-colour precedence now checks the error case after an explicit user text colour and before bold/formula/default, using a new `error_text_color` (default red). Visible-cell status bar: - `GridMetrics::visible_cell_count(viewport_w, viewport_h)` estimates the visible columns/rows from the viewport and default cell sizes, clamped to the grid extent — testable in geometry.rs. - `SpreadsheetGrid::visible_cell_counts` hands the live viewport to it, and the workspace status label now shows "Ready | C × R visible = N cells", cached so it only re-lays-out when the numbers change (scroll or resize). Tests: 2 new (error detection against real/plain values; visible-count estimation + clamping). UI lib tests 73 -> 75; ui-controllers coverage 99.02% (floor 96).	2026-08-20 17:10:57 +00:00
andodeki	0326da8dfe	feat(spreadsheet): ^ power operator and General thousands separators ... Tranche 1 of the gap analysis against the Makepad `work`-branch datagrid example: the two formula-engine features it has that we lacked. `^` exponentiation operator: - New `Caret` token, `BinOp::Pow` (precedence 5), and a right-associative `parse_pow` between multiplicative and unary in the parser. - Unary minus binds tighter than `^`, so `-3^2` is `(-3)^2 = 9` — Excel's precedence, and the exact assertion the datagrid reference pins. - `2*3^2` = 18 (power over multiply) and `2^3^2` = 512 (right-assoc). - Evaluated in `apply_binop`, so it broadcasts element-wise over arrays like every other operator. General-format thousands separators: - `format_number` stays comma-free — computed values and criteria must round-trip through `parse_cell_computed_value` and `parse::<f64>`. - New `format_number_display` (plus a `group_thousands` helper) groups the integer part at the display boundary only: `apply_number_format`'s General arm now shows `1,000,000` while the raw/edit value stays `1000000`. Non-numeric text passes through untouched. Tests: `^` precedence/associativity/evaluation, display grouping, and an end-to-end check that comma display does not break the recalc fast path. Engine unit tests 441 -> 447; UI lib tests still pass. Engine coverage 96.58% (floor 96).	2026-08-20 17:01:46 +00:00
andodeki	a82c8f7ff7	feat(pdf): Unicode-aware search and layout-aware reading order ... Phase 8 bullets one and two. Probing the existing code first, as the workflow requires, found five defects rather than the one the plan names: SPLIT MATCH 'Hello': 0 hits plain_text: "Hello" PRECOMPOSED 'café': 0 hits COLUMNS plain_text: "LeftTopRightTop\nLeftBotRightBot" OUT OF ORDER plain_text: "second\nfirst" `PageText::find` searched one run at a time and documented that as a known limitation. It is a limitation from inside the code and a broken feature from outside it: a writer starts a new run wherever it adjusts kerning, so an ordinary word arrives as two runs, and the find bar says a word plainly visible on the page is not there. `search.rs` indexes the page as one flattened string with a map back to (run, character), so a cross-run match is found and highlighted with one rectangle per run — never a merged box, which across a line break covers half the paragraph. The separator between two runs is a geometric question with three answers: abutting runs join with nothing (one word, split by kerning), separated runs with a space, and a different line or column with a newline. The newline matters as much as the empty join: joining lines with a space lets "one Right" match across a column gutter, text that appears nowhere. Whether two runs share a column is *asked* of the layout analysis rather than re-derived, or the extracted text and the searched text disagree about where a column ends — the original defect wearing a different hat. NFD, never NFC: composition needs the next character, so an NFC fold applied per character composes nothing and the two spellings of an accent stay different. That was a real bug in the first draft. And case *folding*, not lowercasing — Rust lowercases ß to ß, so "Strasse" never found "Straße". Columns are detected before lines, because two columns share their baselines; that is what makes them columns. Bands are separated by a gutter rather than by bare non-overlap, since two abutting runs on a line do not overlap either. Also fixed, found by running the gates rather than by looking: a stream reader trimmed a trailing CR before `endstream` as if it were the writer's separator. Binary data ends in CR about one time in 256, and when it did the reader returned a stream one byte short — no longer AES-block-aligned, so decryption produced garbage and Flate failed. Roughly one encrypted document in 250 was silently corrupt on read. The test failed once under coverage, passed five times in isolation, and failed 2 in 40 when actually counted. A /Length consistent with the file is now the authority; both stream readers are fixed and a test reads one file through each. 1477 tests pass (was 1426), coverage 88.37%, all floors met, external readers pass. 10 mutations across the two modules, all killed. ADR 0034.	2026-08-19 16:12:12 +00:00
andodeki	555c7daaa1	feat(spreadsheet): postfix LAMBDA application and LET ... Completes the first-class-function story started with LAMBDA-as-an- argument. Postfix application: - A new `Expr::Apply { func, args }` node lets a callable expression be invoked directly: `LAMBDA(x, x*2)(3)` and curried `LAMBDA(x, LAMBDA(y, x+y))(1)(2)`. The parser gained a `parse_postfix` loop, so `f(...)(...)` chains bind tightest, after the primary expression. - Arguments are bound with `resolve_bound_arg`: a range or array binds as an array (so `LAMBDA(x, SUM(x))(A1:A4)` aggregates), a scalar binds as a scalar. - Applying a non-callable is `#VALUE!`; an argument-count mismatch is reported. Postfix arguments still register their cell dependencies. LET: - `LET(name1, value1, [name2, value2, ...], body)` binds names to values sequentially — a later value may reference an earlier name — and evaluates the body with the names in scope, reusing the LAMBDA substitution machinery. A range value binds as an array, so `LET(s, A1:A4, SUM(s))` aggregates the whole range. - Duplicate names are rejected in both LET and LAMBDA, matching Excel; an unbound name in the body is `#NAME?`. Tests: 6 unit tests (postfix application, currying, LET binding/range/ error shapes, duplicate-parameter rejection, dependency tracking) + an end-to-end test proving LET and postfix application recalculate through the dependency graph. Engine unit tests 434 -> 441. Engine coverage 96.55% (floor 96).	2026-08-19 12:05:26 +00:00
andodeki	47f645ab58	feat(spreadsheet): resolve the four dynamic-array limits ... Closes the four limits documented when the spill model landed. LAMBDA (was: GROUPBY/PIVOTBY only took an aggregate name): - `LAMBDA(params..., body)` builds a `Value::Callable` without evaluating its body. A new `Expr::BoundValue` node splices a bound argument into the body AST when the callable is applied, so the ordinary evaluator runs the body. - `GROUPBY` and `PIVOTBY` accept either a named aggregate (`SUM`, ...) or a `LAMBDA`, applied per value column / per pivot bucket. This unlocks arbitrary aggregations (`LAMBDA(x, MAX(x)-MIN(x))`). FILTER include + full arithmetic broadcasting: - `FILTER` now accepts a same-shape include: matching cells are kept and non-matching positions become `#N/A`, element-wise, like Excel. - Unary operators broadcast over an array (`-A1#`, `-FILTER(...)`), completing the operator-level arithmetic alongside the existing binary broadcast. Spill formatting: - `SpillRange` records the anchor's `NumberFormat`, and derived cells are formatted at display time through a shared `apply_number_format` (extracted from `write_display_value`). Raw values stay numeric, so `SUM(A1#)` still evaluates correctly. #SPILL! blocking: - A spill that would overwrite an existing cell — or another spill, flowing or blocked — reports `#SPILL!` in the anchor instead of clobbering data. The would-be range is remembered in `blocked_spills`, and clearing the blocking cell retries the spill automatically. Tests: 7 new unit tests (LAMBDA in GROUPBY/PIVOTBY, lambda errors, same-shape FILTER, unary broadcast) + 2 end-to-end tests (NumberFormat inheritance and #SPILL! block-then-retry). Engine unit tests 427 -> 434; UI lib tests still pass. Engine coverage 96.41% (floor 96).	2026-08-19 07:58:38 +00:00
andodeki	1740da3f34	feat(pdf): render a form XObject to pixels — the golden caught what the ... assertions missed `Rasteriser::register_xobject` takes a form's recorded commands from a caller that can resolve the page dictionary, so Phase 7's last golden-corpus criterion is met with pixels instead of with a request recorded by name. The first golden of that page showed the form drawn at the **page origin**, ignoring the `1 0 0 1 20 20 cm` that placed it. The recorded commands' `SetTransform`s are absolute in form space, and replaying them overwrote the page's CTM rather than composing with it. Nested lists now compose against the CTM in force at the `Do`. The colour assertions written next to that golden all passed while the bug was live — a red square two pixels from where it belongs is still a red square somewhere. That is the argument for pixel goldens in one sentence, and it is why the golden is compared after the assertions and not instead of them. The offset now has its own assertion too. The new fixture's form deliberately overflows its own /BBox, so the clip is visible in the golden as an absence rather than being taken on trust. Phase 7's golden-corpus exit criterion is now met in full. The `ui.rs` smoke tests remain blocked on the Makepad headless backend, as they have been since Phase 1, and are still not claimed as done. 1426 tests pass.	2026-08-18 20:06:50 +00:00
andodeki	d3089bc62a	feat(pdf): nested content, the wire codec and tiled rendering — Phase 7 closed ... Three bullets, and the Phase 7 status table rewritten row by row. **Nested content (ADR 0032).** A form XObject and a Type 3 glyph are the same problem: a content stream inside a content stream. Both were parsed completely and then not run. `paint_x_object` reported the name for "the host" to resolve and no host existed, so `Do` painted nothing. Type 3 was worse because it looked more correct — `d0`/`d1` reached the device, so the pen advanced by the declared width and the page rendered an invisible line of text with correct spacing after it. `nested.rs` runs both, in pdf-graphics because the dependency runs graphics → document and this is the only crate that can see the interpreter and the object model at once. Forms get their `/Matrix`, their `/BBox` clip and a save/restore wrapper, because without the wrapper a form's colour leaks into every object after it and looks like a bug in the document. Type 3 composes translate-then-matrix; the other order scales the translation and puts the glyph at (1.7, 16.8) instead of (72, 700). Recursion is bounded in both: unbounded, a self-referencing form is a stack overflow reachable from an untrusted document, which is a denial of service and not a rendering bug. **Wire codec and tiling (ADR 0033).** `worker.rs` moved interpretation off the UI thread only because both ends shared a Vec. Tags are explicit numbers, never declaration order, so reordering the enum cannot silently make old recordings decode as different commands. Truncation is an error rather than a short list — a decoder that stopped early would render a page missing its last few operations, plausible and wrong. The obvious truncation test failed, correctly: `Save` is one byte, so a cut on a command boundary really is a complete list. It now tries every cut position and requires each to be a named error or a genuine prefix. Tile skipping is conservative. A command whose geometry is unknown is kept, because dropping a state change corrupts everything after it in that tile, silently. Only untransformed geometry that provably falls outside is dropped. Every tile is asserted pixel-identical to that region of the whole-page render: tiling that is fast and different is not an optimisation. Eight mutations across the two modules, all killed. Phase 7 status is now two tables — the eight spec bullets and the exit criteria — with what is missing named in the row rather than rounded up. Three rows are not green: Makepad blend compositing needs render-to-texture, the image-XObject pixel golden asserts the request rather than pixels, and the `ui.rs` smoke tests remain blocked on the headless backend they have been blocked on since Phase 1. 1425 tests pass, coverage 88.10% (was 87.60%), all floors met, external readers pass. ADRs 0032 and 0033.	2026-08-18 20:00:18 +00:00
andodeki	f37197781e	feat(pdf): glyph outlines from TrueType and CFF, and glyph-aware text runs ... `sfnt.rs` read the metric tables and nothing else. It could say how wide a glyph was and not what shape it had, so every renderer drew embedded text with a substitute font at the correct advance — the failure mode that looks most like success: the line breaks land right and the letterforms belong to somebody else. `outline.rs` returns one outline type for both formats. TrueType quadratics are degree-elevated to cubics, which is exact, so no format detail leaks to a consumer. Composite glyphs are placed by their offsets and scales, with a depth bound because a font can reference itself. CFF Type 2 charstrings run through an interpreter with biased local and global subroutines, hints, hintmask byte counting, the leading width operand, and the FontMatrix as declared rather than assumed to be 1/1000. Separately, `ShowTextWithMetrics` carried one advance for a whole run — enough to move the pen to the next run and nothing else. So `text.rs` guessed: `seg.advance / char_count`. For "Wi" that puts the boundary between the letters at 5 when it is at 9, and every caret, drag-selection and search highlight in the application was wrong by that much for every proportional font. `GlyphPlacement` now carries per-glyph pen offsets, computed with the same expression as the run total so the two cannot drift. The even-spacing fallback stays for fonts with no width table, which is what `advance_is_measured` has always been for. The fixture story is ADR 0029's, again. `cff_sample.otf` is a fontTools conversion of DejaVu: no subroutines, no hints, no width operands. It proved the interpreter draws the right shapes, and then four mutations of that interpreter survived because nothing in the corpus reached the code they broke — each of which produces a plausible wrong glyph from a font that parses. `cff_subrs.cff` is hand-assembled for exactly those four, and fontTools agrees with every expectation asserted against it. A fifth mutation survived a composite test that counted contours; it is killed now by one that measures where the components land. Coordinates are asserted against fontTools ground truth, not against our own output. Seven mutations, all killed. 1397 tests pass. Deferred and recorded, not claimed: CID-keyed CFF, `seac` accents, rendering outlines through the Makepad device. ADR 0031.	2026-08-18 19:44:18 +00:00
andodeki	0bef30a6d5	feat(pdf): compositing and overprint — the blend maths had no backdrop ... `transparency.rs` implemented all sixteen blend modes and unit-tested them against the specification's formulas. Nothing ever called them with a backdrop. The Makepad renderer's `SetBlendMode` pushed a `TransparencyError::Unsupported` and then painted the source colour, so a /Multiply highlight and a /Normal one produced byte-identical output and every test passed — because every test asked "was the right command issued", not "does the page look right". Overprint had no code at all. /OP, /op and /OPM were not parsed, so an overprinting object knocked out the inks under it. That is not a missing feature, it is the inverse of the instruction: on a press it is the difference between a colour and a hole. - `composite.rs`: a straight-alpha RGBA `Canvas` implementing §11.3.6's union formula, weighted by backdrop alpha so a Multiply over transparency is the source rather than black. Constant alpha and per-pixel soft masks. Transparency groups composite as a unit; knockout groups are refused by name rather than silently treated as non-knockout. - Overprint as `composite_cmyk`, separate from the RGB path rather than a flag on it: overprint is a statement about inks and RGB has none. /op defaults to /OP per table 58 — defaulting it to false makes the common `<< /OP true >>` knock out every fill. §10.7.5's "no effect on an RGB device" is asserted, so our doing nothing there is the spec rather than an omission. - `raster.rs`: a CPU rasteriser that replays a command list onto a canvas. Not on the display path, no anti-aliasing, no fonts; it exists so compositing has a verifiable output. In pdf-graphics and not pdf-makepad because a test that needs a GPU is a test that does not run. - Golden **pixels** for shading, mesh, blend and overprint pages — Phase 7's exit criterion, which the Phase 2 command-text goldens cannot meet. ASCII grids with a colour legend, quantised to quarter steps; each test asserts its exact colours before comparing, so a wrong-but-stable render cannot be blessed by an UPDATE_GOLDEN run. Six mutations, all killed, including the two that describe the old behaviour: discarding the blend result, and ignoring the overprint flag. 1364 tests pass. ADR 0030.	2026-08-18 19:25:15 +00:00
andodeki	728fbc3ad0	fix(pdf): mesh shadings — three bugs in code that had no fixture ... ADR 0028 shipped types 4-7 and said honestly that they were unproven: the uncovered lines of `shading.rs` were exactly `parse_mesh`, "the position `image.rs` was in before ADR 0016 found the JPEG decoder was a stub". Writing the fixtures found three real bugs. - Type 5 has no per-vertex flag; `/VerticesPerRow` delimits it. Reading 8 phantom bits shifted every vertex after the first, decoding plausible coordinates that were entirely wrong. - Types 6 and 7 are patches: 12 or 16 control points carrying no colour, then four corner colours. The old loop read a colour per point, consumed three times too many components, ran off the stream, and the None-on-truncation path swallowed it as "the mesh ended". - A flag-0 triangle is three vertices whose second and third flags are ignored (§8.7.4.5.5). Acting on them cleared the strip every time and produced no triangles at all. Caught in new code, before it shipped. And one omission: `color_at_point` returned None for a mesh, so a mesh that parsed perfectly still painted nothing — indistinguishable from one that failed. `MeshTriangle::color_at` now interpolates the corner colours by barycentric coordinates, None outside, because black is a colour a mesh can legitimately produce. Shared-edge patches (flags 1-3) inherit the previous patch's edge rather than being read as fresh patches, which desynchronised the rest of the stream. Five corpus fixtures, generated from named coordinates and colours so every expected value in the tests is one the generator wrote deliberately. Eight tests, five mutations, all killed. Coons flattening is still an approximation and still reports `is_approximate`. ADR 0029.	2026-08-18 19:10:49 +00:00
andodeki	c9474e2c9f	feat(spreadsheet): dynamic-array spill model — FILTER, GROUPBY, PIVOTBY, A1# ... The engine stored every formula result as one display string per cell. This adds the dynamic-array architecture: a formula can return a grid, which "spills" into the cells below/right of its anchor. Value model: - `Value::Array(Vec<Vec<Value>>)` — a grid result. Scalars coerce to one-cell grids where the shape matters; `to_f64`/`to_bool` refuse an array, `to_display_string` shows the top-left, and `resolve_arg` flattens an array argument so `SUM(FILTER(...))` aggregates its cells. - Element-wise broadcasting: `resolve_array2d` maps a binary operator over a grid when one side is a scalar (or both grids share a shape), so `FILTER(A1:A4, C1:C4 > 15)` builds the boolean include the way Excel does. The binary-op logic was factored into `apply_binop`. Parser and AST: - `#` is now the spill operator: `A1#` and `Sheet2!A1#` parse as `SpillRef` / `SheetSpillRef`, evaluate to the anchored array (so `=A1#` re-spills), flatten in aggregates, and track their anchor in the dependency graph (intra- and cross-sheet). Functions (dynamic arrays): - FILTER(array, include, [if_empty]) — keep rows (column include) or columns (row include); `#N/A` on no match unless `if_empty`. - GROUPBY(row_fields, values, function, [field_headers]) — group rows by field tuples and aggregate each value column (SUM/AVERAGE/COUNT/ MAX/MIN/MEDIAN by name, ETA-reduced-LAMBDA form). - PIVOTBY(row_fields, col_fields, values, function) — a 2D pivot with the aggregate name in the top-left corner. Spill storage (data.rs): - `SpillRange` + `spills` map on `SpreadsheetData`: derived cells read back through `get_display_value`/`get_raw`/`get_edit_value`, are not blank, and are read-only — `set_cell`/`put_cell`/`remove_cell`/ `mutate_cell` refuse to touch them (the UI blocks via `is_spilled`). - Recalc builds the spill from the `Array` result (`apply_formula_result`) and drops stale spills when a formula becomes scalar, is removed, or cycles. Spills are derived state, never serialized — the anchor formula persists and re-derives on load. - Cross-sheet spills read through `get_sheet_spill_values`. Tests: 21 new units (FILTER/GROUPBY/PIVOTBY shapes, broadcasting, Value::Array methods, spill parsing) + 5 end-to-end tests (spill display, read-only cells, `A1#` aggregation and re-spill, stale-spill clearing, cross-sheet spill). Engine unit tests 405 -> 427; UI lib tests still pass. Engine coverage 96.66% (floor 96).	2026-08-18 18:34:22 +00:00
andodeki	a2b05c56c9	feat(makepad-table): opt-in capabilities feature, and raise the matrix_client defect ... The two caveats from the dependency investigation. ## The capabilities feature Camera and location attachments are now available behind `features = ["capabilities"]`, which pulls `nigig-uikit` and supplies `UikitAttachmentProvider`. Measured: 89 crates by default, 275 with the feature on. That cost is real and it is inherent, not packaging waste. `camera_widget` imports `send_geocode_request` and `request_map_tile` from `nigig-core`, both of which call `spawn_async` — the shared Tokio runtime — and the first makes an HTTPS call to Nominatim. A camera that geocodes needs an async runtime and an HTTP client; there is no lighter honest version. It is affordable because it is opt-in, and because any app enabling it already depends on `nigig-core`, so that app's own tree grows by nothing. Everything touching `nigig-uikit` is in one module, so the boundary is a file rather than `#[cfg]` scattered through the widget. The provider holds no widgets of its own: the host owns the `CameraWidget` already in its tree and this asks it to open, because a provider that instantiated a second camera would fight the first for the device. A second request while one is outstanding is refused rather than overwriting. The table turns that refusal into `AttachmentUnavailable`, so the user is told the camera is busy instead of watching their first request vanish. File picking is deliberately declined here — `robius-file-picker` already ships unconditionally and costs nothing, and two paths for one job is one too many. Two CI gates, both verified to fail when they should: the opt-in build must keep compiling, and the default build must pull none of `tokio`, `reqwest`, `hyper`, `clap`, `csv`, `image`, `nigig-uikit` or `nigig-core`. The second checks the resolved `cargo tree` rather than the manifest, because feature unification can switch an optional dependency on from a sibling crate. Tests 99 default, 105 with the feature. Both clippy-clean. ## The matrix_client defect Raised in REVIEWS/MATRIX_CLIENT_FEATURE_GATE.md rather than fixed. It is not my crate, nothing depends on the broken combination, and a blind fix could change behaviour someone relies on. `matrix_client` declares `native = ["dep:tokio", "dep:reqwest", "dep:rusqlite"]` but its source gates on `#[cfg(not(target_arch = "wasm32"))]`. Two switches for the same modules, so on a native target with the feature off the modules compile and their dependencies do not — 19 errors, 26 ungated uses across 7 files. There is no CI job for the crate, which is why it rotted unnoticed. The note corrects an overstatement I made while arguing for the trait hook. I said fixing this would unblock wasm. It would not: `matrix_client` already builds clean for wasm32 with `--no-default-features`, and `nigig-core` has 8 wasm errors of its own (`crate::platform::spawn` missing) that have nothing to do with it. The only broken combination is native-target-with-feature-off, which nothing builds. I also said earlier that `matrix_client` was heavy — it is a 7-dependency local crate, not matrix-sdk. That was wrong and it inflated the case for the trait hook; the note records the measured numbers instead.	2026-08-18 18:03:26 +00:00
andodeki	45efc74106	feat(spreadsheet): pivot and chart aggregates — IFS family, statistics, SUMPRODUCT, LARGE/SMALL/RANK ... Full GROUPBY/PIVOTBY need a dynamic-array "spill" model (a formula returning a grid), which this single-cell engine deliberately does not have. These are the single-cell building blocks that do the same work. Multi-criteria conditional aggregates (the "filter then aggregate" pivot core), resolved positionally so criteria ranges stay aligned: - SUMIFS(sum_range, criteria_range1, criteria1, ...) - AVERAGEIFS(avg_range, criteria_range1, criteria1, ...) (#DIV/0! on no match) - COUNTIFS(criteria_range1, criteria1, ...) - MAXIFS / MINIFS (0 on no match, like Excel) Mismatched range sizes are #VALUE!, not a silent misalignment. Chart statistics (over the flattened numeric arguments): - MEDIAN, MODE (ties keep the smallest value), and the sample/population STDEV / STDEVP / VAR / VARP. Sample forms divide by n-1 (#DIV/0! for a single value), population by n. Pivot/ranking helpers: - SUMPRODUCT(array1, [array2], ...) — the element-wise dot product; text counts as zero, errors propagate, mismatched sizes are #VALUE!. - LARGE / SMALL(array, k) — k-th largest/smallest; k out of range is the new #NUM! error. - RANK(value, array, [order]) — descending by default, ascending on any nonzero order, tied values share a rank (RANK.EQ). `FormulaError` gains `NumError` (`#NUM!`) for out-of-domain numeric arguments, rounding out the error surface after `Na` in the lookup tranche; it round-trips through display/parse and propagates from cached values. Tests: 10 unit tests (multi-criteria aggregation, statistics, dot products, top-N/ranking, argument/size errors) + an end-to-end test proving the dependency graph tracks every range and recalculates the pivot formulas when a source cell is edited. Engine unit tests 395 -> 405. Engine coverage 97.34% (floor 96).	2026-08-18 18:03:00 +00:00
andodeki	c1d1e67f3a	feat(pdf): shadings — the sh operator was parsed and thrown away ... ADR 0028, the first of Phase 7's eight bullets. content.rs contained `PdfOp::Shading(_name) => {}`. The operator was lexed, given its own variant, matched during interpretation, and discarded. A page whose background is a gradient rendered as nothing. Nothing caught it for the usual reason: a blank region is a legal thing for a page to contain, so "drew nothing" and "drew what was asked" are indistinguishable without an assertion naming the expected colour. The golden corpus had no shading page, so there was nothing to be wrong. Two of the three pieces already existed — function.rs evaluates the colour function and colorspace.rs converts it to RGB. What was missing was the geometry between them. Sampling rather than a gradient primitive: a PDF shading is defined by an arbitrary function, possibly a sampled table or a PostScript program, and neither reduces to a stop list without loss. A device with a native gradient can still recognise the two-stop case from the samples. "No colour here" is None, not black. Black is a colour a shading can legitimately produce, so returning it for "outside an unextended shading" would paint a rectangle the author never asked for and the caller could not tell the two apart. Types 1-5 exact. Coons and tensor patches are flattened to their corners, which loses the curvature, and is_approximate says so rather than leaving a caller to assume fidelity. An unknown type is refused by number: a mesh drawn as a flat fill is a plausible-looking wrong answer. paint_shading is a new trait method, so the compiler found every implementor. The Makepad renderer records the request in pending_shadings, mirroring pending_xobjects — it cannot resolve a /Shading resource because it does not own the page dictionary, and recording the request is what stops the operator vanishing a second time. That holds even for types we refuse, so a host can warn the user. Four mutations, all killed. The first — discarding sh again — fails three tests. Stated plainly and left unticked: the mesh path is written but NOT exercised by any real stream. shading.rs is at 68% and the uncovered part is exactly parse_mesh and triangulate. Mesh support should be treated as unproven, not working: the code runs and produces triangles, and nothing yet demonstrates they are the right triangles. That is the position image.rs was in before ADR 0016 found the JPEG decoder was a stub. The Phase 7 status line is a table from the start this time — one row per spec bullet, seven of them saying "not started". Per ADR 0021, written before the work rather than after it. pdf: 1321 passed (was 1291). pdf-ui: 1366. Coverage 87.60%, floors met.	2026-08-18 17:30:39 +00:00
andodeki	cb8912f762	test(pdf): close the two real coverage gaps in the signing module ... Asked to verify Phase 6 was complete *with test coverage*, I measured sign.rs per function rather than trusting the file-level 82%. Most of the apparent gap is error arms inside covered functions — llvm-cov attributes each `map_err` closure separately — but two things were genuinely untested, and one of them was not code that should exist. algorithm_name() was dead. It returned a &'static str describing the algorithm and nothing called it: `algorithm()` supersedes it, returns a type rather than a string, and is what the CMS writer actually uses. Deleted rather than tested, because a test would have preserved code whose only caller was the test. SigningError's Display impl was never exercised. These strings reach a user through a host application. ContentsTooSmall in particular must carry both numbers — a caller cannot raise the reservation without knowing by how much — and that is now driven through the real signing path with a 32-byte reservation rather than by constructing the error. sign.rs 82.07% -> 83.79%. pdf: 1291 passed. Coverage 88.03%, floors met.	2026-08-18 17:08:11 +00:00
andodeki	7737096858	refactor(makepad-table): adopt Robrix's image decode path ... Replaces the hand-rolled try-PNG-then-JPEG with `pageflipnav/src/utils.rs::load_png_or_jpg`, the pattern the Robrix-derived app in this repo already uses. Two things it does better: - It sniffs the header with `imghdr` and calls the matching loader directly, so a JPEG does not decode-and-fail as a PNG first on every cold cache. - It still falls back to trying both when the sniff names something unexpected or nothing at all. `imghdr` is not perfect, and a mislabelled file is more useful decoded than refused. `imghdr` has no transitive dependencies — it reads a header and names a format. It is already a dependency of `pageflipnav` at the same version. The upstream version logs the failure and dumps the bad bytes to disk. That is right for a chat client receiving untrusted media and wrong here: this runs from the draw path for every attached cell, so a broken file would log once per frame. The caller already caches the failure and draws a labelled chip naming the file, which tells the user more than a log line would. Tests 94 -> 99. Verified by removing the sniff and by removing the fallback; each fails the ordering test. `TextOrImage`, the other candidate for reuse, is referenced in `room_screen.rs` but not defined anywhere in this checkout — it is upstream Robrix only, so there was no baseline here to adopt.	2026-08-18 16:51:47 +00:00
andodeki	01ebeeb7fe	feat(makepad-table): real image rendering with a resize anchor, and per-row heights ... Two things: attached images are decoded and drawn rather than shown as a placeholder chip, and row heights become genuinely per-row. **Image rendering.** Decoding follows `pageflipnav/src/utils.rs`, the Robrix-derived app in this repo: try PNG, then JPEG, because a header sniff is not reliable enough to choose on its own. The decoded texture is cached per cell, and a cache entry of `None` records a file that could not be read so a broken path is attempted once rather than every frame — `draw_walk` runs at 60Hz and re-decoding a photo there would be the slowest thing in the widget by a wide margin. One `Image` widget repositioned per cell, matching `cell_editor` and `math_cell`, with the texture swapped from the cache. A pool would let several textures live at once but needs runtime template instantiation and a reuse policy; this is the same number of GPU uploads with far less machinery. On first successful decode the real pixel size is written back to the attachment, so the row is sized from the true aspect ratio instead of the placeholder guess. **The resize anchor.** A grab square at the image's bottom-right corner. Dragging it writes `ImageSizing::Fixed`, which pins the height so a later relayout cannot overrule what the user chose, and the row follows because `row_height_for` reads the same value. The floor is asserted at compile time against the anchor size: an image dragged smaller than its own grab handle could not be grabbed again, and the user would have to delete the attachment to recover it. The anchor is hit-tested before the cell, or dragging it would open the editor instead. Sizing lives on the attachment rather than in widget state, so it survives a column reorder along with the image. **Per-row heights.** `TableRow::height` holds a dragged override and `row_height_for` honours it. Item 5's row resize previously assigned `self.row_height`, which is table-wide — dragging one row's handle resized every row at once. `RowGeometry`, added with the attachments, now carries the consequence: rows below a resized one shift down. Tests 86 -> 94 (140 across the tree), all five crates clippy-clean. Verified by reintroducing four defects. One of those guards did not work first time and the gap was mine. Deleting the per-row override branch from `row_height_for` left the whole suite green: the tests checked that `TableRow::height` could be *stored*, and nothing checked it was ever *read*. Storing a value no one consults is exactly the shape of "the handle does nothing". `the_row_override_is_actually_consulted` now asserts the connection at both ends — that `row_height_for` reads `row.height`, and that the resize writes it rather than the table-wide field.	2026-08-18 15:19:36 +00:00
andodeki	3c751f18bd	feat(makepad-table): cell attachments and per-row heights (item 6) ... The last of the nine. Items 1-5, 7, 8 and 9 shipped in 1887b54 and bbdfe82. **Per-row heights.** Row positions were `index * row_height`, which is only correct while every row is the same. A cell holding an image is not, so `RowGeometry` now accumulates boundaries the way `ColumnGeometry` does, and hit-testing scans them instead of dividing — with unequal heights there is no divisor. Ten call sites moved over. **Attachments.** A long press on a cell opens its menu: copy, paste, clear, add image or PDF, take a photo, add location, remove. Cells stay `String`. An attachment is a side-table on `TableData` keyed by `(row, col)`, so a table without them costs nothing and still round-trips as text — every existing caller builds `TableData` from strings and none of them change. Those keys are positional, which is the part that bites: inserting a row or reordering a column has to move them too, or the data moves and the image stays behind. `shift_attachments_for_row_insert`, `_row_remove` and `_col_move` handle it, and `move_column` swaps rather than shifts because it is a swap. This is the class of bug that only appears once there is real content in the table, so it is tested directly rather than left to review. **Why a provider trait.** Camera and location live in `nigig-uikit`, which pulls `nigig-core` and with it tokio, reqwest, matrix-sdk, clap, image and csv. For a widget whose only dependency is `makepad-widgets`, that is the wrong trade. `CellAttachmentProvider` names the three capabilities and a host that already has them supplies them; the default implementation declines everything, and declining emits `AttachmentUnavailable` rather than leaving a menu entry that silently does nothing. File picking ships with the widget, because `robius-file-picker` is already a dependency of three crates here, works through `rfd` on desktop and the platform picker on Android, and carries none of that weight. Its callback runs off the UI thread, so results are parked in `PENDING_CELL_FILE` and drained on `Event::Signal`, the same shape the invoicer and the SMS bulk import already use. **What is drawn.** A labelled chip, not the image. Decoding a photo or rasterising a PDF page per frame belongs in a texture cache the host owns, and a rasteriser is not something this widget should carry. The chip reports what is attached and takes the space the row grew for it; a host wanting a thumbnail draws over the same rect. Tests 72 -> 86 (132 across the tree), all five crates clippy-clean. Verified by reintroducing four defects: attachments not following a column reorder, not following a row insert, row lookup dividing by a uniform height, and an image ignoring its aspect ratio.	2026-08-18 13:52:41 +00:00
andodeki	bbdfe823f8	feat(makepad-table): row gutter, header select/resize, long-press menus, one input model ... Items 1, 2, 3, 4, 5, 7 and 9 of the nine reported. Item 8 shipped separately in 1887b54; item 6 (cell attachments) is next and needs the trait hook agreed for camera and location. **One input model for mouse and touch (item 9).** The touch path was a parallel implementation with its own tracker, its own long-press timer and its own movement threshold, and the two had already drifted: a long press opened a menu under a finger and did nothing under a mouse. Both paths now share `PressTracker`, `handle_press_timers` and `dispatch_target_click`. Only gesture recognition differs, because a finger reports Start/Move/Stop with no tap count and no hover — double-tap is derived from the interval between taps, the way the CAD viewport derives its gestures from raw touches. A held press schedules its own frame with `new_next_frame`. Neither a mouse nor a finger emits events while held still, so a long press would otherwise only fire if the user happened to move. **Double-click to edit (item 3).** A single click now selects; the second click of a double-click edits. `DOUBLE_TAP_WINDOW` is deliberately shorter than `LONG_PRESS`, so a slow double tap cannot also register as a long press and both edit the cell and open the menu. **Long press opens header menus (item 4).** Previously a single click did, which left no gesture free for selection. **Header selection and resizing (item 5).** A click on a header outlines it and shows a grab handle — right edge for a column, bottom edge for a row. Dragging the handle resizes, with a full-length guide while the drag is live. The size is floored at `MIN_SIZE`, which is not cosmetic: a header dragged below the handle size cannot be grabbed again, so the column would be unrecoverable by dragging. **Row gutter (items 1 and 2).** A leading column numbers the rows and acts as the row header. It is not a data column — no entry in `columns`, so it cannot be reordered or dropped onto — and the numbers are derived from the index each frame rather than stored, so an insert or reorder cannot leave them stale. This replaces the old 16px handle strip, which sat inside the first data cell and stole clicks from it. **Every grid position is drawn (item 1).** The draw loop iterated each row's own `cells`, so a row shorter than the column list simply stopped: the remaining columns had no background, no border and nothing to click. It now iterates the column count and treats a missing entry as empty. The rightmost vertical divider is also drawn; the old range stopped one short and left the last column open. **Header renaming (item 7).** Reached from the header menu rather than bound directly to the long press, so one gesture does not mean two things. It reuses the same `cell_editor` as the cells, which means the colour pinning, caret placement and focus deferral fixed earlier all apply to it for free. Row headers are deliberately not renameable: a row's label is its position, and changing it would mean introducing a stored row title the data model does not have. Tests 63 -> 72 (118 across the tree), all five crates clippy-clean. Verified by reintroducing four defects: removing the resize floor, removing the self-scheduled frame, widening the double-tap window past the long press, and re-forking the touch threshold.	2026-08-18 13:18:31 +00:00
andodeki	1887b54efa	fix(makepad-table): repaint when the cell editor takes focus (item 8) ... The editing cell's text did not appear until the pointer moved. `begin_edit` cannot focus the editor directly. `set_key_focus` takes an `Area`, and the editor only has one once it has been drawn, so focus is deferred: `begin_edit` sets `needs_editor_focus` and the next `handle_event` calls `set_key_focus` on the now-valid area. That deferred step changed how the editor draws — the caret starts blinking and the focused colour states apply — but it never asked for another frame. The editor therefore kept painting its unfocused appearance until something unrelated triggered a redraw. Moving the mouse was that something, which is why the text appeared only after moving the pointer away. One `self.redraw(cx)` after focus is taken. Tests 62 -> 63; verified by removing the call, which fails the new test. This is item 8 of nine reported together. The other eight are new capability — a row-number gutter, header editing, resize handles, a cell context menu with attachments, and a touch path — and are being scoped separately rather than bundled into a bug fix.	2026-08-18 13:04:21 +00:00
andodeki	374af5ccad	feat(pdf): the five Phase 6 bullets the status line omitted ... ADR 0027. Asked whether Phase 6 was 100% complete, I checked the plan's bullets against the code instead of answering from the status line. Five were not implemented and the status line named none of them: detached/ATTACHED signatures /SubFilter hardcoded to adbe.pkcs7.detached PAdES basics ETSI.CAdES.detached was a string in a match external_signing_test.dart absent; SigningIdentity needs an in-memory key OCSP/CRL lookup CRL only; OCSP counted, never parsed Fulcio identity absent (optional in the plan) This is the second time. ADR 0021 recorded the same failure in Phase 4 and wrote the rule meant to prevent it — enumerate criteria from the plan text first, then mark each done or explicitly deferred. I wrote that rule and then produced another prose summary of what I had built. A summary written from the work cannot show what the work omitted. PAdES is a real profile, not a label. CAdES signs a set of signed attributes, one carrying the document digest, and the signature is over those attributes re-tagged as a SET (RFC 5652 5.4) rather than over the [0] IMPLICIT SEQUENCE they are carried in. Verification checks the messageDigest attribute against the document as well as verifying the attribute signature; without that, a signature over somebody else's digest would be accepted. /SubFilter now comes from the profile, so a document cannot claim CAdES while carrying plain PKCS#7. ExternalSigner is a trait: bytes in, signature out. A smartcard or KMS never hands out its key, so SigningIdentity could not represent one. SigningIdentity implements the trait rather than sitting beside it, so there is one signing path — a second path for hardware keys would be a second place the byte range could be computed differently. OCSP is decoded with the der crate already present rather than adding the ocsp crate for two fields. Revoked from any response beats Good from any other. Attached signatures are REFUSED, not deferred. Both attached profiles (adbe.pkcs7.sha1, adbe.x509.rsa_sha1) are SHA-1 based, and SHA-1 is broken for signatures. They are parsed so such documents can be read; they cannot be written, enforced by the absence of a SignatureProfile variant. Same decision as RC4 in ADR 0024. Recorded as refused rather than not-done, because "not done" invites someone to finish it. Four mutations, all killed first attempt: messageDigest not compared, CAdES verified against the wrong bytes, /SubFilter hardcoded again, OCSP revoked read as good. The status line is now the plan's own bullets in a table, one row per spec item, not prose. Two wrong status lines in the same direction is a pattern, and the fix is structural: a missing row is visible, a missing sentence is not. Four rows are left unticked — Fulcio, independent review, Acrobat interoperability, and signing a document that already has an AcroForm. qpdf accepts documents under both profiles. pdf: 1289 passed (was 1276). Coverage 87.96%.	2026-08-18 12:06:59 +00:00
andodeki	780e323674	refactor(spreadsheet-ui): headless formula-bar state machine, wired to the workspace ... The formula bar's state — formula text, target cell, dirty flag, and the "ignore the next change" latch — lived as five scattered fields plus inline logic inside workspace.rs, a `script_mod!` DSL file that no unit test can construct. That made the formula bar the one piece of the UI whose behaviour was untestable headlessly. Extract a `formula_bar` controller module (the same pattern as the earlier geometry.rs extraction) and wire the workspace to it: - `FormulaBar` owns text/target/dirty/latch with `sync_to_cell`, `on_user_input`, `insert_reference`, `take_commit`, and `sync_after_commit`. The workspace now only bridges it to the Makepad text input; the change/return/focus-loss/undo-redo handlers route through the controller unchanged in behaviour. - Point-and-click formula building (new, Excel-style): while the formula bar is focused, selecting a cell appends its reference instead of replacing the formula being edited. `reference_for(sheet, row, col)` and `quote_sheet_name` build `A1`, `Sheet2!A1`, or `'My Sheet'!A1` with Excel's quoting rules (bare identifier unquoted; spaces, punctuation or a leading digit quoted; embedded quotes doubled) — the machinery that makes the cross-sheet reference syntax buildable from the UI. - The coverage script now measures formula_bar.rs alongside the other controller modules. Tests: 11 new headless unit tests (quoting, reference construction, sync/latch/commit round trip, and insertion into empty/value/formula bars). UI lib tests 62 -> 73, all pass; ui-controllers coverage 98.74% (floor 96) with formula_bar.rs at ~98%.	2026-08-18 11:40:49 +00:00
andodeki	d0e7ede0c1	feat(spreadsheet): cross-sheet named ranges (Data!Total) ... Completes the cross-sheet reference feature with `Sheet!Name`, the one shape the previous tranche explicitly left out. Parser and AST: - `Sheet2!Total` and `'My Sheet'!Sales` parse as a new `Expr::SheetNamedRange { sheet, name }` node. In `parse_sheet_qualified_ref`, an identifier after `!` that is not a valid cell reference is treated as a named range; a cell reference wins when both readings are possible (`Sheet2!A1`), matching Excel. Evaluation: - `EvalContext` gains `get_sheet_named_range` (default `None`). `evaluate` resolves a `SheetNamedRange` to its range and returns the first cell in expression context, while `resolve_arg` expands it to every cell for aggregates — so `Data!Total` reads one cell and `SUM(Data!Total)` sums the whole range. An unknown name on a real sheet is `#NAME?`, and `DataEvalContext` looks the range up on the sibling sheet case-insensitively (or on the sheet itself, through the intra-sheet path). Dependencies: - A named-range reference is a coarse sheet-level dependency, because its bounds live on the target sheet. `SpreadsheetData` tracks `cross_sheet_named_refs` (cell -> sheet names), rebuilt by `rebuild_dependency_graphs` / `update_dependency_graph` alongside the cell-level `cross_sheet_refs`, and the workbook folds both into its sheet-index dependency map. Editing a cell inside the named range therefore recalculates the reading sheet. Tests: parser/evaluator units (incl. quoted names, cell-vs-named precedence, dependency extraction) and workbook end-to-end (SUM over a cross-sheet named range with propagation on edit, quoted names, and the #NAME? case). Engine unit tests 390 -> 395; UI lib tests still pass. Engine coverage 97.56% (floor 96).	2026-08-18 11:07:57 +00:00
andodeki	593cd8fee8	feat(spreadsheet): cross-sheet references (Sheet2!A1) ... The engine's dependency graph is per-sheet by design, so this was the structural change: a formula can now read a cell on another sheet, and edits to the source sheet recalculate the readers. Parser and AST: - New tokens: `Bang` (`!`) and `SheetName` (`'My Sheet'`, with `''` as an escaped quote). `!=` still tokenizes as not-equal. - New AST nodes: `SheetCellRef { sheet, cell }` and `SheetRange { sheet, range }`, parsed from `Sheet2!A1`, `Sheet2!A1:B2`, `'My Sheet'!B3`, before the cell-ref fallback (a bare `Sheet2` would otherwise parse as a bogus cell reference). - `evaluate` and `resolve_arg` resolve them through two new `EvalContext` hooks, `get_sheet_cell_value` / `get_sheet_range_values`, which default to `#REF!` in contexts without sibling sheets. Evaluation: - `DataEvalContext` gains a sibling-sheet view plus a shared cross-sheet cycle guard keyed by `(sheet, row, col)`. Reading a sibling routes to a child context pointed at that sibling; re-entering the same cell on the same sheet mid-evaluation reports `#CYCLE!`, and a reference back to the sheet being recalculated is caught through the guard. - `recalculate_all_with` / `evaluate_formula_with` accept the sibling view; the plain per-sheet entry points are unchanged. Recalculation and dependencies: - `SpreadsheetData` tracks `cross_sheet_refs` per cell (rebuilt by `rebuild_dependency_graphs` / `update_dependency_graph`), and the workbook flattens it into sheet-index edges. Editing a sheet recalculates — with sibling access — the transitive closure of sheets that read it, plus the sheet itself (its own formulas may read other sheets). Single-sheet edits with no cross-sheet references keep the fast incremental path. - `evaluate_all` clears computed values across sheets first, then runs one extra pass per sheet, so acyclic chains propagate and mutual cross-sheet cycles terminate with `#CYCLE!`. - Cross-sheet formulas survive save/load: deserialization runs a sibling-aware pass, and a missing sheet renders `#REF!`. Tests: parser/evaluator units (incl. quoted names, dependency extraction) and workbook end-to-end (read + propagation, a three-sheet chain, quoted names, missing sheet, save/load round trip, cycle termination). Engine unit tests 382 -> 390; UI lib tests still pass. Engine coverage 97.55% (floor 96).	2026-08-18 10:53:08 +00:00
andodeki	99aebc202a	fix(pdf): security review of the signing code — a forgery verified as valid ... ADR 0026. Both ADR 0024 and ADR 0025 said this code needed a security review before shipping. This is that review, done adversarially: for each way a signature could be defeated, a test that attempts it. It found a critical vulnerability in the code as shipped last turn. FINDING 1, critical, exploitable with no special access. Verification recovered the certificate and the signature by *scanning* the blob for DER-shaped bytes rather than decoding it. The signature was checked against certificates[0]; trust was checked against ANY certificate present. Two questions, two different certificates. So: the attacker signs a forgery with their own key the attacker appends the victim's trusted certificate to the blob signature_valid = true (their signature over their own content is real) chain_trusted = true (the victim's certificate is present) is_valid() = true Demonstrated before the fix, with the message "I hereby transfer everything to the attacker" verifying as valid. Fixed by decoding the ContentInfo/SignedData structure and finding the certificate the SignerInfo actually names, by issuer AND serial, then evaluating both the signature and the trust path against that one certificate. Trailing data now fails the decode instead of being ignored. The scanning functions are deleted, not left unused: dead code that once returned the wrong answer is an invitation to call it again. FINDING 2, moderate. signer_certificate() returned chain[0] unconditionally, so a chain whose first entry was not the signing key's certificate made the SignerInfo name the wrong one. Not a forgery route — the signature fails — but a UI showing "signed by <somebody trustworthy>" beside a failed check is its own kind of dangerous. Now it finds the entry whose public key matches the key doing the signing. FINDING 3, informational. digest_matches was hardcoded true under a comment claiming it was computed. Not exploitable, because is_valid() also requires signature_valid and the signature covers the bytes — but a field asserting an unperformed check is ADR 0017's pattern exactly. The four items ADR 0025 left unticked are closed: PKIX chain building, with each link's issuer signature verified. A name match alone is not a chain; anyone can put any name in a certificate. Pinning still short-circuits first. Stapled revocation from /DSS, offline only. Unknown is the default and a first-class answer: treating "no information" as "not revoked" is a claim a verifier cannot support. Signature appearances, with the claimed time labelled "Time claimed" because a self-declared /M carries no authority. One-call sign_document. Three things were wrong first: the /ByteRange placeholder was too narrow for real offsets so patching them moved every later byte; /Contents must be a hex string because a literal full of NULs needs escaping and changes length; and a signature dictionary nothing points at is invisible — the first version wrote one and the reader reported zero signatures over a correctly signed document. Four mutations, all killed — two only after strengthening the tests. My first smuggling test put the attacker's certificate first, where certificates[0] finds it anyway, so it passed with or without the issuer/serial match. Putting the TRUSTED certificate first is what distinguishes them, and writing that test is what exposed Finding 2. qpdf --check accepts the signed documents. pdf: 1276 passed. Coverage 87.98%. Left unticked, deliberately: an independent review by someone who did not write the code. This is a self-review; it found two real vulnerabilities, which is evidence the method works and not evidence that nothing remains. Also untested against Acrobat, which is stricter than the spec, and sign_document replaces rather than merges an existing AcroForm.	2026-08-18 10:39:20 +00:00
andodeki	2d7b48b72c	feat(spreadsheet): lookup functions — MATCH, INDEX, VLOOKUP, HLOOKUP, XLOOKUP ... The formula engine could compute over ranges but could not look a value up in one. This adds the Excel lookup family, plus the #N/A error they need to report "not found". New `FormulaError::Na`: - Renders as `#N/A`, parses back in `from_display`, and — a bonus fix — the recalc fast path now propagates a stored `#N/A` as an error instead of turning it into text. Functions (5): - MATCH(lookup_value, lookup_array, [match_type]) — 1-based position. Type 0 exact (case-insensitive text), 1 largest ≤ lookup, -1 smallest ≥ lookup. - INDEX(array, row_num, [col_num]) — cell at a 1-based position; a single index walks the array flat in row-major order. Out of range is #REF!. - VLOOKUP / HLOOKUP — exact or approximate (approximate takes the largest first-column/first-row value ≤ lookup, the sorted-table convention), col/row index out of range is #REF!. - XLOOKUP(lookup_value, lookup_array, return_array, [if_not_found], [match_mode], [search_mode]) — match modes 0 exact, -1 next-smaller, 1 next-larger, 2 wildcard (`*`/`?`, case-insensitive); search modes 1 first-to-last and -1 last-to-first; the `if_not_found` fallback is evaluated lazily, only when nothing matches. Binary search modes are rejected with a clear error rather than silently mishandled. Lookups index ranges by position, so they keep their arguments as AST nodes (a new `range_from_arg` resolves Range and NamedRange expressions) instead of flattening through the aggregate path. Tests: 9 new unit tests (exact/approximate/wildcard/search-direction/ error arms) + an end-to-end test proving the dependency graph tracks the lookup table and recalculates dependents when a table cell is edited. Engine unit tests 373 -> 382.	2026-08-18 10:13:51 +00:00
andodeki	383533da36	feat(email): email the trip report to the finance department ... build_report_email (pure, tested) attaches the report PDF as application/pdf in a multipart message to a comma-separated finance recipient list. spawn_email_trip_report fetches the inbox, extracts trip receipts, builds the report, and emails it via the signed-in SMTP account; the proxy backend reports 'attachments unsupported' honestly rather than failing silently. The Finance card gains a recipients field and an 'Email report to finance' button. An end-to-end test sends the attached PDF through the SMTP sink and asserts the recipient, application/pdf type and filename land in DATA. Domain tests 234 -> 237.	2026-08-18 10:07:54 +00:00
andodeki	26f6d431fb	feat(spreadsheet): date and time functions, with a date display format ... The formula engine had no notion of dates. This adds an Excel-style date/time model and the functions to work with it. Date model (new `dates` module): - A date/time is a single f64: the integer part is a day serial, the fraction is the time of day. Serial 25569 = 1970-01-01, so every modern date agrees with Excel exactly. The calendar is the proleptic Gregorian (Howard Hinnant's days_from_civil/civil_from_days), so Excel's phantom 1900-02-29 (serial 60) reads back as 1900-02-28 and serial 61 = 1900-03-01 — documented rather than reproduced. - Serial <-> calendar conversion, day-of-week, leap-year and days-in-month helpers, date/time string parsing, EDATE/EOMONTH, DATEDIF(Y/M/D), and ISO formatting. Functions (17): - DATE, DATEVALUE, YEAR, MONTH, DAY, HOUR, MINUTE, SECOND, TIME, TIMEVALUE, WEEKDAY (return types 1/2/3), DAYS, EDATE, EOMONTH, DATEDIF — with Excel's month/day rollover in DATE, day clamping in EDATE, and #VALUE! for malformed strings and unknown units. - TODAY and NOW read a wall clock. The engine stays deterministic: `EvalContext` gains a default-none `now_serial` hook, and `SpreadsheetData` carries an optional `now_serial` (never serialized). Without a clock they report "requires a wall clock"; `SpreadsheetData::set_system_now` / `Workbook::set_system_now` supply the system clock, and the UI workspace model wires it on creation. Display: - `NumberFormat::Date` and `NumberFormat::DateTime` render a serial as `YYYY-MM-DD` / `YYYY-MM-DD HH:MM:SS` through write_display_value, with codes that round-trip through the existing serialization. Tests: 18 new unit tests (calendar round-trips across centuries, known serials, leap-year rules, parsing, weekday schemes, EDATE/EOMONTH clamping, DATEDIF, ISO formatting) + 6 evaluator tests + an end-to-end test covering format rendering and recalc through the dependency graph. Engine unit tests 352 -> 373. UI lib tests still 62 pass with the new clock wiring.	2026-08-18 10:00:53 +00:00
andodeki	9a5ce9c0e6	feat(pdf): signing and verification — Valid becomes reachable, with a policy ... ADR 0025, completing Phase 6's functional core. This partially reverses ADR 0010, which refused signing and cryptographic verification outright, and it reverses only the half whose justification expired. ADR 0010 gave two reasons. The first — a parsing library has no business signing — stopped being true when Phase 4 began creating documents and Phase 5 editing them. The second is still true and is preserved intact: deciding which certificate authorities to trust is a policy decision that belongs to the host, not to a parsing library So VerificationStatus::Valid is still not reachable by default. Verification returns three independent booleans and is_valid() needs all three; the third, chain_trusted, can only become true through a caller-supplied TrustAnchors. There is no TrustAnchors::system(), no bundled root store, no Default that trusts anything. A caller with no policy is told "cryptographically intact, signed by somebody you have not said you trust" — a different fact from "forged", and a host that cannot tell them apart shows the wrong thing to a user. RSA PKCS#1 v1.5, ECDSA P-256 and Ed25519, all with SHA-256. PSS is stronger and not universally accepted by PDF verifiers, so v1.5 is what is written. Ed25519 carries an interoperability caveat in the doc comment on the variant itself, because that is where someone choosing it will read it: ISO 32000-2 does not list it and most desktop viewers will reject it. No network. Revocation is not implemented rather than smuggled in: the engine crates are CI-gated against reaching outward, and that gate is a rule about layering, not an obstacle to work around. Every test generates a real key and a real certificate at run time. Nothing asserts against a checked-in blob — a fixed expectation only proves the code still does what it did, which is the wrong question for a signature. The tampering tests assert the signature verifies FIRST, then flip a bit; without that half they could pass by never verifying anything. Four mutations, all killed. The one that matters is the first: making an empty anchor set confer trust is exactly the regression that would turn this back into the thing ADR 0010 refused, and it fails immediately. Two bugs the tests found: UTCTime cannot encode a year past 2049 (RFC 5280 4.1.2.5.1). The first fixture used a 2096 expiry and every certificate failed to encode. The certificate scanner assumed a two-byte DER length. RSA certificates are large enough to use that form, so RSA and P-256 passed while Ed25519 found no certificate at all — its certificate is small enough for the short form. A scanner tested only against the largest input fails silently on the smallest. 72 dependency packages pulled in, zero non-compliant licences, no C. Stated plainly and left unticked in the ADR: chain_trusted is anchor identity matching, not PKIX path building. Correct for certificate pinning, a false negative for a real CA hierarchy. Also outstanding: revocation, signature appearance generation, and one-call incremental signing. pdf: 1247 passed. Coverage 88.08%, floors met.	2026-08-18 09:45:01 +00:00
andodeki	b5ff5dcf40	feat(spreadsheet): text manipulation and IS* info functions ... The formula engine could count, sum, compare and test conditions, but had almost no way to work with text (only CONCAT/LEN/UPPER/LOWER) and no way to ask about a value's type. This adds both. Text functions (operate on the display form of their arguments): - TRIM — collapse internal space runs and drop leading/trailing spaces - LEFT / RIGHT — first/last n characters (default 1) - MID — n characters from a 1-based start - SUBSTITUTE — replace all occurrences, or just the nth instance - FIND / SEARCH — 1-based position, case-sensitive vs case-insensitive, #VALUE! when not found or start is out of range - REPT — repeat n times, capped at Excel's 32767-character result - PROPER — title-case each word Info functions (never propagate their argument's error, like Excel — ISERROR reports it, the others treat it as FALSE): - ISNUMBER / ISTEXT / ISNONTEXT / ISLOGICAL / ISERROR - ISBLANK — TRUE only for an absent cell. This needs the distinction between "missing" and "holds 0", so EvalContext gains a `cell_is_blank` method; DataEvalContext overrides it with a direct cells lookup (an absent cell reads as Number(0.0) through get_cell_value, but is blank, whereas a real 0 is not). Dependencies flow through the existing AST walk, so a TRIM/LEFT/etc. reference is tracked and recalculates when its source cell is edited — pinned by an end-to-end test through SpreadsheetData. Engine unit tests 345 -> 352.	2026-08-18 09:35:19 +00:00
andodeki	b93dc485b9	test(spreadsheet): cover public workbook serialization round trip	2026-08-18 08:31:17 +00:00
andodeki	b7eacb2a94	test(spreadsheet): cover public multi-sheet evaluation	2026-08-18 08:27:13 +00:00
andodeki	4deefabd0a	test(spreadsheet): cover public style commands	2026-08-18 08:24:09 +00:00