nigig-org/BENCH_BASELINE.md

# CAD benchmark baseline

Reproduce with either:

```bash
# needs the Makepad desktop stack (wayland/X11/GL/alsa/polkit)
cargo test --locked --release -p nigig-build --lib -- \
  --ignored --nocapture --test-threads=1 profile_benchmarks

# host-only: isolated toolchain, no desktop, self-cleaning
CAD_BENCH=1 ./tools/test-cad-coverage.sh
```

The second form exists because these benchmarks were unrunnable for ten
review tranches — `REVIEWS/PAY_CAD_IMPLEMENTATION_STATUS.md` recorded the
blocker as "no Cargo toolchain is available in this execution
environment". `profile_benchmarks.rs` turns out to import nothing from
Makepad but the math types, so it builds in the same host-only harness
the coverage run uses.

**Release mode matters.** Debug builds are 3–5× slower here and the ratios
between arms shift, so a debug number is not comparable to anything below.

These are measurements, not assertions — the benchmarks are `#[ignore]`-d
and deliberately contain no timing `assert!`s (a wall-clock assertion in
the normal test run flakes under CI load). Compare against this file by
hand when changing the hot paths.

The frame-submission rows below were recorded on the Phase 5 tree,
x86-64, `--release`. The wall-clock rows above are retained machine
baselines and are not comparable across hosts; the submission rows are
structural counts and are intentionally stable.

| Benchmark | Result |
|---|---|
| `bench_scene_cache_hit_vs_rebuild` | cold 25.9 µs / warm 53 ns (100 parts) — **488×** |
| `bench_scene_cache_scaling` | 10/50/100/500 parts: cold scales linearly, warm stays ~150 ns |
| `bench_mesh_cache_hit_cost` | 11.1 µs/frame, **110 ns per part** (100 parts, all hits) |
| `bench_size_parametric_vs_mesh_derived` | parametric 2 ns vs mesh-derived 886 ns — **443×** |
| `bench_pick_broadphase_mesh_bounds_vs_size` | 63.3 µs vs 131.0 µs per frame — **2.1× saved** |
| `bench_pick_broadphase_world_aabb_recompute_vs_cache` | **230 µs → 39 µs per pick at 500 parts — 5.9×.** Transforming 8 corners per part per pick, versus a `PlacedHash` lookup. Hover picking runs on mouse-move, so this is per-frame. Cached in `SceneCache::world_aabb_for` (Phase 3.2). |
| `bench_parts_script_regeneration_per_drag_frame` | **42 µs (50 parts) / 170 µs (200) / 425 µs (500)** per drag frame, before the editor's own `set_text`. `script_dirty` was set on every motion event; it is now set once on commit (Phase 3.6). |
| `bench_param_hash_cost_per_frame` | Box **50 ns/node** (25 µs/frame at 500) — not worth a cache. Extruded 64-gon **987 ns/node**; that is the vertex data, and bulk-hashing the slice measured no faster. Phase 3.3 left undone deliberately. |
| `bench_parallel_threshold_warm_vs_cold_cache` | 200 nodes: cold **8.80 ms seq / 6.21 ms par (1.42×)**, warm **5.81 / 5.73 (1.01×)**. The 32-node proxy is imprecise but harmless; Phase 3.8 rewrite not worth it. |
| `bench_glb_export_with_and_without_cache` | cold 1.42 ms / warm 1.35 ms (50 parts) |
| `bench_parallel_vs_sequential_export` | sequential 4.83 ms / parallel 3.58 ms (100 parts) — 1.3× |
| `bench_command_execute_overhead` | **0.8 µs per execute, 0.5 µs per undo** (1000 commands over a 1000-node scene). Was 254 µs / 246 µs: `CadCommandCtx::new` eagerly built a scene snapshot no production command reads. Now built on first `scene()` call — roughly **300×**. |
| `bench_gpu_upload_mesh_source` | boxes: rebuild 0.27 µs/part vs warm cache 0.178 µs — **2×**; extruded 24-gons: 1.11 µs vs 0.516 µs — **2×**. Cold cache 0.79 µs/part is **slower** than rebuilding (hash + insert) |
| `bench_delete_invalidation_clear_vs_evict` | deleting 1 of 100 parts: clear+rebuild 52.9 µs vs evict+reuse 14.5 µs — **4×** |
| `bench_viewport_snapshot_sync_per_frame` | deep copies 21.2 µs/frame + forced rebuilds 77.2 µs/frame = **98.5 µs/frame**; generation-checked 0.051 µs — **1930×** (100 parts) |

### Frame submission budget

`bench_frame_submission_budget`, added by Phase 0 of
`REVIEWS/CAD_RENDER_OPTIMISATION_PLAN.md` and extended through Phase 5.
Every other benchmark in this file measures CPU work *behind* the
renderer; none measures a live GPU frame, so the submission report stays
as an explicit structural count rather than pretending to be frame time.

Counts, not timings — counts do not flake under CI load. 1920×1080,
parts on a lattice across a 200 m site (deliberately larger than the
viewport at either zoom: a scene that fits on screen is the case where
culling can do nothing, and reporting only that would flatter it).

**2D, after Phase 1 culling, Phase 4 stroke batching, and Phase 5
sub-pixel LOD.** "Before" is the Phase 0 baseline: every part in the
document submitted, one `stroke()` per part and one per grid line.

| Zoom | Parts | Visible | Full outlines | LOD points | Part calls | Tessellations | Before |
|---|---|---:|---:|---:|---:|---:|---:|
| 5 m (170 grid lines) | 100 | 1 | 1 | 0 | 1 stroke | **3** (2 grid + 1 parts) | 270 |
| | 500 | 4 | 4 | 0 | 2 strokes | **4** | 670 |
| | 2,000 | 12 | 12 | 0 | 2 strokes | **4** | 2,170 |
| 200 m (270 grid lines) | 100 | 100 | 1 | 99 | 1 stroke + 1 fill | **4** | 370 |
| | 500 | 500 | 1 | 499 | 1 stroke + 1 fill | **4** | 770 |
| | 2,000 | 2,000 | 1 | 1,999 | 1 stroke + 1 fill | **4** | 2,270 |
| 500 m (170 grid lines) | 100 | 100 | 1 | 99 | 1 stroke + 1 fill | **4** | 270 |
| | 500 | 500 | 1 | 499 | 1 stroke + 1 fill | **4** | 670 |
| | 2,000 | 2,000 | 1 | 1,999 | 1 stroke + 1 fill | **4** | 2,170 |

The benchmark models one material plus one selected part. Selection is
intentionally kept at full detail, so the first visible part remains an
outline while the ordinary parts cross the LOD threshold. At 5 m, a 1 m
part is comfortably resolvable and every survivor stays full detail. At
200 m, the same part is only about 2.7 logical pixels across; at 500 m it
is about 1.1 pixels, so the ordinary survivors become 2×2 logical-pixel
filled markers.

Phase 5 does **not** claim another draw-call win. The 2D scene remains one
`DrawVector::end()` submission, and the one material still makes one
outline stroke plus one marker fill. The win is geometry volume: a filled
marker is bounded and does not carry a full rectangle/circle outline for
a part whose shape cannot be resolved. `FrameBudget` reports full
outlines and LOD markers separately so this change cannot be mistaken for
another tessellation reduction.

**3D, frustum culling (Phase 1), instanced batching (Phase 3), and
bounding-box LOD (Phase 5)**, 60° fov, camera orbiting the site centre.
"Shapes" is how many distinct full-detail solids the visible parts use —
6 for a catalogue-built model, all distinct for the worst case. LOD boxes
are all instances of one shared unit-cube proxy batch:

| Camera | Parts | Visible | Full meshes | LOD boxes | Draw calls, 6 shapes | Draw calls, all distinct | Before |
|---|---:|---:|---:|---:|---:|---:|---:|
| 20 m | 100 | 39 | 39 | 0 | **6** | 39 | 100 |
| | 500 | 198 | 198 | 0 | **6** | 198 | 500 |
| | 2,000 | 733 | 733 | 0 | **6** | 733 | 2,000 |
| 80 m | 100 | 76 | 76 | 0 | **6** | 76 | 100 |
| | 500 | 386 | 386 | 0 | **6** | 386 | 500 |
| | 2,000 | 1,459 | 1,459 | 0 | **6** | 1,459 | 2,000 |
| 400 m | 100 | 100 | 20 | 80 | **7** | 21 | 100 |
| | 500 | 500 | 63 | 437 | **7** | 64 | 500 |
| | 2,000 | 2,000 | 380 | 1,620 | **7** | 381 | 2,000 |

The bottom row is the one to look at. At 400 m the whole site is on
screen, so culling removes nothing. LOD replaces 1,620 unresolvable
meshes with instances of one proxy, and the six full-shape groups plus
that proxy draw in **7 calls instead of 2,000**. The "all distinct" column
is the same scene with every full-detail part a different size; 380 remain
full and the 1,620 tiny parts still share the proxy, so the count is 381.
The selected/hovered exemption is not present in this synthetic 3-D
benchmark; the policy tests pin that decorated parts remain full detail.

Where culling does work it is decisive on its own: drafting at a 5 m
zoom over a 200 m site, the 2,000-part scene queues 12 outlines instead
of 2,000. Where it cannot help — everything on screen — batching still
holds the call count at 4.

Two different costs, deliberately not summed:

- **Tessellations** are CPU calls into `tessellate_path_stroke`. The
  whole 2D scene is **one** draw call from `DrawVector::end()` no matter
  how many parts, so this — not draw calls — was the number that scaled
  there. Since Phase 4 it scales with colour groups instead.
- **Draw calls** are GPU submissions. 3D used to be one per uploaded,
  visible part; since Phase 3 full-detail parts use one call per distinct
  shape, with the parts riding inside as instance rows. Since Phase 5,
  all tiny parts use one additional shared proxy batch.

The grid line count is unchanged by culling, as it should be: it was
already virtualised, and stays at 170–270 lines across the measured zoom
range because the step adapts. Phase 4 changed what those lines cost —
two `stroke()` calls rather than one each — and Phase 5 changes only the
representation of tiny parts. The tests `culling_shows_up_as_fewer_part_tessellations`,
`the_grid_costs_two_strokes_at_any_zoom`, and the LOD policy tests pin
those separate costs.

Phase 2 (sharing geometry between parts of the same shape, keyed by a
new `ShapeHash` — **not** `ParamHash`, which includes the node id) and
Phase 3 (instancing) brought the draw-call column down in the zoomed-out
case where culling cannot; Phase 4 did the same for the 2-D tessellation
column. Phase 5 addresses the remaining geometry volume in both paths:
the 2,000-part rows at 200 m and 500 m keep one selected outline but
replace 1,999 unresolvable outlines with bounded point markers, while the
400 m 3-D row replaces 1,620 unresolvable meshes with one shared proxy
batch. A real GPU/window visual check is still required before treating
the structural counts as frame-time results.

### GPU geometry buffers, keyed by shape

`bench_geometry_buffers_shared_by_shape`, added by Phase 2. Counts, not
timings. The number it replaces was never measured because it was never
in doubt: **one buffer per part**, however many of those parts were the
same column.

| Model | Parts | Buffers | Ratio |
|---|---|---|---|
| 200 identical walls (`build_house_scene(200)`) | 200 | **1** | 200× |
| 420-part repetitive model (columns, walls in three lengths, two opening types) | 420 | **6** | 70× |
| 420 parts, every one a different size | 420 | **420** | 1× |

Read the last row too. Sharing is a property of the model, not of the
code: a scene where every part differs gets nothing from this phase.
That is the honest ceiling, and it is why the draw-call column is
untouched here — Phase 3 (instancing) is what collapses draw calls, and
it needs the shared buffers this phase produces.

The middle row is the one to plan against. Real architectural models are
catalogue-built: a few column types, a few wall lengths, a door and a
window repeated everywhere. 70× fewer GPU buffers, and 70× fewer mesh
uploads on the first frame after a load.

### Note: two benchmarks had drifted off the caching path

If you ran these between the Phase 5.1 generation-tracked store and the
commit that added this note, `bench_scene_cache_hit_vs_rebuild` reported
**1.2×** against the **488×** in the table above, and
`bench_scene_cache_scaling` reported **1× at every part count**. That
looked exactly like a catastrophic cache regression. It was not.

Both benchmarks called `SceneCache::scene(&[CadNode])`, which is
documented as *always rebuilding* — it takes a bare slice, so it has no
generation to compare and cannot cache. The caching entry point the
editor uses is `SceneCache::scene_for(&PartsStore)`. When the store
landed, these two benchmarks were not moved with it, so their "warm"
sample was a second full rebuild and the printed speedup was allocator
noise.

Repointed at `scene_for`, they reproduce the table: cold 24.6 µs / warm
**48 ns**, **512×**, and the warm read stays flat at ~55 ns from 10 parts
to 500. `bench_scene_cache_hit_vs_rebuild` now asserts `Arc::ptr_eq` on
the two samples, so it fails loudly rather than quietly measuring two
rebuilds if it is ever pointed at a non-caching path again.

`SceneCache::scene()` is not the bug and was left alone: its docstring
says what it does, and nine tests legitimately use it for exactly the
case it describes.

### Note on `bench_gpu_upload_mesh_source`

Phase 5.3 routed the GPU upload path through `MeshCache` instead of
calling `build_solid()` directly. The measured speed win is **2× on a warm
cache and negative on a cold one** — a first-ever build now pays a
`ParamHash` and a map insert on top of the meshing.

That trade is worth taking, but not primarily for speed:

- `ensure_part_geometry` only builds for parts with no uploaded buffer, so
  the cold case is a scene load or an undo/redo, not a frame in a loop.
- The warm case is every part that was already exported, hovered or
  previously uploaded — which, during editing, is most of them.
- The real gain is that there is now **one** node-to-triangles path. Two
  independent `match` arms over the same enum were free to drift, and
  nothing would have caught it: the preview would simply have disagreed
  with the export.

Do not "optimise" this back by special-casing cheap primitives. The 0.5 µs
is not worth reintroducing the second pipeline.

### Note on `bench_viewport_snapshot_sync_per_frame`

This benchmark measures the *primitives* the sync path is built from --
`Vec<CadNode>` clones and `SceneCache::scene()` rebuilds -- not the widget
method itself, which needs a live `Cx` and three real viewports. Its
printed numbers therefore do **not** move when the sync path is fixed;
they quantify what one frame of syncing costs and what the same frame
costs when a generation check is allowed to short-circuit it.

Read it as the size of the prize, not as a regression gate. The Phase 5.2
change is verified by behaviour tests
(`replace_all_bumps_the_generation_so_destinations_can_compare`,
`mesh_cache_self_invalidates_on_parameter_and_transform_edits`,
`geometry_is_retained_for_surviving_ids_only`), each negative-tested.

---

## Reading these

**The scene cache is doing its job.** 488× on a warm hit, and the scaling
run shows the warm path is O(1) while the cold path is linear. No work
needed here.

**`MeshCache` hits are not free.** 186 ns per part is `ParamHash::from_node`
being recomputed on every lookup — it hashes the solid's every field, the
transform and the material, even when the entry is present and valid. At
100 parts that is ~19 µs per hover frame. Not the dominant cost today, but
it is the next thing to fix if picking needs to get faster: memoise the
hash on `CadNode` and invalidate it on mutation.

**`CadNode::size()` is a trap.** 2 ns for a parametric solid, 886 ns for a
CSG or extruded one, because those have no closed-form extent and the
function meshes the solid to derive a bounding box (added in Phase 1.7 to
fix unpickable parts). It reads like a cheap field access at the call site.
Anything calling it in a loop should take bounds from a cached mesh
instead — which is exactly what `pick_part` now does.

**Export is already fast enough.** 1.4 ms for 50 parts, and the mesh cache
barely moves it because JSON/binary serialisation dominates, not meshing.
The parallel path wins only 1.3× at 100 parts. Neither is worth tuning
before the UI hot paths.

---

## What Phase 3 changed

| Change | Effect |
|---|---|
| `pick_part` takes AABB bounds from the mesh it already holds instead of calling `size()` | **2.1×** less broad-phase work per hover frame; also *more correct* — see below |
| Hover picking throttled by `HOVER_PICK_MIN_MOVE_PX` (3 px) | Sub-pixel jitter no longer triggers a full ray cast against every triangle |
| Removed `cx.redraw_all()` from the part-drag `MouseMove` handler | No full-widget-tree relayout per motion event during a drag |

The AABB change also fixed a latent correctness bug. `size()` returns a
*symmetric* extent about the origin, but an extruded polygon grows along
+Y from its base plane — so the old broad-phase box was in the wrong
place and could reject a ray that actually hits. `pick_bounds_tests::
mesh_bounds_are_asymmetric_where_size_is_not` pins this.

---

## Not done, and why

**Memoising `ParamHash` on `CadNode`** (~186 ns/part/frame). It needs a
cache field on the node plus invalidation on every mutation path, which is
the same ownership problem Phase 5.1 solves properly with a generation
counter. Doing it now would add a second thing to keep in sync by hand —
exactly the pattern this codebase already suffers from.

**A BVH or spatial index for picking.** Current cost is linear in part
count with a cheap early-out. At the 10–100 parts these models actually
contain, a BVH would cost more to build than it saves. Revisit if scenes
grow an order of magnitude.

**The other 81 `cx.redraw_all()` calls.** Most are on discrete user
actions (keypress, button, tool change) where a full redraw is once-per-
gesture and harmless. Only the per-motion ones were worth removing, and
`orbit_3d_by_screen_delta` / `pan_3d` were already correct.


## `bench_command_execute_overhead` — resolved

Recorded here because the number moved 300× and the reason is worth
keeping.

`CadCommandCtx::new` eagerly built a scene snapshot:

```rust
let built_scene = scene_cache.scene_for(parts);   // O(nodes)
```

`CommandContext::scene()` needs one, but **no production command reads
it** — `MoveNode`, `ResizeNode`, `RotateNode`, `YawNode`, `ModifyNode`,
`CreateNode` and `DeleteNode` all address a node by id. Every command
mutation bumps the `PartsStore` generation, so the next context
construction was a guaranteed cache miss that cloned every node and
re-registered every material. `move_selected` issues one command per
selected part per frame, so a drag was O(commands × nodes).

The snapshot is now built on the first `scene()` call and memoised, and
dropped by every mutating method (paired with the existing
`mark_dirty()` calls, so a new mutator cannot silently miss it).

The eager build was also a latent **correctness** bug: it captured the
scene *before* the command ran, so a command that mutated and then read
`scene()` saw its own edit missing. Pinned by
`a_lazily_built_scene_reflects_edits_made_earlier_in_the_same_context`.

Two measurement notes:

- The old 0.4 µs figure predated `PartsStore`; the generation counter
  turned a previously accidental cache hit into a guaranteed miss. The
  regression was real, not a measurement artefact.
- The benchmark itself called `ctx.scene()` each iteration to read the
  start position, which no production path does — `move_selected` and
  `finish_part_drag` both read `p.pos()` off the parts list. Fixing the
  lazy build alone moved undo 228 → 0.5 µs but left execute at 231 µs,
  because the benchmark was timing its own scene read. It now mirrors
  the production callers.

---

## Engine baselines (doc + spreadsheet)

Added in PLAN_PERF_OPTIMIZATION.md Phase 0. Release-only, `#[ignore]`-d
benches that **print, never assert** wall-clock numbers (CI-load flakes).
Host: x86-64, macOS, `--release`, `target-cpu` not set. Reproduce:

```bash
cargo test --release -p spreadsheet-engine --lib -- \
  --ignored --nocapture --test-threads=1 bench_
cargo test --release -p doc-engine --lib -- \
  --ignored --nocapture --test-threads=1 bench_
```

| Benchmark | Baseline (this host) | What it measures |
|---|---|---|
| `bench_grid_set_get_10k` | 1.65 µs per get_raw+display | HashMap-backed grid read hot loop |
| `bench_formula_chain_recalc_1k` | 22.6 ms per full recalc | 1000-cell dependency-chain rebuild+eval |
| `bench_column_sum_10k` | 91.8 ms per recalc | SUM(A1:A10000) — aggregate hot loop |
| `bench_workbook_roundtrip_10k` | 97.7 ms per roundtrip | serialize+deserialize 10k cells |
| `bench_crdt_materialize_2k` | **29.9 ms per materialize** | whole-op-log → projection, 2000 chars |
| `bench_crdt_save_wire_2k` | **9.9 ms per to_json+from_json** | full-doc save wire, 2000 chars |
| `bench_insert_char_1k` | **3.65 ms per keystroke** | insert atom + re-materialize per key |

The doc rows confirm the Phase 6 targets are real: typing is ~3.65 ms/key
(materialize dominates), and each save rewrites the whole op log.