andodeki
9989043a37
Phase 0 of REVIEWS/REPO_COVERAGE_100_PLAN.md, and the reason it is Phase 0: no new tests, no new measurement, just ratchets on numbers that were already good and already decaying-capable. **spreadsheet** — `tools/test-spreadsheet-coverage.sh` has had a 96 floor for the engine and another for the UI controllers, and no CI job has ever run it. New `.forgejo/workflows/spreadsheet.yml`, two jobs: engine-coverage 98.83% of lines (floor 96) ui-controller-coverage 98.85% of lines (floor 96) Split in two because the halves cost very differently. The engine is pure Rust and finishes in about three minutes; the UI half has to build Makepad's Linux backend to link a test binary. One job would hide an engine regression behind a ten-minute build. **CAD widget layer** — `cad-widget-coverage` in nigig-build.yml, deliberately REPORT-ONLY. It sits at 13.25% of 10,637 lines with six files at exactly zero, and a floor there would read as a blessing rather than a debt. What the job buys is that the number is printed on every push instead of being rediscovered in six months. The first real input test should set a floor behind it. Also corrects the plan. It claimed the doc workspace module was ungated; it is not — nigig-build.yml has run doc-workspace-coverage since before the plan was written. I had surveyed by grepping workflow files for the word "coverage" and attributed nigig-build's coverage jobs to CAD alone. I nearly committed a duplicate workflow on the strength of it. The census table was right; the prose under it was not, and the correction is in the file. One thing checked and deliberately NOT changed: the spreadsheet script appears to skip its UI half when the native packages are absent. It does not. `makepad-native-libs.sh --check` returns 1, the script runs under `set -e`, and it aborts. What misled me was reading `$?` after piping the script into `tail` — which reports tail's status, not the script's. The same class of mistake this repository's CI comments warn about; no fix was needed and none was made. Verified by running each job's exact command line: COVERAGE_TARGET=engine ./tools/test-spreadsheet-coverage.sh rc=0 COVERAGE_TARGET=ui ./tools/test-spreadsheet-coverage.sh floors met ./tools/test-cad-widget-coverage.sh 13.25%, rc=0
10 KiB
Repository test coverage: a phased plan to 100%
Scope: all 58 workspace members, 310,287 lines of source. A CAD-module
plan already exists at REVIEWS/CAD_COVERAGE_100_PLAN.md; this is the
level above it, and where the two disagree the CAD document is more
specific and wins.
Every number here was produced by a command in this repository. None is estimated unless it says so.
The census
| Tier | Crates | Source lines | State |
|---|---|---|---|
| A | 5 | 45,190 | measured and gated in CI |
| B | 2 | 15,373 | measured, not gated |
| C | 27 | 225,149 | has tests, never measured |
| D | 5 | 4,160 | zero tests |
| E | 19 | 20,415 | templated app shells, zero tests |
| 58 | 310,287 | 5,155 tests, 27,419 lines of test code |
Tier C is 73% of the repository and the whole problem. Everything else is either already handled or arguably not worth handling.
Tier A — measured and gated
| Area | Floor | Tool |
|---|---|---|
CAD engine (nigig-build cad/, 15 files) |
96 total, per-file | test-cad-coverage.sh |
doc-engine |
96 | test-doc-engine-coverage.sh |
email domain (7 nigig-core files) |
88 | test-email-coverage.sh |
| pdf (4 crates) | 80 | test-pdf-coverage.sh |
Tier B — measured, nobody enforces it
spreadsheet-engine + spreadsheet-ui have a coverage script with
per-file floors and no CI job that runs it. The CAD widget layer
likewise: 13.25%, measured, ungated. This is the cheapest gap in the
document.
Correction, made while executing Phase 0: an earlier draft of this
section also listed the doc workspace module here. It is not ungated —
nigig-build.yml has run doc-workspace-coverage since before this
plan was written. The census table above was right; this paragraph was
wrong, because the survey that produced it matched workflow files
containing the word "coverage" and attributed nigig-build's coverage
jobs to CAD alone.
Tier C — the elephant, biggest first
| Crate | Lines | Tests | Measured? |
|---|---|---|---|
nigig-build |
77,758 | 1,485 | only cad/ (~24k of it) |
map |
44,518 | 887 | no |
pageflipnav |
24,117 | 20 | no |
nigig-core |
18,879 | 327 | only the 7 email files |
nigig-sms |
7,127 | 64 | no |
nigig-pay-domain |
6,321 | 161 | CI runs tests, no coverage |
nigig-pay |
6,107 | 72 | no |
nigig-pay-ui |
5,724 | 84 | no |
nigig-mpesa |
5,198 | 45 | no |
robius-sms |
4,045 | 58 | no |
nigig-pay-platform |
2,567 | 64 | no |
nigig-pay-storage |
2,405 | 46 | no |
pageflipnav is the standout: 24,117 lines, 20 tests, no gate, and
nothing in this repository's review documents mentions it.
Tier E — nineteen copies of one file
nigig-shop, nigig-tow, nigig-rider, nigig-book, nigig-health
and fourteen more are 992–1,033 lines each and are templated clones:
diffing two main.rs files after normalising the name yields a
background colour and a root screen identifier. Treating these as
nineteen crates to cover is nineteen times the work for one crate's
worth of risk. See Phase 5.
What 100% would actually cost
The only honest estimate available is the measured one. The CAD engine is 13,752 lines and reaching 97.16% took roughly six concentrated sessions, producing 568 tests and finding four real defects. That is ~2,300 lines brought under coverage per session, on code that was already the easiest half — pure, dependency-light, and already partially tested.
Tier C is 225,149 lines. Straight-line extrapolation is ~98 sessions, and the extrapolation is optimistic because tier C contains the GUI layers, which are strictly harder: the CAD widget half sits at 13.25% after all that work.
That number is not an argument against the goal. It is an argument about sequence: at this scale, the order in which coverage arrives matters more than the destination, because the first 20% of the effort can cover most of the risk if it is pointed at the right code.
The principle this plan is sequenced on
Coverage is a search strategy, not a product. Of the defects this work has found so far — a matrix inverse wrong for every rotate-plus-translate, three exporters that only ever dispatched one geometry variant, a nav-pad hit zone a pixel from its button, two benchmarks measuring a function that cannot cache — not one was found by driving a percentage up. All four came from reading uncovered regions and asking why they were unreachable.
So: measure everything early because measurement is cheap and tells you where to look; close gaps in risk order, not size order; and gate each number the day it is good, because an ungated number decays.
Phase 0 — Gate what is already measured (2–3 days)
Three coverage scripts exist with floors and no CI job. Add them.
spreadsheet(15,373 lines, floors already written) → CI job. Done:.forgejo/workflows/spreadsheet.yml, two jobs, measured 98.83% engine / 98.85% ui-controllers against floors of 96.doc workspace module— already gated bynigig-build.yml; see the correction in the tier B section.- CAD widget layer → CI job, report-only, then floors at
measured-minus-one once a real input test lands. It is at 13.25%; a
floor there today records the debt rather than blessing it.
Done:
cad-widget-coverageinnigig-build.yml.
Exit: every existing coverage script runs in CI. No new tests written. This is pure ratchet installation.
Phase 1 — Measure tier C (1–2 weeks)
You cannot plan 225,149 lines without knowing where they stand. Today there are seven bespoke coverage scripts, each hand-rolled for one area. Fifty-eight crates cannot have fifty-eight scripts.
- Generalise the harness into one parameterised runner: crate list,
file list, floors, exclusions. The existing scripts become config.
test-cad-coverage.shis the most evolved and is the right base — it already handles isolated toolchains, reuse knobs, floors and drift detection. - Run it over every tier C crate. Record a number per crate in
TEST_BASELINE.md. - Re-sort the phases below by what it finds. Phases 2–4 are provisional until this lands.
Exit: one runner, a coverage number for all 58 members, and a table that makes the next argument evidence-based instead of size-based.
Phase 2 — Money and safety first (4–6 weeks)
Regardless of what Phase 1 measures, this goes first. nigig-pay,
nigig-pay-domain, -storage, -platform, nigig-mpesa,
nigig-sms, robius-sms: 34,000 lines that move money and send
messages on a user's behalf. REVIEWS/NIGIG_PAY_CONSOLIDATED_REVIEW.md
and PAYMENT_RISK_REGISTER.md already describe unresolved hazards here
— thread-local coordinator, plaintext pending store, SMS-as-evidence.
Target 100% on the pure domain crates (pay-domain, pay-storage,
pay-platform — 11,293 lines, all host-testable), and ≥90% on the
transport and UI crates.
Exit: the four payment crates gated at 95+, with workflow.md's
nine non-negotiable payment rules each pinned by a named test.
Phase 3 — The large engines (3–6 months)
map (44,518), nigig-build outside cad/ (~53,000), nigig-core
(18,879), pageflipnav (24,117).
Use the pattern that worked on CAD, in this order per crate:
- Split pure logic from widget code and measure the pure half in a host-only harness. CAD's engine/widget split is the template.
- Extract decision logic out of event handlers into pure modules — ~60 lines per extraction, half a day each, and on the one data point available it found a real defect on the first try.
- Accept that the imperative widget glue stays uncovered until Phase 4.
pageflipnav first despite not being the largest: 20 tests over 24,117
lines is the worst ratio in the repository and nothing has ever looked
at it.
Phase 4 — The GUI layers (2–3 months, gated on a spike)
REVIEWS/CAD_COVERAGE_100_PLAN.md Phase 0 is the spike for this whole
phase: 148 #[makepad_test] tests exist in nigig-build/tests/ui.rs,
have never compiled, and run under xvfb-run once a one-line
dev-dependency is added. pdf.yml already runs its UI suite under
xvfb, so the mechanism is proven in this repository.
If that spike succeeds, the same approach applies to map,
spreadsheet-ui, pay-ui and the shells. If it does not, GUI coverage
stays out of reach and this plan tops out around 60% repo-wide — which
is worth knowing before committing three months.
Do not start Phase 4 before the CAD spike reports.
Phase 5 — Decide what the templated shells are (1 week)
Nineteen crates, 20,415 lines, one program. Options, in preference order:
- Extract the template. One shared crate with the app scaffold, 19 thin crates that configure it. Covering the shared crate once covers all nineteen. This is a refactor, not a test task, and it is the only option that makes the number honest.
- Cover one representative and exclude the rest by policy, documented.
- Cover all nineteen. Nineteen times the work for one crate of risk.
Exit: a written decision. Not covering them silently is the only outcome that is wrong.
Phase 6 — Close the last mile and enforce (ongoing)
- Structural exclusions, because
#[coverage(off)]is unstable on the pinned 1.97.1 toolchain — verified,error[E0658]. Genuinely unreachable code must be covered, moved to a file the report excludes, or subtracted openly in the report. There is no annotation. - Raise every floor to measured-minus-one, quarterly.
- Report a repo-wide adjusted total, with the subtracted set listed by file and line count.
What this plan will not deliver, and why
- A literal 100% of 310,287 lines. GL draw calls, native save
dialogs, platform startup and
panic!arms inside tests are either unreachable in a test process or assert nothing when reached. The honest target is 100% of what is worth executing, with the remainder named. In CAD that remainder is ~48 lines out of 13,752; repo-wide it will be larger. - Assurance. Coverage counts lines executed, not assertions made.
The 148 CAD UI tests are mostly
wait_visible()and will move the number a long way while proving that widgets exist. Nothing in this plan prevents that; only review does.
Recommended first action
Phase 0. It is two or three days, writes no tests, and stops the 15,373 already-measured lines in tier B from decaying while the rest of this is argued about.