Compare commits

...

3 commits

Author SHA1 Message Date
arena-agent
228bc2c81f ci(doc-engine): gate the engine coverage, and note it in the doc README
Some checks failed
email.yml / ci(doc-engine): gate the engine coverage, and note it in the doc README (push) Failing after 0s
nigig-build (CAD) / supply-chain (push) Has been cancelled
nigig-build (CAD) / cad-module (push) Has been cancelled
nigig-build (CAD) / full-crate-check (push) Has been cancelled
nigig-build (CAD) / cad-engine-coverage (push) Has been cancelled
repo hygiene / hygiene (push) Has been cancelled
doc-engine / engine (push) Has been cancelled
doc-engine / coverage (push) Has been cancelled
doc-engine / consumer (push) Has been cancelled
New coverage job runs tools/test-doc-engine-coverage.sh on changes to
crates/apps/doc/**, the script itself, or the workflow. A coverage
number nobody gates goes down; the floors (total plus per-file) are the
enforcement. The doc workspace README records the milestone and the two
CRDT-tolerance behaviors the new tests pin.
2026-08-17 04:33:08 +00:00
arena-agent
9d37874453 test(doc-engine): isolated source-coverage harness with floors
Mirror of the CAD engine harness for the doc crate, minus the shim
gymnastics (doc-engine depends only on serde/serde_json, so it
instruments directly): an isolated toolchain + cargo + target dir under
one mktemp directory, removed by a shell trap on every exit path;
nothing enters the host, the workspace target/, or $HOME. Runs the unit
tests plus tests/materialize.rs under -C instrument-coverage, enforces
a 96% total-lines floor against a 99.00% baseline plus per-file floors
(losing one module's tests must not hide in the total), and with
KEEP_COVERAGE=1 writes the uncovered-line listing that makes adding
branch tests directed rather than guesswork. COVERAGE.md records the
baseline, the exclusions, and the arms that are deliberately left
uncovered (defensive CRDT merge arms, one unreachable!, and the
Compensation::inverse arms unreachable through the public API).
2026-08-17 04:33:08 +00:00
arena-agent
1269811b44 test(doc-engine): cover the branches the first coverage report named
A first instrumented run (99.00% -> this branch set is what got it
there) showed the gaps precisely; these tests close the reachable ones:

- insert_text_at_offset / delete_text_at_offset: mid-block splices,
  prepend at 0, append at end, backward/forward deletion and every
  out-of-range guard (both fns were 0% covered).
- set_block_alignment materialization, including the CRDT-tolerance arm
  for a block whose op has not arrived.
- set_table_cells: multi-cell batch undoes and redoes as ONE group;
  empty write lists are rejected.
- CrdtDocument::to_json/from_json wire round trip (the format every
  workspace save rides on) was never exercised end to end.
- toggle_text_style_at_offsets rejects unknown fields and missing
  blocks; a style patch on an EMPTY block is retained rather than
  dropped.

Writing them inverted two expectations and the tests pin the actual --
and correct -- CRDT semantics instead: inserts/cell writes addressed to
anchors that have not arrived yet are ACCEPTED into the op log (they
must be, to merge when the anchor lands) while conjuring nothing into
the rendered document. 96 integration tests pass; clippy stays at -D
warnings clean.
2026-08-17 04:33:08 +00:00
5 changed files with 500 additions and 0 deletions

View file

@ -22,6 +22,7 @@ on:
push:
paths:
- 'crates/apps/doc/**'
- 'tools/test-doc-engine-coverage.sh'
- 'Cargo.lock'
- 'Cargo.toml'
- 'rust-toolchain.toml'
@ -29,6 +30,7 @@ on:
pull_request:
paths:
- 'crates/apps/doc/**'
- 'tools/test-doc-engine-coverage.sh'
- 'Cargo.lock'
- 'Cargo.toml'
- 'rust-toolchain.toml'
@ -52,6 +54,27 @@ jobs:
- name: Reject whitespace errors
run: git diff --check
# Source coverage for the engine, gated, not just printed.
#
# The crate is UI-free (serde + serde_json only), so unlike the CAD
# harness this needs no host-only shim: tools/test-doc-engine-coverage.sh
# copies the crate into a scratch directory, runs the unit tests plus
# tests/materialize.rs under -C instrument-coverage, and enforces a
# total floor and a per-file floor (a lone total would wave through the
# loss of every test in one module). It installs its own toolchain into
# a mktemp directory and deletes everything through a shell trap, so
# nothing is cached between runs and nothing is left in the workspace.
# Exclusions, the baseline table and what the number does NOT mean live
# in crates/apps/doc/doc-engine/COVERAGE.md.
coverage:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- name: Engine coverage, with floors
run: ./tools/test-doc-engine-coverage.sh
consumer:
runs-on: ubuntu-latest
timeout-minutes: 60

View file

@ -0,0 +1,51 @@
# doc-engine coverage baseline
Measured by `tools/test-doc-engine-coverage.sh`, which runs the crate's
unit tests plus `tests/materialize.rs` under `-C instrument-coverage` in
a fully isolated, self-deleting environment (own toolchain, own cargo
home, everything under one `mktemp` directory removed by a shell trap).
Baseline (2026-08-17, toolchain 1.97.1, llvm-cov **lines** percent):
| File | Lines | Regions | Notes |
|---|---|---|---|
| `src/controller.rs` | 98.68% | 93.45% | remaining lines are defensive: batch-compensation arms that only fire on mid-batch op-application failure |
| `src/crdt/document.rs` | 97.63% | 95.13% | remaining arms are split-replay RGA link guards and CRDT-tolerance skips (ops whose anchors never arrive) |
| `src/crdt/operations.rs` | 100% | 100% | |
| `src/crdt/version_vector.rs` | 100% | 100% | |
| `src/history.rs` | 92.00% | 88.34% | the honest floor: `Compensation::inverse` arms unreachable through the public controller API (undo/redo expand groups before materializing) plus one `unreachable!` |
| `src/projection/text.rs` | 100% | 94.19% | |
| **TOTAL** | **99.00%** | **97.54%** | |
`src/lib.rs`, `src/session.rs`, `src/crdt/mod.rs` and `src/projection/mod.rs`
carry no executable lines (module roots and re-exports), so the report
never lists them and the floor table deliberately has no entries for
them — a floor on an unmeasured file fails the run loudly, by design.
## What is NOT measured, and why
- **Widget-level consumers.** This crate is UI-free by construction
(serde + serde_json only); the `CrdtDocEditor` surface in nigig-build
is exercised by its own 800+-test lib suite, not here. Reading
99% as "the doc feature is 99% tested" would be wrong in exactly the
way the CAD baseline warns about: it is the *engine* that is.
- **Defensive CRDT arms.** A handful of match arms exist so that ops
arriving before their anchors (or after duplicate ids) degrade
silently instead of corrupting order. Some are constructible only by
forging op ids below the controller API; they are reported as
uncovered rather than hidden.
## Floors
Total 96% lines; per-file floors a couple of points under the table
above, enforced by the script and the `coverage` job in
`.forgejo/workflows/doc-engine.yml`. Lowering a floor is a reviewable
edit to the script, not something to do quietly.
## Regenerating
```
./tools/test-doc-engine-coverage.sh # gated run
KEEP_COVERAGE=1 ./tools/test-doc-engine-coverage.sh # keeps the env and writes
# an uncovered-line listing
```

View file

@ -1204,3 +1204,191 @@ fn table_cell_special_character_text_survives_sync_and_undo() {
"undo/redo restores the text verbatim"
);
}
/// `insert_text_at_offset` resolves its after-anchor from the projected
/// atom list: mid-block splices, prepend at offset 0, append at the end,
/// and a clean None on an unknown block.
#[test]
fn insert_text_at_offset_maps_mid_block_prepend_and_append() {
let mut c = DocumentController::default();
let b = c.insert_block("a", None, "paragraph").unwrap();
c.insert_text("a", b.clone(), None, "hello");
c.insert_text_at_offset("a", b.clone(), 2, "XX");
assert_eq!(c.projection.blocks[0].text, "heXXllo");
c.insert_text_at_offset("a", b.clone(), 0, ">");
assert_eq!(c.projection.blocks[0].text, ">heXXllo");
let len = ">heXXllo".chars().count();
c.insert_text_at_offset("a", b.clone(), len, "<");
assert_eq!(c.projection.blocks[0].text, ">heXXllo<");
// An insert addressed to an unknown block is ACCEPTED (CRDT store
// tolerance: the block op may still be in flight from a peer) but
// materializes nothing visible.
assert!(
c.insert_text_at_offset("a", OpId { actor: "z".into(), counter: 99 }, 1, "q")
.is_some(),
"unknown-block inserts stay in the op log for convergence"
);
assert_eq!(c.projection.blocks.len(), 1, "nothing projects from a block that never arrives");
}
/// `delete_text_at_offset` addresses one atom in each direction and
/// rejects every out-of-range/guard case instead of guessing.
#[test]
fn delete_text_at_offset_deletes_backward_and_forward_with_guards() {
let mut c = DocumentController::default();
let b = c.insert_block("a", None, "paragraph").unwrap();
c.insert_text("a", b.clone(), None, "abc");
assert!(c.delete_text_at_offset("a", b.clone(), 2, true));
assert_eq!(c.projection.blocks[0].text, "ac", "backward at 2 removes 'b'");
assert!(c.delete_text_at_offset("a", b.clone(), 0, false));
assert_eq!(c.projection.blocks[0].text, "c", "forward at 0 removes 'a'");
assert!(!c.delete_text_at_offset("a", b.clone(), 0, true), "nothing before offset 0");
assert!(
!c.delete_text_at_offset("a", b.clone(), 1, false),
"forward past the last atom finds no target"
);
assert!(
!c.delete_text_at_offset("a", OpId { actor: "z".into(), counter: 9 }, 1, true),
"unknown block"
);
}
/// SetBlockAlignment reaches the projection (the materialization arm a
/// plain text run never touches).
#[test]
fn block_alignment_is_materialized() {
let mut c = DocumentController::default();
let b = c.insert_block("a", None, "paragraph").unwrap();
c.insert_text("a", b.clone(), None, "align me");
assert!(c.set_block_alignment("a", b, "center"));
assert_eq!(c.projection.blocks[0].alignment, "center");
// Alignment on an unknown block is accepted into the log (CRDT
// tolerance) but materializes against nothing.
assert!(c.set_block_alignment("a", OpId { actor: "z".into(), counter: 9 }, "right"));
assert_eq!(c.projection.blocks.len(), 1);
}
/// Styling an empty block stores a span over the empty range instead of
/// discarding the op or panicking (the run-assembly arm a populated
/// block never reaches).
#[test]
fn style_patch_on_empty_block_is_retained() {
let mut c = DocumentController::default();
let b = c.insert_block("a", None, "paragraph").unwrap();
assert!(c.set_text_style_at_offsets(
"a",
b.clone(),
0,
0,
doc_engine::projection::TextStylePatch { bold: Some(true), ..Default::default() }
));
assert_eq!(c.projection.blocks[0].text, "");
// Text typed afterwards materializes normally on top of the styled range.
c.insert_text("a", b, None, "x");
assert_eq!(c.projection.blocks[0].text, "x");
}
/// A multi-cell batch is one undoable group: every write comes back
/// together on undo and reapplies together on redo.
#[test]
fn batched_cell_writes_undo_as_one_group() {
let mut c = DocumentController::default();
let t = c.insert_table("a", None).unwrap();
let r = c.insert_table_row("a", t.clone(), None).unwrap();
let c0 = c.insert_table_column("a", t.clone(), None).unwrap();
let c1 = c.insert_table_column("a", t.clone(), Some(c0.clone())).unwrap();
c.set_table_cell("a", t.clone(), r.clone(), c0.clone(), "old0");
assert!(c.set_table_cells(
"a",
t.clone(),
vec![
(r.clone(), c0.clone(), "new0".to_string()),
(r.clone(), c1.clone(), "new1".to_string())
]
));
let tid = format!("{}:{}", t.actor, t.counter);
let k0 = (format!("{}:{}", r.actor, r.counter), format!("{}:{}", c0.actor, c0.counter));
let k1 = (format!("{}:{}", r.actor, r.counter), format!("{}:{}", c1.actor, c1.counter));
assert_eq!(c.projection.tables[&tid].cells[&k0], "new0");
assert_eq!(c.projection.tables[&tid].cells[&k1], "new1");
assert!(c.undo("a"));
assert_eq!(c.projection.tables[&tid].cells[&k0], "old0");
assert_eq!(
c.projection.tables[&tid].cells.get(&k1).map(String::as_str),
Some(""),
"the single group undo restores both cells"
);
assert!(c.redo("a"));
assert_eq!(c.projection.tables[&tid].cells[&k0], "new0");
assert_eq!(c.projection.tables[&tid].cells[&k1], "new1");
}
/// The batch writer refuses an empty write list outright; writes to
/// cells that do not exist yet are accepted into the op log (CRDT store
/// tolerance: the row/column ops may still be in flight from a peer) but
/// project nothing, and the batch reports its real write count.
#[test]
fn batched_cell_writes_reject_empty_and_tolerate_stray_writes() {
let mut c = DocumentController::default();
let t = c.insert_table("a", None).unwrap();
assert!(!c.set_table_cells("a", t.clone(), vec![]));
let ghost = OpId { actor: "z".into(), counter: 9 };
assert!(
c.set_table_cells("a", t.clone(), vec![(ghost.clone(), ghost.clone(), "x".to_string())]),
"stray cell writes are logged, not lost"
);
let tid = format!("{}:{}", t.actor, t.counter);
let ghost_key = (
format!("{}:{}", ghost.actor, ghost.counter),
format!("{}:{}", ghost.actor, ghost.counter),
);
let table = &c.projection.tables[&tid];
assert!(
table.rows.is_empty() && table.columns.is_empty(),
"stray writes never conjure rows or columns"
);
assert_eq!(
table.cells[&ghost_key],
"x",
"the text itself is retained under the stray key, so it still merges if the cell ever arrives"
);
}
/// The document wire round trip (`to_json`/`from_json`) is the save
/// format every workspace rides on; pin that a full document survives it
/// byte-identical in the projection.
#[test]
fn document_wire_round_trip_preserves_projection() {
let mut c = DocumentController::default();
let b = c.insert_block("a", None, "paragraph").unwrap();
c.insert_text("a", b.clone(), None, "wire");
let t = c.insert_table("a", Some(b)).unwrap();
let r = c.insert_table_row("a", t.clone(), None).unwrap();
let col = c.insert_table_column("a", t.clone(), None).unwrap();
c.set_table_cell("a", t.clone(), r.clone(), col.clone(), "cell(x)");
let json = c.document.to_json().expect("serialize");
let doc = CrdtDocument::from_json(&json).expect("deserialize");
let projection = doc.materialize();
assert_eq!(projection.blocks[0].text, "wire");
let tid = format!("{}:{}", t.actor, t.counter);
let key = (format!("{}:{}", r.actor, r.counter), format!("{}:{}", col.actor, col.counter));
assert_eq!(projection.tables[&tid].cells[&key], "cell(x)");
assert_eq!(projection.blocks.len(), c.projection.blocks.len());
}
/// Text-style toggles reject unknown fields (rather than writing a
/// no-field patch) and a block that is not projected at all.
#[test]
fn toggle_text_style_rejects_unknown_field_and_missing_block() {
let mut c = DocumentController::default();
let b = c.insert_block("a", None, "paragraph").unwrap();
c.insert_text("a", b.clone(), None, "ab");
assert!(!c.toggle_text_style_at_offsets("a", b, 0, 2, "superscript"));
assert!(!c.toggle_text_style_at_offsets(
"a",
OpId { actor: "z".into(), counter: 9 },
0,
1,
"bold"
));
}

View file

@ -1498,3 +1498,27 @@ the 4x3/12-cell table with bold header), and four persistence tests
over temp dirs covering the round trip, store-beats-manifest
precedence, the manifest fallback, and empty-file rejection.
`DEVICE_VERIFICATION.md` section 9 gained the matching hardware rows.
## Engine source coverage (gated)
The doc engine now has what the CAD engine got first: a measured,
gated coverage number instead of an assertion.
`tools/test-doc-engine-coverage.sh` runs the crate's unit tests plus
`tests/materialize.rs` under `-C instrument-coverage` in an isolated,
self-deleting environment and enforces a total floor (96% lines)
against a measured baseline of 99.00% (97.54% regions), with per-file
floors so losing one module's tests cannot hide inside the total. The
harness needed no shim layer: doc-engine is UI-free (serde +
serde_json), which is also why the whole run takes seconds. The run
report named real gaps, closed in the same tranche: offset-addressed
text insert/delete, block alignment materialization, batched cell
group undo/redo, the `#MP_CRDT_V1` wire round trip, and
toggle/batch-reject guards. Two assertions came back inverted and were
pinned as DOCUMENTED behavior instead: writes and style ops
addressed to blocks or cells whose anchors have not arrived are
accepted into the op log (CRDT store tolerance — they must merge when
the anchor lands) while conjuring no blocks, rows or columns into the
rendered document. The baseline, the
exclusions, and what the number does not mean live in
`crates/apps/doc/doc-engine/COVERAGE.md`; the gate runs in the
doc-engine workflow.

214
tools/test-doc-engine-coverage.sh Executable file
View file

@ -0,0 +1,214 @@
#!/usr/bin/env bash
# Temporary LLVM source-coverage run for the standalone CRDT doc engine.
#
# WHAT THIS COVERS
# ----------------
# `doc-engine` is the pure engine the doc workspace builds on: the CRDT
# document/order core, the projection (styled runs, tables, nodes), the
# controller op surface, undo/redo history and the session wire format.
# It is dependency-free beyond serde + serde_json, so unlike nigig-build
# it can be instrumented DIRECTLY -- no host-only shim crate, no Makepad
# checkout, no native packages. The harness copies the crate (sources
# plus tests/materialize.rs, the real integration suite) into a scratch
# directory outside the workspace and runs it under
# -C instrument-coverage.
#
# WHAT IT EXCLUDES FROM THE REPORT (step 4 of the coverage plan)
# - the cargo registry / git dirs -- third-party code
# - the rustc sysroot -- std
# There is no Makepad generated code and no platform startup in this
# crate at all: it is a plain `lib` target, which is exactly why it can
# be measured without the CAD-style shim gymnastics.
#
# USAGE
# ./tools/test-doc-engine-coverage.sh # run, enforce floors, clean up
# KEEP_COVERAGE=1 ./tools/test-doc-engine-coverage.sh # keep env + uncovered lines
# DOC_ENGINE_COVERAGE_REPORT_ONLY=1 ./tools/test-doc-engine-coverage.sh # measure, don't gate
#
# Everything -- toolchain, cargo home, target dir, profraw data, the
# crate copy and the report -- lives under a single mktemp directory a
# shell trap removes on success, failure, interrupt or termination
# (step 7 of the coverage plan: nothing is installed into the host,
# nothing is written into the repository or left in $HOME).
set -Eeuo pipefail
IFS=$'\n\t'
ROOT="$(CDPATH= cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"
# Floors, set a couple of points under the measured baseline so ordinary
# refactoring does not trip them while a real loss of coverage does.
# A single total hides the failure this is meant to catch: losing every
# test in one module moves the total by a point or two and a lone number
# would wave that through. Lowering a floor is a reviewable edit here,
# not something to do quietly.
#
# Baseline (2026-08-17, llvm-cov lines percent):
# controller 98.68 / crdt-document 97.63 / crdt-operations 100 /
# version_vector 100 / history 92.00 / projection-text 100 / TOTAL 98.99.
# lib.rs, session.rs, crdt/mod.rs and projection/mod.rs carry no
# executable lines (re-export and module files), so llvm-cov never
# reports them and they must NOT appear here: an unmeasured file with a
# floor fails the run by design.
#
# history.rs is the honest floor: Compensation::inverse arms unreachable
# through the public controller API (undo/redo decompose groups before
# materializing) sit in its remaining lines.
TOTAL_FLOOR="${DOC_ENGINE_COVERAGE_TOTAL_FLOOR:-96}"
PER_FILE_FLOORS="${DOC_ENGINE_COVERAGE_PER_FILE_FLOORS:-\
src/controller.rs:96
src/history.rs:89
src/crdt/document.rs:95
src/crdt/operations.rs:97
src/crdt/version_vector.rs:97
src/projection/text.rs:97}"
TOOLCHAIN="$(sed -n 's/^channel = "\(.*\)"/\1/p' "$ROOT/rust-toolchain.toml")"
HOST_TRIPLE="${DOC_ENGINE_COV_HOST:-x86_64-unknown-linux-gnu}"
# Same TMPDIR reasoning as tools/test-spreadsheet-coverage.sh: a mktemp
# default under $HOME/.cache -- roomy, outside the workspace, and removed
# on every exit path.
DEFAULT_TMP="${HOME:-/var/tmp}/.cache/nigig-coverage"
mkdir -p "${TMPDIR:-$DEFAULT_TMP}"
WORK="$(mktemp -d "${TMPDIR:-$DEFAULT_TMP}/doc-engine-coverage.XXXXXXXX")"
KEEP_COVERAGE="${KEEP_COVERAGE:-0}"
cleanup() {
local status=$?
if [[ "$KEEP_COVERAGE" == "1" ]]; then
echo "coverage environment retained: $WORK" >&2
else
rm -rf -- "$WORK"
rmdir "$DEFAULT_TMP" 2>/dev/null || true
rmdir "${HOME:-/var/tmp}/.cache" 2>/dev/null || true
echo "cleaned isolated coverage environment" >&2
fi
exit "$status"
}
trap cleanup EXIT HUP INT TERM
CRATE="$ROOT/crates/apps/doc/doc-engine"
# ---------------------------------------------------------------------------
# 1. Isolated toolchain with the coverage instrumentation components
# ---------------------------------------------------------------------------
export RUSTUP_HOME="$WORK/rustup"
export CARGO_HOME="$WORK/cargo"
export CARGO_TARGET_DIR="$WORK/target"
export PATH="$CARGO_HOME/bin:$PATH"
export LLVM_PROFILE_FILE="$WORK/profiles/%p-%m.profraw"
export RUSTFLAGS="-C instrument-coverage -C codegen-units=1 -C opt-level=0"
mkdir -p "$WORK/profiles"
curl --fail --location https://sh.rustup.rs -o "$WORK/rustup-init"
chmod 700 "$WORK/rustup-init"
"$WORK/rustup-init" -y --profile minimal --default-toolchain "$TOOLCHAIN" \
--component llvm-tools-preview --no-modify-path
# ---------------------------------------------------------------------------
# 2. The crate copy, outside the workspace (a copy inside the repository
# would be seen as a workspace member and refuse to build; there is no
# crate-local lockfile -- the workspace root owns the lock -- so plain
# `cargo test` re-resolves serde/serde_json, a two-crate registry hit).
# ---------------------------------------------------------------------------
COPY="$WORK/doc-engine"
mkdir -p "$COPY"
cp "$CRATE/Cargo.toml" "$COPY/"
cp -r "$CRATE/src" "$COPY/src"
cp -r "$CRATE/tests" "$COPY/tests"
# ---------------------------------------------------------------------------
# 3. Instrumented run: unit tests (in-file #[cfg(test)]) + tests/materialize.rs
# ---------------------------------------------------------------------------
cargo test --manifest-path "$COPY/Cargo.toml" --all-targets
# ---------------------------------------------------------------------------
# 4. Report
# ---------------------------------------------------------------------------
LLVM_BIN="$RUSTUP_HOME/toolchains/$TOOLCHAIN-$HOST_TRIPLE/lib/rustlib/$HOST_TRIPLE/bin"
"$LLVM_BIN/llvm-profdata" merge -sparse "$WORK/profiles"/*.profraw \
-o "$WORK/coverage.profdata"
mapfile -t BINS < <(find "$CARGO_TARGET_DIR/debug/deps" -maxdepth 1 -type f -executable \
\( -name 'doc_engine-*' -o -name 'materialize-*' \) ! -name '*.d')
[[ ${#BINS[@]} -gt 0 ]] || { echo "no instrumented test binaries found"; exit 1; }
OBJECTS=("${BINS[0]}")
for b in "${BINS[@]:1}"; do OBJECTS+=(-object "$b"); done
# The explicit source list does double duty: only the engine sources and
# the integration suite contribute to TOTAL, so the number cannot be
# diluted by anything that happens to sit under $WORK, and a file dropped
# from src/ shows up as a missing floor measurement below.
IGNORE="(/cargo/registry|/cargo/git|/rustc/)"
SOURCES=()
while IFS= read -r f; do SOURCES+=("$COPY/$f"); done < <(
cd "$CRATE" && find src tests -name '*.rs' | sort
)
"$LLVM_BIN/llvm-cov" report "${OBJECTS[@]}" \
-instr-profile="$WORK/coverage.profdata" -ignore-filename-regex="$IGNORE" \
"${SOURCES[@]}"
if [[ "$KEEP_COVERAGE" == "1" ]]; then
"$LLVM_BIN/llvm-cov" show "${OBJECTS[@]}" \
-instr-profile="$WORK/coverage.profdata" -ignore-filename-regex="$IGNORE" \
"${SOURCES[@]}" \
| grep -E '^ *[0-9]+\| *0\|' > "$WORK/uncovered-lines.txt" || true
echo "uncovered line report: $WORK/uncovered-lines.txt" >&2
fi
# ---------------------------------------------------------------------------
# 5. Enforce the floors
# ---------------------------------------------------------------------------
"$LLVM_BIN/llvm-cov" export "${OBJECTS[@]}" \
-instr-profile="$WORK/coverage.profdata" -ignore-filename-regex="$IGNORE" \
"${SOURCES[@]}" > "$WORK/coverage.json"
if [[ "${DOC_ENGINE_COVERAGE_REPORT_ONLY:-0}" == "1" ]]; then
echo 'report-only mode: the floors were not enforced'
exit 0
fi
python3 - "$WORK/coverage.json" "$TOTAL_FLOOR" "$PER_FILE_FLOORS" <<'PY'
import json, sys
path, total_floor, per_file = sys.argv[1], float(sys.argv[2]), sys.argv[3]
with open(path) as fh:
data = json.load(fh)
export = data["data"][0]
total = export["totals"]["lines"]["percent"]
measured = {f["filename"]: f["summary"]["lines"]["percent"] for f in export["files"]}
failures = []
if total < total_floor:
failures.append(f" total {total:.2f}% is below the floor of {total_floor:.2f}%")
for line in per_file.split():
if not line.strip():
continue
name, _, floor = line.rpartition(":")
floor = float(floor)
hits = [v for k, v in measured.items() if k.endswith("/doc-engine/" + name)]
if not hits:
failures.append(
f" {name} has a floor but was not measured -- was it renamed, "
"deleted, or dropped from sources? A floor on a file that "
"is not measured silently protects nothing.")
continue
if hits[0] < floor:
failures.append(
f" {name} {hits[0]:.2f}% is below its floor of {floor:.2f}%")
if failures:
print("coverage floors not met:", file=sys.stderr)
print("\n".join(failures), file=sys.stderr)
print(
"\nEither the change removed tested code, or it added untested code.\n"
"Lowering a floor is a reviewable edit to tools/test-doc-engine-coverage.sh,\n"
"not something to do quietly.",
file=sys.stderr,
)
sys.exit(1)
print(f"all coverage floors met (total {total:.2f}%)")
PY