andodeki
0596fc66ed
The number nobody had. The engine harness is structurally blind to viewport*.rs, workspace*.rs and friends, so "97.14% covered" has always been a statement about the smaller half. With the Makepad Linux packages installed, the real crate builds and its tests run under -C instrument-coverage, which makes the other half measurable in about six minutes. 10,714 lines. 13.15% covered. 9,305 never executed by any test. viewport_input.rs 736 lines 0.00% viewport_render.rs 2,083 lines 0.00% workspace_actions.rs 690 lines 0.00% cad_editor_sheet.rs 224 lines 0.00% viewport_2d.rs 138 lines 0.00% code_editor.rs 67 lines 0.00% viewport.rs 3,548 lines 14.37% workspace.rs 2,462 lines 14.18% script_bindings.rs 724 lines 71.27% viewport_input.rs is every click, drag, modifier and keystroke the editor handles, and not one line of it has ever run in a test. viewport_render.rs is every draw call. This reframes the six sessions of engine work above it. 1044 green tests and a 97% engine coexist with an input layer nothing has touched. Those facts were never in tension — they were just never on the same page, because the tool that produced the good number could not see the bad one. A coverage figure that excludes the risky half is not a summary, it is an average with the interesting term deleted. The new script reports the widget files ONLY, on purpose. Folding them into one number would let a 97% engine hide a 0% input layer, which is the arithmetic this exists to prevent. No floors yet, deliberately: a floor at 13% reads as a blessing rather than a debt. The first real input test should set one behind it. Verified: two runs, cold and warm, same numbers; script cleans its profraw data on exit and refuses with a useful message when the native packages or llvm-tools are missing.
138 lines
5.1 KiB
Bash
Executable file
138 lines
5.1 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# Source coverage for the CAD *widget* layer — the half
|
|
# tools/test-cad-coverage.sh cannot see.
|
|
#
|
|
# WHY THIS EXISTS
|
|
#
|
|
# The host-only harness measures the fourteen pure engine files by
|
|
# copying them into a crate with no Makepad in it. That is fast, needs no
|
|
# root, and is what the CI gate uses — but it is structurally blind to
|
|
# viewport*.rs, workspace*.rs, mod.rs, script_bindings.rs,
|
|
# cad_editor_sheet.rs and code_editor.rs, which are the files that handle
|
|
# every click, drag, keystroke and draw call in the editor.
|
|
#
|
|
# Those were called unmeasurable for a long time, including by the
|
|
# tooling's own documentation. They are not. They need the Makepad Linux
|
|
# packages — not a desktop, not a GPU, not a display server. With those
|
|
# installed, `cargo test -p nigig-build --lib` builds and runs, and it
|
|
# runs just as happily under -C instrument-coverage.
|
|
#
|
|
# The first run of this script reported 13.15% of lines over those ten
|
|
# files, with viewport_input.rs, viewport_render.rs, workspace_actions.rs,
|
|
# viewport_2d.rs, cad_editor_sheet.rs and code_editor.rs at exactly zero.
|
|
#
|
|
# COST
|
|
#
|
|
# ~6 minutes single-job from cold, most of it compiling the Makepad graph
|
|
# with instrumentation. Unlike the engine harness this needs apt packages
|
|
# and therefore root, which is why it is a separate script and not a flag
|
|
# on the other one.
|
|
#
|
|
# USAGE
|
|
#
|
|
# bash tools/makepad-native-libs.sh --check # what is missing
|
|
# bash tools/makepad-native-libs.sh --install # opt-in, apt-based
|
|
# ./tools/test-cad-widget-coverage.sh
|
|
#
|
|
# CAD_WIDGET_KEEP=1 ./tools/test-cad-widget-coverage.sh # keep profraw
|
|
# + line report
|
|
#
|
|
# Deliberately NOT gated with floors yet. A floor at 13% reads as a
|
|
# blessing; the number is here to be driven up, and the first person to
|
|
# add a real input test should set one behind them.
|
|
set -Eeuo pipefail
|
|
IFS=$'\n\t'
|
|
|
|
ROOT="$(CDPATH= cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
CAD_REL="crates/apps/nigig-build/src/construction_frame/pages/workspace/cad"
|
|
TOOLCHAIN="$(sed -n 's/^channel = "\(.*\)"/\1/p' "$ROOT/rust-toolchain.toml")"
|
|
HOST_TRIPLE="${CAD_WIDGET_HOST:-x86_64-unknown-linux-gnu}"
|
|
WORK="$(mktemp -d "${TMPDIR:-/tmp}/cad-widget-coverage.XXXXXXXX")"
|
|
KEEP="${CAD_WIDGET_KEEP:-0}"
|
|
|
|
cleanup() {
|
|
local status=$?
|
|
if [[ "$KEEP" == "1" ]]; then
|
|
echo "kept: $WORK" >&2
|
|
else
|
|
rm -rf "$WORK"
|
|
echo "cleaned temporary coverage data" >&2
|
|
fi
|
|
exit "$status"
|
|
}
|
|
trap cleanup EXIT HUP INT TERM
|
|
|
|
# The build cache is reused by default: a cold instrumented build of the
|
|
# Makepad graph is minutes, and throwing it away every run would make
|
|
# this unusable in a loop. Point it somewhere else, or at $WORK, to get
|
|
# a hermetic run.
|
|
export CARGO_TARGET_DIR="${CAD_WIDGET_TARGET_DIR:-$WORK/target}"
|
|
export LLVM_PROFILE_FILE="$WORK/profiles/%p-%m.profraw"
|
|
export RUSTFLAGS="-C instrument-coverage"
|
|
export CARGO_NET_GIT_FETCH_WITH_CLI=true
|
|
export CARGO_BUILD_JOBS="${CAD_WIDGET_JOBS:-1}"
|
|
mkdir -p "$WORK/profiles"
|
|
|
|
if ! command -v cargo >/dev/null 2>&1; then
|
|
echo "ERROR: cargo is not on PATH." >&2
|
|
echo " Unlike tools/test-cad-coverage.sh this script does not install a" >&2
|
|
echo " toolchain: it builds the real crate, so it uses yours." >&2
|
|
exit 1
|
|
fi
|
|
|
|
if ! pkg-config --exists wayland-client x11 alsa 2>/dev/null; then
|
|
echo "ERROR: the Makepad Linux packages are missing." >&2
|
|
echo " bash tools/makepad-native-libs.sh --check" >&2
|
|
echo " bash tools/makepad-native-libs.sh --install" >&2
|
|
exit 1
|
|
fi
|
|
|
|
cd "$ROOT"
|
|
cargo test --locked -p nigig-build --lib
|
|
|
|
LLVM_BIN="$(rustc --print sysroot)/lib/rustlib/$HOST_TRIPLE/bin"
|
|
if [[ ! -x "$LLVM_BIN/llvm-profdata" ]]; then
|
|
echo "ERROR: llvm-tools-preview is missing from the active toolchain." >&2
|
|
echo " rustup component add llvm-tools-preview --toolchain $TOOLCHAIN" >&2
|
|
exit 1
|
|
fi
|
|
|
|
"$LLVM_BIN/llvm-profdata" merge -sparse "$WORK/profiles"/*.profraw \
|
|
-o "$WORK/widget.profdata"
|
|
|
|
BIN="$(find "$CARGO_TARGET_DIR/debug/deps" -maxdepth 1 -type f -executable \
|
|
-name 'nigig_build-*' ! -name '*.d' -printf '%T@ %p\n' \
|
|
| sort -rn | head -1 | cut -d' ' -f2-)"
|
|
[[ -n "$BIN" ]] || { echo "no instrumented test binary found"; exit 1; }
|
|
|
|
# The widget files only. The engine files have their own report with
|
|
# floors; mixing them here would let a 97% engine hide a 0% input layer,
|
|
# which is exactly the arithmetic this script exists to prevent.
|
|
WIDGET_FILES=(
|
|
mod.rs
|
|
viewport.rs
|
|
viewport_input.rs
|
|
viewport_render.rs
|
|
viewport_2d.rs
|
|
workspace.rs
|
|
workspace_actions.rs
|
|
script_bindings.rs
|
|
cad_editor_sheet.rs
|
|
code_editor.rs
|
|
)
|
|
SOURCES=()
|
|
for f in "${WIDGET_FILES[@]}"; do
|
|
SOURCES+=("$ROOT/$CAD_REL/$f")
|
|
done
|
|
|
|
echo
|
|
echo "=== CAD widget layer — the half the engine harness cannot see"
|
|
echo
|
|
"$LLVM_BIN/llvm-cov" report "$BIN" -instr-profile="$WORK/widget.profdata" \
|
|
"${SOURCES[@]}" | sed "s|$ROOT/$CAD_REL/||"
|
|
|
|
if [[ "$KEEP" == "1" ]]; then
|
|
"$LLVM_BIN/llvm-cov" show "$BIN" -instr-profile="$WORK/widget.profdata" \
|
|
"${SOURCES[@]}" | grep -E '^ *[0-9]+\| *0\|' > "$WORK/uncovered-lines.txt" || true
|
|
echo "uncovered line report: $WORK/uncovered-lines.txt" >&2
|
|
fi
|