678 commits

Author	SHA1	Message	Date
andodeki	3a3db4efa6	feat(robius-notification): iOS via UNUserNotificationCenter, and the Apple/Windows logic comes out of the cfg blocks ... Two gaps remained on the Apple and Windows side after the last commit, and they are different in kind. The first was a missing backend. iOS returned PermanentlyUnavailable with a note saying it was not implemented. It now uses UNUserNotificationCenter -- the only option there, since every iOS process is bundled, so the Objective-C exception that rules that API out on macOS cannot occur. The objc2-user-notifications bindings are real and fetchable, so this type-checks for aarch64-apple-ios against the actual framework, and the module was confirmed genuinely reachable by injecting a type error and watching the target build fail. Its is_available() returns a constant false, and that is a limit worth naming rather than burying. The real answer comes from getNotificationSettingsWithCompletionHandler:, which is asynchronous -- it hands the settings to a block on an arbitrary queue. A synchronous is_available() could only produce that by blocking on a completion handler, which on the main thread is a deadlock rather than a delay. Returning false errs toward telling the user delivery is unverified; the alternative is claiming an availability the platform never confirmed, which is the whole failure this crate was written to remove. A correct answer needs an async entry point, which is a change to the public API rather than a bug fix, so it is recorded in the ADR instead of being quietly wrong. The second gap is subtler and, I think, the more valuable fix. The Apple and Windows backends contain pure logic -- XML escaping, tag clamping -- that was sitting inside #[cfg(target_os = "windows")], where cargo test on the only available machine could never reach it. Those functions had zero tests and no prospect of any. They now live in src/payload.rs, which is cfg-free and runs on every target. Eleven tests cover them, and they cover exactly the rules a compiler cannot: an unescaped `&` in a merchant name makes the toast XML malformed and Windows discards the whole notification rather than showing a mangled character, and the Binance P2P book is full of `&`. A byte-wise truncation of the 64-character tag limit panics outright on the full-width names that book also contains -- so the clamp cuts on character boundaries, pinned by a test that would panic if anyone changed it back. One test pins something deliberately counter-intuitive: escaping is not idempotent, and escape_xml("&") is "&". That is correct, and the test exists so nobody "fixes" double-escaping by teaching the function to detect already-escaped input, which is precisely how escaping filters grow holes. The distinction this commit is really about: the last one made the Apple and Windows backends *compile*, which catches wrong selectors -- it found CreateToastNotifier(&HSTRING), which does not exist. It could not catch wrong behaviour. Extracting the logic is what makes the behaviour testable on a machine that will never run either platform. 62 tests, up from 51. Clippy clean with -D warnings on all five targets: linux-gnu, linux-android, apple-darwin, apple-ios, windows-msvc. p2p-intel unchanged at 225.	2026-09-01 20:10:50 +00:00
andodeki	3ef182414e	feat(robius-notification): real macOS and Windows backends, type-checked against the actual frameworks ... The previous commit refused to write these two, on the grounds that code no compiler has ever seen is not an implementation -- it is plausible-looking text that would sit in the same crate as tested code and be read as equally finished. That reasoning holds. The premise behind it did not. `cargo check` needs the *target's standard library*, not a linker or a platform SDK. `rustup target add x86_64-pc-windows-msvc aarch64-apple-darwin` puts the real `windows` and `objc2` crates -- genuine WinRT metadata, genuine Objective-C class definitions -- in front of the type checker on a Linux host. So both backends are now written and both compile against the frameworks they call. Compile-checking earned its place immediately. The Windows backend was written against `ToastNotificationManager::CreateToastNotifier(&HSTRING)`, which does not exist: the AUMID overload is `CreateToastNotifierWithId`. Nothing short of a compiler holding the real metadata would have caught that, and it would have shipped looking entirely correct. Because a cfg-gated module can silently compile to nothing -- leaving a green check that proves only that the module was skipped -- each backend was verified to be genuinely reachable by injecting a type error and confirming the target build failed. Both macOS and Windows were checked this way, then restored. Two platform decisions worth recording. macOS uses NSUserNotification, not UNUserNotificationCenter, and that is a deliberate downgrade to a deprecated API. `UNUserNotificationCenter.current()` raises an Objective-C exception when the process has no bundle identifier; that unwinds through Rust frames and aborts. A plain `cargo run` host has no bundle, so the modern API would crash the caller instead of reporting unavailable -- which is worse than deprecated. The nil check on `defaultUserNotificationCenter` is there for the same reason: msg_send on nil returns zero rather than crashing, so every later call would silently do nothing, which is precisely the failure this crate exists to remove. Windows requires the host to supply an AppUserModelID, because a library cannot invent one. It comes from an MSIX manifest or a Start Menu shortcut, and a fabricated id produces a notifier that constructs happily and then fails at Show. `set_app_user_model_id` is therefore public, a no-op off Windows so portable hosts call it unconditionally, and `is_available()` is false with a reason naming exactly what is missing until it is called. Toast payloads are XML, so text is escaped -- the same bug class as the Telegram MarkdownV2 escaping p2p-intel needed before it dropped Telegram, and a merchant nickname containing `&` is not hypothetical. The support table now has two columns, "compiles" and "executed", because they are different claims. All four backends compile; only Linux has posted a notification. iOS remains unavailable by design: it needs UN* with a bundle and an entitlement, which is an app-packaging concern rather than something this crate can satisfy. Off Windows the module defines no entry points at all rather than stubs. Clippy was right to call them dead: sys/mod.rs dispatches elsewhere, so they existed only to satisfy a symmetry nothing needs. CI gains a step that clippies all four cross-targets with -D warnings, so a cfg-gated backend cannot rot unnoticed -- which is exactly how the CreateToastNotifier mistake would have survived. 51 tests, clippy clean on five targets, p2p-intel still at 225.	2026-09-01 18:54:57 +00:00
andodeki	e16d1d1d44	feat(robius-notification): a zero-dependency D-Bus notifier, and p2p-intel's last limit closes on Linux ... ADR 0035 left one of p2p-intel's four limits open: no OS notification backend, so alerts stopped when the window closed. The seam existed with one implementation that truthfully did nothing. This is the crate that fills it. Posting a notification on Linux is one D-Bus method call. Three ways to make it were measured rather than assumed: notify-rust with libdbus is twelve crates but a C library, which needs pkg-config and breaks the Android and iOS cross-compile this crate family keeps clean; notify-rust with zbus is pure Rust and 169 crates including an async executor; writing the wire format out is about 250 lines and nothing at all. robius-sms deleted polkit and gio for exactly this reason -- its E9 note records they were the sole source of two RUSTSEC advisories and an LGPL question for every consumer -- so pulling a 169-crate tree back into the same family for one method call would reverse that decision for a worse reason. The cost of hand-rolling is that the protocol has to be exactly right, and a mistake makes the daemon disconnect with no diagnostic. That cost was paid in tests: the suite starts a private dbus-daemon per test and talks to it. This matters more than it sounds. The marshaller and the parser were written from the same reading of the specification, so them agreeing with each other proves only that I was consistently wrong or consistently right; only a third party can say which. It found three bugs no unit test would have. The first is the one worth dwelling on. Every error reply parsed as success. The header-field walk assumed all fields were strings, but REPLY_SERIAL is a u32, and reading its four bytes as a string length desynchronised the cursor so ERROR_NAME was never reached. `post` returned Ok against a bus with no notification service running. That is precisely the bug this crate was written to eliminate -- a notifier that reports success and delivers nothing -- reintroduced by accident inside its own parser. I cannot think of a stronger argument for testing against something you did not write. Second, is_available() was true on a bare bus, because NameHasOwner *succeeds* and answers false in its body; checking only for an error reported a working notifier on a machine with no notification daemon. Third, replies were not correlated. The bus sends NameAcquired unprompted right after Hello, so "read the next message" consumed a signal and treated it as the answer. Replies are now matched on REPLY_SERIAL, and a single read carrying several messages is walked rather than truncated. The suite also serialises every test that mutates DBUS_SESSION_BUS_ADDRESS behind a mutex. The variable is process-wide and cargo runs tests in parallel threads; three consecutive parallel runs are now green. --test-threads=1 would have made the failures go away too, and would have hidden a real hazard from whoever reads the file next. On the four platforms, honestly. Linux is implemented and tested. Android is implemented and *compiles* -- cargo check and clippy both pass for aarch64-linux-android -- but has never run on a device, and the module says so in its first paragraph. It handles the two things Android drops silently, missing POST_NOTIFICATIONS on API 33+ and a missing channel on API 26+, because both are the same accepted-and-discarded failure this crate exists to remove. Apple and Windows are deliberately not written. Neither could be compiled here -- no macOS or Windows toolchain and no way to add one -- and objc2 message sends or WinRT calls that no compiler has ever seen are not an implementation. They are plausible-looking text that would sit in the same crate as tested code and be read as equally finished. Both return PermanentlyUnavailable with a reason naming ADR 0036, and their module docs record the call sequence so the next person starts from a design rather than a blank file. The support table says "written" and "verified" in separate columns for the same reason. p2p-intel's dashboard now uses SystemNotifications instead of UnavailableNotifications. The latter stays: on a platform with no backend it is still the truthful answer, and a test needs something that reliably cannot deliver. Alerts are tagged per fiat so a market replaces its own previous notification rather than stacking -- a 30-second poll would otherwise fill the shade, and a full shade is what makes someone turn notifications off for the app entirely, which costs more than the feature is worth. Two new tests pin the invariant that a sink must never report delivery it did not achieve. CI gains a notifications job that installs dbus and sets ROBIUS_NOTIFICATION_REQUIRE_DBUS=1. The bus-backed tests skip when dbus-daemon is absent so the suite stays green on a bare machine, but a silent skip in CI would mean the integration tests quietly stopped running while the build stayed green. I verified the guard fails by hiding dbus-daemon behind a stub that exits 127. 50 tests in the new crate, 225 in p2p-intel, clippy clean on host and Android, and the app still starts under Xvfb.	2026-09-01 18:43:07 +00:00
andodeki	da326142bf	fix(pay-ui): stop re-setting ussd/phone-perm toggles on refresh so manual user state is preserved	2026-09-01 21:40:49 +03:00
andodeki	1d8b3a6053	merge(local): reapply local WIP onto merged main -- cad dashboard/explode/script_parts/xray merged with remote Phase-5 LOD, plus doc-ui extraction, spreadsheet xls-import, android ussd, camera and test work	2026-09-01 21:13:52 +03:00
andodeki	159f3c7fee	Merge remote-tracking branch 'origin/main'	2026-09-01 20:31:44 +03:00
andodeki	8bf62e2644	feat(p2p-intel): the exchange tab becomes spread intelligence, with rails, drift capture, and three of four limits closed ... The exchange tab in nigig-mpesa and nigig-pay was a swap mock-up -- Sell/Buy cards showing hardcoded ETH and AAVE at $38,409.24, wired to nothing -- above six rate labels, each built from the *first* cached advert for its exchange: Binance P2P: USDT BUY @ 129.92 KES min 1000 max 200000 via BANK That is a price, not an opportunity. It never compared the two sides of the book, said nothing about whether the counterparty could be dealt with, and ignored what it costs to move the money. Against the live KES book it would have shown a 134.60 advert from a merchant with three completed trades. Both pages now run the p2p-intel analyzer over the adverts the app already caches. No new endpoint, no new traffic, api.rs untouched -- the data was always there, nothing was being asked of it. The two files are byte-identical and CI now fails if they drift apart, which they already had once: data.rs differs between the apps by 188 lines of tests one copy has and the other does not. **Rails, because a spread alone is a lie.** 29 bps on a 10,000 KES trade is 29 KES of gross margin; M-Pesa Send Money costs 55 KES a leg, 110 round trip. The trade is deeply negative and the spread says nothing about it. rail.rs prices every route and ranks by what survives. M-Pesa fees come from the published Safaricom bands in robius_ussd::mpesa_bands -- real tariff rows, not estimates, and flat rather than percentage, which is exactly why the same spread is ruinous at 500 KES and fine at 200,000. Both legs are charged. An amount outside the tariff reports OutOfRange and an unconfigured bank reports Unknown; neither reports zero, because zero is a claim and it is the wrong one. Bank tariffs ship unconfigured for the same reason: every bank differs and there is no table to default to. **API drift capture.** These endpoints are internal and undocumented. When one changes the symptom is an empty panel -- indistinguishable from a quiet market -- and the response that broke it is gone by the time anyone looks. Every parse failure is now recorded with the payload excerpt that caused it and copyable as a plain-text report. Deduplicated, because a 30-second poll against a changed endpoint fails 120 times an hour and 120 identical rows is a log nobody reads; the excerpt is excluded from the equality check, since two responses differing only in advert ids are the same drift. The exported header states that it carries response excerpts only and never a request, credential or account number -- the type can only be constructed from a response body, so that is enforced rather than promised. Cutting the excerpt is done on character boundaries: Binance really returns names like ＢｅｎｎｙＢｏｓｓ and a byte slice would panic mid-character. The repo's other clipboard code (nigig-build's crdt_widget.rs) answers a Hit::TextCopy, which is the query-driven path the platform uses for Ctrl+C on a focused widget. Right for a text editor, wrong for a button exporting a report the user never selected, so this uses cx.copy_to_clipboard and confirms in the UI -- a copy button with no feedback is one people press three times. On the four limits, the honest scoreboard is two closed, one usefully sidestepped, one open. ADR 0035 records why, because rounding all four up to "addressed" would have been the easy write-up and the wrong one. The poll timer and the host clock are closed outright. cx.start_interval delivers the tick as an ordinary UI-thread event, so there is no runtime and nothing to join at shutdown, and the interval reads through effective_poll_seconds so the fifteen-second floor still applies -- a config file cannot be used to hammer a rate-limited endpoint, which a test pins by name. Staleness is now shown, because a price from four minutes ago is not a price. One ambiguity is recorded rather than hidden: last_scan_ms == 0 is the "never scanned" sentinel, so a scan whose timestamp genuinely is 0 reads as never. That only happens when the host clock is broken, which is exactly when the UI should not claim the data is current. The headless-backend limit is sidestepped, and the distinction matters enough to write down. Xvfb is a real X11 server that draws into memory, so the app gets the display it insists on and tools/test-p2p-app-smoke.sh runs the real binary end to end. That earned its place immediately: it caught `Row = <View> { ... }`, which is not valid in this fork's script language and which cargo build is entirely silent about, because script_mod! is parsed at *runtime* -- a broken widget tree compiles perfectly and then renders nothing. The gate greps for [E] in the log for that reason; the process exits 0. I verified it fails by reintroducing the bad syntax. What it still cannot do is drive widgets: makepad_test's Selector::id(..).click() needs the harness to own the event loop, so the six #[ignore]d interaction tests stay ignored. OS notifications stay open. There is no robius-notification crate beside robius-sms and robius-ussd, and Makepad exposes none on any target; building one means NotificationCompat on Android, UNUserNotification on iOS and a D-Bus call on Linux. What shipped is the seam plus one implementation named UnavailableNotifications -- named for what it is, because a DefaultNotifier that silently discarded every alert would read like a working feature at every call site. is_delivering() returns false and the UI shows the reason, so the user is told that alerts stop with the window instead of assuming they are covered. A no-op reporting success is the exact "declared versus delivered" failure ADR 0017 exists to prevent. 223 tests, coverage 97.03% with fifteen per-file floors -- seven of them new and all measured, after I noticed an earlier edit had silently failed to apply and the new files were being counted in the total but gated by nothing. CI gains the Xvfb smoke job and an exchange-tab job that builds both host apps and diffs their pages.	2026-09-01 16:07:43 +00:00
andodeki	fefde1ecca	refactor(p2p-intel): Makepad's HTTP stack, in-app alerts, and micro_serde -- serde and reqwest are gone ... Three changes that turn out to be one change: the app now uses the platform's own facilities instead of carrying its own. Networking moves to Cx::http_request, alerting moves into the UI, and deserialisation moves to makepad_micro_serde. The default dependency graph drops from serde + serde_derive + serde_json + toml + reqwest + rustls + tokio + hyper to **twenty-five crates total**, none of which is any of those. Networking follows nigig-mpesa/src/pages/exchange/api.rs: build an HttpRequest, hand it to Cx::http_request keyed by a LiveId, match the reply in handle_network_responses. There is no HTTP client, no TLS stack and no async runtime in this workspace any more. That is not only leanness -- on Android and iOS the platform stack is the only one that works without shipping a second TLS implementation, so the `live` feature that gated reqwest has been deleted rather than made default. Correlating replies is where the real trap was. A scan of five markets puts ten requests in flight and the replies come back in whatever order the network gives them, so the LiveId has to say which market and which side. RequestKey encodes both plus a generation counter and round-trips through a u64 with the high bit set, so a LiveId Makepad derived from a name is never decoded as a scan reply, and a late reply from a previous round is dropped instead of folded into fresh data. Six tests cover the codec, including the one that matters most: the two sides of one market must not share an id, or the second reply overwrites the first and every spread is measured against itself. The transport allowlist is lifted from the nigig-mpesa review, which reached the same conclusion I would have: Makepad exposes no certificate pinning -- its only TLS control is set_ignore_ssl_cert, which weakens verification -- so what is enforceable at this layer is that only HTTPS to p2p.binance.com can be dialled at all. The tests cover the two ways a naive check leaks: the lookalike host p2p.binance.com.evil.example, which passes any starts_with test, and the userinfo smuggle https://p2p.binance.com@evil.example/, which resolves to evil.example while reading as Binance. Telegram is gone, as asked, and the app alerts itself: a banner, an unread badge on the status line, an Alerts tab holding the history, and a chime. Removing it removes a bot token from the threat model entirely -- a token in a config file is a bot anyone who reads the file can drive -- and removes a second network dependency from a tool already gated on one endpoint. One test now records the *absence* of a bug rather than its fix: Twin_traders00 is a real merchant from the live capture, and its underscore previously had to be escaped or Telegram rejected the whole message with a 400 and delivered nothing. Rendering in our own UI deletes that failure mode, and the test asserts the name appears unescaped. The trade-off is stated in the README rather than glossed: an in-app alert only reaches you while the app is running. No OS notification is raised, so a minimised window is a missed alert. The chime is synthesised rather than bundled -- a two-note rising blip generated at the device's sample rate, which is a few dozen lines instead of an audio asset shipped on three platforms, and which can therefore be tested. It is, and the tests found the bugs you would expect from writing audio: a freshly rendered chime starts *finished* so opening the output does not announce itself at startup, both note edges fade so neither clicks, the tail pads with silence rather than replaying whatever the buffer last held, and a nonsense sample rate falls back instead of panicking. Rising rather than falling because a falling interval reads as a dismissal and this is an invitation to act. The micro_serde migration surfaced two behaviours that differ from serde and both bit before they were understood. **micro_serde is strict by default.** deserialize_json errors on the first key it does not model. Binance sends about forty fields per advert and we model eight, so the strict parser cannot read the response at all -- and for a config file it means an older build cannot open a file written by a newer one. Everything uses deserialize_json_lenient, and a test pins that the strict call *would* have failed, because the two differ by one word and the strict one looks more correct. **There is no #[serde(default)].** Optional config entries are modelled as Option<T> on a Raw* struct and resolved into the real Config by hand. A few more lines in exchange for two fewer dependency trees. config.toml became config.json for the same arithmetic: micro_serde has no TOML reader, and toml depends on serde, so a single config file would have dragged all of serde back in through the back door. Also worth recording: DeJsonErr implements Debug but not Display, and Debug is the variant carrying line and column, so every error path formats it with {e:?} deliberately rather than by accident. CI gains a check that fails the build if reqwest, tokio, rustls, hyper, serde, serde_derive, serde_json or toml reappears in the default graph. I verified it fails by adding serde back to p2p-core and watching it match serde_derive, then reverted. A gate that cannot fail is decoration. 136 tests in the default feature set and 36 more with --features ui, including the RequestKey codec, which cannot be measured by the coverage job because it lives behind the feature that needs Makepad. Coverage 96.92% with ten per-file floors, up from eight -- chime.rs at 100% and client.rs at 95.69% are new. The desktop binary was built and linked to prove the app half is real.	2026-09-01 09:37:20 +00:00
andodeki	eaeebd3910	feat(p2p-intel): Binance P2P spread intelligence, shaped by what the live book actually contains ... A new nested workspace under crates/apps/p2p-intel: six engine crates, a CLI, and a Makepad dashboard that builds for desktop, Android and iOS. It reads public P2P adverts, measures the spread that is actually fillable, alerts when one is worth acting on, and tracks what the float really cost. It never places an order. Binance publishes no P2P trading API, and automating an escrow release is how a merchant loses their float to chargeback fraud. This is the intelligence layer; execution stays manual. The design came from a live capture rather than a sketch, and the capture contradicted the sketch three times. All three are now pinned by tests against checked-in real payloads. **The best price is routinely the least fillable one.** In the KES book the top sell advert was 134.60 from a merchant with three completed trades, implying a 3.6% spread; the next was 130.30. Another advert showed a 0% completion rate. A best-price scan with no quality floor does not find opportunities, it finds outliers, and outliers on a P2P book are bait or a merchant about to run dry. QualityFilter defaults to 95% completion and 50 orders, and analyse() reports every exclusion with its reason rather than dropping it silently. The honest consequence is recorded in the integration suite: at those defaults **not one sell-side advert in the captured KES book qualified**. There was no fillable arbitrage. A tool that reported the raw best-price number would have sent its user after a trade that does not exist, so the test asserts best_sell is None and net_bps is None rather than asserting a comfortable number. **tradeType is inverted between request and response.** Asking the endpoint for tradeType "BUY" returns adverts whose own adv.tradeType reads "SELL". Both are correct: the request parameter is what you want to do, the response field is what the advertiser is doing. Conflating them inverts every spread and the result still looks plausible, which makes it the most expensive mistake available here. Side keeps the two apart with request_trade_type()/advert_trade_type(), and a test asserts they are never equal. **An empty market answers HTTP 200 with success: true.** NGN returned zero adverts. "No ads" and "no answer" need opposite responses, so is_empty_market() is a named predicate and ScanError separates Malformed (Binance changed the payload; retrying makes it worse) from Network (transient). basis_points_above returns None against a zero base rather than an infinity, so an empty book cannot read as an infinite opportunity at 3am. Money is never a float, following the rule in nigig-pay-domain/src/money.rs. IEEE 754 cannot represent 0.1 and a spread is a difference of two nearly equal numbers, which is exactly where binary floating point loses the digits that matter. Binance sends prices as decimal strings, so Price parses them straight into scaled i128 integers and never passes through f64. i128 rather than i64 because the intermediate in a bps calculation overflows, not the result. Excess precision is refused rather than rounded and a thousands separator is refused rather than dropped: "1,299.92" read as 129992 is a 1000x error that still looks like a price. Tests pin 0.1 + 0.2 == 0.3 and rotate 100 round trips at one price asserting exactly zero P&L. Alerting is mostly restraint. At a 30-second poll one wide spread would fire 120 identical messages an hour, and a channel that cries wolf gets muted, at which point the tool has negative value because the user believes they are covered. AlertGate suppresses repeats inside a cooldown and re-alerts early only when the spread improves materially -- a collapsing spread is not worth waking someone for. Telegram MarkdownV2 escaping is tested against a real merchant name from the capture, Twin_traders00, whose underscore would otherwise make Telegram reject the message with a 400 and deliver nothing. Writing the dashboard view model found a bug in my own comparator: sorting descending by swapping the tuple to (b, a) also silently swaps the meaning of the None arms, which put dead markets at the top of the opportunity list. The test that caught it was written first and named for the behaviour, not the implementation. Networking is behind a non-default `live` feature, so an ordinary cargo test cannot make a request and CI never depends on Binance being reachable. A CI step asserts reqwest is absent from the default dependency graph so this cannot regress quietly. A live scan was run once to confirm the fixtures match reality; it reported a negative spread for KES and an empty NGN book, which is the tool working correctly. Conventions follow the repo rather than the generic layout in the request: .forgejo/workflows/p2p-intel.yml rather than .github, and no Dockerfile, since the stack is pure Rust and nothing else here is containerised. error_set is used instead of anyhow, matching nigig-core. The root Cargo.toml excludes the nested workspace by name, as it already does for makepad_table, so the isolation is intentional rather than dependent on a table inside someone else's manifest. 106 tests, coverage 96.86% with per-file floors enforced by tools/test-p2p-coverage.sh. Both the total and per-file gates were verified to actually fail by running them with impossible floors; a gate that cannot fail is decoration. Two files are excluded and only because they were first emptied of decisions: the Makepad widget, which needs a GPU and a windowing backend this repo has no headless backend for, and the CLI main, which is argument parsing and println. Every rule the widget renders lives in view_model.rs, measured at 97%. That split is deliberate -- spreadsheet-ui/grid.rs once hid 36 pure functions behind a file-level exclusion, and excluding a file you have not emptied of logic is how that happens. The Makepad desktop binary was built and linked in the sandbox to prove the app half is real and not just a compiling stub.	2026-09-01 09:13:43 +00:00
andodeki	c7c0ec7582	makepad: centralize fork deps in workspace + bump to 4a166606c ... Declare the 12 makepad crate deps once in root [workspace.dependencies] pointing at the gitdab fork rev 4a166606c (which now also carries [workspace.dependencies] into the android wrapper manifest). Member crates switch to workspace = true; 5 non-member workspaces get an inline rev bump. Fix 9 nigig-app script_mod indent errors surfaced by the newer upstream macro parser. The fork rev includes: NIGIG test-mode forwarding, custom AndroidManifest hook, ortho camera support, and the wrapper workspace-deps fix.	2026-08-28 10:49:41 +03:00
andodeki	cbce585eb4	perf(cad): complete 2D and 3D LOD -- Phase 5 ... The first Phase 5 tranche correctly reduced sub-pixel 2D outlines, but it stopped short of the plan's full level-of-detail phase. Complete the phase by applying the same measured three-pixel policy to the 3D path. Visible 3D parts below the projected AABB threshold now use one shared unit-cube geometry with an axis-aligned world-bounds transform. Full-detail shape batches, proxy instance rows and proxy draw calls are reported separately. Wireframe and hidden-line overlays use the same policy, and selected or hovered parts remain full detail. Extend the host-testable projection and proxy policy tests, frame benchmark, source guards and documentation. Keep the real GPU/window visual check explicit: compilation, policy arithmetic, submission structure and coverage pass here, but this environment cannot execute Makepad's draw submission.	2026-08-26 16:25:45 +00:00
andodeki	a5ba719d8d	perf(cad): add measured sub-pixel LOD to the 2D path -- Phase 5 ... Phase 4 reduced the number of tessellation calls, but it left the geometry volume unchanged: a zoomed-out 2,000-part plan still queued a full outline for every visible part. At the measured 200 m site view a 1 m part is only about 2.7 logical pixels across, so the remaining geometry was not resolvable detail. Add a pure lod policy that projects plane extents into logical pixels, keeps selected and hovered parts full detail, and conservatively falls back to a full outline for malformed state. The renderer and CadViewport::frame_budget call the same policy. Ordinary sub-pixel parts use one bounded 2x2 filled marker, while FrameBudget reports full outlines, markers, strokes and fills separately. Also fold the remaining 2D 1.2 margins into render_budget::VIEW_MARGIN, extend the structural benchmark and coverage harness, correct the phase documentation, and explicitly leave 3D mesh LOD deferred until a real GPU/window measurement justifies a second geometry policy.	2026-08-26 15:40:02 +00:00
andodeki	7a55f99c3a	Merge remote-tracking branch 'origin/main'	2026-08-26 18:04:15 +03:00
andodeki	6f3a96cf8f	perf(camera): persistent preview texture + area-scoped redraw; add native Video-path variant ... - camera_widget: allocate the preview texture once and update it in place via Texture::set_data_u32 instead of creating a new texture every frame; replace full-tree view.redraw with cx.redraw_area over the feed region. - camera_frames (new): duplicate of camera_widget migrated to makepad's native Video path (set_source_camera + begin_playback, Auto preview mode) so per-frame app work drops to zero; photo capture uses a short-lived CPU frame tap opened only between shutter press and first frame.	2026-08-24 23:34:14 +03:00
andodeki	8d33b6edb8	perf(camera): persistent preview texture + area-scoped redraw; add native Video-path variant ... - camera_widget: allocate the preview texture once and update it in place via Texture::set_data_u32 instead of creating a new texture every frame; replace full-tree view.redraw with cx.redraw_area over the feed region. - camera_frames (new): duplicate of camera_widget migrated to makepad's native Video path (set_source_camera + begin_playback, Auto preview mode) so per-frame app work drops to zero; photo capture uses a short-lived CPU frame tap opened only between shutter press and first frame.	2026-08-24 23:33:28 +03:00
andodeki	4cbb155cb7	perf(cad): stroke per group, not per item -- Phase 4 of the render plan ... A 2,000-part plan view with everything on screen cost 2,270 tessellation calls a frame. It now costs four: two for the grid, two for the parts. `stroke()` tessellates the whole accumulated path and clears it -- `tessellate_path_stroke` ends in `path.clear()` -- so queueing many subpaths and stroking once is one tessellation instead of N. The idiom was already in this file: `queue_dashed_line` has done it for the axis grid since Phase 3.9, guarded by a test. Phase 4 applies it to the two loops that never adopted it. **Base grid: two passes, two strokes.** Minors queued and stroked at 0.55, majors at 1.6 -- the stroke width is the one thing that genuinely needs its own call. `GridRange::has_minor_lines`/`has_major_lines` decide whether a pass runs at all and `frame_budget` counts strokes with the same two predicates, because an empty `stroke()` still enters the tessellator and a budget that assumed two when the renderer made one would be wrong in the direction that hides work. Minors stroke first so majors land on top where they cross; same colour either way, so the only visible difference is that the thicker line wins a crossing, which is the right answer. **Parts grouped by colour.** New `batching::ColorKey` -- the bit pattern, because `f32` is not `Hash` and two colours whose bits differ are two colours -- feeding the same `group_in_first_appearance_order` that Phase 3 groups shapes with. The colour policy moved out of the two draw loops into `constants::part_outline_color`, so the renderer and `frame_budget` cannot disagree about how many groups a frame has; the 2D loop had `vec4(1.0, 0.82, 0.40, 1.0)` written out where `PART_SELECT_COLOR` already existed. **Selected and hovered parts stroke last**, in their own groups, so a highlight is never hidden under a neighbour's outline. They were interleaved in document order before and could be. `FrameBudget` gained `grid_lines` and `part_outlines` beside the call counts. Geometry volume and call count are different numbers now and both are worth reading -- `VectorSubmission { outlines, stroke_calls }` mirrors Phase 3's `MeshSubmission` for the same reason. Measured (bench_frame_submission_budget, 1920x1080, 200 m site): zoom 5 m, 2000 parts: 12 visible outlines -> 4 tessellations (was 2170) zoom 200 m, 2000 parts: 2000 visible outlines -> 4 tessellations (was 2270) The second row is the point, and it is the row Phase 1 could not move: everything is on screen, culling removes nothing, and the frame still costs four calls. WHAT THIS DOES NOT DO: vertex volume is unchanged. The same 2,000 rectangles are tessellated -- in two calls rather than 2,000. What is saved is per-call overhead: tessellator setup, two `std::mem::take`s and an `append_geometry` each time. If a 2,000-part plan view is still slow after this, the remaining cost is triangles, which is Phase 5 and should only happen if a measurement asks for it. One visible-behaviour caveat, stated rather than buried: parts of the same colour are now drawn together, so where two outlines of *different* colours overlap, which is on top can change. They are 1.8 px outlines and the highlight ordering got strictly better, but it is a change to what is drawn, not only to how. Two tests were wrong before the code was, which is becoming this plan's pattern. `constants.rs` fell to 81.82% and the coverage floor caught it -- `part_outline_color` had no tests, and it now has five. And the guard test's first draft looked for a closing brace at a fixed indentation, matched the wrong one, and failed on correct code; it matches braces properly now. Verified: tools/test-cad-coverage.sh green -- total 97.33%, batching.rs 100%, cull.rs 100%, render_budget.rs 99.68%, constants.rs 98.55%, all floors met; cargo check --locked -p nigig-build --lib clean; cargo test --lib 1114 passed (1100 + 14 new); --test cad_integration 154 passed; CAD_BENCH=1 harness green; cargo fmt --check and git diff --check clean.	2026-08-21 05:08:14 +00:00
andodeki	06a3de6126	feat(spreadsheet): dropdown cells and button cells (#9 remainder, complete) ... Finish the widget-in-cell catalog with the two deferred controls, each a plain value plus a render flag (same convention as checkbox/slider/Markdown). - Dropdown: CellStyle.choices (a serialized, pipe-escaped list) makes a cell cycle through its choices on click. The toolbar "Drop" button parses the selected cell's value as a "Low|Med|High" list into choices (undoable SetChoices) and selects the first entry; a dropdown renders its value with a trailing ▾ affordance. dropdown.rs holds next_choice / parse_choice_list. - Button: CellStyle.button makes a cell a button whose value is a TARGET[+N] action spec (A1 ref + optional signed step, default +1). Clicking increments the target cell's numeric value through set_cell (undoable, recalculates dependents) — the spreadsheet-native counterpart of the reference's "+10" boost. button.rs holds parse_button_spec / button_step / format_step_value; the cell renders as a raised box with the spec centred. Toolbar "Btn" toggles the flag. Both flags ride in the render cache so cached cells stay styled, and both commands route through WorkbookCommand (apply + apply_command + dirty marking). Choices serialize after the markdown/slider/button flags; older files default to off/empty. Engine: 491 lib tests (+4) + integration. UI controllers: 142 tests (+8, button 5 + dropdown 4). Coverage: engine 96.59%, ui-controllers 99.45% (floors 96); button.rs and dropdown.rs at 100%.	2026-08-21 04:52:00 +00:00
andodeki	328690dfbf	feat(spreadsheet): 3-state header sort (asc/desc/off) with a timed report ... Close the last sort gap: the spreadsheet grid's header sort now cycles ascending → descending → off like the datagrid reference, and reports how long each sort took. Engine: SpreadsheetData::sort_rows remembers a RowSort { ascending, key_col, restore } where restore[view_row] is the row the data originally lived on, composed across sort chains. New unsort_rows applies that map to restore the pre-sort order (no-op without a sort, clears undo, recalculates moved formulas); sort_state() reports the active (column, direction). Workbook gains unsort_active_sheet and active_sort_state. The sort order is transient — reset by deserialize, never serialized. UI: sort_state::next_sort_state replaces the 2-state next_sort_direction with the asc→desc→off cycle, and the workspace HeaderClicked handler times the sort with std::time::Instant, drives unsort_active_sheet on "off", and appends the report ("sorted 1000 rows by B descending in N ms") to the status bar via a cached sort_status field. Engine: 487 lib tests (+6) + integration. UI controllers: 134 tests. Coverage: engine 96.66%, ui-controllers 99.42% (floors 96).	2026-08-21 04:52:00 +00:00
andodeki	dc1defd7e8	perf(cad): one draw call per shape, not per part -- Phase 3 of the render plan ... A 2,000-part model of six repeated shapes now draws in six calls. It drew in 2,000 before, and in 733 after Phase 1 culling at a working camera distance -- but at 400 m, with the whole site on screen and nothing to cull, it still drew in 2,000. Culling decides which parts are submitted; instancing decides how many calls carry them. They are different axes and this is the one that does not care where the camera is. No shader change was needed and that was the surprise of the phase. `DrawCadMesh`'s `transform`, `color`, `depth_clip` and `display_mode` are `#[live]` fields after `#[deref] draw_vars` in a `#[repr(C)]` struct, which is precisely the per-instance row `DrawVars::as_slice` packs and sends. The loop was already sending instance rows -- one per call. Three methods (`begin_instances`, `push_instance`, `end_instances`), ported from `DrawPbr::begin_many_instances_for_mesh` in the pinned fork, keep the call open across a group instead. New `batching.rs` (pure, 100% covered, floored): group_in_first_appearance_order(items) -> Vec<(K, Vec<T>)> batch_count(keys) -> usize First-appearance order rather than iterating a `HashMap`, deliberately. `HashMap` order is randomised per process, so batch order -- and the order parts reach the GPU -- would differ between runs and after a rehash. For opaque depth-tested geometry that is invisible, which is exactly what makes it a bad thing to depend on: the day someone adds a translucent material it becomes a flicker that reproduces on one machine in five. `FrameBudget` now carries `MeshSubmission { instances, batches }`. Named fields rather than a second positional `usize` because the entire point of the phase is that the two numbers now differ, and a caller that swapped them would report the win backwards. Measured (`bench_frame_submission_budget`, counts): camera 20 m, 2000 parts: 733 visible -> 6 draw calls (was 2000) camera 80 m, 2000 parts: 1459 visible -> 6 draw calls (was 2000) camera 400 m, 2000 parts: 2000 visible -> 6 draw calls (was 2000) ...and with every part a different size, 733/1459/2000 -- the ceiling, which is in the table for the same reason Phase 2's is. WHAT IS NOT VERIFIED, plainly. The instanced submission has never run. There is no GPU, no window and no `Cx` here, and tests/ui.rs still fails at child-build exit 101. What is verified: it compiles against the real Makepad API; the grouping is right (seven tests, including "no item is lost or duplicated", which is the failure mode hardest to see in a screenshot); the budget arithmetic is right; and the batch cannot be left open on any path -- `every_instanced_batch_is_closed_before_the_loop_turns` is a source check in the same style as the redraw_all guard, because a batch left open drops its rows and the parts simply vanish with no error anywhere. Someone with a window needs to open a 3D model and confirm the picture is unchanged. Two things limit the damage if it is not: `begin_instances` returning false falls back to the old one-call-per-part loop (it returns false while the draw shader is still compiling, which happens on the first frames of every window), and batch order is deterministic, so a defect reproduces instead of flickering. Verified: tools/test-cad-coverage.sh green -- batching.rs 100%, cull.rs 100%, total 97.30%, all floors met; cargo check --locked -p nigig-build --lib clean at the 127-warning baseline; cargo test --lib 1100 passed (1091 + 9 new); --test cad_integration 154 passed; CAD_BENCH=1 harness green; cargo fmt --check and git diff --check clean.	2026-08-21 04:41:48 +00:00
andodeki	1e5d014198	perf(cad): one GPU buffer per shape, not per part -- Phase 2 of the render plan ... Two hundred identical columns held two hundred GPU vertex buffers. They build byte-identical triangles -- `CadSolid::build_mesh` is a pure function of the solid payload and returns local-space geometry, and the upload path (`part_mesh_buffers_from_mesh` -> `MeshSpace::Model`) carries no transform and no colour -- so the duplication bought nothing but memory and upload time. `part_geoms` is now `HashMap<ShapeHash, Geometry>`. **ShapeHash**, new in cad_scene.rs: the solid payload and nothing else. No node id, no transform, no material. `ParamHash` is now *defined in terms of it* -- `id` then `ShapeHash` -- rather than repeating the match arm by arm. That is deliberate: last commit corrected a plan written on the assumption that equal `ParamHash` meant "same shape", which the leading `node.id.hash()` quietly made false. With one shared body the two hashes cannot drift apart again, and `param_hash_moves_whenever_the_shape_hash_moves` pins the join. Three consequences worth naming: - **Staleness became structural.** The key IS the content hash, so the draw loop's `.filter(|(hash, _)| *hash == ParamHash::from_node(part))` is gone: an edited part looks up a key that does not exist yet and `ensure_part_geometry` uploads it on the same frame. There is no "entry uploaded from different parameters" state left to guard against, which is the class of bug `subdivide_selected` shipped. - **Eviction is by live shape, not live id.** This is the hazard the change introduces and it is not obvious: deleting one of two hundred identical columns must NOT drop the buffer the other 199 draw from. An eviction written as "remove the deleted node's entry" would blank most of the model. `geometry_is_retained_by_live_shape_not_by_live_id` pins it. - **Uploads deduplicate within the frame.** Without the `queued` set, the first frame of a 200-column scene would call `get_or_build` two hundred times before the map had anything in it. Measured (`bench_geometry_buffers_shared_by_shape`, counts not timings): 200 identical walls 200 parts -> 1 buffer (200x) 420-part repetitive model 420 parts -> 6 buffers (70x) 420 all-distinct parts 420 parts -> 420 buffers (1x) The last row is in the table on purpose. Sharing is a property of the model, not of the code; a scene where every part differs gets nothing from this phase. And draw calls are unchanged -- still one per visible part -- exactly as the plan predicted. Collapsing those is Phase 3, which needed these shared buffers to be possible at all. Estimated at a week, took under a day. Two things the estimate did not know: `MeshCache::get_or_build` was already a pure function of `node.solid`, and the upload path already produced transform-free, colour-free buffers. It assumed both would need untangling; they were built right the first time. Verified: tools/test-cad-coverage.sh green -- total 97.27%, cad_scene.rs 98.56%, cull.rs 100%, all floors met; cargo check --locked -p nigig-build --lib clean (and one warning fewer: the `ParamHash` re-export in mod.rs is no longer needed by the widget layer); cargo test --lib 1091 passed (1085 + 6 new); --test cad_integration 154 passed; CAD_BENCH=1 harness green; cargo fmt --check and git diff --check clean.	2026-08-21 04:18:33 +00:00
andodeki	4aaebafe4e	perf(cad): cull parts against the viewport -- Phase 1 of the render plan ... Before this, `grep -niE "cull|frustum|offscreen|in_view"` over the 2,438 line renderer returned nothing. Every part in the document was tessellated in 2D and issued its own draw call in 3D, on screen or not. At a 5 m drafting zoom over a 200 m site, a 2,000-part model spent 2,000 tessellations and 2,001 draw calls per frame to show twelve parts. New `cull.rs` -- pure, host-testable, 100% covered with a floor in tools/test-cad-coverage.sh: draw_part_2d(centre, half_w, half_h, view, decorated) -> bool Frustum::from_view_projection(view, projection) -> Frustum Frustum::draw_part_3d(aabb, decorated) -> bool world_aabb_from_local_bounds(min, max, model) -> WorldAabb It lives in its own module for the same reason nav_pad.rs and render_budget.rs do: viewport_render.rs is 2,438 lines at 0.00% coverage and needs a Cx, so a predicate written inline there could not be tested at all. Both draw loops AND CadViewport::frame_budget call these same functions, which is what stops the reported budget from drifting away from what is drawn -- the discipline render_budget.rs already applies to the grid loops. Measured (bench_frame_submission_budget, 1920x1080, parts on a 200 m site; full tables in BENCH_BASELINE.md): 2D, 5 m zoom, 2,000 parts: 12 visible. 2,170 -> 182 tessellations, 2,001 -> 13 draw calls. 3D, 20 m camera, 2,000: 733 draw calls, 63% culled. 3D, 80 m camera, 2,000: 1,459 draw calls, 27% culled. Whole site on screen: nothing culled, and nothing should be. That last row is the honest half and it is in the doc too: when the view holds the whole model, all 2,000 parts are genuinely visible and culling cannot help. Phases 2 and 3 (shape-shared geometry, then instancing) are what address that case. Three things came out different from the plan I wrote last commit, and the plan was wrong about each: - **AABB, not bounding sphere.** A sphere around an AABB is looser at the same cost -- a 6 m wall gets a 3 m radius ball. The p-vertex AABB-versus-plane test is strictly tighter. So there is no bounding_sphere helper, and in particular no bounding_sphere FIELD on CadNode: the AABB is already cached by PlacedHash in SceneCache, which invalidates itself on a move or a resize, where a stored field would have to be maintained at 77 construction sites. - **world_aabb_from_local_bounds is shared with pick_part**, replacing the eight-corner transform that was inline there. Two copies would be two chances to disagree about a part's bounds, and picking a part the renderer culled is precisely the bug that disagreement produces. - **The "drawn extent" hazard split into two concrete rules.** The 2D test takes part_to_plane_2d and part_size_on_plane -- the very values the draw uses -- so it tests the drawn rect, not the model extent. And no selected or hovered part is ever culled: the highlight and the tooltip are drawn at the cursor, arbitrarily far from the part. The frustum comes from Gribb-Hartmann row arithmetic on projection * view, deliberately not from mat4_inverse -- that function had a mistyped index in all sixteen cofactors until 2ea5a74, and the terms that were wrong are exactly the ones a perspective matrix makes non-zero. Makepad's Mat4f::perspective is the OpenGL convention (xr/src/scene/xr_root.rs builds the camera with it), so -w <= x,y,z <= w is right; a 0..w depth projection would only make the near plane more permissive, which is the safe direction. Both predicates are conservative by construction: NaN coordinates, NaN AABBs, negative extents and an identity camera all fall through to "draw it". A part kept but invisible costs one submission; a part culled but visible is a bug the user sees. One test of mine was wrong before the code was: the first draft asserted a part 40 m off axis was invisible from a 50 m camera at 60 deg on 16:9. tan(30 deg) * 16/9 is 1.03, so the horizontal half-angle reaches past 45 deg and the part is on screen. The frustum was right; the test is now narrow-fov and says so in its docstring. Verified: tools/test-cad-coverage.sh green -- cull.rs 100.00%, total 97.26% (was 97.20%), all floors met; cargo check --locked -p nigig-build --lib clean; cargo test --locked -p nigig-build --lib 1085 passed (1062 + 23 new); --test cad_integration 154 passed; cargo fmt --check clean; git diff --check clean. Not done here: the two hard-coded 1.2 margins in viewport_render.rs (397-400, 1374-1377) still do not read render_budget::VIEW_MARGIN. cull.rs does, so cull and grid agree by the constant rather than by the code. Folded into Phase 4, which rewrites those loops anyway.	2026-08-20 21:45:45 +00:00
andodeki	4d681dfdbe	feat(spreadsheet): slider cells and inline-Markdown cells (#9 remainder) ... Complete the widget-in-cell gap the same value/style-convention way as the checkbox: two more cell kinds, both plain data with a render flag. Engine: - CellStyle gains `markdown` and `slider` bool flags, serialized as two trailing columns on the CELL line (older files default them off), plus WorkbookCommand::{SetMarkdown, SetSlider} routed through apply and apply_command via mutate_cell — undoable like every style mutation. UI: - markdown.rs (measured): a flat inline parser splitting a cell value into Regular/Bold/Italic/Code runs (`**bold**`, `*italic*`, `` `code` ``); unclosed markers and empty spans stay literal/dropped. The grid draws each run with the bold or regular resource (code tinted like formulas). - slider.rs (measured): 0-100 fraction/value mapping (rounded to whole steps) and track/fill/handle geometry. A slider cell draws the control instead of text, and a press/drag sets the value through set_cell — one undo step per drag (reverse-order ChangeSet application restores the pre-drag value). - Toolbar "Md" and "Slider" buttons toggle the flags on the selection; the render cache carries the two flags so cached cells stay styled. Dropdown and button cells are intentionally not included: a dropdown needs a per-cell choices list and a button needs an action semantic that a spreadsheet does not have — both would require a CellKind in the data model rather than a render flag. Engine: 481 lib tests (+2) + integration. UI controllers: 133 tests (+13, markdown 10 + slider 3). Coverage: engine 96.62%, ui-controllers 99.42% (floors 96); markdown.rs and slider.rs at 100%.	2026-08-20 21:36:21 +00:00
andodeki	42b1e7152a	docs(cad): merge a second render review into the plan, and correct the hash both reviews got wrong ... A second optimisation plan arrived (frustum culling, geometry merging, GPU instancing, octree/LOD). Rather than run two plans, every claim in it was checked against the tree at 14aa0d5 and merged into REVIEWS/CAD_RENDER_OPTIMISATION_PLAN.md with a verdict and the evidence behind it. Three of its ideas change what the phases will do; two would have made things worse; one of them found nothing, but reading it found something in mine. The correction that matters, and it lands on my own plan: ParamHash::from_node hashes node.id FIRST (cad_scene.rs:1871), before a single geometric parameter. My Phase 2 said "parts with equal ParamHash are geometrically identical by definition -- that is what the hash means". True and vacuous: equal ParamHash also means the same node. Two identical columns hash differently, so re-keying part_geoms by ParamHash -- the central move of that phase -- would have shared exactly nothing. The incoming plan inherited the same assumption from the same type name, which is the kind of coincidence that argues for a test rather than a paragraph, so there is now one: param_hash_is_not_a_shape_key_it_includes_the_node_id. The id is not a bug. MeshCache and part_geoms are both keyed by NodeId and only ask "is this entry still valid for this node", where the id is a constant. Sharing geometry needs a second hash, ShapeHash: the same body without the id. Phase 2 now carries that instead. Adopted from the incoming plan: - Frustum culling in 3D. SceneState3D { view, projection } is already captured into self.last_view/last_proj every frame (viewport.rs:3781) immediately before draw_scene, so the planes are a row add/subtract away -- and deliberately not via mat4_inverse, which had a mistyped index in all sixteen cofactors until 2ea5a74. My Phase 1 had only an AABB-versus-rect test, which is right for 2D and wrong for perspective. - Instancing, upgraded from "confirm the machinery exists" to "here it is". DrawPbr::begin_many_instances_for_mesh / push_many_instance_with_transform / end_many_instances (draw/src/shader/draw_pbr.rs:2696-2745 in the pinned fork) do exactly this for a shared mesh geometry, and per-instance data is just the #[live] fields after #[deref] draw_vars -- the layout DrawCadMesh already has (color, transform, depth_clip, display_mode). No shader rewrite; ~30 lines ported. The incoming plan rated it 7 days and high risk with "if supported"; it is neither. DrawCadMesh is also declared alpha_blend: false, so regrouping draw order is safe here. - Its dependency graph: instancing batches per geometry, so shared geometry genuinely gates it. Rejected, with reasons in the doc: - bounding_sphere as a stored field on CadNode. 77 construction sites, and this codebase has already paid for hand-maintained derived state once -- the part_geoms staleness bug (viewport.rs:4629) drew the pre-edit shape because one edit site forgot to invalidate. The sphere is two lines from SceneCache::world_aabb_for, which is keyed by PlacedHash so it invalidates itself on a move or a resize, and is benchmarked at 4.72x. - Merged vertex buffers per PartKind with transforms baked in. MeshCache stores local-space meshes on purpose ("the transform and the material are NOT hashed"); baking transforms inverts that and turns dragging one column into a full re-concatenate and re-upload of every column, every frame of the drag. It also cannot express per-part selection colour. That phase's own draw loop still sets a transform per part, which is still N draw calls -- the two halves contradict each other. - Octree above 500 parts. The cached world-AABB pass is 20 us at 500 parts, 0.12% of a 16.7 ms frame. A tree that replaces 20 us cannot pay for itself. Revisit on a measurement, not a part count. - "Log skipped-part count per frame" as verification. Phase 0 already built the counting seam; a log line nobody reads is a step that cannot fail. Also corrected REVIEWS/CAD_DRAWCALL_STRATEGY_ANALYSIS.md, which is where the ParamHash claim originated: section 0 is now two corrections rather than one. Verified: tools/test-cad-coverage.sh green, total 97.20%, all floors met, cad_scene.rs 98.53%; rustfmt clean; git diff --check clean.	2026-08-20 21:17:07 +00:00
andodeki	14aa0d5017	perf(cad): measure what a frame submits — Phase 0 of the render plan ... Phase 0 of REVIEWS/CAD_RENDER_OPTIMISATION_PLAN.md. No optimisation here; this is the measurement everything after it depends on. 0a first: Makepad does not count draw calls. `Cx::performance_stats` is `PerformanceStats { last_frame_time, max_frame_times }` — frame times only. So the seam had to be built. `render_budget.rs` reports two costs, deliberately not summed: tessellations CPU calls into tessellate_path_stroke draw calls GPU submissions Measured at 1920x1080, now in BENCH_BASELINE.md: zoomed in (0.2 m grid, 170 lines) 2000 parts -> 2170 tess, 2001 draw calls zoomed out (5 m grid, 270 lines) 2000 parts -> 2270 tess, 2001 draw calls The grid column barely moves across a 40x zoom range because the step adapts — that half is already virtualised. The parts column is every part, every frame, on screen or not. **This is not a model of the renderer.** The obvious way to count submissions is to write a second copy of the loop structure and count what it would do, which is exactly how the scene-cache benchmarks ended up timing a function that cannot cache. So `grid_range` owns the decision and `draw_2d_vector_scene` now drives its loops from it: the count and the drawing come from one function and cannot disagree. The ~25 lines of nice-number step arithmetic that were inline in the renderer now live in the pure module, with tests. 11 tests, 100% of the new module, and two of them are there to pin things people get wrong: - the 2D scene is ONE draw call regardless of part count, because DrawVector::end() submits the whole accumulation; - 3D is one draw call per uploaded part, which is where they multiply. One test — culling_shows_up_as_fewer_part_tessellations — asserts the *shape* of Phase 1's improvement before the work starts: parts fall proportionally, the grid column does not move. Verified: 1062 lib tests pass, cad_integration 154, cargo fmt clean, engine coverage 97.20% with every floor met including render_budget at 100%. The benchmark runs host-only under CAD_BENCH=1.	2026-08-20 21:01:52 +00:00
andodeki	8c1a4ad446	feat(spreadsheet-ui): Big Data virtual tab — 1B cells, procedural, timed sort ... Close the UI half of the virtualization gap. A new VirtualGrid widget renders one billion virtual cells (1,000,000 rows × 1,000 columns) from the engine's VirtualSheet: only the visible range is drawn, values derive from a hashed (row, col), and a column-header click sorts the full million rows by permuting an index — with the elapsed time reported in a status strip, as the reference does. - virtual_grid.rs (DSL widget, excluded from coverage like grid.rs): zebra-striped cells, row/column headers with the sort glyph, gridlines, drag-to-pan and wheel scroll, and a 3-state header sort (asc → desc → off) delegated to VirtualSheet::sort_rows/reset_sort. All placement and hit-testing reuses the measured GridMetrics; text widths reuse TextMeasureCache. - workspace.rs: a "BigData" toolbar button overlays the virtual grid over the spreadsheet grid via two child Views toggled with View::set_visible, so the existing spreadsheet path is untouched. Engine: 479 lib tests + integration (grid data-provider commit included). UI controllers: 119 tests. Coverage: engine 96.67% (data_source.rs 100%), ui-controllers 99.35% (floors 96).	2026-08-20 20:48:35 +00:00
andodeki	f14823a6da	feat(spreadsheet): grid data-provider abstraction and a virtual 1B-cell source ... Close the engine half of the virtualization gap (report section A). Add a GridDataSource trait — row/col counts, per-cell display text, sort, and a column label — implemented by SpreadsheetData (the real store, delegating to get_display_value/sort_rows) and by a new VirtualSheet. VirtualSheet derives every cell procedurally from a hashed (row, col), so 1,000,000 rows × 1,000 columns (one billion cells) cost one struct and no stored data. Its rendered text and numeric sort key come from the same hash, so sorting by the key sorts what the user sees; sort_rows permutes a Vec<u32> row index (O(n log n) over the index, not the data) with a deterministic tie-break. Column labels mirror the reference (# / Name / City / Balance / Score / Active, then ·N repeats). Engine: 478 lib tests (+12) + integration. Coverage: data_source.rs 100%, engine total 96.67% (floor 96).	2026-08-20 20:48:35 +00:00
andodeki	7946aa889f	feat(spreadsheet-ui): OS-clipboard TSV copy and toolbar zoom (#13, #7) ... Close the two remaining small UI gaps from the datagrid analysis: - TSV copy to the OS clipboard (#13): clipboard.rs gains a tested tsv_escape/rows_to_tsv pair (tab/newline/quote escaping, Excel-style), and the grid answers Hit::TextCopy with the selection as TSV while Ctrl+C also writes it straight to the OS clipboard via Cx::copy_to_clipboard. The in-app clipboard still feeds Ctrl+V paste. - Zoom (#7): a new measured zoom module holds the step factor (×1.15), 50-400% clamps and the minimum/maximum cell-size clamp. The grid's apply_zoom/reset_zoom rescale the default cell size deterministically from a captured base (no floating-point drift), and the cell text's font_scale follows the zoom — with text widths scaled at draw time so right/centre alignment stays true. The toolbar gains - / 100% / + buttons. Engine untouched. UI controllers: 119 tests (+7). Coverage: ui-controllers 99.35% (floor 96), clipboard.rs and zoom.rs at 100%.	2026-08-20 20:14:51 +00:00
andodeki	2adec957e9	feat(spreadsheet-ui): TrendChart line + candlesticks wired to the selected row ... Close gap #11 (chart integration) from the datagrid gap analysis, split as usual into testable headless logic plus a thin DSL widget: - chart.rs (measured): candle bucketing (OHLC from a price series), series bounds, line-point mapping with the reference's 8% padding, the vertical strips that trace a polyline with axis-aligned quads, candle body/wick geometry with up/down classification, and the 1/2/5×10^k axis tick step. - market.rs (measured): a deterministic live market — the reference's splitmix-style `mix64`, a HISTORY-capped random walk per symbol, lazy per-row symbol growth (so any selected row charts), tick() with roll-off, and the derived stats (last/change/pct_change/day_range/candles). - trend_chart.rs: a `TrendChart` widget (excluded from coverage like grid.rs) that colours and draws chart.rs output — gridlines, a polyline for a series, or candle bodies + wicks — via `set_series`/`set_candles`. - workspace.rs glue: a 220px chart panel under the grid (line + candlesticks side by side) fed from a 0.25s `Timer` ticker; selecting a grid row switches the charted symbol and updates the title label. Engine untouched. UI controllers: 112 tests (+16 chart/market). Coverage: ui-controllers 99.32% (floor 96), chart.rs 100%, market.rs 99.36%.	2026-08-20 19:02:02 +00:00
andodeki	1a36ca5538	feat(spreadsheet-ui): interactive checkbox cells (widget-in-cell) ... Close gap #9 (cells hosting widgets) the spreadsheet-native way. The datagrid reference hosts a live CheckBox widget per visible cell and recycles instances from a per-template pool; our grid instead draws the control from batched quads, so there is no per-cell widget object to instantiate or recycle — the render cache and reused draw buffers already fill the role the pool does. - A plain value cell holding TRUE/FALSE now renders as a checkbox: a centred square box with a checkmark when TRUE, and its TRUE/FALSE label to the right. A single click toggles the value (through the normal set_cell path, so it is undoable and recalculates dependents), selects the cell, and does not open the editor — a checkbox is a button, not a text surface. Formula cells are never checkboxes, so a click can't clobber a formula. - The predicate (is_checkbox), the value flip (toggled) and the box geometry (checkbox_layout) live in a new measured checkbox module with unit tests; grid.rs stays thin glue (draw the box/tick/label, reposition the label, and the click handler's toggle). UI controllers: 96 tests (+6). Coverage: ui-controllers 99.20% (floor 96), checkbox.rs at 100%. No engine changes.	2026-08-20 18:28:46 +00:00
andodeki	1a4b479013	feat(spreadsheet): SPARKLINE(range) formula rendered as in-cell sparkline bars ... Close gap #10 (sparkline cells) from the datagrid gap analysis, split as usual into testable headless logic plus thin grid glue: - Engine: a `SPARKLINE(range)` function resolves its single range argument to a flat numeric series and returns a new `Value::Sparkline(Vec<f64>)`. The variant is not a scalar — arithmetic on it is a `#VALUE:` error, and the aggregation/lookup helpers skip it — so no code path invents a number from a chart. `SpreadsheetData::apply_formula_result` stores the series on `CellData.sparkline` (derived state, like spills: never serialized, re-derived on recalculation) and clears it when the formula stops returning a sparkline. Dependency tracking is inherited from the range reference, so editing a cell in the source range re-derives the bars. - UI: a new measured `sparkline` module computes the bar rectangles (bars rise from the series minimum, tinted up/down by last-vs-first trend, gap-shrunk for narrow cells) — the same geometry as the reference `Sparkline` widget, but testable headlessly. The grid draws the bars with a dedicated `draw_spark` resource (depth 0.4) when a cell carries a sparkline and skips the text path; the series is cached in `CellRenderState` alongside the display text. The in-cell editor keeps its text on top by suppressing the bars while editing. Engine: 467 lib tests + integration (483 total, +6 sparkline tests). UI controllers: 90 tests (+5). Coverage: engine 96.60%, ui-controllers 99.15% (floors 96); sparkline.rs at 100%.	2026-08-20 18:17:27 +00:00
andodeki	e13aa03dca	feat(spreadsheet-ui): header sort, column reorder and row/col/all selection ... Wire up the three missing header interactions from the datagrid gap analysis, split as usual into testable headless logic plus thin grid glue: - Selection kinds: SelectionController gains GridSelectKind (Cells/Rows/Cols/All). Header clicks select a whole column / row, the corner (or Ctrl+A) selects the sheet, and bounds() expands the row/column/all kinds to the sheet's full extent so the fill, copy, delete, format and autofill paths handle every kind without branching. - Header-click sort: clicking a column header toggles the sort direction (same column flips asc/desc, a new column starts ascending) and re-sorts the active sheet; the sorted header shows an ▲/▼ glyph. The toggle lives in the new measured sort_state module; the grid holds the (col, asc) state and the workspace applies Workbook::sort_active_sheet. - Column reorder: dragging a header past the tap threshold moves the column, with a drop-indicator line at the insertion index (computed by the new, tested GridMetrics::col_insert_at). Engine addition SpreadsheetData::move_column(from, to) remaps cells and column-width overrides, rebuilds the dependency graph, recalculates, and clears undo history; Workbook::move_active_column propagates to cross-sheet readers via the new WorkbookCommand::MoveColumn. Selection overlay drawing is clipped to the cell area so a full row/column/sheet selection border no longer paints over the headers and the surrounding workspace. Engine: 461 lib tests + integration (477 total). UI controllers: 85 tests. Coverage: engine 96.66% and ui-controllers 99.11% (floors 96).	2026-08-20 17:54:54 +00:00
andodeki	61d666699b	feat(spreadsheet): sort rows — the engine half of the datagrid sort ... Tranche 3 of the gap analysis: the datagrid example sorts a million rows on a header click. This is the engine-side sort that makes that possible (the UI header wiring and the procedural data provider are later tranches). `SpreadsheetData::sort_rows(ascending, key_col)`: - Sorts every row by the values in one column. Numbers sort before text (case-insensitively), and empty cells always sort last — in both directions, unlike a naive `reverse` which would float blanks to the top on a descending sort. - Stable: equal keys keep their original order via a row-index tie-break. - Formulas move with their rows and recalculate against the sorted positions, matching Excel's reference-by-position semantics. - Undo history is cleared (a sort is a destructive bulk reorder); row heights, column widths and named ranges stay positional, as in Excel. `Workbook::sort_active_sheet(ascending, key_col)`: - Sorts the active sheet and recalculates cross-sheet dependents so readers on other sheets see the sorted values. Tests: 6 (ascending/descending, numbers-vs-text-vs-empty ordering, stable ties, formula recalculation, non-undoability, workbook-level sort with cross-sheet propagation). Engine unit tests 447 -> 453. Engine coverage 96.61% (floor 96).	2026-08-20 17:18:33 +00:00
andodeki	481a0e133c	feat(spreadsheet-ui): red error cells and a visible-cell status bar ... Tranche 2 of the datagrid gap analysis — the small UI polish. (Zebra stripes were already present as `cell_alt_bg_color` alternating rows.) Error cells render red: - `render_cache::is_error_text` matches a display value against the engine's own `FormulaError::from_display` surface (plus `#SPILL!`), so real errors are red while a user value like `#hashtag` stays plain text. - The grid's text-colour precedence now checks the error case after an explicit user text colour and before bold/formula/default, using a new `error_text_color` (default red). Visible-cell status bar: - `GridMetrics::visible_cell_count(viewport_w, viewport_h)` estimates the visible columns/rows from the viewport and default cell sizes, clamped to the grid extent — testable in geometry.rs. - `SpreadsheetGrid::visible_cell_counts` hands the live viewport to it, and the workspace status label now shows "Ready | C × R visible = N cells", cached so it only re-lays-out when the numbers change (scroll or resize). Tests: 2 new (error detection against real/plain values; visible-count estimation + clamping). UI lib tests 73 -> 75; ui-controllers coverage 99.02% (floor 96).	2026-08-20 17:10:57 +00:00
andodeki	0326da8dfe	feat(spreadsheet): ^ power operator and General thousands separators ... Tranche 1 of the gap analysis against the Makepad `work`-branch datagrid example: the two formula-engine features it has that we lacked. `^` exponentiation operator: - New `Caret` token, `BinOp::Pow` (precedence 5), and a right-associative `parse_pow` between multiplicative and unary in the parser. - Unary minus binds tighter than `^`, so `-3^2` is `(-3)^2 = 9` — Excel's precedence, and the exact assertion the datagrid reference pins. - `2*3^2` = 18 (power over multiply) and `2^3^2` = 512 (right-assoc). - Evaluated in `apply_binop`, so it broadcasts element-wise over arrays like every other operator. General-format thousands separators: - `format_number` stays comma-free — computed values and criteria must round-trip through `parse_cell_computed_value` and `parse::<f64>`. - New `format_number_display` (plus a `group_thousands` helper) groups the integer part at the display boundary only: `apply_number_format`'s General arm now shows `1,000,000` while the raw/edit value stays `1000000`. Non-numeric text passes through untouched. Tests: `^` precedence/associativity/evaluation, display grouping, and an end-to-end check that comma display does not break the recalc fast path. Engine unit tests 441 -> 447; UI lib tests still pass. Engine coverage 96.58% (floor 96).	2026-08-20 17:01:46 +00:00
andodeki	a82c8f7ff7	feat(pdf): Unicode-aware search and layout-aware reading order ... Phase 8 bullets one and two. Probing the existing code first, as the workflow requires, found five defects rather than the one the plan names: SPLIT MATCH 'Hello': 0 hits plain_text: "Hello" PRECOMPOSED 'café': 0 hits COLUMNS plain_text: "LeftTopRightTop\nLeftBotRightBot" OUT OF ORDER plain_text: "second\nfirst" `PageText::find` searched one run at a time and documented that as a known limitation. It is a limitation from inside the code and a broken feature from outside it: a writer starts a new run wherever it adjusts kerning, so an ordinary word arrives as two runs, and the find bar says a word plainly visible on the page is not there. `search.rs` indexes the page as one flattened string with a map back to (run, character), so a cross-run match is found and highlighted with one rectangle per run — never a merged box, which across a line break covers half the paragraph. The separator between two runs is a geometric question with three answers: abutting runs join with nothing (one word, split by kerning), separated runs with a space, and a different line or column with a newline. The newline matters as much as the empty join: joining lines with a space lets "one Right" match across a column gutter, text that appears nowhere. Whether two runs share a column is *asked* of the layout analysis rather than re-derived, or the extracted text and the searched text disagree about where a column ends — the original defect wearing a different hat. NFD, never NFC: composition needs the next character, so an NFC fold applied per character composes nothing and the two spellings of an accent stay different. That was a real bug in the first draft. And case *folding*, not lowercasing — Rust lowercases ß to ß, so "Strasse" never found "Straße". Columns are detected before lines, because two columns share their baselines; that is what makes them columns. Bands are separated by a gutter rather than by bare non-overlap, since two abutting runs on a line do not overlap either. Also fixed, found by running the gates rather than by looking: a stream reader trimmed a trailing CR before `endstream` as if it were the writer's separator. Binary data ends in CR about one time in 256, and when it did the reader returned a stream one byte short — no longer AES-block-aligned, so decryption produced garbage and Flate failed. Roughly one encrypted document in 250 was silently corrupt on read. The test failed once under coverage, passed five times in isolation, and failed 2 in 40 when actually counted. A /Length consistent with the file is now the authority; both stream readers are fixed and a test reads one file through each. 1477 tests pass (was 1426), coverage 88.37%, all floors met, external readers pass. 10 mutations across the two modules, all killed. ADR 0034.	2026-08-19 16:12:12 +00:00
andodeki	555c7daaa1	feat(spreadsheet): postfix LAMBDA application and LET ... Completes the first-class-function story started with LAMBDA-as-an- argument. Postfix application: - A new `Expr::Apply { func, args }` node lets a callable expression be invoked directly: `LAMBDA(x, x*2)(3)` and curried `LAMBDA(x, LAMBDA(y, x+y))(1)(2)`. The parser gained a `parse_postfix` loop, so `f(...)(...)` chains bind tightest, after the primary expression. - Arguments are bound with `resolve_bound_arg`: a range or array binds as an array (so `LAMBDA(x, SUM(x))(A1:A4)` aggregates), a scalar binds as a scalar. - Applying a non-callable is `#VALUE!`; an argument-count mismatch is reported. Postfix arguments still register their cell dependencies. LET: - `LET(name1, value1, [name2, value2, ...], body)` binds names to values sequentially — a later value may reference an earlier name — and evaluates the body with the names in scope, reusing the LAMBDA substitution machinery. A range value binds as an array, so `LET(s, A1:A4, SUM(s))` aggregates the whole range. - Duplicate names are rejected in both LET and LAMBDA, matching Excel; an unbound name in the body is `#NAME?`. Tests: 6 unit tests (postfix application, currying, LET binding/range/ error shapes, duplicate-parameter rejection, dependency tracking) + an end-to-end test proving LET and postfix application recalculate through the dependency graph. Engine unit tests 434 -> 441. Engine coverage 96.55% (floor 96).	2026-08-19 12:05:26 +00:00
andodeki	47f645ab58	feat(spreadsheet): resolve the four dynamic-array limits ... Closes the four limits documented when the spill model landed. LAMBDA (was: GROUPBY/PIVOTBY only took an aggregate name): - `LAMBDA(params..., body)` builds a `Value::Callable` without evaluating its body. A new `Expr::BoundValue` node splices a bound argument into the body AST when the callable is applied, so the ordinary evaluator runs the body. - `GROUPBY` and `PIVOTBY` accept either a named aggregate (`SUM`, ...) or a `LAMBDA`, applied per value column / per pivot bucket. This unlocks arbitrary aggregations (`LAMBDA(x, MAX(x)-MIN(x))`). FILTER include + full arithmetic broadcasting: - `FILTER` now accepts a same-shape include: matching cells are kept and non-matching positions become `#N/A`, element-wise, like Excel. - Unary operators broadcast over an array (`-A1#`, `-FILTER(...)`), completing the operator-level arithmetic alongside the existing binary broadcast. Spill formatting: - `SpillRange` records the anchor's `NumberFormat`, and derived cells are formatted at display time through a shared `apply_number_format` (extracted from `write_display_value`). Raw values stay numeric, so `SUM(A1#)` still evaluates correctly. #SPILL! blocking: - A spill that would overwrite an existing cell — or another spill, flowing or blocked — reports `#SPILL!` in the anchor instead of clobbering data. The would-be range is remembered in `blocked_spills`, and clearing the blocking cell retries the spill automatically. Tests: 7 new unit tests (LAMBDA in GROUPBY/PIVOTBY, lambda errors, same-shape FILTER, unary broadcast) + 2 end-to-end tests (NumberFormat inheritance and #SPILL! block-then-retry). Engine unit tests 427 -> 434; UI lib tests still pass. Engine coverage 96.41% (floor 96).	2026-08-19 07:58:38 +00:00
andodeki	90f25641d6	feat(tests): add doc workspace device verification tests (sections 1–5, 9) ... Add navigate_to_doc_workspace helper for the home → work → construction_grid → build_workspace → m_workspace_docs_btn → crdt_editor navigation chain. Tests covering DEVICE_VERIFICATION.md: - 1.1–1.4: interaction mode View ↔ Edit (edit_mode_btn toggle) - 1.2: View mode scroll by touch (touch_down/move/up) - 2.1: IME text input in Edit mode - 2.2: IME composition sends text - 3.1: Long-press arms selection - 5.1–5.2: Scroll handoff to parent ScrollYView - 9.0: Boot content renders (status bar + editor visible) Also fixes 3 pre-existing test compilation bugs: - show_password_toggle_works: moved value on Locator - login_status_modal: wait_not_visible doesn't exist - sso_buttons: &&str not Into<String> Updates makepad rev to ce899827a across all crates for consistency.	2026-08-19 02:41:25 +03:00
andodeki	72b0ce250b	feat(doc-ui): extract doc module from nigig-build into standalone crate ... Move 40+ files (~17K lines) from nigig-build/src/construction_frame/pages/workspace/doc/ to crates/apps/doc/doc-ui/src/. Rewrite internal paths from crate::construction_frame::pages::workspace::doc:: to crate::. doc-ui depends on doc-engine, makepad-widgets, nigig-core, serde, serde_json. nigig-build now depends on doc-ui instead of doc-engine directly.	2026-08-19 00:55:05 +03:00
andodeki	1740da3f34	feat(pdf): render a form XObject to pixels — the golden caught what the ... assertions missed `Rasteriser::register_xobject` takes a form's recorded commands from a caller that can resolve the page dictionary, so Phase 7's last golden-corpus criterion is met with pixels instead of with a request recorded by name. The first golden of that page showed the form drawn at the **page origin**, ignoring the `1 0 0 1 20 20 cm` that placed it. The recorded commands' `SetTransform`s are absolute in form space, and replaying them overwrote the page's CTM rather than composing with it. Nested lists now compose against the CTM in force at the `Do`. The colour assertions written next to that golden all passed while the bug was live — a red square two pixels from where it belongs is still a red square somewhere. That is the argument for pixel goldens in one sentence, and it is why the golden is compared after the assertions and not instead of them. The offset now has its own assertion too. The new fixture's form deliberately overflows its own /BBox, so the clip is visible in the golden as an absence rather than being taken on trust. Phase 7's golden-corpus exit criterion is now met in full. The `ui.rs` smoke tests remain blocked on the Makepad headless backend, as they have been since Phase 1, and are still not claimed as done. 1426 tests pass.	2026-08-18 20:06:50 +00:00
andodeki	d3089bc62a	feat(pdf): nested content, the wire codec and tiled rendering — Phase 7 closed ... Three bullets, and the Phase 7 status table rewritten row by row. **Nested content (ADR 0032).** A form XObject and a Type 3 glyph are the same problem: a content stream inside a content stream. Both were parsed completely and then not run. `paint_x_object` reported the name for "the host" to resolve and no host existed, so `Do` painted nothing. Type 3 was worse because it looked more correct — `d0`/`d1` reached the device, so the pen advanced by the declared width and the page rendered an invisible line of text with correct spacing after it. `nested.rs` runs both, in pdf-graphics because the dependency runs graphics → document and this is the only crate that can see the interpreter and the object model at once. Forms get their `/Matrix`, their `/BBox` clip and a save/restore wrapper, because without the wrapper a form's colour leaks into every object after it and looks like a bug in the document. Type 3 composes translate-then-matrix; the other order scales the translation and puts the glyph at (1.7, 16.8) instead of (72, 700). Recursion is bounded in both: unbounded, a self-referencing form is a stack overflow reachable from an untrusted document, which is a denial of service and not a rendering bug. **Wire codec and tiling (ADR 0033).** `worker.rs` moved interpretation off the UI thread only because both ends shared a Vec. Tags are explicit numbers, never declaration order, so reordering the enum cannot silently make old recordings decode as different commands. Truncation is an error rather than a short list — a decoder that stopped early would render a page missing its last few operations, plausible and wrong. The obvious truncation test failed, correctly: `Save` is one byte, so a cut on a command boundary really is a complete list. It now tries every cut position and requires each to be a named error or a genuine prefix. Tile skipping is conservative. A command whose geometry is unknown is kept, because dropping a state change corrupts everything after it in that tile, silently. Only untransformed geometry that provably falls outside is dropped. Every tile is asserted pixel-identical to that region of the whole-page render: tiling that is fast and different is not an optimisation. Eight mutations across the two modules, all killed. Phase 7 status is now two tables — the eight spec bullets and the exit criteria — with what is missing named in the row rather than rounded up. Three rows are not green: Makepad blend compositing needs render-to-texture, the image-XObject pixel golden asserts the request rather than pixels, and the `ui.rs` smoke tests remain blocked on the headless backend they have been blocked on since Phase 1. 1425 tests pass, coverage 88.10% (was 87.60%), all floors met, external readers pass. ADRs 0032 and 0033.	2026-08-18 20:00:18 +00:00
andodeki	f37197781e	feat(pdf): glyph outlines from TrueType and CFF, and glyph-aware text runs ... `sfnt.rs` read the metric tables and nothing else. It could say how wide a glyph was and not what shape it had, so every renderer drew embedded text with a substitute font at the correct advance — the failure mode that looks most like success: the line breaks land right and the letterforms belong to somebody else. `outline.rs` returns one outline type for both formats. TrueType quadratics are degree-elevated to cubics, which is exact, so no format detail leaks to a consumer. Composite glyphs are placed by their offsets and scales, with a depth bound because a font can reference itself. CFF Type 2 charstrings run through an interpreter with biased local and global subroutines, hints, hintmask byte counting, the leading width operand, and the FontMatrix as declared rather than assumed to be 1/1000. Separately, `ShowTextWithMetrics` carried one advance for a whole run — enough to move the pen to the next run and nothing else. So `text.rs` guessed: `seg.advance / char_count`. For "Wi" that puts the boundary between the letters at 5 when it is at 9, and every caret, drag-selection and search highlight in the application was wrong by that much for every proportional font. `GlyphPlacement` now carries per-glyph pen offsets, computed with the same expression as the run total so the two cannot drift. The even-spacing fallback stays for fonts with no width table, which is what `advance_is_measured` has always been for. The fixture story is ADR 0029's, again. `cff_sample.otf` is a fontTools conversion of DejaVu: no subroutines, no hints, no width operands. It proved the interpreter draws the right shapes, and then four mutations of that interpreter survived because nothing in the corpus reached the code they broke — each of which produces a plausible wrong glyph from a font that parses. `cff_subrs.cff` is hand-assembled for exactly those four, and fontTools agrees with every expectation asserted against it. A fifth mutation survived a composite test that counted contours; it is killed now by one that measures where the components land. Coordinates are asserted against fontTools ground truth, not against our own output. Seven mutations, all killed. 1397 tests pass. Deferred and recorded, not claimed: CID-keyed CFF, `seac` accents, rendering outlines through the Makepad device. ADR 0031.	2026-08-18 19:44:18 +00:00
andodeki	0bef30a6d5	feat(pdf): compositing and overprint — the blend maths had no backdrop ... `transparency.rs` implemented all sixteen blend modes and unit-tested them against the specification's formulas. Nothing ever called them with a backdrop. The Makepad renderer's `SetBlendMode` pushed a `TransparencyError::Unsupported` and then painted the source colour, so a /Multiply highlight and a /Normal one produced byte-identical output and every test passed — because every test asked "was the right command issued", not "does the page look right". Overprint had no code at all. /OP, /op and /OPM were not parsed, so an overprinting object knocked out the inks under it. That is not a missing feature, it is the inverse of the instruction: on a press it is the difference between a colour and a hole. - `composite.rs`: a straight-alpha RGBA `Canvas` implementing §11.3.6's union formula, weighted by backdrop alpha so a Multiply over transparency is the source rather than black. Constant alpha and per-pixel soft masks. Transparency groups composite as a unit; knockout groups are refused by name rather than silently treated as non-knockout. - Overprint as `composite_cmyk`, separate from the RGB path rather than a flag on it: overprint is a statement about inks and RGB has none. /op defaults to /OP per table 58 — defaulting it to false makes the common `<< /OP true >>` knock out every fill. §10.7.5's "no effect on an RGB device" is asserted, so our doing nothing there is the spec rather than an omission. - `raster.rs`: a CPU rasteriser that replays a command list onto a canvas. Not on the display path, no anti-aliasing, no fonts; it exists so compositing has a verifiable output. In pdf-graphics and not pdf-makepad because a test that needs a GPU is a test that does not run. - Golden **pixels** for shading, mesh, blend and overprint pages — Phase 7's exit criterion, which the Phase 2 command-text goldens cannot meet. ASCII grids with a colour legend, quantised to quarter steps; each test asserts its exact colours before comparing, so a wrong-but-stable render cannot be blessed by an UPDATE_GOLDEN run. Six mutations, all killed, including the two that describe the old behaviour: discarding the blend result, and ignoring the overprint flag. 1364 tests pass. ADR 0030.	2026-08-18 19:25:15 +00:00
andodeki	728fbc3ad0	fix(pdf): mesh shadings — three bugs in code that had no fixture ... ADR 0028 shipped types 4-7 and said honestly that they were unproven: the uncovered lines of `shading.rs` were exactly `parse_mesh`, "the position `image.rs` was in before ADR 0016 found the JPEG decoder was a stub". Writing the fixtures found three real bugs. - Type 5 has no per-vertex flag; `/VerticesPerRow` delimits it. Reading 8 phantom bits shifted every vertex after the first, decoding plausible coordinates that were entirely wrong. - Types 6 and 7 are patches: 12 or 16 control points carrying no colour, then four corner colours. The old loop read a colour per point, consumed three times too many components, ran off the stream, and the None-on-truncation path swallowed it as "the mesh ended". - A flag-0 triangle is three vertices whose second and third flags are ignored (§8.7.4.5.5). Acting on them cleared the strip every time and produced no triangles at all. Caught in new code, before it shipped. And one omission: `color_at_point` returned None for a mesh, so a mesh that parsed perfectly still painted nothing — indistinguishable from one that failed. `MeshTriangle::color_at` now interpolates the corner colours by barycentric coordinates, None outside, because black is a colour a mesh can legitimately produce. Shared-edge patches (flags 1-3) inherit the previous patch's edge rather than being read as fresh patches, which desynchronised the rest of the stream. Five corpus fixtures, generated from named coordinates and colours so every expected value in the tests is one the generator wrote deliberately. Eight tests, five mutations, all killed. Coons flattening is still an approximation and still reports `is_approximate`. ADR 0029.	2026-08-18 19:10:49 +00:00
andodeki	c9474e2c9f	feat(spreadsheet): dynamic-array spill model — FILTER, GROUPBY, PIVOTBY, A1# ... The engine stored every formula result as one display string per cell. This adds the dynamic-array architecture: a formula can return a grid, which "spills" into the cells below/right of its anchor. Value model: - `Value::Array(Vec<Vec<Value>>)` — a grid result. Scalars coerce to one-cell grids where the shape matters; `to_f64`/`to_bool` refuse an array, `to_display_string` shows the top-left, and `resolve_arg` flattens an array argument so `SUM(FILTER(...))` aggregates its cells. - Element-wise broadcasting: `resolve_array2d` maps a binary operator over a grid when one side is a scalar (or both grids share a shape), so `FILTER(A1:A4, C1:C4 > 15)` builds the boolean include the way Excel does. The binary-op logic was factored into `apply_binop`. Parser and AST: - `#` is now the spill operator: `A1#` and `Sheet2!A1#` parse as `SpillRef` / `SheetSpillRef`, evaluate to the anchored array (so `=A1#` re-spills), flatten in aggregates, and track their anchor in the dependency graph (intra- and cross-sheet). Functions (dynamic arrays): - FILTER(array, include, [if_empty]) — keep rows (column include) or columns (row include); `#N/A` on no match unless `if_empty`. - GROUPBY(row_fields, values, function, [field_headers]) — group rows by field tuples and aggregate each value column (SUM/AVERAGE/COUNT/ MAX/MIN/MEDIAN by name, ETA-reduced-LAMBDA form). - PIVOTBY(row_fields, col_fields, values, function) — a 2D pivot with the aggregate name in the top-left corner. Spill storage (data.rs): - `SpillRange` + `spills` map on `SpreadsheetData`: derived cells read back through `get_display_value`/`get_raw`/`get_edit_value`, are not blank, and are read-only — `set_cell`/`put_cell`/`remove_cell`/ `mutate_cell` refuse to touch them (the UI blocks via `is_spilled`). - Recalc builds the spill from the `Array` result (`apply_formula_result`) and drops stale spills when a formula becomes scalar, is removed, or cycles. Spills are derived state, never serialized — the anchor formula persists and re-derives on load. - Cross-sheet spills read through `get_sheet_spill_values`. Tests: 21 new units (FILTER/GROUPBY/PIVOTBY shapes, broadcasting, Value::Array methods, spill parsing) + 5 end-to-end tests (spill display, read-only cells, `A1#` aggregation and re-spill, stale-spill clearing, cross-sheet spill). Engine unit tests 405 -> 427; UI lib tests still pass. Engine coverage 96.66% (floor 96).	2026-08-18 18:34:22 +00:00
andodeki	a2b05c56c9	feat(makepad-table): opt-in capabilities feature, and raise the matrix_client defect ... The two caveats from the dependency investigation. ## The capabilities feature Camera and location attachments are now available behind `features = ["capabilities"]`, which pulls `nigig-uikit` and supplies `UikitAttachmentProvider`. Measured: 89 crates by default, 275 with the feature on. That cost is real and it is inherent, not packaging waste. `camera_widget` imports `send_geocode_request` and `request_map_tile` from `nigig-core`, both of which call `spawn_async` — the shared Tokio runtime — and the first makes an HTTPS call to Nominatim. A camera that geocodes needs an async runtime and an HTTP client; there is no lighter honest version. It is affordable because it is opt-in, and because any app enabling it already depends on `nigig-core`, so that app's own tree grows by nothing. Everything touching `nigig-uikit` is in one module, so the boundary is a file rather than `#[cfg]` scattered through the widget. The provider holds no widgets of its own: the host owns the `CameraWidget` already in its tree and this asks it to open, because a provider that instantiated a second camera would fight the first for the device. A second request while one is outstanding is refused rather than overwriting. The table turns that refusal into `AttachmentUnavailable`, so the user is told the camera is busy instead of watching their first request vanish. File picking is deliberately declined here — `robius-file-picker` already ships unconditionally and costs nothing, and two paths for one job is one too many. Two CI gates, both verified to fail when they should: the opt-in build must keep compiling, and the default build must pull none of `tokio`, `reqwest`, `hyper`, `clap`, `csv`, `image`, `nigig-uikit` or `nigig-core`. The second checks the resolved `cargo tree` rather than the manifest, because feature unification can switch an optional dependency on from a sibling crate. Tests 99 default, 105 with the feature. Both clippy-clean. ## The matrix_client defect Raised in REVIEWS/MATRIX_CLIENT_FEATURE_GATE.md rather than fixed. It is not my crate, nothing depends on the broken combination, and a blind fix could change behaviour someone relies on. `matrix_client` declares `native = ["dep:tokio", "dep:reqwest", "dep:rusqlite"]` but its source gates on `#[cfg(not(target_arch = "wasm32"))]`. Two switches for the same modules, so on a native target with the feature off the modules compile and their dependencies do not — 19 errors, 26 ungated uses across 7 files. There is no CI job for the crate, which is why it rotted unnoticed. The note corrects an overstatement I made while arguing for the trait hook. I said fixing this would unblock wasm. It would not: `matrix_client` already builds clean for wasm32 with `--no-default-features`, and `nigig-core` has 8 wasm errors of its own (`crate::platform::spawn` missing) that have nothing to do with it. The only broken combination is native-target-with-feature-off, which nothing builds. I also said earlier that `matrix_client` was heavy — it is a 7-dependency local crate, not matrix-sdk. That was wrong and it inflated the case for the trait hook; the note records the measured numbers instead.	2026-08-18 18:03:26 +00:00
andodeki	45efc74106	feat(spreadsheet): pivot and chart aggregates — IFS family, statistics, SUMPRODUCT, LARGE/SMALL/RANK ... Full GROUPBY/PIVOTBY need a dynamic-array "spill" model (a formula returning a grid), which this single-cell engine deliberately does not have. These are the single-cell building blocks that do the same work. Multi-criteria conditional aggregates (the "filter then aggregate" pivot core), resolved positionally so criteria ranges stay aligned: - SUMIFS(sum_range, criteria_range1, criteria1, ...) - AVERAGEIFS(avg_range, criteria_range1, criteria1, ...) (#DIV/0! on no match) - COUNTIFS(criteria_range1, criteria1, ...) - MAXIFS / MINIFS (0 on no match, like Excel) Mismatched range sizes are #VALUE!, not a silent misalignment. Chart statistics (over the flattened numeric arguments): - MEDIAN, MODE (ties keep the smallest value), and the sample/population STDEV / STDEVP / VAR / VARP. Sample forms divide by n-1 (#DIV/0! for a single value), population by n. Pivot/ranking helpers: - SUMPRODUCT(array1, [array2], ...) — the element-wise dot product; text counts as zero, errors propagate, mismatched sizes are #VALUE!. - LARGE / SMALL(array, k) — k-th largest/smallest; k out of range is the new #NUM! error. - RANK(value, array, [order]) — descending by default, ascending on any nonzero order, tied values share a rank (RANK.EQ). `FormulaError` gains `NumError` (`#NUM!`) for out-of-domain numeric arguments, rounding out the error surface after `Na` in the lookup tranche; it round-trips through display/parse and propagates from cached values. Tests: 10 unit tests (multi-criteria aggregation, statistics, dot products, top-N/ranking, argument/size errors) + an end-to-end test proving the dependency graph tracks every range and recalculates the pivot formulas when a source cell is edited. Engine unit tests 395 -> 405. Engine coverage 97.34% (floor 96).	2026-08-18 18:03:00 +00:00
andodeki	c1d1e67f3a	feat(pdf): shadings — the sh operator was parsed and thrown away ... ADR 0028, the first of Phase 7's eight bullets. content.rs contained `PdfOp::Shading(_name) => {}`. The operator was lexed, given its own variant, matched during interpretation, and discarded. A page whose background is a gradient rendered as nothing. Nothing caught it for the usual reason: a blank region is a legal thing for a page to contain, so "drew nothing" and "drew what was asked" are indistinguishable without an assertion naming the expected colour. The golden corpus had no shading page, so there was nothing to be wrong. Two of the three pieces already existed — function.rs evaluates the colour function and colorspace.rs converts it to RGB. What was missing was the geometry between them. Sampling rather than a gradient primitive: a PDF shading is defined by an arbitrary function, possibly a sampled table or a PostScript program, and neither reduces to a stop list without loss. A device with a native gradient can still recognise the two-stop case from the samples. "No colour here" is None, not black. Black is a colour a shading can legitimately produce, so returning it for "outside an unextended shading" would paint a rectangle the author never asked for and the caller could not tell the two apart. Types 1-5 exact. Coons and tensor patches are flattened to their corners, which loses the curvature, and is_approximate says so rather than leaving a caller to assume fidelity. An unknown type is refused by number: a mesh drawn as a flat fill is a plausible-looking wrong answer. paint_shading is a new trait method, so the compiler found every implementor. The Makepad renderer records the request in pending_shadings, mirroring pending_xobjects — it cannot resolve a /Shading resource because it does not own the page dictionary, and recording the request is what stops the operator vanishing a second time. That holds even for types we refuse, so a host can warn the user. Four mutations, all killed. The first — discarding sh again — fails three tests. Stated plainly and left unticked: the mesh path is written but NOT exercised by any real stream. shading.rs is at 68% and the uncovered part is exactly parse_mesh and triangulate. Mesh support should be treated as unproven, not working: the code runs and produces triangles, and nothing yet demonstrates they are the right triangles. That is the position image.rs was in before ADR 0016 found the JPEG decoder was a stub. The Phase 7 status line is a table from the start this time — one row per spec bullet, seven of them saying "not started". Per ADR 0021, written before the work rather than after it. pdf: 1321 passed (was 1291). pdf-ui: 1366. Coverage 87.60%, floors met.	2026-08-18 17:30:39 +00:00
andodeki	cb8912f762	test(pdf): close the two real coverage gaps in the signing module ... Asked to verify Phase 6 was complete *with test coverage*, I measured sign.rs per function rather than trusting the file-level 82%. Most of the apparent gap is error arms inside covered functions — llvm-cov attributes each `map_err` closure separately — but two things were genuinely untested, and one of them was not code that should exist. algorithm_name() was dead. It returned a &'static str describing the algorithm and nothing called it: `algorithm()` supersedes it, returns a type rather than a string, and is what the CMS writer actually uses. Deleted rather than tested, because a test would have preserved code whose only caller was the test. SigningError's Display impl was never exercised. These strings reach a user through a host application. ContentsTooSmall in particular must carry both numbers — a caller cannot raise the reservation without knowing by how much — and that is now driven through the real signing path with a 32-byte reservation rather than by constructing the error. sign.rs 82.07% -> 83.79%. pdf: 1291 passed. Coverage 88.03%, floors met.	2026-08-18 17:08:11 +00:00
andodeki	7737096858	refactor(makepad-table): adopt Robrix's image decode path ... Replaces the hand-rolled try-PNG-then-JPEG with `pageflipnav/src/utils.rs::load_png_or_jpg`, the pattern the Robrix-derived app in this repo already uses. Two things it does better: - It sniffs the header with `imghdr` and calls the matching loader directly, so a JPEG does not decode-and-fail as a PNG first on every cold cache. - It still falls back to trying both when the sniff names something unexpected or nothing at all. `imghdr` is not perfect, and a mislabelled file is more useful decoded than refused. `imghdr` has no transitive dependencies — it reads a header and names a format. It is already a dependency of `pageflipnav` at the same version. The upstream version logs the failure and dumps the bad bytes to disk. That is right for a chat client receiving untrusted media and wrong here: this runs from the draw path for every attached cell, so a broken file would log once per frame. The caller already caches the failure and draws a labelled chip naming the file, which tells the user more than a log line would. Tests 94 -> 99. Verified by removing the sniff and by removing the fallback; each fails the ordering test. `TextOrImage`, the other candidate for reuse, is referenced in `room_screen.rs` but not defined anywhere in this checkout — it is upstream Robrix only, so there was no baseline here to adopt.	2026-08-18 16:51:47 +00:00