diff --git a/AGENTS.md b/AGENTS.md index 8aacca936..1ad2250c4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,6 +7,11 @@ > studio websocket bridge for all agent work. Full spec: [App Remote Control](#app-remote-control---remote). ## Execution Policy + +- **Designs stay local.** Design documents, plans, and reports are local files + (`local/agent_state//DESIGN.md`) that lanes read from disk. Never + publish them to the web (no Artifacts, no hosted pages); summarize in the + terminal instead. - Launch UI programs as standalone release binaries from this checkout. Do not use the Studio remote bridge, `ObserveMount`, `RunItem`, or any `cargo-makepad studio` websocket client. @@ -31,8 +36,56 @@ can be run directly in the shell. - A standalone app's built-in screenshot/capture hook is valid for visual inspection. +- **System-level screenshots are FORBIDDEN.** Never run `screencapture`, + `CGWindowListCreateImage`/`CGDisplayCreateImage` scripts, `xcap`, + `import`, `scrot`, `grim`, `xwd`, PowerShell/Win32 screen grabs, or any + other OS screen capture — not of the display, not of a window, not + "just the caption". The user's screen is private. The only image of a + running app you may ever take is the app's own `--remote` grab (`/g`, + `/gq`, `/tweak/grab`), which renders the app's own drawable and nothing + else. If something only shows in the OS layer (native caption buttons, + other apps, the desktop), ask the user for a screenshot instead of + taking one. - When adding a new example crate, update both the Cargo workspace and `makepad.splash`. +- **Zero locking on the UI thread, one mechanism everywhere.** The UI + thread never takes a `Mutex`/`RwLock`/`Condvar` that another thread can + hold, and never blocks on a channel. UI → workers/audio is commands over + a channel (bounded, non-blocking send; a full queue is reported and + retried next frame). Workers/audio → UI is snapshots over atomics, a + triple buffer, or a channel read with `try_recv`. Large payloads (PCM, + stems, grids, images) travel as `Arc` through the channel; a replaced + payload is handed back so the UI thread does the drop, never a realtime + thread. A realtime callback (audio) owns its state, never takes a lock + the UI can hold, never allocates on the hot path. This is ONE code path + for native and wasm — no `cfg` fork where desktop keeps shared mutexes. + On wasm both the browser UI thread and the AudioWorklet thread abort on + `Atomics.wait`, and a spinning fallback against a busy audio callback is + a 100 % CPU feedback loop that kills audio and frame rate together (DJ + web, 2026-09-03). `lock_from_ui` is only acceptable on state provably + touched by the UI thread alone. +- **Standard operating flow — who does what.** The main session (Fable) + designs, briefs, manages and reviews; it does not write the code itself + except one-line fixes. **Codex writes the code**: every implementation lane + is a Codex lane with a precise brief (observations, files, rules, the + verification commands) launched through `local/tools/delegate` / + `local/agent_state/webdemos/tasks/queue.sh` and landed through + `local/tools/integrate`. **Grok does the tests and the token-heavy work**: + test suites, audits, surveys, log reading, conflict resolution passes, + reviews of large diffs (`delegate grok` / `research-grok` / `review-grok`). + A Fable subagent is the exception, only for a design-level change the + other two cannot carry (a new platform mechanism), and it stops as soon as + the API is fixed so Codex can do the conversions. Keep at most six lanes + per provider; land everything through the integrator; the user tries the + result — no routine captures. +- **No temporary threads — use the pool.** Never spawn a thread for one + job (`std::thread::spawn` is unsupported on wasm anyway; the platform + spawner works everywhere). Background work goes to the platform thread + pool (`cx.thread_spawner()` / the pool `TaskHandle` API) or to a + long-lived worker created once at start-up and fed over a channel. On + the web a Web Worker takes hundreds of milliseconds to come up, so a + per-job thread is a stall; on desktop it is still churn. One mechanism + on both targets. ## Standalone Launch 1. `cargo build --release -p ` from this checkout. @@ -44,6 +97,22 @@ ## App Remote Control (`--remote`) +> **Focus law.** A `--remote` app opens its window VISIBLE BUT UNFOCUSED and +> stays that way: it never activates, never becomes key, and bridge clicks +> never raise it. The user keeps typing wherever they were. Everything the +> bridge does (grabs, `/m`, `/k`, `/t`, `/snap`) works without focus because +> input is injected through the app's event loop, not the OS. Do not work +> around this (`MAKEPAD_FOCUS=1` exists only for a run the user asks to see +> in front); `MAKEPAD_NO_FOCUS=1` gives a non-remote launch the same manners. + +> **Who may open a visible window.** Subagent/lane verification runs HIDDEN: +> launch with `MAKEPAD_HIDE_WINDOWS=1 --remote` — the window never +> appears, grabs (`/g`), `/snap`, `/m`, `/k`, `/t` all still work offscreen. +> Only the integrating session opens the one visible, unfocused window the +> user watches; several look-alike windows on screen made the user "go +> insane" (2026-08-26). + + Any makepad app launched with `--remote` runs a localhost HTTP server inside the process and prints one line before the UI appears: @@ -111,6 +180,8 @@ this pattern as an executable end-to-end test across three example apps. `GET /gq` (or `/close` each window, then `/quit`). Never leave test windows on the user's screen, and never `pkill` when the protocol is available. - **Never touch an instance the user is running.** Launch your own. +- **`/g` is the only camera.** No OS-level screen capture of any kind (see + Execution Policy) — the remote grab is what you get. - **A vanished window or app with `[makepad-remote] user closed …` in the log means the human dismissed it — it was in their way.** Do **not** treat that as a crash and do **not** relaunch it. The app prints @@ -151,6 +222,59 @@ this pattern as an executable end-to-end test across three example apps. - **Cost when idle is zero.** The event loop only upshifts its paint clock while a remote request is in flight. +### The TWEAKER (`/tweak/*`) — design feedback and live styling + +Every `--remote` app carries a design-feedback overlay (plan of record: +repo-root `tweaker.md`; implementation: `widgets/src/tweaker.rs`). Off it +costs nothing. On, the person (or you) points at the UI: pointer events over +the window body are swallowed before widget dispatch — **clicking a Button in +tweak mode outlines it and never fires it** — and the window grows a property +sidebar next to the (compressed) app UI. Shift+F10 toggles it in-app; every edit, +theirs or yours, lands in one shared diff log. + +| Route | Answer | Notes | +|---|---|---| +| `/tweak` `?on=1\|0&annotate=1\|0` | `{"on":1,"annotate":0}` | toggle the overlay / the freehand draw mode (Alt-drag draws too) | +| `/tweak/state` | `{"on":1,"sel":{path,ty,r,band},"props":[{n,v,set}],"hover":…,"diff":[…],"ann":[…]}` | the STRUCTURE feedback: pinned selection, its real reflected properties (`set:1` = explicitly applied), the edit log, annotation strokes with the widget paths they touch | +| `/tweak/apply` (POST) | `{"ok":1,"path":…,"changed":[{path,prop,old,new}]}` | body `{"path":"a.b.c","splash":"{padding: Inset{left: 20}}"}` or the one-property shorthand `{"path":…,"prop":"draw_bg.border_radius","value":"8"}`. Evaluates the chunk onto that ONE instance through the ordinary apply machinery (`+:` merge rules intact) and triggers a full relayout. Answers after the next drawn frame | +| `/tweak/diff` | `{"diff":[{path,prop,old,new}…]}` | the raw edit log, in order | +| `/tweak/clear` | `{"ok":1}` | reset diff + annotations | +| `/tweak/final` | `{"final":[…coalesced…],"ann":[…],"drew":0\|1,"png":path?}` | **read this when tweaking is done**: per (path, prop) only the original and final value, churn collapsed. When the user drew, `png` is the composited screenshot — look at it, the strokes mean something | +| `/tweak/grab` | like `/g` | the overlay (outlines, strokes, sidebar) draws in the window's own pass, so any grab is already composited | + +`local/tools/tweak` wraps all of this: +`tweak PORT on`, `tweak PORT state`, `tweak PORT apply PATH PROP VALUE`, +`tweak PORT splash PATH 'CHUNK'`, `tweak PORT final`, … + +**How to listen.** Sidebar edits push to you: each one emits a marked +`TWEAK sidebar -> ` line into the app log — the +`/log` tail is your ear; you never poll `/tweak/state` for changes. Talk back +on `/tweak/apply` (values or whole shader chunks) to the same selected +instance. + +**Write-back (you do this part — the overlay never writes source).** When the +session is done, take `/tweak/final` and edit the splash source: + +1. Resolve each entry's widget path to its DSL site: the dotted path mirrors + the `script_mod!` tree (`/d` shows the same ids). `-` segments are + anonymous containers — skip them when searching the source. +2. Write each property at the **most specific existing site** — the widget's + own `name := Type{…}` block if it has one; create one only when none + exists. +3. Respect the merge law: a property inside a typed sub-struct goes through + `+:` (`draw_bg +: { border_radius: 8 }`), never a replacing + `draw_bg: {…}`. Plain walk/layout values (`padding`, `margin`, `width`) + are set directly (`padding: Inset{left: 20}`). +4. Values come back in source spelling (`#rrggbbaa` colors, plain numbers) — + paste them as-is. Mind the Rust-tokenizer hex-`e` trap in `script_mod!`: + `#1e1e2e` must be written `#x1e1e2e`. +5. Rebuild and relaunch; verify the value survived with `/tweak/state` or + `/snap` before calling it done. + +Reflection truth: `props` come from the widget's live Rust fields plus the +type's DSL-declared shader inputs (`instance()`/`uniform()`), so the list is +what the widget actually exposes — there is no synthetic schema to drift. + ### Studio remote bridge (the older path) The studio (`studio/desktop` + `studio/hub`) drives a hosted app over a diff --git a/Cargo.toml b/Cargo.toml index 926bee45f..ae7ff2f01 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,17 +1,20 @@ workspace.members = [ # === app === - "apps/mpbrowser", - "apps/mpterm", - "apps/mpwm", + "apps/browser", + "apps/terminal", + "apps/wm", + "apps/aichat", "apps/finance", - "apps/mpsheets", - "libs/mp_theme", - "libs/mp_wm_api", - "apps/mptask", - "apps/mpimage", - "apps/mpvideo", - "apps/mppdf", - "apps/mpfiles", + "apps/sheets", + "apps/photos", + "libs/wm_theme", + "libs/wm_api", + "libs/app_module", + "apps/task", + "apps/image", + "apps/video", + "apps/pdf", + "apps/files", "apps/route", # === arcade (game.md) — networked AI game sandbox === "apps/arcade", @@ -27,6 +30,10 @@ workspace.members = [ "libs/game/assets", "apps/asset-ui", "apps/asset-server", + "apps/flow-server", + "apps/flow-ui", + "libs/flowgraph", + "libs/media_view", "apps/ai-hub", "apps/vj", "apps/mixer", @@ -40,6 +47,8 @@ workspace.members = [ "libs/asset/store", "libs/asset/chat", "libs/chat_ui", + "libs/ai/hub_ui", + "libs/ai/services", "libs/asset/annotate", "libs/render", "libs/raytrace", @@ -50,6 +59,8 @@ workspace.members = [ "libs/sim/math", "libs/show_control", "libs/asset/creator", + "libs/bounded_http", + "libs/flow", "libs/strict_json", # === remesh (FaithC port) / xatlas (jpcy port) === "libs/remesh", @@ -94,6 +105,9 @@ workspace.members = [ "examples/render_to_texture", # === digital-fabrication product === "apps/fab", + "apps/fabric", + "libs/fabric/measure", + "libs/fabric/draft", # === xr app === "xr", # === studio === @@ -106,6 +120,8 @@ workspace.members = [ "libs/score_layout", "libs/score_play", "libs/score_render", + "libs/score_view", + "libs/score_view/tests/embed_app", "libs/score_ai", "libs/score_import", "libs/score_pdf", @@ -114,6 +130,8 @@ workspace.members = [ "libs/midi_file", "libs/soundfont", "libs/piano_model", + "libs/drumkit", + "libs/drumkit_phys", "libs/musicxml", # === own MP3 / Ogg Vorbis decoders === "libs/audio_decode", @@ -123,6 +141,7 @@ workspace.members = [ "libs/audio_lyrics", # === stems + lyrics side-channel bake/publish (asset-ui + VJ) === "libs/audio_sidechannels", + "libs/vj_analysis", "libs/teamtalk", # === pictures of audio (spectrogram, wave strip, composite) === "libs/audio_picture", @@ -135,16 +154,23 @@ workspace.members = [ "libs/frametween", # === archive.org search + download content input (VJ / asset-ui) === "libs/archive_org", + # === image tile wall: HEVC tape atlases + baker CLI + TileGrid widget + # (the engine extracted from the Source Library picture wall) === + "libs/image_tiles", + "examples/image_tiles", # === mp4 sample index for range-streaming playback === "libs/mp4_index", # === necessary tools === "platform/video", "tools/cargo_makepad", + "tools/makepad_loader", # === OSM PBF -> tile archive + nav artifact bake passes (CLI + in-app) === "libs/map_build", "tools/map_tiles", "tools/map_bake", "tools/remote", + "tools/dj_pack", + "libs/system_speech", # === tests === "platform/script/test", ] @@ -163,7 +189,6 @@ workspace.exclude = [ "libs/diffusion", # the AI model workspace (loader + cuda/metal stores + model crates) — aiarch.md "libs/ai", - "libs/voice", "widgets/test", "libs/stitch", "libs/wasm_bridge/test", @@ -280,6 +305,9 @@ strip = true inherits = "release" debug = true +[profile.test.package.makepad-stitch] +opt-level = 1 + #[profile.dev.package.makepad-live-tokenizer] #opt-level = 3 #[profile.dev.package.makepad-live-compiler] diff --git a/apps/ai-hub/Cargo.toml b/apps/ai-hub/Cargo.toml index 0ecb86897..4fd7773e7 100644 --- a/apps/ai-hub/Cargo.toml +++ b/apps/ai-hub/Cargo.toml @@ -14,7 +14,7 @@ makepad-ai-hub = { path = "../../libs/ai/hub", default-features = false } # Feature passthrough: the headless node exposes exactly the library's # feature set, so box launch scripts keep their --features flags working. [features] -default = ["flux", "paint", "paint-cuda", "llm", "tts", "indextts", "video", "interpolate", "audio", "mesh", "matte-native", "depth-native", "segment-native", "upscale-native", "motion-native", "rig-native", "splat-native"] +default = ["flux", "paint", "paint-cuda", "llm", "tts", "indextts", "video", "interpolate", "audio", "mesh", "matte-native", "depth-native", "segment-native", "body-native", "upscale-native", "motion-native", "rig-native", "splat-native", "stems-native"] python-backends = ["makepad-ai-hub/python-backends"] flux = ["makepad-ai-hub/flux"] paint = ["makepad-ai-hub/paint"] @@ -29,7 +29,9 @@ mesh = ["makepad-ai-hub/mesh"] matte-native = ["makepad-ai-hub/matte-native"] depth-native = ["makepad-ai-hub/depth-native"] segment-native = ["makepad-ai-hub/segment-native"] +body-native = ["makepad-ai-hub/body-native"] upscale-native = ["makepad-ai-hub/upscale-native"] motion-native = ["makepad-ai-hub/motion-native"] splat-native = ["makepad-ai-hub/splat-native"] rig-native = ["makepad-ai-hub/rig-native"] +stems-native = ["makepad-ai-hub/stems-native"] diff --git a/apps/ai-hub/src/main.rs b/apps/ai-hub/src/main.rs index 6d42b5fbe..edabd1e59 100644 --- a/apps/ai-hub/src/main.rs +++ b/apps/ai-hub/src/main.rs @@ -36,10 +36,14 @@ fn run() -> Result<(), AssetAiError> { let mut cache_dir: Option = None; let mut registry_path: Option = None; let mut machine = false; + let mut activity_probe = None; let mut args = std::env::args().skip(1); while let Some(arg) = args.next() { match arg.as_str() { + "--activity-probe" => { + activity_probe = Some(args.next().ok_or_else(|| AssetAiError::Io("--activity-probe needs seconds (1..3600)".into()))?.parse::().map_err(|_| AssetAiError::Io("invalid activity probe seconds".into()))?); + } "--port" => { let value = args .next() @@ -78,7 +82,7 @@ fn run() -> Result<(), AssetAiError> { } "--help" | "-h" => { println!( - "{SERVICE_NAME} {SERVICE_VERSION}\nusage: {SERVICE_NAME} [--port N] [--host ADDR] [--fleet NAME] [--cache-dir PATH] [--registry PATH] [--machine]" + "{SERVICE_NAME} {SERVICE_VERSION}\nusage: {SERVICE_NAME} [--port N] [--host ADDR] [--fleet NAME] [--cache-dir PATH] [--registry PATH] [--machine] [--activity-probe SECONDS]" ); return Ok(()); } @@ -88,6 +92,10 @@ fn run() -> Result<(), AssetAiError> { } } + if let Some(seconds) = activity_probe { + return makepad_ai_hub::activity::run_probe(seconds); + } + let port = match port { Some(port) => port, None => match std::env::var("MAKEPAD_ASSET_AI_PORT") { diff --git a/apps/aichat/Cargo.toml b/apps/aichat/Cargo.toml new file mode 100644 index 000000000..2791ed8c2 --- /dev/null +++ b/apps/aichat/Cargo.toml @@ -0,0 +1,46 @@ +# aichat — the assistant, as an app. +# +# One conversation that drives every application through the AI services +# layer. It is hosted three ways with one code base: under the window +# manager as a special child seated in the pane slot (this binary, +# `--stdin-loop`), standalone as its own window (this binary), and inside +# any app's Window as the F10 overlay or inside the mobile/web superbuild +# (this crate's lib, `makepad_aichat::script_mod` + `AiChatPanel{}`). +# +# The panel widget OWNS the engine: the service registry, the model, the +# transcript. Hosts only feed it links (in-process) or bus frames (the +# window manager's studio Custom frames) and give it a place to draw. +# +# Plan of record: repo-root aicontrol.md. + +[package] +name = "makepad-aichat" +version = "0.1.0" +edition = "2021" +default-run = "aichat" + +[lib] +name = "makepad_aichat" +path = "src/lib.rs" + +[[bin]] +name = "aichat" +path = "src/main.rs" + +[features] +default = ["engine"] +# The real models (the hub's local runtime, the cloud providers). Off for a +# build without a model runtime — the web page answers with NoModel and the +# tool console still works. +engine = ["makepad-ai-services/engine", "dep:makepad-asset-creator"] + +[dependencies] +makepad-widgets = { path = "../../widgets" } +makepad-wm-theme = { path = "../../libs/wm_theme" } +makepad-wm-api = { path = "../../libs/wm_api" } +makepad-ai-services = { path = "../../libs/ai/services", default-features = false } +makepad-strict-json = { path = "../../libs/strict_json" } +# The generative pipelines behind the assistant's own `gen` service +# (src/gen.rs): the creator runner picks a fleet node and brings the +# picture back. Native only, with the engine. +makepad-asset-creator = { path = "../../libs/asset/creator", optional = true } diff --git a/apps/aichat/src/bus.rs b/apps/aichat/src/bus.rs new file mode 100644 index 000000000..a50eb2846 --- /dev/null +++ b/apps/aichat/src/bus.rs @@ -0,0 +1,213 @@ +//! The client half of the window manager's service bus. +//! +//! Under the WM the other apps are not in this process. The WM forwards +//! their up-frames to the aichat child as studio `Custom` frames, each +//! stamped with the endpoint the WM issued to the sender, and forwards +//! the aichat child's down-frames (which name their target endpoint) back +//! to the right client. This adapter turns those frames into ordinary +//! [`ServiceLink`]s in the panel's registry, so the engine never knows +//! whether a service is a channel away or a process away. +//! +//! One link per endpoint. A `Register` from an endpoint the registry does +//! not know creates the link and registers it under the WM's endpoint id +//! (`register_as`); a later `Register` from the same endpoint is just the +//! manifest going down the existing link, where the registry answers it. +//! The WM tells us about a dead client by sending `Unregister` on its +//! behalf. Everything the registry sends down a bus link is drained here +//! and put on the wire to the WM. + +use makepad_ai_services::engine::ServiceRegistry; +use makepad_ai_services::port::{ServiceLink, ServiceLinkHost}; +use makepad_ai_services::wire::*; +use makepad_widgets::makepad_platform::studio::AppToStudio; +use makepad_widgets::*; +use std::collections::HashMap; + +#[derive(Default)] +pub struct ServiceBus { + hosts: HashMap, +} + +impl ServiceBus { + /// An up-frame from the WM. `None` for frames that are not the bus's. + pub fn on_custom(&mut self, registry: &ServiceRegistry, json: &str) -> bool { + let Some(frame) = HostedUp::parse(json) else { return false }; + let Some(from) = frame.from.clone() else { return true }; + match (&frame.msg, self.hosts.get(&from)) { + (ServiceUp::Register { manifest, .. }, None) => { + let (link, host) = ServiceLink::pair(manifest.clone()); + if registry.register_as(link, from.clone(), "", None).is_ok() { + let _ = host.up.send(frame); + self.hosts.insert(from, host); + } + } + (_, Some(host)) => { + let _ = host.up.send(frame); + } + (_, None) => {} + } + true + } + + /// Put every frame the registry sent down a bus link on the wire. + /// Links whose registry side is gone are dropped. + pub fn relay_down(&mut self, registry: &ServiceRegistry) { + for frame in self.drain_down(registry) { + Cx::send_studio_message(AppToStudio::Custom(frame.to_json())); + } + } + + fn drain_down(&mut self, registry: &ServiceRegistry) -> Vec { + let mut gone: Vec = Vec::new(); + let mut frames = Vec::new(); + for (endpoint, host) in &self.hosts { + loop { + match host.down.try_recv() { + Ok(mut frame) => { + // `Registered` travels without a target; the WM + // routes it by the endpoint we register as. + if frame.to.is_none() { + frame.to = Some(endpoint.clone()); + } + frames.push(frame); + } + Err(std::sync::mpsc::TryRecvError::Empty) => break, + Err(std::sync::mpsc::TryRecvError::Disconnected) => { + gone.push(endpoint.clone()); + break; + } + } + } + } + for endpoint in gone { + self.hosts.remove(&endpoint); + registry.unregister(&endpoint); + } + frames + } + + pub fn len(&self) -> usize { + self.hosts.len() + } + + pub fn is_empty(&self) -> bool { + self.hosts.is_empty() + } +} + +#[cfg(test)] +mod tests { + use super::*; + use makepad_ai_services::engine::{EngineCore, NoModelWithReason, RegistryUp}; + + #[test] + fn hosted_subscriptions_and_messages_cross_the_adapter_unchanged() { + let registry = ServiceRegistry::new(); + let mut bus = ServiceBus::default(); + let endpoint = EndpointId("w4".into()); + let manifest = ServiceManifest::new("flow", "Flow", "A flow.") + .with_topic(TopicDef::new("run", "Run events.")); + let register = HostedUp { + from: Some(endpoint.clone()), + msg: ServiceUp::Register { manifest, port_tag: 4 }, + }; + assert!(bus.on_custom(®istry, ®ister.to_json())); + registry.pump(); + let host = bus.hosts.get(&endpoint).unwrap(); + let _registered = host.down.try_recv().unwrap(); + assert!(registry.send( + &endpoint, + ServiceDown::Subscribe { + sub_id: "s1".into(), + topic: "run".into(), + filter: None, + }, + )); + assert!(matches!( + host.down.try_recv().unwrap().msg, + ServiceDown::Subscribe { sub_id, topic, filter: None } + if sub_id == "s1" && topic == "run" + )); + let message = HostedUp { + from: Some(endpoint.clone()), + msg: ServiceUp::Message { + sub_id: "s1".into(), + topic: "run".into(), + text: "finished".into(), + data: None, + final_: true, + }, + }; + assert!(bus.on_custom(®istry, &message.to_json())); + assert!(matches!( + registry.pump().as_slice(), + [RegistryUp::Message { endpoint: from, sub_id, message }] + if from == &endpoint && sub_id == "s1" && message.final_ + )); + } + + #[test] + fn engine_shutdown_is_flushed_through_the_hosted_service_bus() { + let registry = ServiceRegistry::new(); + let mut bus = ServiceBus::default(); + let endpoint = EndpointId("w4".into()); + let manifest = ServiceManifest::new("flow", "Flow", "A flow.") + .with_tool(ToolDef::new( + "watch", + "Watch a run.", + r#"{"type":"object","properties":{}}"#, + Risk::Read, + )) + .with_topic(TopicDef::new("run", "Run events.")); + assert!(bus.on_custom( + ®istry, + &HostedUp { + from: Some(endpoint.clone()), + msg: ServiceUp::Register { manifest, port_tag: 4 }, + } + .to_json(), + )); + let mut engine = EngineCore::new( + registry.clone(), + Box::new(NoModelWithReason::new("not used by the tool console")), + None, + 0x44, + ); + engine.send("/flow.watch {}", 0.0); + let call_id = bus + .drain_down(®istry) + .into_iter() + .find_map(|frame| match frame.msg { + ServiceDown::Call(call) => Some(call.call_id), + _ => None, + }) + .expect("the hosted service receives the call"); + assert!(bus.on_custom( + ®istry, + &HostedUp { + from: Some(endpoint.clone()), + msg: ServiceUp::Result( + ToolResult::ok(call_id, "watching", "") + .with_subscription(SubscriptionRequest::new("run")), + ), + } + .to_json(), + )); + engine.pump(0.1); + let sub_id = bus + .drain_down(®istry) + .into_iter() + .find_map(|frame| match frame.msg { + ServiceDown::Subscribe { sub_id, .. } => Some(sub_id), + _ => None, + }) + .expect("the hosted service receives the subscription"); + assert_eq!(sub_id, "l44-s1"); + engine.shutdown(); + assert!(matches!( + bus.drain_down(®istry).as_slice(), + [HostedDown { to: Some(to), msg: ServiceDown::Unsubscribe { sub_id: ended } }] + if to == &endpoint && ended == &sub_id + )); + } +} diff --git a/apps/aichat/src/gen.rs b/apps/aichat/src/gen.rs new file mode 100644 index 000000000..647c22d26 --- /dev/null +++ b/apps/aichat/src/gen.rs @@ -0,0 +1,297 @@ +//! The assistant's own generative service: `gen.image{prompt}`. +//! +//! The hub's pipelines are not an app, so no app registers them; the +//! panel registers this service in-process beside whatever apps join. +//! An `image` call runs the creator pipeline on a worker thread (the +//! runner is blocking: node pick over the LAN fleet, request, poll, +//! fetch), streams the node's progress into the card, writes the picture +//! under the makepad home's `gen` folder and answers with the path — the +//! model then hands that path to `photos.add`, which puts it on the wall. +//! Nothing goes through the asset store. Without the `engine` feature (the +//! web page) the service still exists and says it cannot. + +use makepad_ai_services::engine::ServiceRegistry; +use makepad_ai_services::port::{AiServicePort, PortEvent}; +use makepad_ai_services::wire::{Risk, ServiceCall, ServiceManifest, ToolDef, ToolResult}; +use makepad_widgets::*; +use std::path::PathBuf; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::mpsc::{Receiver, TryRecvError}; +use std::sync::Arc; + +/// The service id on the bus. +pub const SERVICE_ID: &str = "gen"; + +pub fn manifest() -> ServiceManifest { + ServiceManifest::new( + SERVICE_ID, + "Generate", + "The machine's generative pipelines, on the fleet's GPU nodes: a \ + picture from a prompt. The picture is saved on this machine under \ + the makepad home's gen folder and the answer carries its path. To \ + SHOW it, call photos.add with that path (launch Photos first with \ + os.launch if it is not running) — the wall then glides onto it. A \ + generation takes half a minute on a warm node, longer when a node \ + must load the model.", + ) + .with_tool(ToolDef::new( + "image", + "Generate one picture from a text prompt on a fleet image node; saves it under the makepad home's gen folder and returns the path.", + r#"{"type":"object","properties":{"prompt":{"type":"string","description":"what the picture shows, in plain words"},"width":{"type":"integer","description":"pixels, optional (default 1024)"},"height":{"type":"integer","description":"pixels, optional (default 1024)"}},"required":["prompt"]}"#, + Risk::Act, + )) +} + +/// What the worker reports back. +enum GenMsg { + Progress(String, u16), + Done(Result), +} + +struct GenDone { + path: PathBuf, + node: String, +} + +struct Job { + call_id: String, + cancel: Arc, + rx: Receiver, +} + +/// The in-process port plus the jobs in flight. +pub struct GenService { + port: AiServicePort, + jobs: Vec, +} + +impl GenService { + /// Open the service and register it in the panel's registry. `None` + /// only when the manifest does not validate (a programming error). + pub fn open(registry: &ServiceRegistry) -> Option { + let (port, link) = AiServicePort::in_process(manifest()).ok()?; + registry.register(link, "built in", None).ok()?; + Some(GenService { port, jobs: Vec::new() }) + } + + /// Drain the port and the workers; called on every panel event. + pub fn handle_event(&mut self, cx: &mut Cx, event: &Event) { + for ev in self.port.handle_event(cx, event) { + match ev { + PortEvent::Call(call) => self.start(call), + PortEvent::Cancel { call_id } => { + if let Some(job) = self.jobs.iter().find(|j| j.call_id == call_id) { + job.cancel.store(true, Ordering::Relaxed); + } + } + PortEvent::Registered(_) + | PortEvent::ChatOpen { .. } + | PortEvent::Subscribe { .. } + | PortEvent::Unsubscribe { .. } => {} + } + } + self.poll(); + } + + fn start(&mut self, call: ServiceCall) { + let id = call.call_id.clone(); + if call.tool != "image" { + self.port.reply(ToolResult::refused(&id, format!("gen has no tool `{}`; it has image", call.tool))); + return; + } + let args = match parse_image_args(&call.args) { + Ok(a) => a, + Err(why) => { + self.port.reply(ToolResult::refused(&id, why)); + return; + } + }; + #[cfg(not(feature = "engine"))] + { + let _ = args; + self.port.reply(ToolResult::unavailable(&id, "this build has no pipeline runtime; pictures need the native app")); + } + #[cfg(feature = "engine")] + { + use makepad_widgets::makepad_platform::thread::SignalToUI; + let cancel = Arc::new(AtomicBool::new(false)); + let (tx, rx) = std::sync::mpsc::channel(); + let worker_cancel = cancel.clone(); + let spawned = std::thread::Builder::new().name("gen-image".into()).spawn(move || { + let progress_tx = tx.clone(); + let mut progress = |note: &str, permille: u16| { + let _ = progress_tx.send(GenMsg::Progress(note.to_string(), permille)); + SignalToUI::set_ui_signal(); + }; + let result = run_image(&args, &worker_cancel, &mut progress); + let _ = tx.send(GenMsg::Done(result)); + SignalToUI::set_ui_signal(); + }); + match spawned { + Ok(_) => self.jobs.push(Job { call_id: id, cancel, rx }), + Err(e) => self.port.reply(ToolResult::failed(&id, format!("could not start the generation: {e}"))), + } + } + } + + fn poll(&mut self) { + let mut done: Vec = Vec::new(); + for (i, job) in self.jobs.iter().enumerate() { + loop { + match job.rx.try_recv() { + Ok(GenMsg::Progress(note, permille)) => self.port.progress(&job.call_id, ¬e, permille), + Ok(GenMsg::Done(Ok(out))) => { + let path = out.path.to_string_lossy().to_string(); + self.port.reply( + ToolResult::ok( + &job.call_id, + format!("saved {path} (made on {}). Show it on the wall with photos.add {{\"path\":\"{path}\"}}.", out.node), + "saved", + ) + .with_data(format!("{{\"path\":{},\"node\":{}}}", json_string(&path), json_string(&out.node))), + ); + done.push(i); + break; + } + Ok(GenMsg::Done(Err(e))) => { + let result = if e == "cancelled" { ToolResult::cancelled(&job.call_id) } else { ToolResult::failed(&job.call_id, e) }; + self.port.reply(result); + done.push(i); + break; + } + Err(TryRecvError::Empty) => break, + Err(TryRecvError::Disconnected) => { + self.port.reply(ToolResult::failed(&job.call_id, "the generation worker died")); + done.push(i); + break; + } + } + } + } + for i in done.into_iter().rev() { + self.jobs.remove(i); + } + } +} + +/// The arguments of one `image` call, checked. +#[derive(Clone, Debug, PartialEq)] +pub struct ImageArgs { + pub prompt: String, + pub width: u32, + pub height: u32, +} + +/// Sizes are clamped to what the image nodes serve; a prompt is required. +pub fn parse_image_args(args: &str) -> Result { + use makepad_strict_json as json; + let fields = match json::parse(args.as_bytes()) { + Ok(json::Value::Obj(fields)) => fields, + _ => return Err("image needs a JSON object with a `prompt`".to_string()), + }; + let get = |key: &str| fields.iter().find(|(k, _)| k == key).map(|(_, v)| v.clone()); + let prompt = get("prompt").and_then(|v| v.as_str().map(|s| s.trim().to_string())).unwrap_or_default(); + if prompt.is_empty() { + return Err("image needs a `prompt`".to_string()); + } + if prompt.len() > 4000 { + return Err("the prompt is longer than 4000 bytes".to_string()); + } + let dim = |key: &str| -> Result { + match get(key) { + None => Ok(1024), + Some(json::Value::Int(i)) if (256..=2048).contains(&i) => Ok(i as u32), + Some(_) => Err(format!("`{key}` must be an integer from 256 to 2048")), + } + }; + Ok(ImageArgs { prompt, width: dim("width")?, height: dim("height")? }) +} + +/// A short file-name-safe slug of the prompt. +pub fn slug(prompt: &str) -> String { + let mut out = String::new(); + for ch in prompt.chars().flat_map(|c| c.to_lowercase()) { + if ch.is_ascii_alphanumeric() { + out.push(ch); + } else if !out.ends_with('-') && !out.is_empty() { + out.push('-'); + } + if out.len() >= 40 { + break; + } + } + let out = out.trim_matches('-').to_string(); + if out.is_empty() { "picture".to_string() } else { out } +} + +/// The file extension for a content type the nodes produce. +pub fn extension_for(content_type: &str) -> &'static str { + match content_type.split(';').next().unwrap_or("").trim() { + "image/jpeg" => "jpg", + "image/webp" => "webp", + "image/gif" => "gif", + _ => "png", + } +} + +fn json_string(s: &str) -> String { + makepad_strict_json::s(s.to_string()).to_json() +} + +#[cfg(feature = "engine")] +fn run_image(args: &ImageArgs, cancel: &Arc, progress: &mut dyn FnMut(&str, u16)) -> Result { + use makepad_asset_creator::makepad_ai_hub::home::makepad_home; + use makepad_asset_creator::makepad_strict_json as json; + use makepad_asset_creator::runner::generate_bytes; + let body = json::obj(vec![ + ("prompt", json::s(args.prompt.clone())), + ("width", json::Value::Int(args.width as i64)), + ("height", json::Value::Int(args.height as i64)), + ]); + let seed = (Cx::time_now().max(0.0) * 1_000_000_000.0) as u64; + progress("finding an image node", 0); + let generated = generate_bytes("image.generate", &body, seed, cancel, progress)?; + let artifact = generated.artifact.ok_or("the node returned no picture")?; + let dir = makepad_home().join("gen"); + std::fs::create_dir_all(&dir).map_err(|e| format!("cannot make {}: {e}", dir.display()))?; + let stamp = Cx::time_now().max(0.0) as u64; + let path = dir.join(format!("{stamp}-{}.{}", slug(&args.prompt), extension_for(&artifact.content_type))); + std::fs::write(&path, &artifact.bytes).map_err(|e| format!("cannot write {}: {e}", path.display()))?; + Ok(GenDone { path, node: generated.node }) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn the_manifest_validates_and_the_tool_acts() { + let m = manifest(); + m.validate().expect("a manifest the wire accepts"); + assert_eq!(m.id, "gen"); + assert_eq!(m.tools.len(), 1); + assert_eq!(m.tools[0].name, "image"); + assert_eq!(m.tools[0].risk, Risk::Act); + } + + #[test] + fn image_args_need_a_prompt_and_sane_sizes() { + let a = parse_image_args(r#"{"prompt":"a red bicycle on the moon"}"#).unwrap(); + assert_eq!((a.width, a.height), (1024, 1024)); + let b = parse_image_args(r#"{"prompt":"x","width":512,"height":768}"#).unwrap(); + assert_eq!((b.width, b.height), (512, 768)); + assert!(parse_image_args(r#"{"width":512}"#).unwrap_err().contains("prompt")); + assert!(parse_image_args(r#"{"prompt":"x","width":16}"#).unwrap_err().contains("256")); + assert!(parse_image_args("nope").is_err()); + } + + #[test] + fn file_names_are_slugs_with_the_right_extension() { + assert_eq!(slug("A red bicycle, on the Moon!"), "a-red-bicycle-on-the-moon"); + assert_eq!(slug(" "), "picture"); + assert!(slug(&"word ".repeat(30)).len() <= 41); + assert_eq!(extension_for("image/png"), "png"); + assert_eq!(extension_for("image/jpeg; charset=binary"), "jpg"); + assert_eq!(extension_for("application/octet-stream"), "png"); + } +} diff --git a/apps/aichat/src/lib.rs b/apps/aichat/src/lib.rs new file mode 100644 index 000000000..383a19b66 --- /dev/null +++ b/apps/aichat/src/lib.rs @@ -0,0 +1,89 @@ +//! aichat as a library: the panel widget that owns the engine, the WM-bus +//! client half, the settings, and the module root. A host links this, +//! calls [`script_mod`] after the widgets' own, and puts `AiChatPanel{}` +//! where the chat goes — the standalone binary and the WM pane child do +//! that themselves; a plain `Window` does it for free through its F10 +//! slot, which instantiates `mod.widgets.AiChatOverlay{}` by name; the +//! superbuild seats the same overlay in its pane. +//! +//! Linking this crate also gives the bridge its `/ai` routes: `script_mod` +//! installs `Cx::ai_callback`, the way the widgets crate installs the +//! tweaker's, so `/ai?on=1`, `/ai?say=…` and `/ai/transcript` drive and +//! read the chat in a hidden instance. + +pub use makepad_widgets; +use makepad_widgets::ai_slot::AiSlotRequests; +use makepad_widgets::makepad_platform::ScriptVmCx; +use makepad_widgets::*; + +pub mod bus; +pub mod gen; +pub mod overlay; +pub mod panel; +pub mod settings; + +pub use bus::ServiceBus; +pub use overlay::{AiChatOverlay, AiTranscript}; +pub use panel::{AiChatPanel, AiChatPanelAction}; +pub use settings::AiSettings; + +/// Register the panel and the overlay, and give the bridge its `/ai` +/// routes. Call once after `makepad_widgets::script_mod`. +pub fn script_mod(vm: &mut ScriptVm) { + crate::panel::script_mod(vm); + crate::overlay::script_mod(vm); + vm.cx_mut().ai_callback = Some(ai_callback); +} + +fn arg<'a>(args: &'a [(String, String)], keys: &[&str]) -> Option<&'a str> { + for key in keys { + if let Some((_, value)) = args.iter().find(|(k, _)| k == key) { + return Some(value.as_str()); + } + } + None +} + +/// The bridge's `/ai` dispatcher. `toggle` (`on=1|0`, or flip) and `say` +/// (`say=TEXT`, opening the overlay if it is closed) are requests the +/// slot and the overlay take on their next event; `transcript` is what +/// the overlay last published. Never borrows a widget. +fn ai_callback(cx: &mut Cx, op: &str, args: &[(String, String)]) -> Result { + match op { + "toggle" => { + let is_open = cx.global::().is_open; + let on = match arg(args, &["on"]) { + Some(value) => !matches!(value, "0" | "false" | "off" | "no"), + None => !is_open, + }; + cx.global::().open = Some(on); + // The slot takes the request on its next event: make one. + cx.new_next_frame(); + cx.redraw_all(); + Ok(format!("{{\"on\":{}}}", on as u8)) + } + "say" => { + let text = arg(args, &["say", "t"]).unwrap_or("").trim().to_string(); + if text.is_empty() { + return Err("need say=TEXT".into()); + } + let req = cx.global::(); + if !req.is_open { + req.open = Some(true); + } + req.say.push(text); + cx.new_next_frame(); + cx.redraw_all(); + Ok("{\"ok\":1}".into()) + } + "transcript" => { + let json = cx.global::().json.clone(); + if json.is_empty() { + Ok("{\"status\":\"closed\",\"provider\":\"\",\"apps\":[],\"entries\":[],\"generation\":0}".into()) + } else { + Ok(json) + } + } + other => Err(format!("no ai op `{other}`; there are toggle, say, transcript")), + } +} diff --git a/apps/aichat/src/main.rs b/apps/aichat/src/main.rs new file mode 100644 index 000000000..7f1a4762e --- /dev/null +++ b/apps/aichat/src/main.rs @@ -0,0 +1,79 @@ +//! aichat: the assistant as its own window, and as the window manager's +//! special child. Both are this binary: standalone it is a window with +//! the panel in it and whatever services are linked in-process (none, +//! until an app embeds it); under the WM (`--stdin-loop`) the WM seats it +//! in the pane slot and every other app's service reaches it over the bus +//! as studio `Custom` frames, which the panel turns into registry links. + +pub use makepad_widgets; +use makepad_aichat::AiChatPanelAction; +use makepad_widgets::*; + +app_main!(App); + +script_mod! { + use mod.prelude.widgets.* + use mod.widgets.* + + startup() do #(App::script_component(vm)){ + ui: Root{ + main_window := Window{ + window.inner_size: vec2(460, 760) + window.title: "AI" + pass +: { clear_color: theme.color_bg_app } + body +: { + panel := AiChatPanel{ + width: Fill + height: Fill + } + } + } + } + } +} + +#[derive(Script, ScriptHook)] +pub struct App { + #[live] + ui: WidgetRef, +} + +impl MatchEvent for App { + fn handle_startup(&mut self, cx: &mut Cx) { + makepad_wm_api::set_title(cx, "AI"); + } + + fn handle_actions(&mut self, cx: &mut Cx, actions: &Actions) { + for action in actions { + if let Some(widget_action) = action.as_widget_action() { + if let AiChatPanelAction::Close = widget_action.cast() { + // Under the WM the pane hides; standalone the window stays. + let _ = makepad_wm_api::send(cx, &makepad_wm_api::WmRequest::Close); + } + } + } + } +} + +impl AppMain for App { + fn script_mod(vm: &mut ScriptVm) -> ScriptValue { + crate::makepad_widgets::script_mod(vm); + makepad_wm_theme::apply(vm); + makepad_aichat::script_mod(vm); + self::script_mod(vm) + } + + fn handle_event(&mut self, cx: &mut Cx, event: &Event) { + // Bus frames reach the panel through its own handle_event; of the + // WM's own frames only the polite close is ours (the pane hides us + // by leaving us running; this is the desktop going down). + if let Event::Custom(json) = event { + if let Some(makepad_wm_api::WmEvent::CloseRequested) = makepad_wm_api::WmEvent::parse(json) { + cx.quit(); + return; + } + } + self.match_event(cx, event); + self.ui.handle_event(cx, event, &mut Scope::empty()); + } +} diff --git a/apps/aichat/src/overlay.rs b/apps/aichat/src/overlay.rs new file mode 100644 index 000000000..93774b7a5 --- /dev/null +++ b/apps/aichat/src/overlay.rs @@ -0,0 +1,261 @@ +//! The chat's module root: what a host seats in-process. +//! +//! `mod.widgets.AiChatOverlay{}` is what the Window's AI slot +//! (`widgets/src/ai_slot.rs`) instantiates by name on F10, and what the +//! superbuild seats in its pane. It is a `View` with the panel in it and +//! three duties around it: adopt the in-process service links the apps +//! parked on `Cx` ([`PendingServiceLinks`]) into the panel's registry, +//! send the lines the bridge asked to say ([`AiSlotRequests::say`]), +//! and publish the transcript as JSON ([`AiTranscript`]) after each draw +//! so `/ai/transcript` answers without touching a widget. The panel's +//! Escape (an idle, empty composer) asks the slot to close. + +use crate::panel::{AiChatPanel, AiChatPanelAction}; +use makepad_ai_services::port::PendingServiceLinks; +use makepad_ai_services::state::{Entry, EngineState, Status, ToolStatus}; +use makepad_widgets::ai_slot::AiSlotRequests; +use makepad_widgets::makepad_micro_serde::*; +use makepad_widgets::*; + +script_mod! { + use mod.prelude.widgets_internal.* + use mod.widgets.* + + mod.widgets.AiChatOverlayBase = #(AiChatOverlay::register_widget(vm)) + mod.widgets.AiChatOverlay = set_type_default() do mod.widgets.AiChatOverlayBase{ + width: Fill + height: Fill + panel := AiChatPanel{ + width: Fill + height: Fill + } + } +} + +/// The transcript as the bridge reads it: a `Cx` global the overlay +/// rewrites after every draw of the panel. +#[derive(Default)] +pub struct AiTranscript { + pub json: String, +} + +#[derive(SerJson)] +struct TranscriptRow { + kind: String, + text: String, + title: String, + status: String, + note: String, +} + +#[derive(SerJson)] +struct Transcript { + status: String, + provider: String, + apps: Vec, + entries: Vec, + generation: u64, +} + +/// The transcript JSON for a state: `status`, `provider`, the connected +/// apps, and one row per entry with what its card says. +pub fn transcript_json(state: &EngineState) -> String { + let status = match &state.status { + Status::Idle => "idle".to_string(), + Status::Loading { phase, fraction } => format!("loading {phase} {:.0}%", fraction * 100.0), + Status::Thinking => "thinking".to_string(), + Status::Streaming => "streaming".to_string(), + Status::WaitingForTool => "waiting_for_tool".to_string(), + Status::Error(e) => format!("error: {e}"), + }; + let entries = state + .entries + .iter() + .map(|entry| match entry { + Entry::User { text } => TranscriptRow { + kind: "user".into(), + text: text.clone(), + title: String::new(), + status: String::new(), + note: String::new(), + }, + Entry::Event(event) => TranscriptRow { + kind: "event".into(), + text: event.text.clone(), + title: format!("{} · {}", event.service_label, event.topic), + status: if event.final_ { "final".into() } else { "message".into() }, + note: if event.dropped == 0 { + format!("sub_id: {}", event.sub_id) + } else { + format!("sub_id: {} · dropped: {}", event.sub_id, event.dropped) + }, + }, + Entry::Assistant { text, streaming } => TranscriptRow { + kind: "assistant".into(), + text: text.clone(), + title: String::new(), + status: if *streaming { "streaming".into() } else { "done".into() }, + note: String::new(), + }, + Entry::Tool(t) => { + let (status, note, text) = match &t.status { + ToolStatus::Confirm => ("confirm".to_string(), String::new(), String::new()), + ToolStatus::Running { note, permille } => ("running".to_string(), format!("{note} {permille}‰"), String::new()), + ToolStatus::Done { outcome, note, text } => (outcome.slug().to_string(), note.clone(), text.clone()), + }; + TranscriptRow { kind: "tool".into(), text, title: t.title.clone(), status, note } + } + Entry::System { text } => TranscriptRow { + kind: "system".into(), + text: text.clone(), + title: String::new(), + status: String::new(), + note: String::new(), + }, + }) + .collect(); + Transcript { + status, + provider: state.provider_label.clone(), + apps: state.services.iter().filter(|s| s.connected).map(|s| s.label.clone()).collect(), + entries, + generation: state.generation, + } + .serialize_json() +} + +#[derive(Script, ScriptHook, Widget)] +pub struct AiChatOverlay { + #[source] + source: ScriptObjectRef, + #[deref] + view: View, + /// The state generation last published as JSON. + #[rust] + published: Option, +} + +impl AiChatOverlay { + /// The transcript JSON, rewritten when the engine's state moved — on + /// every event as well as every draw, so a closed slot (no draws) still + /// answers `/ai/transcript` with the turn that finished behind it. + fn publish_transcript(&mut self, cx: &mut Cx) { + let json = self + .view + .widget(cx, ids!(panel)) + .borrow::() + .and_then(|panel| { + let state = panel.state()?; + if self.published == Some(state.generation) { + return None; + } + Some((state.generation, transcript_json(state))) + }); + if let Some((generation, json)) = json { + self.published = Some(generation); + cx.global::().json = json; + } + } + + /// The links the apps parked and the lines the bridge asked for, into + /// the panel. + fn adopt_requests(&mut self, cx: &mut Cx) { + let links = cx.global::().take(); + let says = std::mem::take(&mut cx.global::().say); + if links.is_empty() && says.is_empty() { + return; + } + let panel = self.view.widget(cx, ids!(panel)); + let Some(mut panel) = panel.borrow_mut::() else { + return; + }; + for link in links { + let label = link.manifest.label.clone(); + match panel.registry().register(link, "in this window", None) { + Ok(endpoint) => log!("aichat: {label} joined in-process as {}", endpoint.as_str()), + Err(e) => log!("aichat: {label} refused: {e}"), + } + } + for text in says { + panel.say(cx, text); + } + } +} + +impl Widget for AiChatOverlay { + fn handle_event(&mut self, cx: &mut Cx, event: &Event, scope: &mut Scope) { + self.adopt_requests(cx); + self.view.handle_event(cx, event, scope); + self.publish_transcript(cx); + if let Event::Actions(actions) = event { + for action in actions { + if let Some(widget_action) = action.as_widget_action() { + if let AiChatPanelAction::Close = widget_action.cast() { + cx.global::().open = Some(false); + cx.redraw_all(); + } + } + } + } + } + + fn draw_walk(&mut self, cx: &mut Cx2d, scope: &mut Scope, walk: Walk) -> DrawStep { + let step = self.view.draw_walk(cx, scope, walk); + if step.is_done() { + self.publish_transcript(cx); + } + step + } +} + +#[cfg(test)] +mod tests { + use super::*; + use makepad_ai_services::state::{EventEntry, ServiceInfo, ToolEntry}; + use makepad_ai_services::wire::ToolOutcome; + + #[test] + fn the_transcript_reads_as_json_rows() { + let mut state = EngineState::default(); + state.provider_label = "No model".into(); + state.services.push(ServiceInfo { + id: "sheets".into(), + endpoint: "e1".into(), + label: "Sheets".into(), + parent: None, + location: "in this window".into(), + connected: true, + launchable: false, + tool_count: 1, + }); + state.entries.push(Entry::User { text: "/sheets.summary {}".into() }); + state.entries.push(Entry::Event(EventEntry { + sub_id: "s1".into(), + service_label: "Sheets".into(), + topic: "changes".into(), + text: "A1 changed".into(), + data: None, + dropped: 2, + final_: false, + })); + state.entries.push(Entry::Tool(ToolEntry { + call_id: "c1".into(), + service: "sheets".into(), + service_label: "Sheets".into(), + tool: "summary".into(), + title: "Sheets · summary".into(), + args: "{}".into(), + status: ToolStatus::Done { outcome: ToolOutcome::Ok, note: "3 × 4".into(), text: "Sheet1, 3 rows".into() }, + preview: false, + expanded: false, + })); + state.generation = 7; + let json = transcript_json(&state); + assert!(json.contains(r#""status":"idle""#), "{json}"); + assert!(json.contains(r#""apps":["Sheets"]"#), "{json}"); + assert!(json.contains(r#""kind":"user""#) && json.contains(r#""kind":"tool""#), "{json}"); + assert!(json.contains(r#""kind":"event""#) && json.contains("sub_id: s1") && json.contains("dropped: 2"), "{json}"); + assert!(json.contains(r#""title":"Sheets · summary""#) && json.contains(r#""status":"ok""#), "{json}"); + assert!(json.contains(r#""generation":7"#), "{json}"); + } +} diff --git a/apps/aichat/src/panel.rs b/apps/aichat/src/panel.rs new file mode 100644 index 000000000..3a27cf844 --- /dev/null +++ b/apps/aichat/src/panel.rs @@ -0,0 +1,589 @@ +//! The chat panel: the one widget every host shows. It owns the engine — +//! the service registry, the model, the transcript — and draws +//! `EngineState` as a transcript of user lines, assistant text, tool cards +//! (running, done, waiting for a confirm), and system lines, over a +//! composer. +//! +//! Hosts talk to it in two ways: they hand it service links +//! (`registry()`) or bus frames (`on_custom`), and they listen for +//! [`AiChatPanelAction`]s. The engine runs on the panel's own events: every +//! event pumps it, and while a turn is in flight the panel asks for the +//! next frame so streaming, deadlines and the cloud provider's polling all +//! advance without a host timer. + +use crate::bus::ServiceBus; +use crate::gen::GenService; +use crate::settings::AiSettings; +#[cfg(feature = "engine")] +use makepad_ai_services::engine::models::{build_model, provider_rows}; +use makepad_ai_services::engine::NoModelWithReason; +use makepad_ai_services::engine::{EngineCore, EngineEvent, ServiceRegistry}; +use makepad_ai_services::state::*; +use makepad_ai_services::wire::ToolOutcome; +use makepad_widgets::*; + +script_mod! { + use mod.prelude.widgets_internal.* + use mod.widgets.* + + mod.widgets.AiChatPanelBase = #(AiChatPanel::register_widget(vm)) + + let Line = Label{ + width: Fill + height: Fit + draw_text +: { + color: theme.color_text + text_style: theme.font_regular{font_size: 9.5} + } + } + + let Row = View{ + width: Fill + height: Fit + flow: Down + padding: Inset{left: 14 right: 14 top: 4 bottom: 4} + } + + mod.widgets.AiChatPanel = set_type_default() do mod.widgets.AiChatPanelBase{ + width: Fill + height: Fill + flow: Down + draw_bg +: { color: theme.color_bg_app } + + header := SolidView{ + width: Fill + height: 36 + flow: Right + spacing: 10 + padding: Inset{left: 14 right: 8} + align: Align{y: 0.5} + draw_bg +: { color: theme.color_bg_container } + title := Label{ + text: "AI" + draw_text +: { + color: theme.color_text + text_style: theme.font_bold{font_size: 10.5} + } + } + provider := Label{ + width: Fill + text: "" + draw_text +: { + color: theme.color_text_meta + text_style: theme.font_regular{font_size: 8.5} + } + } + clear_button := ButtonFlatter{ text: "Clear" } + } + + apps_row := Label{ + width: Fill + height: Fit + padding: Inset{left: 14 right: 14 top: 6 bottom: 2} + max_lines: 2 + text: "" + draw_text +: { + color: theme.color_text_meta + text_style: theme.font_regular{font_size: 8.5} + } + } + + transcript := PortalList{ + width: Fill + height: Fill + auto_tail: true + + UserRow := Row{ + padding: Inset{left: 14 right: 14 top: 10 bottom: 4} + user_text := Line{ + draw_text +: { + color: theme.color_text + text_style: theme.font_bold{font_size: 9.5} + } + } + } + EventRow := Row{ + margin: Inset{left: 10 right: 10 top: 5 bottom: 5} + padding: Inset{left: 10 right: 10 top: 7 bottom: 7} + draw_bg +: { color: theme.color_bg_container } + event_title := Line{ + draw_text +: { + color: theme.color_text_hl + text_style: theme.font_bold{font_size: 8.5} + } + } + event_text := Line{} + event_meta := Line{ + draw_text +: { + color: theme.color_text_meta + text_style: theme.font_regular{font_size: 8.0} + } + } + } + AssistantRow := Row{ + assistant_md := Markdown{ + width: Fill + height: Fit + body: "" + } + } + StreamRow := Row{ + stream_text := Line{} + } + ToolRow := Row{ + padding: Inset{left: 22 right: 14 top: 3 bottom: 3} + tool_head := View{ + width: Fill + height: Fit + cursor: MouseCursor.Hand + tool_title := Line{ + draw_text +: { + color: theme.color_text_meta + text_style: theme.font_regular{font_size: 8.5} + } + } + } + tool_note := Line{ + draw_text +: { + color: theme.color_text_meta + text_style: theme.font_regular{font_size: 8.5} + } + } + tool_bar := SolidView{ + width: Fill + height: 2 + margin: Inset{top: 3} + draw_bg +: { color: theme.color_text_hl } + } + tool_detail := Line{ + visible: false + draw_text +: { + color: theme.color_text_meta + text_style: theme.font_regular{font_size: 8.0} + } + } + } + ConfirmRow := Row{ + padding: Inset{left: 22 right: 14 top: 6 bottom: 6} + confirm_title := Line{ + draw_text +: { + color: theme.color_text + text_style: theme.font_regular{font_size: 9.0} + } + } + confirm_buttons := View{ + width: Fill + height: Fit + flow: Right + spacing: 8 + margin: Inset{top: 4} + run_button := Button{ text: "Run" } + deny_button := ButtonFlat{ text: "Cancel" } + } + } + SystemRow := Row{ + system_text := Line{ + draw_text +: { + color: theme.color_text_hl + text_style: theme.font_regular{font_size: 8.5} + } + } + } + } + + status := Label{ + width: Fill + height: Fit + padding: Inset{left: 14 right: 14 top: 4 bottom: 2} + max_lines: 2 + text: "" + draw_text +: { + color: theme.color_text_meta + text_style: theme.font_regular{font_size: 8.5} + } + } + + composer := View{ + width: Fill + height: Fit + flow: Right + spacing: 8 + padding: Inset{left: 12 right: 12 top: 6 bottom: 10} + align: Align{y: 0.5} + input := TextInput{ + width: Fill + height: Fit + empty_text: "Ask AI" + // The prompt is a hint, not text: a dark grey in every state, + // never the typed colour (the composer is always focused). + draw_text +: { + color_empty: #666666 + color_empty_hover: #777777 + color_empty_focus: #666666 + } + } + send_button := Button{ text: "Send" } + } + } +} + +/// What the panel tells its host. +#[derive(Clone, Debug, PartialEq, Default)] +pub enum AiChatPanelAction { + /// Esc with an empty composer and no turn in flight: the host may hide + /// the pane. + Close, + #[default] + None, +} + +#[derive(Script, ScriptHook, Widget)] +pub struct AiChatPanel { + #[source] + source: ScriptObjectRef, + #[deref] + view: View, + #[rust] + engine: Option, + #[rust] + registry: ServiceRegistry, + #[rust] + bus: ServiceBus, + /// The assistant's own `gen` service (pictures from the fleet), joined + /// to the registry with the engine. + #[rust] + gen: Option, + #[rust] + settings: Option, + #[rust] + drawn_generation: u64, + #[rust] + next_frame: NextFrame, + /// The composer took the keyboard once it existed on screen — a + /// focus set before the first draw lands on no area at all. + #[rust] + composer_focused: bool, +} + +impl AiChatPanel { + /// The registry a host plugs in-process links into. + pub fn registry(&self) -> &ServiceRegistry { + &self.registry + } + + /// A studio `Custom` frame that may be a bus frame from the WM. + pub fn on_custom(&mut self, json: &str) -> bool { + self.bus.on_custom(&self.registry, json) + } + + pub fn state(&self) -> Option<&EngineState> { + self.engine.as_ref().map(|e| e.state()) + } + + fn settings(&mut self) -> &AiSettings { + if self.settings.is_none() { + self.settings = Some(AiSettings::load()); + } + self.settings.as_ref().unwrap() + } + + /// The engine comes up on first use — a local model is a long load + /// and no host pays it before the person opens the pane. + fn ensure_engine(&mut self) -> &mut EngineCore { + if self.engine.is_none() { + let lease_id = self.widget_uid().0; + let settings = self.settings().clone(); + // The real models ride the `engine` feature; a build without a + // runtime (the web page) says so and keeps the tool console. + #[cfg(feature = "engine")] + let (model, rows): (Box, Vec) = ( + match build_model(&settings.provider, settings.local_only) { + Ok(m) => m, + Err(reason) => Box::new(NoModelWithReason::new(reason)), + }, + provider_rows(settings.local_only), + ); + #[cfg(not(feature = "engine"))] + let (model, rows): (Box, Vec) = + (Box::new(NoModelWithReason::new("this build has no model runtime")), Vec::new()); + let mut core = EngineCore::new(self.registry.clone(), model, None, lease_id); + // The state is the panel's window into the core; the core owns + // it, so provider facts go in through the core. + core.set_provider_facts(settings.provider.clone(), rows, settings.local_only); + self.engine = Some(core); + self.gen = GenService::open(&self.registry); + } + self.engine.as_mut().unwrap() + } + + fn now(cx: &Cx) -> f64 { + cx.seconds_since_app_start() + } + + /// A line as if typed and sent — the bridge's `/ai?say=`. + pub fn say(&mut self, cx: &mut Cx, text: String) { + self.send(cx, text); + } + + fn send(&mut self, cx: &mut Cx, text: String) { + let now = Self::now(cx); + self.ensure_engine().send(&text, now); + let input = self.view.text_input(cx, ids!(input)); + input.set_text(cx, ""); + // The widget drops the keyboard on submit; a chat composer keeps + // it, so the next line can be typed straight away. + input.take_key_focus(cx); + self.view.redraw(cx); + } + + fn apps_line(&self, state: &EngineState) -> String { + if state.services.is_empty() { + return "No apps connected.".into(); + } + state + .services + .iter() + .map(|s| if s.connected { s.label.clone() } else { format!("{} (not running)", s.label) }) + .collect::>() + .join(" · ") + } + + fn status_line(state: &EngineState) -> String { + let rate = state.rate.map(|r| format!(" · {r:.0} tok/s")).unwrap_or_default(); + match &state.status { + Status::Idle => rate.trim_start_matches(" · ").to_string(), + Status::Loading { phase, fraction } => format!("loading {phase} {:.0}%", fraction * 100.0), + Status::Thinking => { + if state.thinking.is_empty() { + "thinking…".to_string() + } else { + let tail: String = state.thinking.chars().rev().take(120).collect::>().into_iter().rev().collect(); + format!("thinking… {tail}") + } + } + Status::Streaming => format!("writing{rate}"), + Status::WaitingForTool => "waiting for the app…".to_string(), + Status::Error(e) => e.clone(), + } + } +} + +impl Drop for AiChatPanel { + fn drop(&mut self) { + // The engine only enqueues Unsubscribe frames. Flush them while the + // hosted bus and registry still exist; field destruction is too late. + if let Some(engine) = self.engine.as_mut() { + engine.shutdown(); + } + self.bus.relay_down(&self.registry); + } +} + +impl Widget for AiChatPanel { + fn handle_event(&mut self, cx: &mut Cx, event: &Event, scope: &mut Scope) { + if let Event::Custom(json) = event { + self.on_custom(json); + } + // Esc: stop a running turn; with nothing running and an empty + // composer, ask the host to hide the pane. + if let Event::KeyDown(ke) = event { + if ke.key_code == KeyCode::Escape { + let busy = self.engine.as_ref().map(|e| e.state().status.is_busy()).unwrap_or(false); + let empty = self.view.text_input(cx, ids!(input)).text().trim().is_empty(); + if busy { + let now = Self::now(cx); + self.ensure_engine().cancel(now); + self.view.redraw(cx); + } else if empty { + cx.widget_action(self.widget_uid(), AiChatPanelAction::Close); + } + } + } + // The built-in services answer before the engine pumps, so a + // finished picture lands in this same event. + if let Some(gen) = self.gen.as_mut() { + gen.handle_event(cx, event); + } + // Drive the engine on every event; the bus relays what it sent. + let now = Self::now(cx); + let mut changed = false; + let mut busy = false; + if let Some(engine) = self.engine.as_mut() { + for ev in engine.pump(now) { + match ev { + EngineEvent::Changed => changed = true, + EngineEvent::Confirm { .. } => changed = true, + } + } + busy = engine.needs_pump() + || engine.state().status.is_busy() + || matches!(engine.state().status, Status::Loading { .. }); + } + self.bus.relay_down(&self.registry); + if changed { + self.view.redraw(cx); + } + if busy { + self.next_frame = cx.new_next_frame(); + } + self.view.handle_event(cx, event, scope); + self.widget_match_event(cx, event, scope); + } + + fn draw_walk(&mut self, cx: &mut Cx2d, scope: &mut Scope, walk: Walk) -> DrawStep { + // The engine exists once the panel is on screen. + let _ = self.ensure_engine(); + let state = self.engine.as_ref().unwrap().state().clone(); + self.drawn_generation = state.generation; + self.view.label(cx, ids!(provider)).set_text(cx, &format!("{}{}", state.provider_label, if state.local_only { " · local only" } else { "" })); + self.view.label(cx, ids!(apps_row)).set_text(cx, &self.apps_line(&state)); + self.view.label(cx, ids!(status)).set_text(cx, &Self::status_line(&state)); + while let Some(item) = self.view.draw_walk(cx, scope, walk).step() { + let Some(mut list) = item.borrow_mut::() else { continue }; + let total = state.entries.len(); + list.set_item_range(cx, 0, total); + while let Some(index) = list.next_visible_item(cx) { + if index >= total { + continue; + } + let entry = &state.entries[index]; + let row = match entry { + Entry::User { text } => { + let row = list.item(cx, index, id!(UserRow)); + row.label(cx, ids!(user_text)).set_text(cx, text); + row + } + Entry::Event(event) => { + let row = list.item(cx, index, id!(EventRow)); + row.label(cx, ids!(event_title)) + .set_text(cx, &format!("→ {} · {}", event.service_label, event.topic)); + row.label(cx, ids!(event_text)).set_text(cx, &event.text); + let mut meta = format!("sub_id: {}", event.sub_id); + if event.dropped != 0 { + meta.push_str(&format!(" · dropped: {}", event.dropped)); + } + if event.final_ { + meta.push_str(" · final"); + } + if let Some(data) = &event.data { + meta.push_str(&format!("\n{data}")); + } + row.label(cx, ids!(event_meta)).set_text(cx, &meta); + row + } + Entry::Assistant { text, streaming: false } => { + let row = list.item(cx, index, id!(AssistantRow)); + // No text, no row: a blank block above a card is a gap. + row.set_visible(cx, !text.trim().is_empty()); + if let Some(mut md) = row.widget(cx, ids!(assistant_md)).borrow_mut::() { + md.set_text(cx, text); + } + row + } + Entry::Assistant { text, streaming: true } => { + let row = list.item(cx, index, id!(StreamRow)); + row.set_visible(cx, !text.trim().is_empty()); + row.label(cx, ids!(stream_text)).set_text(cx, text); + row + } + Entry::Tool(t) if matches!(t.status, ToolStatus::Confirm) => { + let row = list.item(cx, index, id!(ConfirmRow)); + row.label(cx, ids!(confirm_title)).set_text(cx, &format!("{} — this changes things outside the app. Run it?", t.title)); + row + } + Entry::Tool(t) => { + let row = list.item(cx, index, id!(ToolRow)); + let (glyph, note, permille, detail) = match &t.status { + ToolStatus::Running { note, permille } => ("›", note.clone(), *permille, String::new()), + ToolStatus::Done { outcome, note, text } => { + let glyph = if outcome.is_ok() { "✓" } else { "✗" }; + let note = if note.is_empty() { outcome.slug().to_string() } else { note.clone() }; + (glyph, note, 1000, text.clone()) + } + ToolStatus::Confirm => ("?", String::new(), 0, String::new()), + }; + row.label(cx, ids!(tool_title)).set_text(cx, &format!("{glyph} {}", t.title)); + row.label(cx, ids!(tool_note)).set_text(cx, ¬e); + let bar_visible = matches!(t.status, ToolStatus::Running { .. }); + row.view(cx, ids!(tool_bar)).set_visible(cx, bar_visible); + if bar_visible { + // The bar is a fraction of the row's inner width + // (the row's rect is last frame's; the first + // frame of a card draws it at a token width). + let inner = (row.area().rect(cx).size.x - 36.0).max(24.0); + let px = inner * (permille as f64 / 1000.0).max(0.05); + let mut bar = row.view(cx, ids!(tool_bar)); + script_apply_eval!(cx, bar, { width: #(px) }); + } + let detail_label = row.label(cx, ids!(tool_detail)); + detail_label.set_visible(cx, t.expanded && !detail.is_empty()); + if t.expanded { + detail_label.set_text(cx, &format!("{}\n{}", t.args, detail)); + } + let _ = matches!(t.status, ToolStatus::Done { outcome: ToolOutcome::Ok, .. }); + row + } + Entry::System { text } => { + let row = list.item(cx, index, id!(SystemRow)); + row.label(cx, ids!(system_text)).set_text(cx, text); + row + } + }; + row.draw_all(cx, &mut Scope::empty()); + } + } + if !self.composer_focused { + self.composer_focused = true; + self.view.text_input(cx, ids!(input)).take_key_focus(cx); + } + DrawStep::done() + } +} + +impl WidgetMatchEvent for AiChatPanel { + fn handle_actions(&mut self, cx: &mut Cx, actions: &Actions, scope: &mut Scope) { + let input = self.view.text_input(cx, ids!(input)); + let returned = input.returned(actions).map(|(text, _)| text); + if let Some(text) = returned { + self.send(cx, text); + } else if self.view.button(cx, ids!(send_button)).clicked(actions) { + let text = input.text(); + self.send(cx, text); + } + if self.view.button(cx, ids!(clear_button)).clicked(actions) { + let now = Self::now(cx); + self.ensure_engine().clear(now); + self.view.redraw(cx); + } + // Cards: confirm buttons and click-to-expand. + let list = self.view.portal_list(cx, ids!(transcript)); + let items = list.items_with_actions(actions); + if !items.is_empty() { + let now = Self::now(cx); + let call_ids: Vec> = { + let state = self.ensure_engine().state(); + items + .iter() + .map(|(index, _)| match state.entries.get(*index) { + Some(Entry::Tool(t)) => Some(t.call_id.clone()), + _ => None, + }) + .collect() + }; + for ((_, item), call_id) in items.iter().zip(call_ids) { + let Some(call_id) = call_id else { continue }; + if item.button(cx, ids!(run_button)).clicked(actions) { + self.ensure_engine().confirm(&call_id, true, now); + self.view.redraw(cx); + } else if item.button(cx, ids!(deny_button)).clicked(actions) { + self.ensure_engine().confirm(&call_id, false, now); + self.view.redraw(cx); + } else if item.view(cx, ids!(tool_head)).finger_up(actions).is_some() { + self.ensure_engine().toggle_tool(&call_id); + self.view.redraw(cx); + } + } + } + let _ = scope; + } +} diff --git a/apps/aichat/src/settings.rs b/apps/aichat/src/settings.rs new file mode 100644 index 000000000..d153ae42f --- /dev/null +++ b/apps/aichat/src/settings.rs @@ -0,0 +1,101 @@ +//! What the person chose: which model answers, and whether cloud models +//! are locked out. Two lines in `~/.makepad/aichat/settings`, the lock a +//! promise kept in code (a cloud choice under the lock normalises to +//! local at load and is refused at set), never a menu filter. + +use makepad_ai_services::state::ProviderChoice; +use std::path::PathBuf; + +#[derive(Clone, Debug, PartialEq)] +pub struct AiSettings { + pub provider: ProviderChoice, + /// Default ON: the person must switch it off before any cloud model + /// can be picked. + pub local_only: bool, +} + +impl Default for AiSettings { + fn default() -> Self { + AiSettings { provider: ProviderChoice::Local, local_only: true } + } +} + +impl AiSettings { + /// The lock, applied: a cloud provider under the lock becomes local. + /// "none" (no model) is always allowed — it reaches nothing. + pub fn normalized(mut self) -> Self { + if self.local_only { + if let ProviderChoice::Cloud(slug) = &self.provider { + if slug != "none" { + self.provider = ProviderChoice::Local; + } + } + } + self + } + + /// Can this choice be made under the current lock? + pub fn allows(&self, choice: &ProviderChoice) -> Result<(), String> { + match choice { + ProviderChoice::Cloud(slug) if self.local_only && slug != "none" => Err("Local AI only is on".into()), + _ => Ok(()), + } + } + + pub fn parse(text: &str) -> AiSettings { + let mut s = AiSettings::default(); + for line in text.lines() { + let Some((k, v)) = line.split_once('=') else { continue }; + match k.trim() { + "provider" => s.provider = ProviderChoice::from_slug(v.trim()), + "local_only" => s.local_only = v.trim() != "false", + _ => {} + } + } + s.normalized() + } + + pub fn render(&self) -> String { + format!("provider={}\nlocal_only={}\n", self.provider.slug(), self.local_only) + } + + pub fn path() -> Option { + if cfg!(target_arch = "wasm32") { + return None; + } + let home = std::env::var_os("HOME").or_else(|| std::env::var_os("USERPROFILE"))?; + Some(PathBuf::from(home).join(".makepad").join("aichat").join("settings")) + } + + pub fn load() -> AiSettings { + match Self::path().and_then(|p| std::fs::read_to_string(p).ok()) { + Some(text) => Self::parse(&text), + None => AiSettings::default(), + } + } + + pub fn save(&self) -> Result<(), String> { + let Some(path) = Self::path() else { return Ok(()) }; + if let Some(dir) = path.parent() { + std::fs::create_dir_all(dir).map_err(|e| e.to_string())?; + } + std::fs::write(&path, self.render()).map_err(|e| e.to_string()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn the_lock_is_kept_at_load_and_at_set() { + let s = AiSettings::parse("provider=claude-api\nlocal_only=true\n"); + assert_eq!(s.provider, ProviderChoice::Local, "a cloud pref under the lock cannot resurrect a cloud model"); + assert!(s.allows(&ProviderChoice::Cloud("claude-api".into())).is_err()); + assert!(s.allows(&ProviderChoice::Cloud("none".into())).is_ok()); + let open = AiSettings::parse("provider=claude-api\nlocal_only=false\n"); + assert_eq!(open.provider, ProviderChoice::Cloud("claude-api".into())); + assert_eq!(AiSettings::parse(&open.render()), open); + assert_eq!(AiSettings::parse("garbage"), AiSettings::default()); + } +} diff --git a/apps/asset-ui/Cargo.toml b/apps/asset-ui/Cargo.toml index a5d4b4649..1509d5f6e 100644 --- a/apps/asset-ui/Cargo.toml +++ b/apps/asset-ui/Cargo.toml @@ -19,6 +19,7 @@ makepad-render = { path = "../../libs/render" } makepad-gltf = { path = "../../libs/gltf" } # Splat viewer: ViewSplat + XrSceneView desktop host. makepad-xr = { path = "../../xr" } +makepad-media-view = { path = "../../libs/media_view" } # REAL Asset Server frontend: shared session lifecycle (discovery/auth/ # retry), catalog runtimes, committed-event subscriber, verified cache. The # VJ worker owns these crates; this app consumes the public API only. diff --git a/apps/asset-ui/src/analysis.rs b/apps/asset-ui/src/analysis.rs index f1b4a7db6..7d44c218a 100644 --- a/apps/asset-ui/src/analysis.rs +++ b/apps/asset-ui/src/analysis.rs @@ -44,7 +44,7 @@ use makepad_audio_lyrics::bake::LyricsBaker; use makepad_audio_lyrics::TrackLyrics; use makepad_audio_sidechannels::{encode_stem_oggs, publish_side_channels}; use makepad_widgets::log; -use makepad_widgets::makepad_platform::thread::SignalToUI; +use makepad_widgets::makepad_platform::thread::{SignalToUI, ThreadOptions, ThreadSpawner}; use std::collections::HashSet; use std::path::PathBuf; use std::str::FromStr; @@ -545,14 +545,8 @@ pub struct AnalysisQueue { fetch_generation: u64, } -impl Default for AnalysisQueue { - fn default() -> Self { - AnalysisQueue::start() - } -} - impl AnalysisQueue { - pub fn start() -> AnalysisQueue { + pub fn start(spawner: ThreadSpawner) -> AnalysisQueue { let (bake_tx, bake_requests) = channel::(); let (bake_done, bake_rx) = channel::(); let (fetch_tx, fetch_requests) = channel::(); @@ -561,12 +555,26 @@ impl AnalysisQueue { let worker_batch = Arc::clone(&batch); // A failed spawn is not fatal: the queue simply never runs, and // every enqueue reports it instead of pretending to work. - let _ = std::thread::Builder::new() - .name("asset-ui-stem-bake".into()) - .spawn(move || bake_loop(bake_requests, bake_done, worker_batch)); - let _ = std::thread::Builder::new() - .name("asset-ui-stem-fetch".into()) - .spawn(move || fetch_loop(fetch_requests, fetch_done)); + match spawner.spawn_worker( + ThreadOptions { + name: Some("asset-ui-stem-bake".into()), + ..Default::default() + }, + move || bake_loop(bake_requests, bake_done, worker_batch), + ) { + Ok(handle) => handle.detach(), + Err(error) => log!("analysis bake worker unavailable: {error}"), + } + match spawner.spawn_worker( + ThreadOptions { + name: Some("asset-ui-stem-fetch".into()), + ..Default::default() + }, + move || fetch_loop(fetch_requests, fetch_done), + ) { + Ok(handle) => handle.detach(), + Err(error) => log!("analysis fetch worker unavailable: {error}"), + } AnalysisQueue { bake_tx, bake_rx, @@ -1473,7 +1481,8 @@ mod tests { #[test] fn the_queue_counts_a_batch_and_keeps_its_verdict() { - let mut queue = AnalysisQueue::start(); + let cx = makepad_widgets::Cx::new(Box::new(|_, _| {})); + let mut queue = AnalysisQueue::start(cx.thread_spawner()); assert!(!queue.busy()); assert_eq!(queue.status_line(), ""); assert_eq!(queue.progress_fraction(), 0.0); diff --git a/apps/asset-ui/src/artifact_io.rs b/apps/asset-ui/src/artifact_io.rs index 052f97668..17a36b4a1 100644 --- a/apps/asset-ui/src/artifact_io.rs +++ b/apps/asset-ui/src/artifact_io.rs @@ -13,12 +13,11 @@ //! - gallery preview decodes run on a small worker pool (2..=8 threads) //! pulling a last-in-first-out stack, capped so old off-screen work drops. -use makepad_widgets::makepad_platform::thread::SignalToUI; +use makepad_widgets::makepad_platform::thread::{SignalToUI, ThreadOptions, ThreadSpawner}; use makepad_widgets::{decode_image_from_data, ImageBuffer}; use std::collections::HashSet; use std::path::PathBuf; use std::sync::mpsc::{channel, Receiver, Sender}; -use std::sync::{Arc, Condvar, Mutex}; /// What one background read is for. pub enum IoPurpose { @@ -164,35 +163,39 @@ pub enum IoDone { pub struct ArtifactIo { tx: Sender, rx: Receiver, - gallery: Arc, } impl ArtifactIo { - pub fn start() -> Self { + pub fn start(spawner: ThreadSpawner) -> Self { let (request_tx, request_rx) = channel::(); let (done_tx, done_rx) = channel::(); - let gallery = Arc::new(GalleryStack::new()); - std::thread::Builder::new() - .name("asset-ui-artifact-io".into()) - .spawn({ - let done_tx = done_tx.clone(); - let gallery = Arc::clone(&gallery); - move || dispatch_loop(request_rx, done_tx, gallery) - }) - .expect("artifact io dispatcher"); - let n = gallery_worker_count(); - for i in 0..n { - let done_tx = done_tx.clone(); - let gallery = Arc::clone(&gallery); - std::thread::Builder::new() - .name(format!("asset-ui-preview-{i}")) - .spawn(move || gallery_loop(gallery, done_tx)) - .expect("gallery decode worker"); - } + let (gallery_tx, gallery_rx) = channel::(); + spawner + .spawn_worker( + ThreadOptions { + name: Some("asset-ui-artifact-io".into()), + ..Default::default() + }, + { + let done_tx = done_tx.clone(); + move || dispatch_loop(request_rx, done_tx, gallery_tx) + }, + ) + .expect("artifact io dispatcher") + .detach(); + spawner + .spawn_worker( + ThreadOptions { + name: Some("asset-ui-preview".into()), + ..Default::default() + }, + move || gallery_loop(gallery_rx, done_tx), + ) + .expect("gallery decode worker") + .detach(); Self { tx: request_tx, rx: done_rx, - gallery, } } @@ -206,12 +209,6 @@ impl ArtifactIo { } } -impl Drop for ArtifactIo { - fn drop(&mut self) { - self.gallery.shutdown(); - } -} - fn is_gallery(purpose: &IoPurpose) -> bool { matches!( purpose, @@ -221,13 +218,13 @@ fn is_gallery(purpose: &IoPurpose) -> bool { ) } -fn dispatch_loop(rx: Receiver, tx: Sender, gallery: Arc) { +fn dispatch_loop(rx: Receiver, tx: Sender, gallery: Sender) { // One connected client per session, reused across opens: against the // local server a fresh connect costs more than the payload. let mut store: Option<(crate::import::ServerSession, makepad_asset_client::AssetClient)> = None; while let Ok(request) = rx.recv() { if is_gallery(&request.purpose) { - gallery.push_latest(request); + let _ = gallery.send(request); continue; } let done = process_with_store(request, &mut store); @@ -236,18 +233,27 @@ fn dispatch_loop(rx: Receiver, tx: Sender, gallery: Arc, tx: Sender) { - while let Some(request) = gallery.pop_latest() { - let file = request.file.clone(); - let done = process(request); - gallery.finish(&file); - if tx.send(done).is_err() { - return; +fn gallery_loop(rx: Receiver, tx: Sender) { + let mut gallery = GalleryStack::new(); + while let Ok(request) = rx.recv() { + gallery.push_latest(request); + for request in rx.try_iter() { + gallery.push_latest(request); + } + while let Some(request) = gallery.pop_latest() { + let file = request.file.clone(); + let done = process(request); + gallery.finish(&file); + if tx.send(done).is_err() { + return; + } + SignalToUI::set_ui_signal(); + for request in rx.try_iter() { + gallery.push_latest(request); + } } - SignalToUI::set_ui_signal(); } } @@ -264,24 +270,22 @@ struct GalleryInner { } struct GalleryStack { - inner: Mutex, - cv: Condvar, + inner: GalleryInner, } impl GalleryStack { fn new() -> Self { Self { - inner: Mutex::new(GalleryInner { + inner: GalleryInner { stack: Vec::new(), decoding: HashSet::new(), shutdown: false, - }), - cv: Condvar::new(), + }, } } - fn push_latest(&self, request: IoRequest) { - let mut g = self.inner.lock().expect("gallery stack"); + fn push_latest(&mut self, request: IoRequest) { + let g = &mut self.inner; if g.shutdown { return; } @@ -294,43 +298,30 @@ impl GalleryStack { let drop_n = g.stack.len() - GALLERY_STACK_CAP; g.stack.drain(0..drop_n); } - self.cv.notify_one(); } - fn pop_latest(&self) -> Option { - let mut g = self.inner.lock().expect("gallery stack"); - loop { - if g.shutdown { - return None; - } - while let Some(request) = g.stack.pop() { - if g.decoding.insert(request.file.clone()) { - return Some(request); - } - } - g = self.cv.wait(g).expect("gallery stack"); + fn pop_latest(&mut self) -> Option { + let g = &mut self.inner; + if g.shutdown { + return None; } + while let Some(request) = g.stack.pop() { + if g.decoding.insert(request.file.clone()) { + return Some(request); + } + } + None } - fn finish(&self, file: &str) { - let mut g = self.inner.lock().expect("gallery stack"); - g.decoding.remove(file); + fn finish(&mut self, file: &str) { + self.inner.decoding.remove(file); } - fn shutdown(&self) { - let mut g = self.inner.lock().expect("gallery stack"); - g.shutdown = true; - self.cv.notify_all(); + fn shutdown(&mut self) { + self.inner.shutdown = true; } } -fn gallery_worker_count() -> usize { - std::thread::available_parallelism() - .map(|n| n.get()) - .unwrap_or(4) - .clamp(2, 8) -} - /// A read whose bytes may come from the store. Everything past the fetch is /// the same decode the local-file path uses — one viewer, two sources. fn process_with_store( @@ -865,7 +856,8 @@ mod tests { std::fs::write(&payload, b"mp4-bytes").unwrap(); let copy = dir.join("viewer-open.mp4"); - let io = ArtifactIo::start(); + let cx = makepad_widgets::Cx::new(Box::new(|_, _| {})); + let io = ArtifactIo::start(cx.thread_spawner()); io.request(IoRequest { file: "clip.mp4".into(), path: payload.clone(), @@ -1143,7 +1135,7 @@ mod tests { #[test] fn gallery_stack_is_last_requested_first_and_rebumps() { - let stack = GalleryStack::new(); + let mut stack = GalleryStack::new(); let mk = |file: &str| IoRequest { file: file.into(), path: PathBuf::from(file), @@ -1168,7 +1160,7 @@ mod tests { #[test] fn gallery_stack_drops_oldest_when_capped() { - let stack = GalleryStack::new(); + let mut stack = GalleryStack::new(); for i in 0..(GALLERY_STACK_CAP + 10) { stack.push_latest(IoRequest { file: format!("f{i}"), diff --git a/apps/asset-ui/src/asset_store_state.rs b/apps/asset-ui/src/asset_store_state.rs index 37a007e5f..8f033fecb 100644 --- a/apps/asset-ui/src/asset_store_state.rs +++ b/apps/asset-ui/src/asset_store_state.rs @@ -56,6 +56,9 @@ use makepad_asset_client::{ use makepad_asset_data::{AssetId, AssetRevisionId}; pub use makepad_asset_data::AssetKind; use makepad_widgets::log; +use makepad_widgets::makepad_platform::thread::{ + Lane, TaskHandle, TaskPool, ThreadOptions, ThreadSpawner, +}; use std::collections::VecDeque; use std::net::{IpAddr, SocketAddr}; use std::path::{Path, PathBuf}; @@ -360,6 +363,8 @@ pub struct SearchResults { /// happens through `start`/`poll`/`submit_search`/`select` only. #[derive(Default)] pub struct AssetStore { + pool: Option, + spawner: Option, /// Continuous ai-content-library → catalog publisher. Declared BEFORE /// `embedded` so it is joined while the server it publishes into is /// still alive. @@ -397,7 +402,7 @@ pub struct AssetStore { /// boxes directly (the store advertises nothing any more — generation /// is client-driven, aicore §9). pub profiles: Remote>, - profiles_rx: Option>>, + profiles_task: Option>>, /// Committed catalog events, newest first, capped. pub events: VecDeque, /// The event feed delivered its initial cursor and is following commits. @@ -480,7 +485,7 @@ pub struct AssetStore { succession_note: Option, /// The previous session is being torn down off-thread. Flips when its /// cache roots are free for the next session to open. - releasing: Option>, + releasing: Option>, /// What to connect to once `releasing` flips. pending_session: Option, } @@ -498,11 +503,13 @@ impl AssetStore { /// the client finds it through the same discovery/health path any LAN /// peer would. Set the env var to skip embed and talk to a standalone /// server instead. - pub fn start(&mut self, library_dir: PathBuf) { + pub fn start(&mut self, library_dir: PathBuf, pool: TaskPool, spawner: ThreadSpawner) { if self.connector.is_some() || self.server.is_some() { return; } self.library_dir = library_dir; + self.pool = Some(pool); + self.spawner = Some(spawner); self.server_root = default_asset_server_root(); self.beacon = beacon_from_env(); self.embed = embed_policy_from_env(); @@ -642,7 +649,7 @@ impl AssetStore { label: handles.server_label.clone(), server_id: handles.server_id, }); - self.endpoints = Some(handles.endpoints); + self.endpoints = handles.endpoints; self.token = handles.token.clone(); self.handles = Some(*handles); self.connector = None; @@ -655,17 +662,15 @@ impl AssetStore { } } // Fleet-built generation profiles landing from their worker thread. - if let Some(rx) = &self.profiles_rx { - match rx.try_recv() { + if let Some(result) = self.profiles_task.as_mut().and_then(TaskHandle::try_take) { + self.profiles_task = None; + match result { Ok(profiles) => { - self.profiles_rx = None; self.profiles = Remote::Ready(profiles); changed = true; } - Err(std::sync::mpsc::TryRecvError::Empty) => {} - Err(std::sync::mpsc::TryRecvError::Disconnected) => { - self.profiles_rx = None; - self.profiles = Remote::Failed("fleet probe thread died".to_string()); + Err(error) => { + self.profiles = Remote::Failed(format!("fleet profile job failed: {error}")); changed = true; } } @@ -806,13 +811,23 @@ impl AssetStore { /// Start the embedded server's background loops and become the host. fn begin_hosting(&mut self, server: makepad_asset_store::AssetServer, token: &str) { if self.host_loops == HostLoops::Run { - self.publish = - start_publish_loop(&server, token, self.library_dir.clone()); + let Some(spawner) = self.spawner.as_ref() else { + log!("asset store: host workers unavailable (thread runtime not configured)"); + self.role = ServerRole::Host; + self.embedded = Some(server); + return; + }; + self.publish = start_publish_loop( + spawner, + &server, + token, + self.library_dir.clone(), + ); // LIVECODING: observed origin directories, catalogued in place. // Only the HOST runs this — reference admission is loopback // privilege, so an attached client never observes for somebody // else's store. - self.observe = start_observe_loop(&server, token); + self.observe = start_observe_loop(spawner, &server, token); } self.role = ServerRole::Host; self.embedded = Some(server); @@ -856,39 +871,39 @@ impl AssetStore { self.search_continuation = false; self.next_cursor = None; self.detail_req = None; - self.profiles_rx = None; + self.profiles_task = None; self.probe_req = None; self.gc_req = None; self.gc_cancel_req = None; self.retire_reqs.clear(); - let released = Arc::new(AtomicBool::new(false)); - self.releasing = Some(released.clone()); - let Some(handles) = self.handles.take() else { - released.store(true, Ordering::Release); + let Some(pool) = &self.pool else { + log!("asset store: session release refused (runtime pool unavailable)"); return; }; - let done = released.clone(); - let spawned = std::thread::Builder::new() - .name("asset-ui-session-release".to_string()) - .spawn(move || { - handles.shutdown(); - done.store(true, Ordering::Release); - }); - if let Err(error) = spawned { - // The closure (and the session inside it) was dropped, which - // already joined the runtimes right here. Nothing is left to - // wait for, so never leave the swap parked on a thread that - // does not exist. - log!("asset store: session release thread refused ({error}); released inline"); - released.store(true, Ordering::Release); - } + let slot = match pool.reserve(Lane::Heavy) { + Ok(slot) => slot, + Err(error) => { + log!("asset store: session release delayed ({error})"); + return; + } + }; + let Some(handles) = self.handles.take() else { + self.releasing = None; + return; + }; + self.releasing = Some(slot.submit(move || handles.shutdown())); } /// Open the decided session once the previous one has let go. fn finish_swap(&mut self) -> bool { - if let Some(released) = &self.releasing { - if !released.load(Ordering::Acquire) { - return false; + if self.releasing.is_none() && self.handles.is_some() { + self.begin_release(); + return false; + } + if let Some(releasing) = &mut self.releasing { + let Some(result) = releasing.try_take() else { return false }; + if let Err(error) = result { + log!("asset store: session release failed: {error}"); } } self.releasing = None; @@ -1150,18 +1165,20 @@ impl AssetStore { /// they can execute right now. Runs on its own thread — LAN probes must /// never stall a frame — and lands through `profiles_rx` in [`Self::poll`]. fn submit_profiles(&mut self) { - let (tx, rx) = std::sync::mpsc::channel(); - self.profiles_rx = Some(rx); self.profiles = Remote::Loading; - let _ = std::thread::Builder::new() - .name("asset-ui-profiles".to_string()) - .spawn(move || { + let Some(pool) = &self.pool else { + self.profiles = Remote::Failed("runtime task pool unavailable".into()); + return; + }; + match pool.submit(Lane::Light, move || { let snapshots = makepad_asset_creator::runner::fleet_snapshots(); - let profiles = makepad_asset_importer::gen_profiles::build_profiles( - &snapshots, "gen", - ); - let _ = tx.send(profiles); - }); + makepad_asset_importer::gen_profiles::build_profiles(&snapshots, "gen") + }) { + Ok(handle) => self.profiles_task = Some(handle), + Err(error) => { + self.profiles = Remote::Failed(format!("fleet profile job refused: {error}")); + } + } } fn on_catalog_event(&mut self, event: ClientEvent) -> bool { @@ -1673,21 +1690,17 @@ fn start_embedded_asset_server_at( Ok((server, token)) } -/// Stop flag for the single in-process publish loop. A `static` (not an -/// `Arc`) because `watch::run` borrows it for the thread's whole life and -/// there is at most one loop per process. -static PUBLISH_STOP: AtomicBool = AtomicBool::new(false); - /// Owns the publisher thread; dropping the store stops and joins it. struct PublishLoop { - join: Option>, + stop: Arc, + task: Option>, } impl Drop for PublishLoop { fn drop(&mut self) { - PUBLISH_STOP.store(true, Ordering::Release); - if let Some(join) = self.join.take() { - let _ = join.join(); + self.stop.store(true, Ordering::Release); + if let Some(task) = self.task.take() { + task.detach(); } } } @@ -1697,6 +1710,7 @@ impl Drop for PublishLoop { /// slow or refused connection can never stall the UI, and every failure is /// a log line — never a panic. fn start_publish_loop( + spawner: &ThreadSpawner, server: &makepad_asset_store::AssetServer, token: &str, library_dir: PathBuf, @@ -1705,10 +1719,14 @@ fn start_publish_loop( let server_id = server.server_id(); let token = token.to_string(); let cache = asset_ui_home().join("publish-cache"); - PUBLISH_STOP.store(false, Ordering::Release); - let join = std::thread::Builder::new() - .name("asset-ui-publish".to_string()) - .spawn(move || { + let stop = Arc::new(AtomicBool::new(false)); + let worker_stop = stop.clone(); + let task = spawner.spawn_worker( + ThreadOptions { + name: Some("asset-ui-publish".into()), + ..Default::default() + }, + move || { let mut config = makepad_asset_client::ClientConfig::new(cache); config.token = Some(token); let mut client = match makepad_asset_client::AssetClient::connect( @@ -1736,12 +1754,13 @@ fn start_publish_loop( // Log publications, failures and retries; out-of-scope rows // (the pack-import bulk) stay silent by design. true, - &PUBLISH_STOP, + &worker_stop, ); log!("publish loop: stopped"); - }); - match join { - Ok(join) => Some(PublishLoop { join: Some(join) }), + }, + ); + match task { + Ok(task) => Some(PublishLoop { stop, task: Some(task) }), Err(error) => { log!("publish loop: could not spawn: {error}"); None @@ -1749,19 +1768,17 @@ fn start_publish_loop( } } -/// Stop flag for the single in-process observe loop. -static OBSERVE_STOP: AtomicBool = AtomicBool::new(false); - /// Owns the observer thread; dropping the store stops and joins it. struct ObserveLoop { - join: Option>, + stop: Arc, + task: Option>, } impl Drop for ObserveLoop { fn drop(&mut self) { - OBSERVE_STOP.store(true, Ordering::Release); - if let Some(join) = self.join.take() { - let _ = join.join(); + self.stop.store(true, Ordering::Release); + if let Some(task) = self.task.take() { + task.detach(); } } } @@ -1789,6 +1806,7 @@ fn observe_origins() -> Vec { /// Observe effect-document origins into the server this process hosts. /// Connect + watch happen on the thread; every failure is a log line. fn start_observe_loop( + spawner: &ThreadSpawner, server: &makepad_asset_store::AssetServer, token: &str, ) -> Option { @@ -1797,10 +1815,14 @@ fn start_observe_loop( let token = token.to_string(); let cache = asset_ui_home().join("observe-cache"); let origins = observe_origins(); - OBSERVE_STOP.store(false, Ordering::Release); - let join = std::thread::Builder::new() - .name("asset-ui-observe".to_string()) - .spawn(move || { + let stop = Arc::new(AtomicBool::new(false)); + let worker_stop = stop.clone(); + let task = spawner.spawn_worker( + ThreadOptions { + name: Some("asset-ui-observe".into()), + ..Default::default() + }, + move || { let mut config = makepad_asset_client::ClientConfig::new(cache); config.token = Some(token); let mut client = match makepad_asset_client::AssetClient::connect( @@ -1817,12 +1839,13 @@ fn start_observe_loop( makepad_asset_store::observe::run( &mut client, &makepad_asset_store::observe::ObserveConfig::vjfx(origins), - &OBSERVE_STOP, + &worker_stop, ); log!("observe loop: stopped"); - }); - match join { - Ok(join) => Some(ObserveLoop { join: Some(join) }), + }, + ); + match task { + Ok(task) => Some(ObserveLoop { stop, task: Some(task) }), Err(error) => { log!("observe loop: could not spawn: {error}"); None @@ -2078,6 +2101,13 @@ mod tests { namespace: "game".into(), kind: None, title: title.into(), + creator: String::new(), + artist: String::new(), + artist_url: String::new(), + album: String::new(), + source_url: String::new(), + license: String::new(), + license_url: String::new(), snippet: String::new(), score: 0, live: true, @@ -2368,6 +2398,14 @@ mod tests { (server, token) } + fn ensure_test_runtime(store: &mut AssetStore) { + if store.pool.is_none() { + let cx = makepad_widgets::Cx::new(Box::new(|_, _| {})); + store.pool = Some(cx.task_pool()); + store.spawner = Some(cx.thread_spawner()); + } + } + /// Publish one minimal real asset directly (the synchronous /// `AssetClient`, not through `AssetStore` — publishing is the import /// pipeline's job, already covered elsewhere; this test starts from @@ -2412,6 +2450,7 @@ mod tests { /// `AssetStore::poll()` loop (no discovery — explicit endpoints), and /// wait for the initial auto-search `poll()` fires on connect to land. fn connect_store_to(store: &mut AssetStore, server: &makepad_asset_store::AssetServer, token: &str) { + ensure_test_runtime(store); let config = SessionConfig { endpoints: Some(ApiEndpoints { control: server.control_addr(), data: server.data_addr() }), server_id: Some(server.server_id()), @@ -2448,6 +2487,7 @@ mod tests { secs: u64, ready: F, ) { + ensure_test_runtime(store); let deadline = std::time::Instant::now() + std::time::Duration::from_secs(secs); loop { store.poll(); diff --git a/apps/asset-ui/src/audio.rs b/apps/asset-ui/src/audio.rs index 717e0cb85..75336bf3d 100644 --- a/apps/asset-ui/src/audio.rs +++ b/apps/asset-ui/src/audio.rs @@ -1,15 +1,16 @@ //! Artifact playback + waveform strip. //! -//! Same shape as the sandbox's `VideoAudio` mixer (apps/sandbox/src/ -//! video_player.rs): a process-global resampling stereo queue mixed -//! additively from the `cx.audio_output` callback, so playback needs no -//! plumbing through the widget tree. The service emits PCM16 WAV (kokoro: -//! mono 24kHz, sa3-sfx: stereo 44.1kHz), decoded here with a minimal RIFF -//! parser (libs/asset/ai wav.rs is encode-only). +//! The UI owns a transport handle and the device callback owns the playback +//! engine. Commands and atomic snapshots cross between them; neither side +//! waits on a mutex. The service emits PCM16 WAV (kokoro: mono 24kHz, +//! sa3-sfx: stereo 44.1kHz), decoded here with a minimal RIFF parser +//! (libs/asset/ai wav.rs is encode-only). use makepad_widgets::makepad_platform::audio::AudioBuffer; +use makepad_widgets::makepad_platform::thread::{Lane, TaskHandle, TaskPool}; +use std::cell::RefCell; use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; -use std::sync::{Arc, LazyLock, Mutex}; +use std::sync::{mpsc, Arc}; #[derive(Clone)] pub struct WavPcm { @@ -103,24 +104,22 @@ pub fn parse_wav(bytes: &[u8]) -> Result { /// advances it; UI code only loads, pauses and seeks. const FP_ONE: u64 = 1 << 32; -struct WavMixer { - clip: Mutex>>, +struct AudioSnapshot { cursor_fp: AtomicU64, playing: AtomicBool, + ack: AtomicU64, } -impl Default for WavMixer { +impl Default for AudioSnapshot { fn default() -> Self { Self { - clip: Mutex::new(None), cursor_fp: AtomicU64::new(0), playing: AtomicBool::new(false), + ack: AtomicU64::new(0), } } } -static WAV_MIXER: LazyLock = LazyLock::new(WavMixer::default); - // --------------------------------------------------------------------------- // Separated layers ("split audio layers") // --------------------------------------------------------------------------- @@ -150,33 +149,301 @@ pub const STEM_LANES: usize = 4; /// files are published and fetched in, so no index ever has to be remapped. pub const STEM_LANE_NAMES: [&str; STEM_LANES] = ["drums", "bass", "vocals", "other"]; -struct StemMixer { - lanes: Mutex>>, - /// Per-lane mute. Read by the audio callback, written by the UI. - mute: [AtomicBool; STEM_LANES], - /// Whether the layers are what the transport plays. Kept separate from - /// the lock so the callback can tell "no stems" from "stems, but the UI - /// holds the lock this quantum" — the second must be silence, not a - /// blip of the mixed track. - active: AtomicBool, - /// The clip generation these layers belong to. The mixed audio and the - /// four stems arrive on two different workers in either order; this is - /// what lets the second one to land know it is the same track. - generation: AtomicU64, +enum AudioCommand { + InstallClip { + serial: u64, + clip: Arc, + }, + ClearClip { + serial: u64, + }, + InstallStems { + serial: u64, + lanes: Arc<[StemPcm; STEM_LANES]>, + }, + ClearStems { + serial: u64, + }, + Play { + serial: u64, + }, + Pause { + serial: u64, + }, + Stop { + serial: u64, + }, + Seek { + serial: u64, + cursor_fp: u64, + }, + MuteLane { + serial: u64, + lane: usize, + muted: bool, + }, } -impl Default for StemMixer { - fn default() -> Self { +enum RetiredAudio { + Clip(Arc), + Stems(Arc<[StemPcm; STEM_LANES]>), +} + +struct PendingDecode { + generation: u64, + task: TaskHandle>, +} + +/// UI-thread transport handle. It owns the requested state and communicates +/// with the realtime callback exclusively through commands and atomics. +struct AudioMixer { + commands: mpsc::Sender, + retired: mpsc::Receiver, + snapshot: Arc, + engine: Option, + clip: Option>, + stems: Option>, + stem_generation: u64, + muted: [bool; STEM_LANES], + cursor_fp: u64, + playing: bool, + serial: u64, + load_generation: u64, + pending_decodes: Vec, +} + +/// Realtime-owned state. Once installed in `cx.audio_output`, only the audio +/// callback touches these payloads and cursors. +pub struct AudioEngine { + commands: mpsc::Receiver, + retired: mpsc::Sender, + snapshot: Arc, + clip: Option>, + stems: Option>, + muted: [bool; STEM_LANES], + cursor_fp: u64, + playing: bool, + ack: u64, +} + +impl AudioMixer { + fn new() -> Self { + let (command_tx, command_rx) = mpsc::channel(); + let (retired_tx, retired_rx) = mpsc::channel(); + let snapshot = Arc::new(AudioSnapshot::default()); + let engine = AudioEngine { + commands: command_rx, + retired: retired_tx, + snapshot: snapshot.clone(), + clip: None, + stems: None, + muted: [false; STEM_LANES], + cursor_fp: 0, + playing: false, + ack: 0, + }; Self { - lanes: Mutex::new(None), - mute: Default::default(), - active: AtomicBool::new(false), - generation: AtomicU64::new(u64::MAX), + commands: command_tx, + retired: retired_rx, + snapshot, + engine: Some(engine), + clip: None, + stems: None, + stem_generation: u64::MAX, + muted: [false; STEM_LANES], + cursor_fp: 0, + playing: false, + serial: 0, + load_generation: 0, + pending_decodes: Vec::new(), } } + + fn take_engine(&mut self) -> Option { + self.engine.take() + } + + fn next_serial(&mut self) -> u64 { + self.serial = self.serial.wrapping_add(1).max(1); + self.serial + } + + fn send(&self, command: AudioCommand) { + let _ = self.commands.send(command); + } + + fn refresh_snapshot(&mut self) { + let ack = self.snapshot.ack.load(Ordering::Acquire); + if ack >= self.serial { + self.cursor_fp = self.snapshot.cursor_fp.load(Ordering::Acquire); + self.playing = self.snapshot.playing.load(Ordering::Acquire); + } + } + + fn pump(&mut self) { + self.refresh_snapshot(); + for retired in self.retired.try_iter() { + match retired { + RetiredAudio::Clip(clip) => drop(clip), + RetiredAudio::Stems(stems) => drop(stems), + } + } + let pending_decodes = std::mem::take(&mut self.pending_decodes); + let mut waiting = Vec::with_capacity(pending_decodes.len()); + for mut pending in pending_decodes { + let Some(result) = pending.task.try_take() else { + waiting.push(pending); + continue; + }; + match result { + Ok(Ok(pcm)) if pending.generation == self.load_generation => { + self.install(pcm, pending.generation); + } + Ok(Err(error)) if pending.generation == self.load_generation => { + makepad_widgets::log!("audio decode failed: {error}"); + } + Err(error) if pending.generation == self.load_generation => { + makepad_widgets::log!("audio decode task failed: {error}"); + } + _ => {} + } + } + self.pending_decodes = waiting; + } + + fn clear_stems(&mut self) { + self.stems = None; + self.stem_generation = u64::MAX; + self.muted = [false; STEM_LANES]; + let serial = self.next_serial(); + self.send(AudioCommand::ClearStems { serial }); + } + + fn clear(&mut self) { + self.clear_stems(); + self.clip = None; + self.cursor_fp = 0; + self.playing = false; + let serial = self.next_serial(); + self.send(AudioCommand::ClearClip { serial }); + } + + fn install(&mut self, pcm: WavPcm, generation: u64) -> bool { + if self.load_generation != generation { + return false; + } + if pcm.frames.is_empty() || pcm.sample_rate == 0 { + self.clear(); + return false; + } + if self.stem_generation != generation { + self.clear_stems(); + } + let clip = Arc::new(pcm); + self.clip = Some(clip.clone()); + self.cursor_fp = 0; + self.playing = false; + let serial = self.next_serial(); + self.send(AudioCommand::InstallClip { serial, clip }); + true + } + + fn load(&mut self, pcm: WavPcm) -> bool { + self.load_generation = self.load_generation.wrapping_add(1); + self.install(pcm, self.load_generation) + } + + fn set_stems(&mut self, lanes: [StemPcm; STEM_LANES], generation: u64) -> bool { + if self.load_generation != generation { + return false; + } + if lanes + .iter() + .any(|lane| lane.frames.is_empty() || lane.sample_rate == 0) + { + self.clear_stems(); + return false; + } + let lanes = Arc::new(lanes); + self.stems = Some(lanes.clone()); + self.stem_generation = generation; + self.muted = [false; STEM_LANES]; + let serial = self.next_serial(); + self.send(AudioCommand::InstallStems { serial, lanes }); + true + } + + fn play(&mut self) { + let Some(clip) = self.clip.as_ref() else { + return; + }; + let end = (clip.frames.len() as u64) << 32; + if self.cursor_fp >= end { + self.cursor_fp = 0; + } + self.playing = true; + let serial = self.next_serial(); + self.send(AudioCommand::Play { serial }); + } + + fn pause(&mut self) { + self.playing = false; + let serial = self.next_serial(); + self.send(AudioCommand::Pause { serial }); + } + + fn stop(&mut self) { + self.playing = false; + self.cursor_fp = 0; + let serial = self.next_serial(); + self.send(AudioCommand::Stop { serial }); + } + + fn seek_fraction(&mut self, fraction: f64) { + let Some(clip) = self.clip.as_ref() else { + return; + }; + let frame = (fraction.clamp(0.0, 1.0) * clip.frames.len() as f64) as u64; + self.cursor_fp = frame.min(clip.frames.len() as u64) << 32; + let serial = self.next_serial(); + self.send(AudioCommand::Seek { + serial, + cursor_fp: self.cursor_fp, + }); + } + + fn set_lane_muted(&mut self, lane: usize, muted: bool) { + let Some(slot) = self.muted.get_mut(lane) else { + return; + }; + *slot = muted; + let serial = self.next_serial(); + self.send(AudioCommand::MuteLane { + serial, + lane, + muted, + }); + } } -static STEM_MIXER: LazyLock = LazyLock::new(StemMixer::default); +thread_local! { + static AUDIO_MIXER: RefCell = RefCell::new(AudioMixer::new()); +} + +fn with_mixer(f: impl FnOnce(&mut AudioMixer) -> T) -> T { + AUDIO_MIXER.with(|mixer| f(&mut mixer.borrow_mut())) +} + +/// Move the engine into the app's one audio callback. It is intentionally +/// one-shot: there must never be a second owner of realtime state. +pub fn take_engine() -> AudioEngine { + with_mixer(|mixer| mixer.take_engine()).expect("audio engine is installed once") +} + +/// Poll background decodes and reclaim callback-retired payloads on the UI. +pub fn pump() { + with_mixer(AudioMixer::pump); +} /// Install four separated layers over the loaded clip. From here the /// transport plays their SUM instead of the mixed track — which is also the @@ -190,63 +457,32 @@ static STEM_MIXER: LazyLock = LazyLock::new(StemMixer::default); /// Refused (and the layers cleared) when a lane is empty: half a stem set is /// a lie about what the asset carries. pub fn set_stems(lanes: [StemPcm; STEM_LANES], generation: u64) -> bool { - if LOAD_GENERATION.load(Ordering::Acquire) != generation { - return false; - } - if lanes - .iter() - .any(|lane| lane.frames.is_empty() || lane.sample_rate == 0) - { - clear_stems(); - return false; - } - for mute in &STEM_MIXER.mute { - mute.store(false, Ordering::Release); - } - *STEM_MIXER.lanes.lock().unwrap() = Some(Arc::new(lanes)); - STEM_MIXER.generation.store(generation, Ordering::Release); - STEM_MIXER.active.store(true, Ordering::Release); - true + with_mixer(|mixer| mixer.set_stems(lanes, generation)) } /// Back to the mixed track. Called whenever the clip changes, so a new /// selection can never play the previous track's layers. pub fn clear_stems() { - STEM_MIXER.active.store(false, Ordering::Release); - STEM_MIXER.generation.store(u64::MAX, Ordering::Release); - *STEM_MIXER.lanes.lock().unwrap() = None; - for mute in &STEM_MIXER.mute { - mute.store(false, Ordering::Release); - } + with_mixer(AudioMixer::clear_stems); } /// True when the transport is playing separated layers. pub fn stems_ready() -> bool { - STEM_MIXER.active.load(Ordering::Acquire) + with_mixer(|mixer| mixer.stems.is_some()) } pub fn lane_muted(lane: usize) -> bool { - STEM_MIXER - .mute - .get(lane) - .is_some_and(|mute| mute.load(Ordering::Acquire)) + with_mixer(|mixer| mixer.muted.get(lane).copied().unwrap_or(false)) } pub fn set_lane_muted(lane: usize, muted: bool) { - if let Some(mute) = STEM_MIXER.mute.get(lane) { - mute.store(muted, Ordering::Release); - } + with_mixer(|mixer| mixer.set_lane_muted(lane, muted)); } /// Length of the installed layers, for the honest "these are this track's /// stems" check a host wants before it draws the toggles. pub fn stems_seconds() -> f64 { - STEM_MIXER - .lanes - .lock() - .unwrap() - .as_ref() - .map_or(0.0, |lanes| lanes[0].seconds()) + with_mixer(|mixer| mixer.stems.as_ref().map_or(0.0, |lanes| lanes[0].seconds())) } /// One lane at a fixed-point cursor, linearly interpolated — the same @@ -276,88 +512,44 @@ fn sample_lane(lane: &StemPcm, cursor: u64) -> (f32, f32) { /// a new clip generation, which invalidates any separated layers installed /// for the previous one. pub fn load(pcm: WavPcm) -> bool { - let generation = LOAD_GENERATION.fetch_add(1, Ordering::AcqRel) + 1; - install(pcm, generation) -} - -/// Install a decoded clip under the generation it was decoded FOR. A newer -/// pick having happened in the meantime drops it. -/// -/// The layers are cleared unless they were installed for THIS clip: the -/// fetch of a track's stems and the decode of its mixed audio run on two -/// workers, and whichever finishes second must not wipe the first. -fn install(pcm: WavPcm, generation: u64) -> bool { - if LOAD_GENERATION.load(Ordering::Acquire) != generation { - return false; - } - if pcm.frames.is_empty() || pcm.sample_rate == 0 { - clear(); - return false; - } - if STEM_MIXER.generation.load(Ordering::Acquire) != generation { - clear_stems(); - } - WAV_MIXER.playing.store(false, Ordering::Release); - *WAV_MIXER.clip.lock().unwrap() = Some(Arc::new(pcm)); - WAV_MIXER.cursor_fp.store(0, Ordering::Release); - true + with_mixer(|mixer| mixer.load(pcm)) } /// Discard the loaded clip and make the transport unavailable. pub fn clear() { - clear_stems(); - WAV_MIXER.playing.store(false, Ordering::Release); - *WAV_MIXER.clip.lock().unwrap() = None; - WAV_MIXER.cursor_fp.store(0, Ordering::Release); + with_mixer(AudioMixer::clear); } -/// Which load request is current. A user clicking down a list starts several; -/// only the newest may install itself, or a slow decode of the track before -/// last lands on top of the one they are looking at. -static LOAD_GENERATION: AtomicU64 = AtomicU64::new(0); - /// Take a track's bytes — WAV, MP3 or Ogg — and make the transport play them. /// -/// Decoding happens on a worker: the music library is MP3s, and turning six -/// minutes of one into PCM on the frame thread is a visible stall. The mixer -/// is process-global, so the worker installs the result itself; there is -/// nothing to plumb back through the widget tree. +/// Decoding happens in the runtime pool: the music library is MP3s, and +/// turning six minutes of one into PCM on the frame thread is a visible +/// stall. The UI polls the task and sends the accepted result to the engine. /// /// The transport goes unavailable immediately, because the previous track is /// no longer what the well is showing — a stale clip left loaded is a play /// button that plays the wrong song. /// Returns the clip generation this request claimed, which is what a /// side-channel fetch for the same track carries back into [`set_stems`]. -pub fn load_clip_async(bytes: Vec) -> u64 { - let generation = LOAD_GENERATION.fetch_add(1, Ordering::AcqRel) + 1; - clear(); - let bytes = Arc::new(bytes); - let spawned = std::thread::Builder::new() - .name("asset-ui-audio-decode".into()) - .spawn({ - let bytes = Arc::clone(&bytes); - move || { - let Ok(pcm) = decode_clip(&bytes) else { - return; - }; - // A newer pick happened while this was decoding: drop it. - install(pcm, generation); - } - }); - if spawned.is_err() { - // No worker to be had: decode here rather than leave a dead - // transport under a drawn waveform. - if let Ok(pcm) = decode_clip(&bytes) { - install(pcm, generation); +pub fn load_clip_async(pool: &TaskPool, bytes: Vec) -> u64 { + with_mixer(|mixer| { + mixer.load_generation = mixer.load_generation.wrapping_add(1); + let generation = mixer.load_generation; + mixer.clear(); + match pool.submit(Lane::Heavy, move || decode_clip(&bytes)) { + Ok(task) => mixer + .pending_decodes + .push(PendingDecode { generation, task }), + Err(error) => makepad_widgets::log!("audio decode job refused: {error}"), } - } - generation + generation + }) } /// The clip generation currently claimed. A side-channel fetch records it /// with the request and hands it back to [`set_stems`]. pub fn clip_generation() -> u64 { - LOAD_GENERATION.load(Ordering::Acquire) + with_mixer(|mixer| mixer.load_generation) } /// Any container the catalog carries, in the mixer's shape. RIFF is parsed @@ -388,48 +580,45 @@ pub fn decode_clip(bytes: &[u8]) -> Result { /// Start or resume. Starting from the end restarts at zero. pub fn play() { - let clip = WAV_MIXER.clip.lock().unwrap(); - let Some(clip) = clip.as_ref() else { return }; - let end = (clip.frames.len() as u64) << 32; - if WAV_MIXER.cursor_fp.load(Ordering::Acquire) >= end { - WAV_MIXER.cursor_fp.store(0, Ordering::Release); - } - WAV_MIXER.playing.store(true, Ordering::Release); + with_mixer(AudioMixer::play); } pub fn pause() { - WAV_MIXER.playing.store(false, Ordering::Release); + with_mixer(AudioMixer::pause); } /// Stop returns to the start but retains the decoded clip for replay. pub fn stop() { - pause(); - WAV_MIXER.cursor_fp.store(0, Ordering::Release); + with_mixer(AudioMixer::stop); } pub fn is_ready() -> bool { - WAV_MIXER.clip.lock().unwrap().is_some() + with_mixer(|mixer| mixer.clip.is_some()) } pub fn is_playing() -> bool { - WAV_MIXER.playing.load(Ordering::Acquire) && !at_end() + with_mixer(|mixer| { + mixer.refresh_snapshot(); + let end = mixer + .clip + .as_ref() + .map_or(0, |clip| (clip.frames.len() as u64) << 32); + mixer.playing && mixer.cursor_fp < end + }) } pub fn duration_secs() -> f64 { - WAV_MIXER - .clip - .lock() - .unwrap() - .as_ref() - .map_or(0.0, |clip| clip.seconds()) + with_mixer(|mixer| mixer.clip.as_ref().map_or(0.0, |clip| clip.seconds())) } /// Truthful device-clocked playhead, never derived from a UI timer. pub fn playhead_secs() -> f64 { - let clip = WAV_MIXER.clip.lock().unwrap(); - let Some(clip) = clip.as_ref() else { return 0.0 }; - (WAV_MIXER.cursor_fp.load(Ordering::Acquire) as f64 / FP_ONE as f64) - / clip.sample_rate as f64 + with_mixer(|mixer| { + mixer.refresh_snapshot(); + mixer.clip.as_ref().map_or(0.0, |clip| { + (mixer.cursor_fp as f64 / FP_ONE as f64) / clip.sample_rate as f64 + }) + }) } /// Normalized playhead across the loaded clip for the waveform overlay: @@ -443,19 +632,18 @@ pub fn playhead_fraction() -> f64 { } pub fn at_end() -> bool { - let clip = WAV_MIXER.clip.lock().unwrap(); - let Some(clip) = clip.as_ref() else { return false }; - WAV_MIXER.cursor_fp.load(Ordering::Acquire) >= (clip.frames.len() as u64) << 32 + with_mixer(|mixer| { + mixer.refresh_snapshot(); + mixer + .clip + .as_ref() + .is_some_and(|clip| mixer.cursor_fp >= (clip.frames.len() as u64) << 32) + }) } /// Sample-accurate fractional seek, clamped to the decoded clip. pub fn seek_fraction(frac: f64) { - let clip = WAV_MIXER.clip.lock().unwrap(); - let Some(clip) = clip.as_ref() else { return }; - let frame = (frac.clamp(0.0, 1.0) * clip.frames.len() as f64) as u64; - WAV_MIXER - .cursor_fp - .store((frame.min(clip.frames.len() as u64)) << 32, Ordering::Release); + with_mixer(|mixer| mixer.seek_fraction(frac)); } /// Long-form threshold for the audition policy below: at/under this a voice @@ -484,95 +672,170 @@ pub fn format_time(secs: f64) -> String { format!("{minutes}:{:04.1}", secs - minutes as f64 * 60.0) } -/// One additive source in the app's single `cx.audio_output` callback. -/// The callback never blocks on a UI load/seek: a contended quantum is silent. -pub fn mix_into(output: &mut AudioBuffer, device_rate: f64) { - if !WAV_MIXER.playing.load(Ordering::Acquire) || device_rate <= 0.0 { - return; +impl AudioEngine { + fn retire(&self, retired: RetiredAudio) { + let _ = self.retired.send(retired); } - let Ok(clip) = WAV_MIXER.clip.try_lock() else { return }; - let Some(clip) = clip.as_ref() else { - WAV_MIXER.playing.store(false, Ordering::Release); - return; - }; - let end = (clip.frames.len() as u64) << 32; - let mut cursor = WAV_MIXER.cursor_fp.load(Ordering::Acquire); - if cursor >= end { - WAV_MIXER.playing.store(false, Ordering::Release); - return; - } - let step = ((clip.sample_rate as f64 / device_rate) * FP_ONE as f64) as u64; - if step == 0 { - return; - } - const GAIN: f32 = 0.9; - // Separated layers REPLACE the mixed track while they are installed. - // A contended lane lock is silence for this quantum, never a blip of - // the original — the same rule the clip lock above follows. - if STEM_MIXER.active.load(Ordering::Acquire) { - let Ok(guard) = STEM_MIXER.lanes.try_lock() else { return }; - let Some(lanes) = guard.as_ref() else { return }; - // ONE cursor for all four lanes — they cannot drift from each other - // — re-derived from the track cursor at every quantum, so they - // cannot drift from the timeline the waveform and the transport - // draw either. The layers are at the model's rate; the clip may not - // be, hence the second step. - let stem_rate = lanes[0].sample_rate.max(1) as f64; - let stem_step = ((stem_rate / device_rate) * FP_ONE as f64) as u64; - let secs = (cursor as f64 / FP_ONE as f64) / clip.sample_rate as f64; - let mut stem_cursor = (secs * stem_rate * FP_ONE as f64) as u64; - // The mute flags are read ONCE per quantum, not per sample: a - // toggle lands on the next buffer, which is inaudible, and the - // callback stays free of per-sample atomics. - let audible: [bool; STEM_LANES] = - std::array::from_fn(|lane| !STEM_MIXER.mute[lane].load(Ordering::Relaxed)); + fn drain_commands(&mut self) { + while let Ok(command) = self.commands.try_recv() { + let serial = match command { + AudioCommand::InstallClip { serial, clip } => { + if let Some(old) = self.clip.replace(clip) { + self.retire(RetiredAudio::Clip(old)); + } + self.cursor_fp = 0; + self.playing = false; + serial + } + AudioCommand::ClearClip { serial } => { + if let Some(old) = self.clip.take() { + self.retire(RetiredAudio::Clip(old)); + } + self.cursor_fp = 0; + self.playing = false; + serial + } + AudioCommand::InstallStems { serial, lanes } => { + if let Some(old) = self.stems.replace(lanes) { + self.retire(RetiredAudio::Stems(old)); + } + self.muted = [false; STEM_LANES]; + serial + } + AudioCommand::ClearStems { serial } => { + if let Some(old) = self.stems.take() { + self.retire(RetiredAudio::Stems(old)); + } + self.muted = [false; STEM_LANES]; + serial + } + AudioCommand::Play { serial } => { + if let Some(clip) = self.clip.as_ref() { + let end = (clip.frames.len() as u64) << 32; + if self.cursor_fp >= end { + self.cursor_fp = 0; + } + self.playing = true; + } + serial + } + AudioCommand::Pause { serial } => { + self.playing = false; + serial + } + AudioCommand::Stop { serial } => { + self.playing = false; + self.cursor_fp = 0; + serial + } + AudioCommand::Seek { serial, cursor_fp } => { + self.cursor_fp = self.clip.as_ref().map_or(0, |clip| { + cursor_fp.min((clip.frames.len() as u64) << 32) + }); + serial + } + AudioCommand::MuteLane { + serial, + lane, + muted, + } => { + if let Some(slot) = self.muted.get_mut(lane) { + *slot = muted; + } + serial + } + }; + self.ack = serial; + } + } + + fn publish(&self) { + self.snapshot.cursor_fp.store(self.cursor_fp, Ordering::Relaxed); + self.snapshot.playing.store(self.playing, Ordering::Relaxed); + self.snapshot.ack.store(self.ack, Ordering::Release); + } + + /// Add this transport to the app's output. Commands are drained first; + /// no application lock or wait occurs on the realtime callback. + pub fn mix_into(&mut self, output: &mut AudioBuffer, device_rate: f64) { + self.drain_commands(); + if !self.playing || device_rate <= 0.0 { + self.publish(); + return; + } + let Some(clip) = self.clip.as_ref() else { + self.playing = false; + self.publish(); + return; + }; + let end = (clip.frames.len() as u64) << 32; + if self.cursor_fp >= end { + self.playing = false; + self.publish(); + return; + } + let step = ((clip.sample_rate as f64 / device_rate) * FP_ONE as f64) as u64; + if step == 0 { + self.publish(); + return; + } + const GAIN: f32 = 0.9; + + if let Some(lanes) = self.stems.as_ref() { + let stem_rate = lanes[0].sample_rate.max(1) as f64; + let stem_step = ((stem_rate / device_rate) * FP_ONE as f64) as u64; + let secs = (self.cursor_fp as f64 / FP_ONE as f64) / clip.sample_rate as f64; + let mut stem_cursor = (secs * stem_rate * FP_ONE as f64) as u64; + for frame in 0..output.frame_count() { + if self.cursor_fp >= end { + self.playing = false; + self.cursor_fp = end; + break; + } + let (mut l, mut r) = (0.0f32, 0.0f32); + for (index, lane) in lanes.iter().enumerate() { + if self.muted[index] { + continue; + } + let (ll, rr) = sample_lane(lane, stem_cursor); + l += ll; + r += rr; + } + l *= GAIN; + r *= GAIN; + for channel in 0..output.channel_count() { + output.channel_mut(channel)[frame] += if channel == 0 { l } else { r }; + } + self.cursor_fp = self.cursor_fp.saturating_add(step); + stem_cursor = stem_cursor.saturating_add(stem_step); + } + self.cursor_fp = self.cursor_fp.min(end); + self.publish(); + return; + } + for frame in 0..output.frame_count() { - if cursor >= end { - WAV_MIXER.playing.store(false, Ordering::Release); - cursor = end; + if self.cursor_fp >= end { + self.playing = false; + self.cursor_fp = end; break; } - let (mut l, mut r) = (0.0f32, 0.0f32); - for (lane, on) in lanes.iter().zip(audible.iter()) { - if !on { - continue; - } - let (ll, rr) = sample_lane(lane, stem_cursor); - l += ll; - r += rr; - } - l *= GAIN; - r *= GAIN; + let index = (self.cursor_fp >> 32) as usize; + let fraction = (self.cursor_fp & (FP_ONE - 1)) as f32 / FP_ONE as f32; + let next = (index + 1).min(clip.frames.len() - 1); + let (al, ar) = clip.frames[index]; + let (bl, br) = clip.frames[next]; + let l = (al + (bl - al) * fraction) * GAIN; + let r = (ar + (br - ar) * fraction) * GAIN; for channel in 0..output.channel_count() { output.channel_mut(channel)[frame] += if channel == 0 { l } else { r }; } - cursor = cursor.saturating_add(step); - stem_cursor = stem_cursor.saturating_add(stem_step); + self.cursor_fp = self.cursor_fp.saturating_add(step); } - WAV_MIXER.cursor_fp.store(cursor.min(end), Ordering::Release); - return; + self.cursor_fp = self.cursor_fp.min(end); + self.publish(); } - - for frame in 0..output.frame_count() { - if cursor >= end { - WAV_MIXER.playing.store(false, Ordering::Release); - cursor = end; - break; - } - let index = (cursor >> 32) as usize; - let fraction = (cursor & (FP_ONE - 1)) as f32 / FP_ONE as f32; - let next = (index + 1).min(clip.frames.len() - 1); - let (al, ar) = clip.frames[index]; - let (bl, br) = clip.frames[next]; - let l = (al + (bl - al) * fraction) * GAIN; - let r = (ar + (br - ar) * fraction) * GAIN; - for channel in 0..output.channel_count() { - output.channel_mut(channel)[frame] += if channel == 0 { l } else { r }; - } - cursor = cursor.saturating_add(step); - } - WAV_MIXER.cursor_fp.store(cursor.min(end), Ordering::Release); } // --------------------------------------------------------------------------- @@ -660,7 +923,10 @@ pub fn waveform_bgra(pcm: &WavPcm, width: usize, height: usize) -> Vec { mod tests { use super::*; - static TRANSPORT_TEST_LOCK: Mutex<()> = Mutex::new(()); + fn reset_transport() -> AudioEngine { + AUDIO_MIXER.with(|slot| *slot.borrow_mut() = AudioMixer::new()); + take_engine() + } fn transport_pcm() -> WavPcm { WavPcm { @@ -715,7 +981,7 @@ mod tests { #[test] fn transport_is_device_clocked_pauseable_seekable_and_restarts_at_end() { - let _serial = TRANSPORT_TEST_LOCK.lock().unwrap(); + let mut engine = reset_transport(); clear(); assert!(load(transport_pcm())); assert!(is_ready()); @@ -728,7 +994,7 @@ mod tests { assert_eq!(playhead_secs(), 0.0); assert_eq!(playhead_fraction(), 0.0); let mut output = AudioBuffer::new_with_size(2, 2); - mix_into(&mut output, 10.0); + engine.mix_into(&mut output, 10.0); assert!((playhead_secs() - 0.2).abs() < 1e-9); // The drawn playhead tracks the same device-clocked cursor. assert!((playhead_fraction() - 0.5).abs() < 1e-9); @@ -737,7 +1003,7 @@ mod tests { pause(); let paused_at = playhead_secs(); let mut silent = AudioBuffer::new_with_size(2, 2); - mix_into(&mut silent, 10.0); + engine.mix_into(&mut silent, 10.0); assert_eq!(playhead_secs(), paused_at); assert!(silent.channel(0).iter().all(|sample| *sample == 0.0)); @@ -752,7 +1018,7 @@ mod tests { assert_eq!(playhead_secs(), 0.0); let mut to_end = AudioBuffer::new_with_size(8, 2); - mix_into(&mut to_end, 10.0); + engine.mix_into(&mut to_end, 10.0); assert!(at_end()); assert!(!is_playing()); clear(); @@ -768,7 +1034,7 @@ mod tests { #[test] fn layers_replace_the_mixed_track_and_mute_one_at_a_time() { - let _serial = TRANSPORT_TEST_LOCK.lock().unwrap(); + let mut engine = reset_transport(); clear(); assert!(!stems_ready(), "no clip, no layers"); assert!(load(transport_pcm())); @@ -788,7 +1054,7 @@ mod tests { play(); let mut all = AudioBuffer::new_with_size(1, 2); - mix_into(&mut all, 10.0); + engine.mix_into(&mut all, 10.0); assert!( (all.channel(0)[0] - expect(1_000 + 2_000 + 4_000 + 8_000)).abs() < 1e-4, "all four layers sum: {}", @@ -803,7 +1069,7 @@ mod tests { set_lane_muted(2, true); assert!(lane_muted(2) && !lane_muted(0)); let mut without_vocals = AudioBuffer::new_with_size(1, 2); - mix_into(&mut without_vocals, 10.0); + engine.mix_into(&mut without_vocals, 10.0); assert!( (without_vocals.channel(0)[0] - expect(1_000 + 2_000 + 8_000)).abs() < 1e-4, "vocals muted: {}", @@ -816,7 +1082,7 @@ mod tests { set_lane_muted(index, true); } let mut silent = AudioBuffer::new_with_size(1, 2); - mix_into(&mut silent, 10.0); + engine.mix_into(&mut silent, 10.0); assert!(silent.channel(0)[0].abs() < 1e-6, "{}", silent.channel(0)[0]); // Clearing the layers hands playback back to the mixed track. @@ -824,7 +1090,7 @@ mod tests { assert!(!stems_ready()); seek_fraction(0.5); let mut mixed = AudioBuffer::new_with_size(1, 2); - mix_into(&mut mixed, 10.0); + engine.mix_into(&mut mixed, 10.0); assert!( (mixed.channel(0)[0] - 0.8 * GAIN).abs() < 1e-4, "the clip's own third frame: {}", @@ -854,7 +1120,7 @@ mod tests { #[test] fn an_empty_layer_is_refused_rather_than_played_as_a_hole() { - let _serial = TRANSPORT_TEST_LOCK.lock().unwrap(); + let _engine = reset_transport(); clear(); assert!(load(transport_pcm())); assert!(!set_stems( @@ -909,7 +1175,7 @@ mod tests { #[test] fn empty_clip_is_unavailable() { - let _serial = TRANSPORT_TEST_LOCK.lock().unwrap(); + let _engine = reset_transport(); clear(); assert!(!load(WavPcm { frames: Vec::new(), diff --git a/apps/asset-ui/src/chat.rs b/apps/asset-ui/src/chat.rs index dfc6365b3..48adbd1c1 100644 --- a/apps/asset-ui/src/chat.rs +++ b/apps/asset-ui/src/chat.rs @@ -30,6 +30,7 @@ use makepad_chat_ui::{ChatFeed, ClientTools, FeedConfig}; use makepad_asset_client::dto::ChatToolOutcomeDto; use makepad_asset_client::json::{self, Value}; use makepad_asset_client::{ApiEndpoints, ChatAttachment}; +use makepad_widgets::Cx; use std::path::PathBuf; use std::sync::mpsc::{self, Receiver, Sender}; use std::sync::{Arc, Mutex}; @@ -493,7 +494,13 @@ impl ChatBridge { /// The store session is up: open the chat on its broker. The session /// itself is created lazily, on the first turn. - pub fn connect(&mut self, endpoints: ApiEndpoints, token: Option, cache: PathBuf) { + pub fn connect( + &mut self, + cx: &Cx, + endpoints: ApiEndpoints, + token: Option, + cache: PathBuf, + ) { let tools = AppTools { defaults: self.defaults.clone(), fleet: self.fleet.clone(), @@ -503,6 +510,7 @@ impl ChatBridge { self.feed = Some(ChatFeed::start( FeedConfig::new(endpoints, token, cache, "gen", "gen"), Box::new(tools), + cx.thread_spawner(), )); } @@ -939,12 +947,13 @@ mod tests { fn snap(url: &str, domain: &str, id: &str, state: &str) -> BoxSnapshot { BoxSnapshot { base_url: url.into(), - health: Some(HealthJson { realtime: None, + health: Some(HealthJson { realtime: None, activity: None, service: "makepad-asset-ai".into(), version: "t".into(), gpu: Some("RTX".into()), vram_free_mb: Some(20000), vram_total_mb: Some(24576), + vram_usable_mb: None, models_loaded: vec![id.into()], jobs_pending: Some(0), node_id: Some(1), @@ -953,6 +962,7 @@ mod tests { capabilities: Some(vec![domain.into()]), vram_reserve_mb: Some(1024), queue_limit: Some(4), + max_job_body_bytes: None, fleet: None, lanes: None, }), diff --git a/apps/asset-ui/src/import.rs b/apps/asset-ui/src/import.rs index eccbb5645..69e4647a5 100644 --- a/apps/asset-ui/src/import.rs +++ b/apps/asset-ui/src/import.rs @@ -18,13 +18,13 @@ use makepad_asset_client::{ AnnotationUpload, ApiEndpoints, AssetClient, ClientConfig, }; use makepad_asset_data::{sha256, AssetKind, BlobId}; +use makepad_widgets::makepad_platform::thread::{Lane, TaskPool}; use makepad_widgets::{log, vec3f}; use std::collections::BTreeSet; use std::path::{Path, PathBuf}; use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::mpsc::{self, Receiver, TryRecvError}; use std::sync::Arc; -use std::thread; /// Import-card preview strip size. /// How many imported items the LOAD grid remembers a picture for. A grid, @@ -835,6 +835,7 @@ pub struct ImportPage { /// status prefix — without it the bar restarted per pack, which over a /// 38-pack run read as noise. all_run: Option<(usize, usize)>, + pool: Option, } impl Default for ImportPage { @@ -854,11 +855,16 @@ impl Default for ImportPage { rx: None, icon_resume: IconResumeGate::default(), all_run: None, + pool: None, } } } impl ImportPage { + pub fn set_task_pool(&mut self, pool: TaskPool) { + self.pool = Some(pool); + } + /// Selected full Kenney kit (name, official page). pub fn selected_pack_id(&self) -> (String, String) { let packs = on_disk_kenney_packs(); @@ -1532,9 +1538,10 @@ impl ImportPage { self.icon_resume = icon_resume; self.kenney_phase = ImportPhase::compiling(pack_name.clone()); let cancel = self.cancel.clone(); - thread::Builder::new() - .name("asset-ui-kenney-import".into()) - .spawn(move || { + self.pool + .as_ref() + .ok_or("runtime task pool is not configured")? + .submit(Lane::Heavy, move || { let phase = run_kenney_import( &dir, &out, @@ -1548,7 +1555,8 @@ impl ImportPage { ); let _ = tx.send(phase); }) - .map_err(|e| format!("failed to start compile thread: {e}"))?; + .map(|handle| handle.detach()) + .map_err(|e| format!("failed to submit compile job: {e}"))?; Ok(()) } @@ -1580,13 +1588,15 @@ impl ImportPage { self.icon_resume = icon_resume; self.kenney_phase = ImportPhase::compiling("kaykit"); let cancel = self.cancel.clone(); - thread::Builder::new() - .name("asset-ui-kaykit-import".into()) - .spawn(move || { + self.pool + .as_ref() + .ok_or("runtime task pool is not configured")? + .submit(Lane::Heavy, move || { let phase = run_kaykit_import(&dir, &out, spec, server, &tx, &cancel, &icon_resume_rx); let _ = tx.send(phase); }) - .map_err(|e| format!("failed to start KayKit import thread: {e}"))?; + .map(|handle| handle.detach()) + .map_err(|e| format!("failed to submit KayKit import job: {e}"))?; Ok(()) } @@ -1619,9 +1629,10 @@ impl ImportPage { self.kenney_phase = ImportPhase::compiling("all"); self.all_run = Some((0, present.len())); let cancel = self.cancel.clone(); - thread::Builder::new() - .name("asset-ui-kenney-import-all".into()) - .spawn(move || { + self.pool + .as_ref() + .ok_or("runtime task pool is not configured")? + .submit(Lane::Heavy, move || { let total = present.len(); let mut ok = Vec::new(); let mut failed = Vec::new(); @@ -1749,7 +1760,8 @@ impl ImportPage { skipped, }); }) - .map_err(|e| format!("failed to start import-all thread: {e}"))?; + .map(|handle| handle.detach()) + .map_err(|e| format!("failed to submit import-all job: {e}"))?; Ok(()) } @@ -4003,6 +4015,12 @@ fn publish_compiled_pack( categories: vec!["kenney".into(), pack_name.to_string()], tags, creator: KENNEY_CREDITS.to_string(), + artist: String::new(), + artist_url: String::new(), + album: String::new(), + source_url: String::new(), + license: String::new(), + license_url: String::new(), generator: "pack_import".into(), backend: "asset-ui".into(), model: pack_name.to_string(), diff --git a/apps/asset-ui/src/import_classic.rs b/apps/asset-ui/src/import_classic.rs index 8c37a107c..e44b2912d 100644 --- a/apps/asset-ui/src/import_classic.rs +++ b/apps/asset-ui/src/import_classic.rs @@ -30,7 +30,6 @@ use std::path::{Path, PathBuf}; use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::mpsc::{self, Receiver, TryRecvError}; use std::sync::Arc; -use std::thread; use crate::import::{KENNEY_MODULE, KAYKIT_MODULE, NASA_SKY_MODULE, PackModule}; use makepad_asset_importer::tdm_zipsync::{ @@ -306,6 +305,7 @@ pub struct ClassicImportCard { /// a classic pack publishes real thumbnails and its staging can be /// reclaimed the moment publish succeeds. See [`IconResumeGate`]. icon_resume: IconResumeGate, + pool: Option, } struct IsoSync { @@ -358,6 +358,7 @@ impl ClassicImportCard { tdm: None, iso: None, icon_resume: IconResumeGate::default(), + pool: None, } } @@ -658,6 +659,7 @@ impl ClassicImportCard { path_override: String, server: Option, ) -> Result<(), String> { + self.pool = Some(cx.task_pool()); if self.compiling() { return Err(format!( "a {} import is already running", @@ -1540,9 +1542,10 @@ impl ClassicImportCard { // until the UI has taken every landing for icon rendering. let (gate, icon_resume_rx) = IconResumeGate::armed(); self.icon_resume = gate; - thread::Builder::new() - .name(format!("asset-ui-{}-import", source.id())) - .spawn(move || { + self.pool + .as_ref() + .ok_or("runtime task pool is not configured")? + .submit(Lane::Heavy, move || { let phase = run_classic_import( &dir, &out, @@ -1555,7 +1558,8 @@ impl ClassicImportCard { ); let _ = tx.send(phase); }) - .map_err(|e| format!("failed to start classic import thread: {e}"))?; + .map(|handle| handle.detach()) + .map_err(|e| format!("failed to submit classic import job: {e}"))?; Ok(()) } @@ -2267,6 +2271,12 @@ fn publish_classic_pack( categories: vec![source.id().into(), pack_name.to_string()], tags, creator: source.credits().to_string(), + artist: String::new(), + artist_url: String::new(), + album: String::new(), + source_url: String::new(), + license: String::new(), + license_url: String::new(), generator: "classic_import".into(), backend: "asset-ui".into(), model: pack_name.to_string(), diff --git a/apps/asset-ui/src/main.rs b/apps/asset-ui/src/main.rs index ea73ff321..bca839933 100644 --- a/apps/asset-ui/src/main.rs +++ b/apps/asset-ui/src/main.rs @@ -55,7 +55,9 @@ mod import_classic; mod store_content; mod library; mod mask_paint; -mod mesh_view; +mod mesh_view { + pub use makepad_media_view::mesh_view::*; +} mod music_page; use crate::mask_paint::{MaskPaint, MaskPaintAction}; mod pipeline; @@ -63,7 +65,9 @@ mod runs_chip; mod scheduler; mod store_views; mod thumbnail_renderer; -mod video_player; +mod video_player { + pub use makepad_media_view::{FileVideoPlayer as VideoPlayer, VideoDecoder}; +} mod webcam; use crate::artifact_io::{ @@ -244,7 +248,7 @@ use crate::store_views::{ StoreListPanel, StoreRow, TileDelete, }; -use crate::video_player::VideoPlayer; +use crate::video_player::{VideoDecoder, VideoPlayer}; use makepad_micro_serde::SerJson; use makepad_widgets::*; @@ -4010,7 +4014,7 @@ pub struct App { fleet_timer: Timer, /// LAN beacon listener; polled on the fleet timer. #[rust] - discovered: Option, + discovered: Option, #[rust] job_timer: Timer, #[rust] @@ -4036,6 +4040,8 @@ pub struct App { library: Option, #[rust] video: Option, + #[rust] + video_decoder: Option, /// The file the viewer's current video came from — Restart and the loop /// toggle re-open it. video_path: Option, @@ -4323,8 +4329,16 @@ impl App { // -- setup --------------------------------------------------------------- fn setup(&mut self, cx: &mut Cx) { + let pool = cx.task_pool(); + let spawner = cx.thread_spawner(); + let (video_decoder, mut video_audio) = + VideoDecoder::start(spawner.clone()).expect("asset-ui video decoder worker"); + self.video_decoder = Some(video_decoder); let _ = std::fs::create_dir_all(artifacts_dir()); - self.artifact_io = Some(ArtifactIo::start()); + self.artifact_io = Some(ArtifactIo::start(spawner.clone())); + self.analysis = Some(AnalysisQueue::start(spawner.clone())); + self.import_page.set_task_pool(pool.clone()); + self.music_import_page.set_task_pool(pool.clone()); self.load_fleet_prefs(); self.library = Some(Library::open(repo_path("local/ai_content_library"))); self.saved_presets = fast_presets::load(&fast_presets::store_path()); @@ -4349,7 +4363,11 @@ impl App { // The store hosts the embedded Asset Server; hand it the library it // must publish. Library::open ran above, so the product backfill is // already on disk when the watcher's first poll reads index.json. - self.store.start(PathBuf::from(repo_path("local/ai_content_library"))); + self.store.start( + PathBuf::from(repo_path("local/ai_content_library")), + pool, + spawner, + ); self.asset_store_timer = cx.start_interval(0.2); // Opening stays metadata-only; every missing preview is queued here // and regenerated a bounded slice at a time once frames are flowing. @@ -4548,11 +4566,12 @@ impl App { self.refresh_voice_ui(cx); self.sync_preset_name_box(cx); - // Speakers: wav artifacts + video soundtrack. + // Speakers: both engines move into the callback and own their state. + let mut audio_engine = crate::audio::take_engine(); cx.audio_output(0, move |info, output| { output.zero(); - crate::audio::mix_into(output, info.sample_rate); - crate::video_player::mix_into(output, info.sample_rate); + audio_engine.mix_into(output, info.sample_rate); + video_audio.mix_into(output, info.sample_rate); }); // Headless drive. @@ -7541,7 +7560,7 @@ impl App { // WAV must not call play() or a 200ms DS_* / Quake // shot becomes a loop (play-at-end restarts). if audition && audio::autoplay_one_shot(domain, pcm.seconds()) { - crate::video_player::stop_audio(); + self.stop_video_audio(); audio::play(); self.arm_audio_pump(cx); } @@ -7589,7 +7608,8 @@ impl App { // behind a new open or an error state. self.stop_video_playback(); self.clear_video_frame(cx); - match VideoPlayer::new(&path.to_string_lossy()) { + let decoder = self.video_decoder.as_ref().expect("video decoder started"); + match VideoPlayer::new(&path.to_string_lossy(), decoder) { Ok(player) => { self.ui.label(cx, ids!(video_info)).set_text( cx, @@ -9101,7 +9121,7 @@ impl App { && audio::is_ready() && !audio::is_playing() { - crate::video_player::stop_audio(); + self.stop_video_audio(); audio::play(); self.arm_audio_pump(cx); self.sync_audio_ui(cx); @@ -9963,7 +9983,7 @@ impl App { if !self.chat.is_linked() { if let Some(endpoints) = self.store.endpoints { let cache = session_config_from_env().cache_parent.join("cache-chat"); - self.chat.connect(endpoints, self.store.token.clone(), cache); + self.chat.connect(cx, endpoints, self.store.token.clone(), cache); // The pane says "waiting for the asset server" until // something redraws it, and the feed only marks itself // dirty once a turn runs — so the line would sit there @@ -10714,7 +10734,8 @@ impl App { if let Some(path) = item.as_ref().and_then(|item| item.payload.clone()) { self.stop_video_playback(); self.clear_video_frame(cx); - match VideoPlayer::new(&path.to_string_lossy()) { + let decoder = self.video_decoder.as_ref().expect("video decoder started"); + match VideoPlayer::new(&path.to_string_lossy(), decoder) { Ok(player) => { self.library_video_file = Some(file.clone()); self.video = Some(player); @@ -10789,7 +10810,8 @@ impl App { self.library_audio_file = Some(file.clone()); // The transport: decoded off the frame thread and // installed when it lands. - let clip_gen = crate::audio::load_clip_async(bytes.clone()); + let pool = cx.task_pool(); + let clip_gen = crate::audio::load_clip_async(&pool, bytes.clone()); // And, exactly once per track, what the store // already holds BESIDE the mixed audio: the four // separated layers and the transcript. The clip @@ -10930,11 +10952,9 @@ impl App { // -- "Split audio layers": the bake queue and its consumers ----------- - /// The bake + fetch lanes, started on first use. Two threads parked on - /// a channel is the whole cost of having them. + /// The bake + fetch lanes, started once with the app and fed by channels. fn analysis(&mut self) -> &mut analysis::AnalysisQueue { - self.analysis - .get_or_insert_with(analysis::AnalysisQueue::start) + self.analysis.as_mut().expect("analysis workers started") } /// The selected catalog hit when it is an AUDIO asset: id and title. @@ -12400,7 +12420,13 @@ impl App { /// [`Self::clear_video_frame`] is also called. fn stop_video_playback(&mut self) { self.video = None; - crate::video_player::stop_audio(); + self.stop_video_audio(); + } + + fn stop_video_audio(&self) { + if let Some(decoder) = &self.video_decoder { + decoder.stop_audio(); + } } /// Blank the actual video WIDGET texture (not only the app-side handle), @@ -12792,7 +12818,8 @@ impl App { fn restart_viewer_video(&mut self, cx: &mut Cx) -> bool { let Some(path) = self.video_path.clone() else { return false }; self.stop_video_playback(); - match VideoPlayer::new(&path.to_string_lossy()) { + let decoder = self.video_decoder.as_ref().expect("video decoder started"); + match VideoPlayer::new(&path.to_string_lossy(), decoder) { Ok(player) => { self.video = Some(player); self.sync_video_transport(cx); @@ -14133,7 +14160,7 @@ impl MatchEvent for App { } else { // A user-resumed WAV preview wins over a stale video // soundtrack in the shared device callback. - crate::video_player::stop_audio(); + self.stop_video_audio(); audio::play(); self.arm_audio_pump(cx); } @@ -14434,13 +14461,11 @@ impl AppMain for App { fn script_mod(vm: &mut ScriptVm) -> ScriptValue { crate::makepad_widgets::script_mod(vm); // Draw shaders must register before the widgets that declare them. - makepad_render::script_mod(vm); - makepad_xr::script_mod(vm); + makepad_media_view::script_mod(vm); // Shared preview widgets (ContentPreview / AudioView): the pool this // app draws catalog content with, and the same one the VJ and DJ // surfaces adopt. makepad_asset_widgets::script_mod(vm); - crate::mesh_view::script_mod(vm); crate::mask_paint::script_mod(vm); crate::billboard_view::script_mod(vm); crate::thumbnail_renderer::script_mod(vm); @@ -14691,14 +14716,17 @@ impl AppMain for App { } } self.scrub_audio(cx, event); - if self.audio_timer.is_event(event).is_some() && audio::is_ready() { - self.sync_audio_ui(cx); - // The Library rail has its own transport over the same mixer. - if self.surface == Surface::Library && self.library_audio_file.is_some() { - self.refresh_library_audio(cx); - // Playback that started from the transport re-arms the - // transcript's own per-frame follow. - self.arm_lyrics_pump(cx); + if self.audio_timer.is_event(event).is_some() { + audio::pump(); + if audio::is_ready() { + self.sync_audio_ui(cx); + // The Library rail has its own transport over the same mixer. + if self.surface == Surface::Library && self.library_audio_file.is_some() { + self.refresh_library_audio(cx); + // Playback that started from the transport re-arms the + // transcript's own per-frame follow. + self.arm_lyrics_pump(cx); + } } } if self.audio_timer.is_event(event).is_some() && self.webcam.capturing { @@ -15918,6 +15946,13 @@ mod world_style_tests { namespace: namespace.to_string(), kind: Some(makepad_asset_data::AssetKind::World), title: title.to_string(), + creator: String::new(), + artist: String::new(), + artist_url: String::new(), + album: String::new(), + source_url: String::new(), + license: String::new(), + license_url: String::new(), snippet: String::new(), score: 0, live: true, diff --git a/apps/asset-ui/src/music_page.rs b/apps/asset-ui/src/music_page.rs index c4dc26767..97cdd647c 100644 --- a/apps/asset-ui/src/music_page.rs +++ b/apps/asset-ui/src/music_page.rs @@ -12,11 +12,11 @@ use crate::import::{ImportPhase, ServerSession}; use makepad_asset_client::{AssetClient, ClientConfig}; use makepad_asset_importer::music_import::{self, MusicReport}; +use makepad_widgets::makepad_platform::thread::{Lane, TaskPool}; use std::path::PathBuf; use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::mpsc::{self, Receiver, TryRecvError}; use std::sync::Arc; -use std::thread; /// The card's whole state: which folder, what the worker is doing, and the /// one-line result of the last run. @@ -44,6 +44,7 @@ pub struct MusicImportPage { /// Aliases the finished run landed, waiting to be handed to the /// analysis queue. Non-empty only when `split_layers` was on. pending_analysis: Vec, + pool: Option, } /// What the worker sends back: live progress, then exactly one verdict. @@ -65,6 +66,7 @@ impl Default for MusicImportPage { split_layers: false, bake_lyrics: false, pending_analysis: Vec::new(), + pool: None, } } } @@ -74,6 +76,10 @@ impl Default for MusicImportPage { pub const MUSIC_NAMESPACE: &str = "music"; impl MusicImportPage { + pub fn set_task_pool(&mut self, pool: TaskPool) { + self.pool = Some(pool); + } + /// The job this card would enqueue right now, or `None` while no folder /// has been picked. pub fn job(&self) -> Option { @@ -209,13 +215,18 @@ impl MusicImportPage { total: 0, current: String::new(), }; - thread::Builder::new() - .name("asset-ui-music-import".into()) - .spawn(move || { + let Some(pool) = self.pool.clone() else { + return Err(self.refuse("runtime task pool is not configured".into())); + }; + match pool.submit(Lane::Heavy, move || { let msg = run_music_import(&dir, server, &tx, &cancel); let _ = tx.send(msg); - }) - .map_err(|e| self.refuse(format!("failed to start music import thread: {e}")))?; + }) { + Ok(handle) => handle.detach(), + Err(error) => { + return Err(self.refuse(format!("failed to submit music import job: {error}"))); + } + } Ok(()) } diff --git a/apps/asset-ui/src/pipeline.rs b/apps/asset-ui/src/pipeline.rs index a3c07e0fc..5c2609175 100644 --- a/apps/asset-ui/src/pipeline.rs +++ b/apps/asset-ui/src/pipeline.rs @@ -218,16 +218,16 @@ impl Preset { /// Python rigger, or another arbitrary model merely because it is resident. // This is the warm, local Makepad-Llama model advertised by the Mac fleet // node. Keeping it as fallback avoids an accidental large-model cold pull. -const CHARACTER_LLM_MODEL: &str = "qwen3.5-9b"; +use makepad_asset_creator::character::CHARACTER_LLM_MODEL; /// Fleet-wide default for text expansion once a node has the audited weights /// ready. This is a preference, not a hard pin: first-run provisioning stays /// explicit and the warm 9B lane remains immediately usable. const PREFERRED_EXPAND_MODEL: &str = "qwen3.8-27b"; -const CHARACTER_IMAGE_MODEL: &str = "flux1-dev"; -const CHARACTER_MATTE_MODEL: &str = "birefnet-hr"; -const CHARACTER_MESH_MODEL: &str = "trellis-2"; -const CHARACTER_RIG_MODEL: &str = "skintokens"; -const CHARACTER_MOTION_MODEL: &str = "hy-motion"; +use makepad_asset_creator::character::CHARACTER_IMAGE_MODEL; +use makepad_asset_creator::character::CHARACTER_MATTE_MODEL; +use makepad_asset_creator::character::CHARACTER_MESH_MODEL; +use makepad_asset_creator::character::CHARACTER_RIG_MODEL; +use makepad_asset_creator::character::CHARACTER_MOTION_MODEL; /// Instruction image editing (reference image + "change …" prompt). const EDIT_MODEL: &str = "flux2-klein-4b"; /// Sprite enhancement runs on the 32B dev DiT, NOT the 4-step distilled @@ -257,7 +257,7 @@ const SPLAT_MODEL: &str = "triposplat"; /// A character expansion substantially shorter than the 40-90 words asked /// for by `expand_rig.txt` is not a usable rig-safe brief. Refuse to quietly /// continue with it; the user can see and retry the failed LLM stage. -const CHARACTER_EXPANSION_MIN_WORDS: usize = 24; +// The shared character contract owns brief validation. /// A character reconstruction gets two deterministic second chances when the /// rig or animated-skin quality gate rejects it. The matte/image are @@ -544,7 +544,7 @@ pub const PRESETS: &[Preset] = &[ // (idle/walk/jump locomotion, see mesh_view play mode). Preset::linear( "character (playable)", - &["text", "image", "matte", "mesh", "rig", "motion"], + makepad_asset_creator::character::CHARACTER_DOMAINS, // Character geometry is downstream of this one image: Schnell's // four-step distillation is useful for previews, but it is the wrong // silent affinity fallback for the rig master. Pin the validated @@ -1329,21 +1329,7 @@ impl Pipeline { return unusable("was empty"); } if self.is_character_pipeline() { - let words = text.split_whitespace().count(); - if words < CHARACTER_EXPANSION_MIN_WORDS { - return Err(format!( - "LLM character brief is too short ({words} words, need at least {CHARACTER_EXPANSION_MIN_WORDS}); refusing to start image generation" - )); - } - if !text - .to_lowercase() - .contains(&self.prompt.trim().to_lowercase()) - { - return Err(format!( - "LLM character brief dropped identity anchor {:?}; refusing to start image generation", - self.prompt.trim() - )); - } + makepad_asset_creator::character::validate_brief(&self.prompt, text)?; } return Ok(text.to_string()); } @@ -1501,16 +1487,7 @@ impl Pipeline { let is_music_target = target == "music"; request.target_domain = Some(target); if self.is_character_pipeline() { - request.identity_anchor = Some(self.prompt.trim().to_string()); - // Named-character identity and rig-safe presentation are - // constraints, not a variant hunt. Keep this expansion - // low-temperature and deterministic enough to avoid - // inventing conflicting signature traits. - request.temperature = Some(0.0); - request.style = Some( - "When the intent names an established character, preserve the exact named identity and canonical official design unchanged. Do not redesign, genericize, or guess traits. If a visual trait is uncertain, omit it instead of inventing it; it is better to say 'canonical official design unchanged' and spend the remaining prompt on full-body framing, a relaxed wide A-pose with straight diagonal arms and hands clear above the hips, visible gaps between every limb and the torso, even studio light, a uniform plain background, and a clean separated silhouette. Rigging constraints may change pose and spacing but never delete canonical anatomy or worn pieces." - .to_string(), - ); + makepad_asset_creator::character::configure_expansion(&mut request, &self.prompt); } // Music expansion carries a compact structured production // brief AND original section-tagged lyrics. Scale its budget @@ -3381,7 +3358,7 @@ impl Pipeline { } => { let bytes = response .filter(|response| !failed && response.status_code == 200) - .and_then(|response| response.body.clone()); + .and_then(|response| response.body.as_deref().map(<[u8]>::to_vec)); let Some(bytes) = bytes else { return self.candidate_failed( cx, @@ -3589,7 +3566,7 @@ impl Pipeline { Req::Artifact(stage, artifact) => { let bytes = response .filter(|r| !failed && r.status_code == 200) - .and_then(|r| r.body.clone()); + .and_then(|r| r.body.as_deref().map(<[u8]>::to_vec)); let Some(bytes) = bytes else { return self.fail_stage_or_skip_expander( cx, @@ -3950,12 +3927,13 @@ mod tests { use makepad_ai_hub::protocol::{HealthJson, ModelInfoJson, MODEL_STATE_LOADED}; BoxSnapshot { base_url: url.to_string(), - health: Some(HealthJson { realtime: None, + health: Some(HealthJson { realtime: None, activity: None, service: "test".to_string(), version: "1".to_string(), gpu: Some("GPU".to_string()), vram_free_mb: Some(24_000), vram_total_mb: Some(24_000), + vram_usable_mb: None, models_loaded: vec!["flux1-schnell".to_string()], jobs_pending: Some(0), node_id: Some(1), @@ -3964,6 +3942,7 @@ mod tests { capabilities: Some(vec!["image".to_string()]), vram_reserve_mb: Some(0), queue_limit: Some(8), + max_job_body_bytes: None, fleet: None, lanes: None, }), @@ -3995,12 +3974,13 @@ mod tests { use makepad_ai_hub::protocol::{HealthJson, ModelInfoJson, MODEL_STATE_LOADED}; BoxSnapshot { base_url: url.to_string(), - health: Some(HealthJson { realtime: None, + health: Some(HealthJson { realtime: None, activity: None, service: "test".to_string(), version: "1".to_string(), gpu: Some("24 GB GPU".to_string()), vram_free_mb: Some(24 * 1024), vram_total_mb: Some(24 * 1024), + vram_usable_mb: None, models_loaded: models .iter() .filter(|(_, state)| *state == MODEL_STATE_LOADED) @@ -4013,6 +3993,7 @@ mod tests { capabilities: Some(vec!["text".to_string()]), vram_reserve_mb: Some(0), queue_limit: Some(8), + max_job_body_bytes: None, fleet: None, lanes: None, }), @@ -4957,7 +4938,10 @@ Arrangement: Pulsing bass, gated drums and widening analog pads." let mut want: Vec<(String, String)> = [ ("audio", "moss-sfx"), ("audio", "sa3-sfx"), + ("audio", "salamander-drumkit"), ("audio", "woosh-sfx"), + ("beats", "beat-this"), + ("body", "sam3dbody"), ("control", "flux1-canny-dev"), ("control", "flux1-depth-dev"), ("depth", "da3-metric-large"), @@ -4975,6 +4959,8 @@ Arrangement: Pulsing bass, gated drums and widening analog pads." ("music", "minimax-music3"), ("music", "minimax-music3-q4"), ("music", "ace-step-1.5-xl"), + ("notes", "basic-pitch"), + ("ocr", "chandra-ocr-2"), ("paint", "hunyuan3d-paint-2.1"), ("rig", "skintokens"), ("rig", "skintokens-oracle"), @@ -4982,6 +4968,8 @@ Arrangement: Pulsing bass, gated drums and widening analog pads." ("splat", "triposplat"), ("speech", "indextts-2.5"), ("speech", "kokoro"), + ("stems", "bs-roformer-4stem"), + ("stt", "whisper-large-v3-turbo"), ("text", "qwen3.8-27b"), ("upscale", "realesrgan-x4plus"), ("vision", "qwen3.8-27b-vision"), @@ -5042,7 +5030,7 @@ Arrangement: Pulsing bass, gated drums and widening analog pads." /// fails visibly as a service gap instead of rerouting. const DOCUMENTED_OVERRIDE_PINS: &[(&str, &str)] = &[("text", CHARACTER_LLM_MODEL)]; - /// Every pin must reference a model the registry actually has in the + /// Every pin must reference a model the registry can serve for the /// pinned domain — or be a documented cache-registry override above. /// Catches typos and silent registry drift. #[test] @@ -5057,7 +5045,13 @@ Arrangement: Pulsing bass, gated drums and widening analog pads." registry .models .iter() - .any(|entry| entry.id == *model && entry.domain.as_str() == *domain), + .any(|entry| { + entry.id == *model + && (entry.domain.as_str() == *domain + || (*domain == "edit" + && entry.domain.as_str() == "image" + && model.starts_with("flux2-dev"))) + }), "preset {:?} pins unknown model {domain}/{model}", preset.name ); diff --git a/apps/asset-ui/src/store_views.rs b/apps/asset-ui/src/store_views.rs index b2c998f90..0ee5fd3b9 100644 --- a/apps/asset-ui/src/store_views.rs +++ b/apps/asset-ui/src/store_views.rs @@ -2542,6 +2542,13 @@ mod tests { namespace: "gen".into(), kind: Some(AssetKind::Prop), title: title.into(), + creator: String::new(), + artist: String::new(), + artist_url: String::new(), + album: String::new(), + source_url: String::new(), + license: String::new(), + license_url: String::new(), snippet: "a thing".into(), score: 10, live, diff --git a/apps/asset-ui/src/thumbnail_renderer.rs b/apps/asset-ui/src/thumbnail_renderer.rs index 1677b5638..34538197b 100644 --- a/apps/asset-ui/src/thumbnail_renderer.rs +++ b/apps/asset-ui/src/thumbnail_renderer.rs @@ -823,6 +823,7 @@ impl ThumbnailRenderer { color_adjust: vec4(0.0, 1.0, 1.0, 0.0), dynamic: true, depth_order: 0.0, + custom_material: None, part_poses: Vec::new(), }), frame, diff --git a/apps/asset-ui/src/video_player.rs b/apps/asset-ui/src/video_player.rs deleted file mode 100644 index 6184fa5f5..000000000 --- a/apps/asset-ui/src/video_player.rs +++ /dev/null @@ -1,637 +0,0 @@ -//! Video artifact playback — the sandbox's proven decode pattern -//! (apps/sandbox/src/video_player.rs): a decode thread pulls frames + audio -//! from the platform video-file seam (`makepad_platform::video_file`, -//! hardware codecs), a small ring buffer hands BGRA frames to the render -//! thread paced by pts against a wall clock, and the audio track mixes into -//! this app's `cx.audio_output` closure. -//! -//! Playback is PLAY-ONCE: at end-of-stream the remaining audio drains and -//! the clip stops (the sandbox pattern's loop-forever reopen was what users -//! heard as "the soundtrack never ends"). Loading a new artifact drops the -//! previous player (its `Drop` silences the queue); `stop_audio()` silences -//! immediately and stays muted until the next clip starts. -//! -//! Copied rather than imported: the sandbox is an app crate under active -//! concurrent development, not a library — and this pattern is ~250 lines. - -use makepad_widgets::log; -use makepad_widgets::makepad_platform::audio::AudioBuffer; -use makepad_widgets::makepad_platform::video_file::{nv12, VideoFileDecoder}; -use std::collections::VecDeque; -use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU64, Ordering}; -use std::sync::{Arc, Mutex}; -use std::time::{Duration, Instant}; - -const RING_FRAMES: usize = 3; -const AUDIO_AHEAD_SECS: f64 = 1.0; - -struct Frame { - pts_100ns: i64, - bgra: Vec, -} - -struct Shared { - frames: Mutex>, - stop: AtomicBool, - /// The decode thread exited for good — a honored stop or a fatal - /// decode error. End-of-stream no longer ends the thread: it PARKS - /// (see `eos`) so seeks — loop restarts, scrubs — stay instant. - done: AtomicBool, - /// The stream is fully decoded and the thread is parked waiting for a - /// seek or a stop. With an empty ring this is the player's EOS state. - eos: AtomicBool, - /// Requested playback position in 100ns units; -1 = none. The decode - /// thread consumes it (an in-place decoder seek — no reopen). - seek_100ns: AtomicI64, -} - -/// Soundtrack-queue ownership ticket. Every player claims a fresh epoch; a -/// DETACHED decode thread of a dropped player still holds its old epoch and -/// its pushes bounce off the queue (see [`VideoPlayer::drop`]). -static NEXT_CLIP_EPOCH: AtomicU64 = AtomicU64::new(1); - -/// Half a frame at 60 fps: a seek target inside a frame's span lands on -/// that frame instead of the next. -const FRAME_EPS_100NS: i64 = 83_000; - -pub struct VideoPlayer { - pub width: u32, - pub height: u32, - /// Container-reported duration; 0 when the container does not say. - pub duration_100ns: i64, - shared: Arc, - started: Option, - last_pts: i64, - /// While paused: when the pause began. The clock rebases by the paused - /// span on resume, so playback continues where it stopped instead of - /// skipping the frames "missed" on the wall clock. - paused_at: Option, - epoch: u64, -} - -impl VideoPlayer { - pub fn new(path: &str) -> Result { - let info = VideoFileDecoder::open(path) - .map_err(|e| e.to_string())? - .info() - .clone(); - if info.width == 0 || info.height == 0 { - return Err(format!( - "video reports zero size: {}x{}", - info.width, info.height - )); - } - log!( - "video: {} {}x{} {}/{} fps codec {:?} audio {} ({} Hz, {} ch)", - path, - info.width, - info.height, - info.fps_num, - info.fps_den, - info.video_codec, - info.has_audio, - info.audio_sample_rate, - info.audio_channels - ); - // Fresh clip: take ownership of the soundtrack queue. Claiming a new - // epoch drops any previous tail, lifts the sticky stop_audio() mute, - // and locks every straggler push from an older detached decode - // thread out of the queue. - let epoch = NEXT_CLIP_EPOCH.fetch_add(1, Ordering::Relaxed); - { - let mut audio = video_audio().lock().unwrap(); - audio.clear(); - audio.muted = false; - audio.owner = epoch; - } - let shared = Arc::new(Shared { - frames: Mutex::new(VecDeque::new()), - stop: AtomicBool::new(false), - done: AtomicBool::new(false), - eos: AtomicBool::new(false), - seek_100ns: AtomicI64::new(-1), - }); - let thread_shared = shared.clone(); - let thread_path = path.to_string(); - // The JoinHandle is deliberately dropped: teardown must never join a - // possibly wedged hardware decoder on the UI thread. The thread is - // detached; `stop` + the audio epoch make that safe. - std::thread::Builder::new() - .name("asset-ui-video-decode".into()) - .spawn(move || { - match VideoFileDecoder::open(&thread_path) { - Ok(decoder) => decode_loop(thread_path, decoder, &thread_shared, epoch), - Err(e) => log!("video: decode thread open failed: {}", e), - } - thread_shared.done.store(true, Ordering::Release); - }) - .map_err(|e| e.to_string())?; - Ok(Self { - width: info.width, - height: info.height, - duration_100ns: info.duration_100ns, - shared, - started: None, - last_pts: 0, - paused_at: None, - epoch, - }) - } - - /// The newest frame whose pts has been reached; `None` keeps whatever is - /// on the texture. Call once per render frame. - pub fn is_paused(&self) -> bool { - self.paused_at.is_some() - } - - /// Freeze the picture; the soundtrack mutes with it. Idempotent. - pub fn pause(&mut self) { - if self.paused_at.is_none() { - self.paused_at = Some(Instant::now()); - video_audio().lock().unwrap().muted = true; - } - } - - /// Continue from the paused position by pushing the clock base forward - /// by the paused span. Idempotent. - pub fn resume(&mut self) { - if let Some(paused_at) = self.paused_at.take() { - if let Some(started) = &mut self.started { - *started += paused_at.elapsed(); - } - video_audio().lock().unwrap().muted = false; - } - } - - pub fn take_due_frame(&mut self) -> Option> { - if self.paused_at.is_some() { - // Paused normally: hold the picture. Freshly seeked while - // paused (clock unset): show the seek target frame once. - if self.started.is_some() { - return None; - } - let mut frames = self.shared.frames.lock().unwrap(); - let frame = frames.pop_front()?; - self.last_pts = frame.pts_100ns; - // Keep the clock unset: resume rebases at the next frame. - return Some(frame.bgra); - } - let mut frames = self.shared.frames.lock().unwrap(); - let first_pts = frames.front()?.pts_100ns; - let started = *self.started.get_or_insert_with(|| { - Instant::now() - Duration::from_nanos(first_pts.max(0) as u64 * 100) - }); - let media_100ns = (started.elapsed().as_nanos() / 100) as i64; - let mut due = None; - while frames.front().is_some_and(|f| f.pts_100ns <= media_100ns) { - due = frames.pop_front(); - } - if let Some(frame) = &due { - self.last_pts = frame.pts_100ns; - } - due.map(|f| f.bgra) - } - - pub fn position_secs(&self) -> f64 { - self.last_pts as f64 / 10_000_000.0 - } - - pub fn duration_secs(&self) -> f64 { - self.duration_100ns as f64 / 10_000_000.0 - } - - /// Jump playback to `secs`. The decode thread reopens the file and - /// discards up to the target; the picture clock rebases on the first - /// frame that arrives, so play continues from there (paused stays - /// paused, showing the seeked frame). - pub fn seek(&mut self, secs: f64) { - let target = (secs.max(0.0) * 10_000_000.0) as i64; - self.shared.seek_100ns.store(target, Ordering::Release); - self.shared.frames.lock().unwrap().clear(); - self.started = None; - self.last_pts = target; - let mut audio = video_audio().lock().unwrap(); - if audio.owner == self.epoch { - audio.clear(); - } - } - - /// True end-of-playback for the frame pump: the decode thread has exited - /// (end of stream, decode error, or stop) and every buffered frame has - /// been taken. The soundtrack tail may still be draining in the audio - /// callback — that needs no frame pump. - pub fn at_end(&self) -> bool { - (self.shared.eos.load(Ordering::Acquire) || self.shared.done.load(Ordering::Acquire)) - && self.shared.frames.lock().unwrap().is_empty() - } - - /// True while a seek request is still unconsumed by the decode thread — - /// the host coalesces scrub drags on this instead of flooding. - pub fn seek_pending(&self) -> bool { - self.shared.seek_100ns.load(Ordering::Acquire) >= 0 - } -} - -impl Drop for VideoPlayer { - fn drop(&mut self) { - self.shared.stop.store(true, Ordering::Relaxed); - // NEVER join the decode thread here: a wedged hardware decoder call - // would hang the UI thread. The detached thread observes `stop` - // between packets and exits on its own; until then the epoch guard - // keeps its audio pushes out of the queue, and its frame ring dies - // with the last Arc. - let mut audio = video_audio().lock().unwrap(); - if audio.owner == self.epoch { - audio.clear(); - audio.owner = 0; - } - } -} - -fn decode_loop(_path: String, mut decoder: VideoFileDecoder, shared: &Shared, epoch: u64) { - let info = decoder.info().clone(); - let mut audio_eos = false; - let mut rgb_scratch = Vec::new(); - loop { - if shared.stop.load(Ordering::Relaxed) { - return; - } - let seek = shared.seek_100ns.swap(-1, Ordering::AcqRel); - if seek >= 0 { - shared.eos.store(false, Ordering::Release); - // In-place decoder seek (SetCurrentPosition / reader rebuild in - // the platform layer, ~10 ms) — never a full reopen, which is - // what makes SCRUBBING realtime and a loop restart seamless. - match decoder.seek(seek) { - Ok(()) => { - audio_eos = !info.has_audio; - shared.frames.lock().unwrap().clear(); - loop { - if shared.stop.load(Ordering::Relaxed) { - return; - } - if shared.seek_100ns.load(Ordering::Acquire) >= 0 { - break; // newer scrub target supersedes this one - } - match decoder.next_frame() { - Ok(Some(frame)) if frame.pts_100ns + FRAME_EPS_100NS < seek => {} - Ok(Some(frame)) => { - nv12::nv12_to_rgb8( - &frame.nv12, - frame.width, - frame.height, - &mut rgb_scratch, - ); - let mut bgra = - Vec::with_capacity((frame.width * frame.height) as usize); - for px in rgb_scratch.chunks_exact(3) { - bgra.push( - 0xff00_0000 - | ((px[0] as u32) << 16) - | ((px[1] as u32) << 8) - | px[2] as u32, - ); - } - shared - .frames - .lock() - .unwrap() - .push_back(Frame { pts_100ns: frame.pts_100ns, bgra }); - break; - } - Ok(None) => break, - Err(e) => { - log!("video: seek decode error: {}", e); - break; - } - } - } - // Audio follows the picture: drop queued samples and - // skip the soundtrack forward to the target. - if info.has_audio { - video_audio().lock().unwrap().clear_for(epoch); - loop { - if shared.stop.load(Ordering::Relaxed) { - return; - } - match decoder.next_audio() { - Ok(Some(chunk)) if chunk.pts_100ns < seek => {} - Ok(Some(chunk)) => { - video_audio().lock().unwrap().push_i16( - epoch, - &chunk.samples, - chunk.channels, - chunk.sample_rate, - ); - break; - } - Ok(None) => { - audio_eos = true; - break; - } - Err(_) => { - audio_eos = true; - break; - } - } - } - } - } - Err(e) => log!("video: decoder seek failed: {}", e), - } - continue; - } - if info.has_audio && !audio_eos { - while video_audio().lock().unwrap().buffered_secs() < AUDIO_AHEAD_SECS { - match decoder.next_audio() { - Ok(Some(chunk)) => video_audio().lock().unwrap().push_i16( - epoch, - &chunk.samples, - chunk.channels, - chunk.sample_rate, - ), - Ok(None) => { - audio_eos = true; - break; - } - Err(e) => { - log!("video: audio decode error: {}", e); - audio_eos = true; - break; - } - } - } - } - if shared.frames.lock().unwrap().len() >= RING_FRAMES { - std::thread::sleep(Duration::from_millis(4)); - continue; - } - match decoder.next_frame() { - Ok(Some(frame)) => { - nv12::nv12_to_rgb8(&frame.nv12, frame.width, frame.height, &mut rgb_scratch); - let mut bgra = Vec::with_capacity((frame.width * frame.height) as usize); - for px in rgb_scratch.chunks_exact(3) { - bgra.push( - 0xff00_0000 - | ((px[0] as u32) << 16) - | ((px[1] as u32) << 8) - | px[2] as u32, - ); - } - shared.frames.lock().unwrap().push_back(Frame { - pts_100ns: frame.pts_100ns, - bgra, - }); - } - Ok(None) => { - // End of stream: drain the soundtrack tail, then PARK. The - // thread stays alive serving seeks — a loop restart or a - // scrub back into the clip is a ~10 ms decoder seek, not a - // teardown and reopen. - if info.has_audio && !audio_eos { - loop { - if shared.stop.load(Ordering::Relaxed) { - return; - } - match decoder.next_audio() { - Ok(Some(chunk)) => video_audio().lock().unwrap().push_i16( - epoch, - &chunk.samples, - chunk.channels, - chunk.sample_rate, - ), - Ok(None) => break, - Err(e) => { - log!("video: audio tail decode error: {}", e); - break; - } - } - } - audio_eos = true; - } - shared.eos.store(true, Ordering::Release); - while shared.eos.load(Ordering::Acquire) { - if shared.stop.load(Ordering::Relaxed) { - return; - } - if shared.seek_100ns.load(Ordering::Acquire) >= 0 { - break; // the top of the loop consumes it - } - std::thread::sleep(Duration::from_millis(10)); - } - continue; - } - Err(e) => { - log!("video: decode error: {}", e); - return; - } - } - } -} - -// --------------------------------------------------------------------------- -// Soundtrack queue (identical mixer shape to the wav player in audio.rs) -// --------------------------------------------------------------------------- - -impl VideoAudio { - /// Epoch-guarded queue drop for a seek: only the owning player's decode - /// thread may flush what it queued. - fn clear_for(&mut self, epoch: u64) { - if self.owner == epoch { - self.frames.clear(); - self.cursor = 0.0; - } - } -} - -pub struct VideoAudio { - frames: VecDeque<(f32, f32)>, - cursor: f64, - source_rate: f64, - /// Sticky mute raised by [`stop_audio`]: the decode thread may still be - /// refilling the queue, so a plain clear would go audible again ~a - /// second later. Cleared when the next clip starts. - muted: bool, - /// Epoch of the ONE player allowed to push (0 = none). Detached decode - /// threads of dropped players carry stale epochs and are locked out. - owner: u64, -} - -impl VideoAudio { - const fn new() -> Self { - Self { - frames: VecDeque::new(), - cursor: 0.0, - source_rate: 0.0, - muted: false, - owner: 0, - } - } - - fn clear(&mut self) { - self.frames.clear(); - self.cursor = 0.0; - } - - fn buffered_secs(&self) -> f64 { - if self.source_rate <= 0.0 { - return 0.0; - } - (self.frames.len() as f64 - self.cursor).max(0.0) / self.source_rate - } - - fn push_i16(&mut self, epoch: u64, samples: &[i16], channels: u16, rate: u32) { - if self.muted || self.owner != epoch { - return; - } - self.source_rate = rate as f64; - let ch = channels.max(1) as usize; - const GAIN: f32 = 0.9; - for frame in samples.chunks_exact(ch) { - let l = frame[0] as f32 / 32768.0 * GAIN; - let r = frame[ch - 1] as f32 / 32768.0 * GAIN; - self.frames.push_back((l, r)); - } - } - - fn mix(&mut self, output: &mut AudioBuffer, device_rate: f64) { - if self.frames.is_empty() || self.source_rate <= 0.0 || device_rate <= 0.0 { - return; - } - let step = self.source_rate / device_rate; - let channels = output.channel_count(); - for frame in 0..output.frame_count() { - let index = self.cursor as usize; - if index + 1 >= self.frames.len() { - break; - } - let fraction = (self.cursor - index as f64) as f32; - let (al, ar) = self.frames[index]; - let (bl, br) = self.frames[index + 1]; - let l = al + (bl - al) * fraction; - let r = ar + (br - ar) * fraction; - for channel in 0..channels { - let s = if channel == 0 { l } else { r }; - output.channel_mut(channel)[frame] += s; - } - self.cursor += step; - } - let consumed = self.cursor as usize; - if consumed > 0 { - self.frames.drain(..consumed.min(self.frames.len())); - self.cursor -= consumed as f64; - } - // A lone trailing frame can never be interpolated: once the queue is - // down to it the clip is over (or hard-underrun) — release it so the - // soundtrack ends instead of pinning the final sample forever. - if self.frames.len() <= 1 { - self.clear(); - } - } -} - -static VIDEO_AUDIO: Mutex = Mutex::new(VideoAudio::new()); - -fn video_audio() -> &'static Mutex { - &VIDEO_AUDIO -} - -/// Mix queued video audio into the device buffer (one line in the app's -/// `cx.audio_output` closure). -pub fn mix_into(output: &mut AudioBuffer, device_rate: f64) { - if let Ok(mut audio) = video_audio().lock() { - audio.mix(output, device_rate); - } -} - -/// Silences the video soundtrack immediately and keeps it silent (the -/// decode thread may still be refilling) until the next clip starts — -/// stop-button / tab-switch hook. Revoking ownership locks every live -/// decode thread out of the queue, not just muting it. -pub fn stop_audio() { - if let Ok(mut audio) = video_audio().lock() { - audio.clear(); - audio.muted = true; - audio.owner = 0; - } -} - -// --------------------------------------------------------------------------- - -#[cfg(test)] -mod tests { - use super::*; - - /// Both tests touch the process-global soundtrack queue. - static VIDEO_TEST_LOCK: Mutex<()> = Mutex::new(()); - - #[test] - fn at_end_needs_decode_exit_and_a_drained_ring() { - let _serial = VIDEO_TEST_LOCK.lock().unwrap(); - let shared = Arc::new(Shared { - frames: Mutex::new(VecDeque::new()), - stop: AtomicBool::new(false), - done: AtomicBool::new(false), - eos: AtomicBool::new(false), - seek_100ns: AtomicI64::new(-1), - }); - shared.frames.lock().unwrap().push_back(Frame { - pts_100ns: 0, - bgra: vec![0xff00_0000; 4], - }); - let mut player = VideoPlayer { - width: 2, - height: 2, - duration_100ns: 0, - shared: shared.clone(), - started: None, - last_pts: 0, - paused_at: None, - epoch: u64::MAX, - }; - // Still decoding: never EOS, with or without buffered frames. - assert!(!player.at_end()); - // Decode exited, but a due frame is still buffered: pump keeps going. - shared.done.store(true, Ordering::Release); - assert!(!player.at_end()); - // The pts-0 frame is immediately due; taking it drains the ring. - assert!(player.take_due_frame().is_some()); - assert!(player.at_end(), "decode done + drained ring is EOS"); - // A stopped-but-undrained ring is also not EOS until taken. - assert!(player.take_due_frame().is_none()); - } - - #[test] - fn soundtrack_queue_ignores_stale_epochs_and_revoked_ownership() { - let _serial = VIDEO_TEST_LOCK.lock().unwrap(); - { - let mut audio = video_audio().lock().unwrap(); - audio.clear(); - audio.muted = false; - audio.owner = 7; - audio.source_rate = 0.0; - } - // A dropped player's detached thread (stale epoch) cannot push. - video_audio() - .lock() - .unwrap() - .push_i16(6, &[1000, 1000], 2, 48_000); - assert_eq!(video_audio().lock().unwrap().frames.len(), 0); - // The owning epoch pushes fine. - video_audio() - .lock() - .unwrap() - .push_i16(7, &[1000, 1000, 2000, 2000], 2, 48_000); - assert_eq!(video_audio().lock().unwrap().frames.len(), 2); - // stop_audio revokes ownership: even the former owner is locked out. - stop_audio(); - video_audio() - .lock() - .unwrap() - .push_i16(7, &[1000, 1000], 2, 48_000); - let audio = video_audio().lock().unwrap(); - assert_eq!(audio.frames.len(), 0); - assert_eq!(audio.owner, 0); - assert!(audio.muted); - } -} diff --git a/apps/browser/Cargo.toml b/apps/browser/Cargo.toml new file mode 100644 index 000000000..356824fff --- /dev/null +++ b/apps/browser/Cargo.toml @@ -0,0 +1,29 @@ +# browser — a Chrome-like browser as a plain full-window Makepad app. +# +# Chromium (CEF, libs/cef) renders ONLY the page, GPU-accelerated straight +# into an IOSurface-backed Makepad texture. All browser chrome — the tab +# strip, the toolbar with back/forward/reload, the omnibox and the menu — +# is Makepad splash UI, styled from the wm theme when hosted by +# makepad-wm (`MAKEPAD_WM_THEME_SPLASH`) and from a Chrome-dark palette otherwise. +# +# Runs standalone, and unmodified inside makepad-wm / Studio tiles via the +# shared --stdin-loop client runtime every Makepad app has. + +[package] +name = "makepad-browser" +version = "0.1.0" +edition = "2021" +default-run = "browser" + +[[bin]] +name = "browser" +path = "src/main.rs" + +[dependencies] +makepad-widgets = { path = "../../widgets", features = ["cef"] } +makepad-cef = { path = "../../libs/cef" } +makepad-ai-services = { path = "../../libs/ai/services" } +makepad-strict-json = { path = "../../libs/strict_json" } +makepad-wm-theme = { path = "../../libs/wm_theme" } +# The window manager's vocabulary: the bar's title, the polite close. +makepad-wm-api = { path = "../../libs/wm_api" } diff --git a/apps/mpbrowser/resources/icons/back.svg b/apps/browser/resources/icons/back.svg similarity index 100% rename from apps/mpbrowser/resources/icons/back.svg rename to apps/browser/resources/icons/back.svg diff --git a/apps/mpbrowser/resources/icons/close.svg b/apps/browser/resources/icons/close.svg similarity index 100% rename from apps/mpbrowser/resources/icons/close.svg rename to apps/browser/resources/icons/close.svg diff --git a/apps/mpbrowser/resources/icons/forward.svg b/apps/browser/resources/icons/forward.svg similarity index 100% rename from apps/mpbrowser/resources/icons/forward.svg rename to apps/browser/resources/icons/forward.svg diff --git a/apps/mpbrowser/resources/icons/globe.svg b/apps/browser/resources/icons/globe.svg similarity index 100% rename from apps/mpbrowser/resources/icons/globe.svg rename to apps/browser/resources/icons/globe.svg diff --git a/apps/mpbrowser/resources/icons/menu.svg b/apps/browser/resources/icons/menu.svg similarity index 100% rename from apps/mpbrowser/resources/icons/menu.svg rename to apps/browser/resources/icons/menu.svg diff --git a/apps/mpbrowser/resources/icons/plus.svg b/apps/browser/resources/icons/plus.svg similarity index 100% rename from apps/mpbrowser/resources/icons/plus.svg rename to apps/browser/resources/icons/plus.svg diff --git a/apps/mpbrowser/resources/icons/reload.svg b/apps/browser/resources/icons/reload.svg similarity index 100% rename from apps/mpbrowser/resources/icons/reload.svg rename to apps/browser/resources/icons/reload.svg diff --git a/apps/mpbrowser/resources/icons/search.svg b/apps/browser/resources/icons/search.svg similarity index 100% rename from apps/mpbrowser/resources/icons/search.svg rename to apps/browser/resources/icons/search.svg diff --git a/apps/mpbrowser/resources/icons/star.svg b/apps/browser/resources/icons/star.svg similarity index 100% rename from apps/mpbrowser/resources/icons/star.svg rename to apps/browser/resources/icons/star.svg diff --git a/apps/browser/src/ai.rs b/apps/browser/src/ai.rs new file mode 100644 index 000000000..246e982c3 --- /dev/null +++ b/apps/browser/src/ai.rs @@ -0,0 +1,220 @@ +//! The browser on the desktop's AI bus. +//! +//! The current CEF wrapper mirrors navigation metadata but does not bind its +//! frame text/source callbacks, so `page` reports the active title and URL. + +use makepad_ai_services::wire::{Risk, ServiceCall, ServiceManifest, ToolDef, ToolResult}; +use makepad_strict_json::{self as json, Value}; + +pub struct PageState { + pub title: String, + pub url: String, +} + +pub struct TabState { + pub title: String, + pub url: String, + pub active: bool, +} + +/// The bus-facing subset of the webview. Keeping it as a trait makes the +/// closed dispatcher testable without starting CEF or a window. +pub trait BrowserTarget { + fn page(&self) -> Option; + fn tabs(&self) -> Vec; + fn navigate(&mut self, url: &str) -> bool; + fn new_tab(&mut self, url: &str); +} + +pub fn manifest() -> ServiceManifest { + ServiceManifest::new( + "browser", + "Browser", + "The live web browser. Its read tools report the active page and all tabs; its action tools steer the active tab or open a new one.", + ) + .with_tool(ToolDef::new( + "page", + "Read the active tab's displayed title and URL. The current CEF binding does not expose visible page text or source, so this tool cannot return page text.", + r#"{"type":"object","properties":{},"additionalProperties":false}"#, + Risk::Read, + )) + .with_tool(ToolDef::new( + "tabs", + "Read every open tab's displayed title and URL, with the active tab marked.", + r#"{"type":"object","properties":{},"additionalProperties":false}"#, + Risk::Read, + )) + .with_tool(ToolDef::new( + "navigate", + "Navigate the active tab to an http:// or https:// URL, or to about:blank.", + r#"{"type":"object","properties":{"url":{"type":"string"}},"required":["url"],"additionalProperties":false}"#, + Risk::Act, + )) + .with_tool(ToolDef::new( + "new_tab", + "Open and activate a new tab at an http:// or https:// URL, or at about:blank.", + r#"{"type":"object","properties":{"url":{"type":"string"}},"required":["url"],"additionalProperties":false}"#, + Risk::Act, + )) +} + +/// Answer one browser call through a closed match over the four advertised +/// names. URL actions use the webview's existing navigation methods. +pub fn answer(call: &ServiceCall, target: &mut impl BrowserTarget) -> ToolResult { + match call.tool.as_str() { + "page" => { + if let Err(error) = empty_args(&call.args) { + return ToolResult::refused(&call.call_id, error); + } + match target.page() { + Some(page) => ToolResult::ok( + &call.call_id, + format!("title: {}\nurl: {}", page.title, page.url), + format!("{} — {}", page.title, page.url), + ), + None => ToolResult::unavailable(&call.call_id, "there is no active browser tab"), + } + } + "tabs" => { + if let Err(error) = empty_args(&call.args) { + return ToolResult::refused(&call.call_id, error); + } + let tabs = target.tabs(); + let mut text = String::new(); + for (index, tab) in tabs.iter().enumerate() { + if index > 0 { + text.push('\n'); + } + text.push_str(if tab.active { "[active] " } else { " " }); + text.push_str(&tab.title); + text.push_str(" — "); + text.push_str(&tab.url); + } + if text.is_empty() { + text.push_str("no tabs open"); + } + ToolResult::ok(&call.call_id, text, format!("{} tabs", tabs.len())) + } + "navigate" => { + let url = match url_arg(&call.args, "navigate") { + Ok(url) => url, + Err(error) => return ToolResult::refused(&call.call_id, error), + }; + if target.navigate(&url) { + ToolResult::ok(&call.call_id, format!("navigating to {url}"), "navigating") + } else { + ToolResult::unavailable(&call.call_id, "there is no active browser tab") + } + } + "new_tab" => { + let url = match url_arg(&call.args, "new_tab") { + Ok(url) => url, + Err(error) => return ToolResult::refused(&call.call_id, error), + }; + target.new_tab(&url); + ToolResult::ok(&call.call_id, format!("opened a new tab at {url}"), "navigating") + } + other => ToolResult::refused( + &call.call_id, + format!("browser has no tool `{other}`; it has page, tabs, navigate, new_tab"), + ), + } +} + +fn empty_args(args: &str) -> Result<(), String> { + let fields = object_args(args)?; + if let Some((key, _)) = fields.first() { + return Err(format!("unknown argument `{key}`")); + } + Ok(()) +} + +fn url_arg(args: &str, tool: &str) -> Result { + let fields = object_args(args)?; + if let Some((key, _)) = fields.iter().find(|(key, _)| key != "url") { + return Err(format!("unknown argument `{key}`")); + } + let Some(url) = fields + .iter() + .find(|(key, _)| key == "url") + .and_then(|(_, value)| value.as_str()) + else { + return Err(format!("{tool}.url must be a string")); + }; + if !allowed_url(url) { + return Err(format!( + "{tool}.url must start with http:// or https://, or be exactly about:blank" + )); + } + Ok(url.to_string()) +} + +fn object_args(args: &str) -> Result, String> { + match json::parse(args.as_bytes()) { + Ok(Value::Obj(fields)) => Ok(fields), + Ok(_) => Err("tool arguments must be a JSON object".to_string()), + Err(error) => Err(format!("invalid tool arguments: {error}")), + } +} + +fn allowed_url(url: &str) -> bool { + if url == "about:blank" { + return true; + } + if url.chars().any(char::is_whitespace) || url.chars().any(char::is_control) { + return false; + } + url.strip_prefix("http://") + .or_else(|| url.strip_prefix("https://")) + .is_some_and(|rest| !rest.is_empty()) +} + +#[cfg(test)] +mod tests { + use super::*; + use makepad_ai_services::wire::ToolOutcome; + + #[derive(Default)] + struct FakeTarget { + navigated: Vec, + } + + impl BrowserTarget for FakeTarget { + fn page(&self) -> Option { + None + } + + fn tabs(&self) -> Vec { + Vec::new() + } + + fn navigate(&mut self, url: &str) -> bool { + self.navigated.push(url.to_string()); + true + } + + fn new_tab(&mut self, url: &str) { + self.navigated.push(url.to_string()); + } + } + + #[test] + fn browser_manifest_validates() { + manifest().validate().expect("a valid browser manifest"); + } + + #[test] + fn navigate_refuses_non_http_urls() { + let mut target = FakeTarget::default(); + for url in ["file:///etc/passwd", "javascript:alert(1)", "data:text/plain,no"] { + let call = ServiceCall { + call_id: "c1".into(), + tool: "navigate".into(), + args: format!(r#"{{"url":"{url}"}}"#), + }; + let result = answer(&call, &mut target); + assert_eq!(result.outcome, ToolOutcome::Refused); + } + assert!(target.navigated.is_empty()); + } +} diff --git a/apps/mpbrowser/src/chrome.rs b/apps/browser/src/chrome.rs similarity index 90% rename from apps/mpbrowser/src/chrome.rs rename to apps/browser/src/chrome.rs index b457defc0..79c4d6182 100644 --- a/apps/mpbrowser/src/chrome.rs +++ b/apps/browser/src/chrome.rs @@ -1,7 +1,7 @@ //! The browser chrome: a custom-drawn Chrome-style tab strip (favicon + //! title tabs with a close x, a + for a new tab) and the toolbar //! (back / forward / reload, the omnibox with search icon and bookmark star, -//! the menu button). Hard-square Omarchy look, colours from `mod.mpb_theme`. +//! the menu button). Hard-square Omarchy look, colours from `mod.browser_theme`. use crate::tabs::{TabId, TabSummary}; use makepad_widgets::image::DrawImage; @@ -18,62 +18,62 @@ script_mod! { height: 36 draw_bg +: { - color: uniform(mod.mpb_theme.darker_background) + color: uniform(mod.browser_theme.darker_background) pixel: fn() { return self.color } } draw_tab +: { - color: uniform(mod.mpb_theme.darker_background) + color: uniform(mod.browser_theme.darker_background) pixel: fn() { return self.color } } draw_tab_hover +: { - color: uniform(mod.mpb_theme.dark_background) + color: uniform(mod.browser_theme.dark_background) pixel: fn() { return self.color } } draw_tab_active +: { - color: uniform(mod.mpb_theme.background) + color: uniform(mod.browser_theme.background) pixel: fn() { return self.color } } draw_sep +: { - color: uniform(mod.mpb_theme.muted) + color: uniform(mod.browser_theme.muted) pixel: fn() { return self.color } } draw_text +: { - color: mod.mpb_theme.foreground + color: mod.browser_theme.foreground text_style: theme.font_regular{ font_size: 9.5 } } draw_text_dim +: { - color: mod.mpb_theme.dark_foreground + color: mod.browser_theme.dark_foreground text_style: theme.font_regular{ font_size: 9.5 } } draw_close +: { svg: crate_resource("self:resources/icons/close.svg") - color: mod.mpb_theme.dark_foreground + color: mod.browser_theme.dark_foreground } draw_close_active +: { svg: crate_resource("self:resources/icons/close.svg") - color: mod.mpb_theme.foreground + color: mod.browser_theme.foreground } draw_plus +: { svg: crate_resource("self:resources/icons/plus.svg") - color: mod.mpb_theme.foreground + color: mod.browser_theme.foreground } draw_globe +: { svg: crate_resource("self:resources/icons/globe.svg") - color: mod.mpb_theme.dark_foreground + color: mod.browser_theme.dark_foreground } } @@ -87,14 +87,14 @@ script_mod! { align: Align{x: 0.5 y: 0.5} icon_walk: Walk{width: 16 height: 16} draw_icon +: { - color: mod.mpb_theme.foreground + color: mod.browser_theme.foreground } draw_bg +: { border_radius: 0.0 border_size: 0.0 color: #00000000 - color_hover: mod.mpb_theme.lighter_background - color_down: mod.mpb_theme.muted + color_hover: mod.browser_theme.lighter_background + color_down: mod.browser_theme.muted color_focus: #00000000 } } @@ -109,8 +109,8 @@ script_mod! { padding: Inset{left: 14 right: 14 top: 0 bottom: 0} margin: Inset{left: 0 right: 0 top: 0 bottom: 0} draw_text +: { - color: mod.mpb_theme.foreground - color_hover: mod.mpb_theme.bright_foreground + color: mod.browser_theme.foreground + color_hover: mod.browser_theme.bright_foreground text_style: theme.font_regular{ font_size: 10 } @@ -119,8 +119,8 @@ script_mod! { border_radius: 0.0 border_size: 0.0 color: #00000000 - color_hover: mod.mpb_theme.lighter_background - color_down: mod.mpb_theme.muted + color_hover: mod.browser_theme.lighter_background + color_down: mod.browser_theme.muted } } @@ -137,7 +137,7 @@ script_mod! { padding: Inset{left: 6 right: 6 top: 4 bottom: 4} spacing: 2 draw_bg +: { - color: mod.mpb_theme.background + color: mod.browser_theme.background } back_btn := MpToolButton{ @@ -162,14 +162,14 @@ script_mod! { padding: Inset{left: 10 right: 2 top: 0 bottom: 0} spacing: 6 draw_bg +: { - color: mod.mpb_theme.darker_background + color: mod.browser_theme.darker_background } Icon{ margin: Inset{top: 9 bottom: 0 left: 0 right: 0} icon_walk: Walk{width: 14 height: 14} draw_icon +: { svg: crate_resource("self:resources/icons/search.svg") - color: mod.mpb_theme.dark_foreground + color: mod.browser_theme.dark_foreground } } omnibox := TextInputFlat{ @@ -197,14 +197,14 @@ script_mod! { border_color_empty: #00000000 } draw_cursor +: { - color: mod.mpb_theme.bright_foreground + color: mod.browser_theme.bright_foreground } draw_text +: { - color: mod.mpb_theme.foreground - color_hover: mod.mpb_theme.foreground - color_focus: mod.mpb_theme.bright_foreground - color_empty: mod.mpb_theme.dark_foreground - color_empty_hover: mod.mpb_theme.dark_foreground + color: mod.browser_theme.foreground + color_hover: mod.browser_theme.foreground + color_focus: mod.browser_theme.bright_foreground + color_empty: mod.browser_theme.dark_foreground + color_empty_hover: mod.browser_theme.dark_foreground text_style: theme.font_regular{ font_size: 10.5 } @@ -217,7 +217,7 @@ script_mod! { icon_walk: Walk{width: 15 height: 15} draw_icon +: { svg: crate_resource("self:resources/icons/star.svg") - color: mod.mpb_theme.dark_foreground + color: mod.browser_theme.dark_foreground } } } diff --git a/apps/mpbrowser/src/main.rs b/apps/browser/src/main.rs similarity index 80% rename from apps/mpbrowser/src/main.rs rename to apps/browser/src/main.rs index 1b587687f..4f9f01bc6 100644 --- a/apps/mpbrowser/src/main.rs +++ b/apps/browser/src/main.rs @@ -1,12 +1,14 @@ -//! mpbrowser: a Chrome-like browser as a plain full-window Makepad app. +//! browser: a Chrome-like browser as a plain full-window Makepad app. //! CEF renders the page (GPU-accelerated into a shared IOSurface texture); //! every bit of chrome is Makepad. Runs standalone or inside makepad-wm / //! Studio tiles via the shared --stdin-loop client runtime. pub use makepad_widgets; +use makepad_ai_services::port::{AiServicePort, PortEvent}; use makepad_cef::BootstrapResult; use makepad_widgets::*; +mod ai; mod chrome; mod tabs; mod theme; @@ -27,7 +29,7 @@ script_mod! { ui: Root{ main_window := Window{ window.inner_size: vec2(1280, 860) - window.title: "mpbrowser" + window.title: "browser" body +: { flow: Overlay View{ @@ -52,14 +54,14 @@ script_mod! { flow: Down padding: Inset{top: 4 bottom: 4 left: 0 right: 0} draw_bg +: { - color: mod.mpb_theme.background + color: mod.browser_theme.background } menu_new_tab := MpMenuItem{text: "New tab"} menu_close_tab := MpMenuItem{text: "Close tab"} menu_reload := MpMenuItem{text: "Reload"} Hr{} menu_gpu := MpMenuItem{text: "chrome://gpu"} - menu_about := MpMenuItem{text: "About mpbrowser"} + menu_about := MpMenuItem{text: "About browser"} } } } @@ -155,8 +157,46 @@ pub struct App { focus_frame: NextFrame, #[rust] focus_retries: u32, + #[rust] + ai_port: Option, + #[rust] + ai_context: String, } +struct AiBrowserTarget<'a> { + cx: &'a mut Cx, + webview: &'a mut WebView, +} + +impl ai::BrowserTarget for AiBrowserTarget<'_> { + fn page(&self) -> Option { + let info = self.webview.active_info(); + info.id.map(|_| ai::PageState { + title: info.title, + url: info.url, + }) + } + + fn tabs(&self) -> Vec { + self.webview + .ai_tabs() + .into_iter() + .map(|(title, url, active)| ai::TabState { title, url, active }) + .collect() + } + + fn navigate(&mut self, url: &str) -> bool { + if self.webview.active_id().is_none() { + return false; + } + self.webview.navigate(self.cx, url); + true + } + + fn new_tab(&mut self, url: &str) { + self.webview.new_tab(self.cx, url, true); + } +} impl App { fn with_webview(&self, cx: &mut Cx, f: impl FnOnce(&mut Cx, &mut WebView) -> R) -> Option { @@ -221,6 +261,8 @@ impl App { .with_webview(cx, |_cx, wv| (wv.summaries(), wv.active_info())) .unwrap_or_default(); + self.refresh_ai_context(&info); + if let Some(mut strip) = self.ui.widget(cx, ids!(tab_strip)).borrow_mut::() { strip.set_tabs(cx, summaries); } @@ -260,16 +302,18 @@ impl App { }); let title = if info.title.is_empty() { - "mpbrowser".to_string() + "browser".to_string() } else { - format!("{} — mpbrowser", info.title) + format!("{} — browser", info.title) }; self.ui.window(cx, ids!(main_window)).set_title(cx, &title); + // The bar shows the page title too when the window manager hosts us. + makepad_wm_api::set_title(cx, &title); if info.render_mode != self.reported_mode && info.render_mode != "None" { self.reported_mode = info.render_mode.clone(); log!( - "mpbrowser: page rendering is {} (accelerated frames so far: {}, last blit {}us)", + "browser: page rendering is {} (accelerated frames so far: {}, last blit {}us)", info.render_mode, info.accelerated_frames, info.last_blit_micros @@ -278,6 +322,63 @@ impl App { self.ui.redraw(cx); } + fn refresh_ai_context(&mut self, info: &webview::ActiveInfo) { + let context = if info.id.is_some() { + format!("active tab: {} — {}", info.title, info.url) + } else { + "no active tab".to_string() + }; + if context == self.ai_context { + return; + } + self.ai_context = context.clone(); + if let Some(port) = self.ai_port.as_ref() { + port.set_context(&context); + } + } + + fn refresh_ai_context_from_webview(&mut self, cx: &mut Cx) { + let info = self + .with_webview(cx, |_cx, webview| webview.active_info()) + .unwrap_or_default(); + self.refresh_ai_context(&info); + } + + fn drain_ai_port(&mut self, cx: &mut Cx, event: &Event) { + let events = match self.ai_port.as_mut() { + Some(port) => port.handle_event(cx, event), + None => return, + }; + for event in events { + match event { + PortEvent::Registered(endpoint) => { + log!("browser: AI service registered as {}", endpoint.as_str()); + self.ai_context.clear(); + self.refresh_ai_context_from_webview(cx); + } + PortEvent::Call(call) => { + let result = self + .with_webview(cx, |cx, webview| { + let mut target = AiBrowserTarget { cx, webview }; + ai::answer(&call, &mut target) + }) + .unwrap_or_else(|| { + makepad_ai_services::wire::ToolResult::unavailable( + &call.call_id, + "the browser view is not ready", + ) + }); + if let Some(port) = self.ai_port.as_ref() { + port.reply(result); + } + self.refresh_chrome(cx); + } + PortEvent::Cancel { .. } | PortEvent::ChatOpen { .. } => {} + PortEvent::Subscribe { .. } | PortEvent::Unsubscribe { .. } => {} + } + } + } + fn navigate_from_omnibox(&mut self, cx: &mut Cx, text: &str) { let url = tabs::resolve_omnibox(text); if url.is_empty() { @@ -361,14 +462,20 @@ impl App { impl MatchEvent for App { fn handle_startup(&mut self, cx: &mut Cx) { + // A warm-pool standby is not a running Browser: its service opens on + // `WmEvent::Adopted`, never while dormant — the assistant must not + // steer a page nobody can see. + if !makepad_wm_api::warm_start() { + self.ai_port = AiServicePort::open(cx, ai::manifest()); + } match makepad_cef::startup_phases() { Some((bundle_ms, exec_gap_ms)) => log!( - "mpbrowser: window up at {} ms after main (app bundle prepared in {} ms, exec-to-main gap {} ms)", + "browser: window up at {} ms after main (app bundle prepared in {} ms, exec-to-main gap {} ms)", uptime_ms(), bundle_ms, exec_gap_ms ), - None => log!("mpbrowser: window up at {} ms after main", uptime_ms()), + None => log!("browser: window up at {} ms after main", uptime_ms()), } let urls = initial_urls(); if urls.is_empty() { @@ -463,8 +570,8 @@ impl MatchEvent for App { .with_webview(cx, |_cx, wv| wv.active_info()) .unwrap_or_default(); let html = format!( - "About mpbrowser\ -

mpbrowser

\ + "About browser\ +

browser

\

Makepad chrome, Chromium Embedded Framework {cef} page rendering.

\

Page rendering path: {mode} (accelerated frames: {frames}, last GPU blit: {blit}µs)

\

ANGLE backend: {angle}

", @@ -527,8 +634,8 @@ impl AppMain for App { fn script_mod(vm: &mut ScriptVm) -> ScriptValue { crate::makepad_widgets::script_mod(vm); // The family theme bridge retints the stock widgets from the WM - // theme; the chrome roles go into mod.mpb_theme. - mp_theme::apply(vm); + // theme; the chrome roles go into mod.browser_theme. + makepad_wm_theme::apply(vm); palette().apply(vm); chrome::script_mod(vm); webview::script_mod(vm); @@ -536,6 +643,22 @@ impl AppMain for App { } fn handle_event(&mut self, cx: &mut Cx, event: &Event) { + // The window manager asked politely (SUPER+W): go now, ahead of the + // kill that follows its grace. A warm-pool browser needs no waking — + // CEF paints nothing until a tile presents it. + if let Event::Custom(json) = event { + match makepad_wm_api::WmEvent::parse(json) { + Some(makepad_wm_api::WmEvent::Adopted) if self.ai_port.is_none() => { + self.ai_port = AiServicePort::open(cx, ai::manifest()); + } + _ => {} + } + if let Some(makepad_wm_api::WmEvent::CloseRequested) = makepad_wm_api::WmEvent::parse(json) { + cx.quit(); + return; + } + } + self.drain_ai_port(cx, event); if let Event::KeyDown(ke) = event { if self.handle_shortcut(cx, ke) { return; @@ -622,5 +745,5 @@ pub fn app_main() { #[cfg(any(target_arch = "wasm32", target_os = "android", target_env = "ohos"))] pub fn app_main() { - panic!("mpbrowser is desktop-only"); + panic!("browser is desktop-only"); } diff --git a/apps/mpbrowser/src/tabs.rs b/apps/browser/src/tabs.rs similarity index 100% rename from apps/mpbrowser/src/tabs.rs rename to apps/browser/src/tabs.rs diff --git a/apps/mpbrowser/src/theme.rs b/apps/browser/src/theme.rs similarity index 90% rename from apps/mpbrowser/src/theme.rs rename to apps/browser/src/theme.rs index 1483559bd..e6389a9d5 100644 --- a/apps/mpbrowser/src/theme.rs +++ b/apps/browser/src/theme.rs @@ -1,14 +1,14 @@ //! The browser-chrome palette. Theming lives in splash: the chrome reads -//! `mod.mpb_theme.*` (tab strip, toolbar, omnibox, icon roles), which this +//! `mod.browser_theme.*` (tab strip, toolbar, omnibox, icon roles), which this //! module evaluates into the VM before the UI modules. //! //! Under makepad-wm the roles come from the WM's theme.splash -//! (`MPWM_THEME_SPLASH`, line-scanned by `mp_theme`, the family bridge); +//! (`MAKEPAD_WM_THEME_SPLASH`, line-scanned by `makepad_wm_theme`, the family bridge); //! standalone runs get Chrome's own dark palette. use makepad_widgets::*; -/// Chrome-dark roles, keyed like the mpwm theme so one mapping serves both. +/// Chrome-dark roles, keyed like the wm theme so one mapping serves both. #[derive(Clone, Debug)] pub struct Palette { /// Tab strip background (the "frame"). @@ -43,10 +43,10 @@ impl Palette { } } - /// The WM palette when mpwm exported one, else Chrome dark. + /// The WM palette when wm exported one, else Chrome dark. pub fn current() -> Self { let fallback = Self::chrome_dark(); - let Some(p) = mp_theme::current() else { + let Some(p) = makepad_wm_theme::current() else { return fallback; }; Self { @@ -63,11 +63,11 @@ impl Palette { } } - /// The `mod.mpb_theme = {...}` splash source. Runtime-evaluated, so plain + /// The `mod.browser_theme = {...}` splash source. Runtime-evaluated, so plain /// `#hex` (the `#x` escape is a proc-macro-only hazard). pub fn splash_source(&self) -> String { format!( - "mod.mpb_theme = {{\n\ + "mod.browser_theme = {{\n\ \x20 darker_background: {}\n\ \x20 background: {}\n\ \x20 dark_background: {}\n\ @@ -93,13 +93,13 @@ impl Palette { ) } - /// Evaluate `mod.mpb_theme` into the VM. Call after + /// Evaluate `mod.browser_theme` into the VM. Call after /// `makepad_widgets::script_mod(vm)` and before the chrome modules. pub fn apply(&self, vm: &mut ScriptVm) { let script_mod_id = ScriptMod { cargo_manifest_path: env!("CARGO_MANIFEST_DIR").to_string(), - module_path: "mpb_theme".to_string(), - file: "mpb_theme.splash".to_string(), + module_path: "browser_theme".to_string(), + file: "browser_theme.splash".to_string(), line: 0, column: 0, code: self.splash_source(), @@ -107,7 +107,7 @@ impl Palette { }; vm.eval(script_mod_id); for e in vm.take_errors() { - log!("mpbrowser theme: {}", e); + log!("browser theme: {}", e); } } @@ -129,7 +129,7 @@ impl Palette { .c{{display:flex;height:100%;align-items:center;justify-content:center;\ flex-direction:column;gap:10px}}.n{{font-size:28px;letter-spacing:1px;color:{fgb}}}\ .s{{color:{fgd}}}
\ -
mpbrowser
Type a URL or search in the box above
\ +
browser
Type a URL or search in the box above
\
", bg = self.darker_background, fg = self.foreground, diff --git a/apps/mpbrowser/src/webview.rs b/apps/browser/src/webview.rs similarity index 97% rename from apps/mpbrowser/src/webview.rs rename to apps/browser/src/webview.rs index eff0b6166..fa79f1ad2 100644 --- a/apps/mpbrowser/src/webview.rs +++ b/apps/browser/src/webview.rs @@ -11,9 +11,9 @@ use crate::tabs::{TabId, TabModel, TabSummary}; // The surface policy is the stock Browser widget's — one source of truth for // how a CEF page survives a resize. -use makepad_widgets::browser::{ - needs_new_surface, surface_alloc, Browser as BrowserKeys, RESIZE_INTERVAL, SETTLE, -}; +#[cfg(target_os = "macos")] +use makepad_widgets::browser::{needs_new_surface, surface_alloc, SETTLE}; +use makepad_widgets::browser::{Browser as BrowserKeys, RESIZE_INTERVAL}; use makepad_widgets::image::DrawImage; use makepad_widgets::*; @@ -27,13 +27,13 @@ script_mod! { width: Fill height: Fill draw_empty +: { - color: uniform(mod.mpb_theme.darker_background) + color: uniform(mod.browser_theme.darker_background) pixel: fn() { return self.color } } draw_status +: { - color: mod.mpb_theme.dark_foreground + color: mod.browser_theme.dark_foreground text_style: theme.font_regular{ font_size: 11 } @@ -102,11 +102,11 @@ pub struct WebView { pump_started: bool, } -/// Env-gated resize tracing (`MPB_TRACE=1`): timestamps + sizes on every +/// Env-gated resize tracing (`MAKEPAD_BROWSER_TRACE=1`): timestamps + sizes on every /// draw, resize and target swap. Debug rig — not for committing. pub fn trace_on() -> bool { static ON: std::sync::OnceLock = std::sync::OnceLock::new(); - *ON.get_or_init(|| std::env::var_os("MPB_TRACE").is_some()) + *ON.get_or_init(|| std::env::var_os("MAKEPAD_BROWSER_TRACE").is_some()) } macro_rules! trace { @@ -177,6 +177,18 @@ impl WebView { self.tabs.summaries() } + /// Title, URL and active marker for the browser's AI `tabs` tool. + pub fn ai_tabs(&self) -> Vec<(String, String, bool)> { + self.tabs + .tabs + .iter() + .enumerate() + .map(|(index, tab)| { + (tab.display_title(), tab.url.clone(), index == self.tabs.active) + }) + .collect() + } + pub fn active_info(&self) -> ActiveInfo { let Some(tab) = self.tabs.active() else { return ActiveInfo::default(); @@ -520,7 +532,7 @@ impl WebView { } if !self.first_frame_logged { self.first_frame_logged = true; - log!("mpbrowser: first page frame at {} ms", crate::uptime_ms()); + log!("browser: first page frame at {} ms", crate::uptime_ms()); } } let generation = browser.nav_generation(); @@ -705,7 +717,7 @@ impl Widget for WebView { Ok(()) => { self.cef_ready = true; log!( - "mpbrowser: CEF {} initialized at {} ms", + "browser: CEF {} initialized at {} ms", makepad_cef::CEF_VERSION, crate::uptime_ms() ); diff --git a/apps/fab/Cargo.toml b/apps/fab/Cargo.toml index e2c28f59a..b186378f0 100644 --- a/apps/fab/Cargo.toml +++ b/apps/fab/Cargo.toml @@ -18,7 +18,7 @@ gltf = ["dep:makepad-fab-loader-gltf"] makepad-widgets = { path = "../../widgets", version = "2.0.0" } makepad-fab-shell = { path = "../../libs/fab", version = "0.1.0" } makepad-fab-loader-gltf = { path = "loaders/gltf", version = "0.1.0", optional = true } -makepad-video = { path = "../../platform/video", version = "1.0.0" } +makepad-video = { package = "makepad-platform-video", path = "../../platform/video", version = "1.0.0" } makepad-zune-png = { path = "../../libs/zune/zune-png", version = "0.5.2" } [dev-dependencies] diff --git a/apps/fab/src/bin/frames_to_mp4.rs b/apps/fab/src/bin/frames_to_mp4.rs index 4685acbef..3739b386f 100644 --- a/apps/fab/src/bin/frames_to_mp4.rs +++ b/apps/fab/src/bin/frames_to_mp4.rs @@ -1,5 +1,5 @@ //! Encode a directory of `frame_%06d.png` files into an H.264 mp4 via the -//! platform hardware encoder (`makepad-video` / VideoToolbox on macOS). +//! platform hardware encoder (`makepad-platform-video` / VideoToolbox on macOS). //! //! ```text //! frames_to_mp4 [--fps 24] [--start N] [--end M] [--bitrate BPS] [--crf N] diff --git a/apps/fabric/Cargo.toml b/apps/fabric/Cargo.toml new file mode 100644 index 000000000..5eae91e76 --- /dev/null +++ b/apps/fabric/Cargo.toml @@ -0,0 +1,18 @@ +[package] +name = "makepad-fabric" +version = "0.1.0" +edition = "2021" +license = "MIT OR Apache-2.0" +description = "Fabric: a photo in, a fitted sewing pattern out — body model, measurements, draft, cut sheet" + +[dependencies] +# The window manager's vocabulary: the bar's title and the polite close. +makepad-wm-api = { path = "../../libs/wm_api" } +makepad-widgets = { path = "../../widgets" } +makepad-fabric-measure = { path = "../../libs/fabric/measure" } +makepad-fabric-draft = { path = "../../libs/fabric/draft" } +# The body model runs in-process (Metal on the Mac, CUDA on a box); the +# hub does install, licence acknowledgement and weight location. +makepad-ai-body = { path = "../../libs/ai/models/body" } +makepad-ai-hub = { path = "../../libs/ai/hub", default-features = false, features = ["local", "body-native"] } +makepad-ai-hub-ui = { path = "../../libs/ai/hub_ui" } diff --git a/apps/fabric/src/body_view.rs b/apps/fabric/src/body_view.rs new file mode 100644 index 000000000..8c11abc15 --- /dev/null +++ b/apps/fabric/src/body_view.rs @@ -0,0 +1,539 @@ +use makepad_fabric_measure::{BodyMesh, Line, Measured, Ring}; +use makepad_widgets::*; +use std::sync::Arc; + +script_mod! { + use mod.prelude.widgets_internal.* + use mod.widgets.* + + set_type_default() do #(DrawBodyPoint::script_shader(vm)) { + ..mod.draw.DrawQuad + pixel: fn() { + let d = length(self.pos - vec2(0.5, 0.5)) + let a = clamp((0.5 - d) * 7.0, 0.0, 1.0) + let near = #x80e7ff + let far = #x27435d + let c = far.mix(near, 1.0 - self.depth) + return vec4(c.xyz * a, a) + } + } + + set_type_default() do #(DrawFabricLine::script_shader(vm)) { + ..mod.draw.DrawQuad + pixel: fn() { + // A line as a distance field inside its bounding quad. The + // endpoints are LOCAL to the quad (the turtle may still shift + // rect_pos after the instance is written), like the chart's + // segment shader. + let p = self.pos * self.rect_size + let ab = self.p1 - self.p0 + let t = clamp(dot(p - self.p0, ab) / max(dot(ab, ab), 0.0001), 0.0, 1.0) + let d = length(p - (self.p0 + ab * t)) + let aa = 1.0 - smoothstep(self.half_width - 0.6, self.half_width + 0.6, d) + let alpha = aa * self.color.w + return vec4(self.color.xyz * alpha, alpha) + } + } + + mod.widgets.FabricBodyViewBase = #(FabricBodyView::register_widget(vm)) + mod.widgets.FabricBodyView = set_type_default() do mod.widgets.FabricBodyViewBase { + width: Fill + height: Fill + draw_bg +: {color: #x11161d} + draw_point +: {} + draw_line +: {} + draw_text +: { + color: #x9aa8b7 + text_style: theme.font_regular{font_size: 9.0} + } + } +} + +#[derive(Script, ScriptHook)] +#[repr(C)] +pub struct DrawBodyPoint { + #[deref] + draw_super: DrawQuad, + #[live] + depth: f32, +} + +#[derive(Script, ScriptHook)] +#[repr(C)] +pub struct DrawFabricLine { + #[deref] + draw_super: DrawQuad, + #[live] + pub color: Vec4f, + #[live] + p0: Vec2f, + #[live] + p1: Vec2f, + #[live] + half_width: f32, +} + +impl DrawFabricLine { + pub fn segment(&mut self, cx: &mut Cx2d, from: DVec2, to: DVec2, width: f64) { + if (to - from).length() < 0.01 { + return; + } + let half = width * 0.5; + let pad = half + 1.0; + let min = dvec2(from.x.min(to.x) - pad, from.y.min(to.y) - pad); + let max = dvec2(from.x.max(to.x) + pad, from.y.max(to.y) + pad); + self.p0 = v2f(from - min); + self.p1 = v2f(to - min); + self.half_width = half as f32; + self.draw_abs(cx, Rect { pos: min, size: max - min }); + } +} + +fn v2f(value: DVec2) -> Vec2f { + Vec2f { + x: value.x as f32, + y: value.y as f32, + } +} + +#[derive(Clone, Copy)] +struct BodyDrag { + from: DVec2, + yaw: f64, + pitch: f64, + pan: DVec2, + panning: bool, +} + +#[derive(Clone, Debug, Default)] +pub(crate) struct BodyPoseMapping { + ring_vertices: Vec>, + line_vertices: Vec<[usize; 2]>, +} + +pub(crate) fn map_measurements_to_vertices( + mesh: &BodyMesh, + measured: &Measured, +) -> BodyPoseMapping { + let nearest = |point| nearest_vertex_index(&mesh.vertices, measured.scale, point); + BodyPoseMapping { + ring_vertices: measured + .rings + .iter() + .map(|ring| ring.points.iter().copied().map(nearest).collect()) + .collect(), + line_vertices: measured + .lines + .iter() + .map(|line| [nearest(line.from), nearest(line.to)]) + .collect(), + } +} + +fn nearest_vertex_index(vertices: &[[f32; 3]], scale: f32, point: [f32; 3]) -> usize { + vertices + .iter() + .enumerate() + .map(|(index, vertex)| { + let dx = vertex[0] * scale - point[0]; + let dy = vertex[1] * scale - point[1]; + let dz = vertex[2] * scale - point[2]; + (index, dx * dx + dy * dy + dz * dz) + }) + .min_by(|left, right| left.1.total_cmp(&right.1)) + .map(|(index, _)| index) + .unwrap_or(0) +} + +fn mirror_x(value: f64, mirrored: bool) -> f64 { + if mirrored { -value } else { value } +} + +fn bounds(points: &[[f32; 3]], scale: f32) -> Option<([f32; 3], f32)> { + let mut min = [f32::INFINITY; 3]; + let mut max = [f32::NEG_INFINITY; 3]; + for point in points { + for axis in 0..3 { + let value = point[axis] * scale; + min[axis] = min[axis].min(value); + max[axis] = max[axis].max(value); + } + } + if !min[0].is_finite() { + return None; + } + let centre = [ + (min[0] + max[0]) * 0.5, + (min[1] + max[1]) * 0.5, + (min[2] + max[2]) * 0.5, + ]; + let dx = max[0] - min[0]; + let dy = max[1] - min[1]; + let dz = max[2] - min[2]; + let radius = (dx * dx + dy * dy + dz * dz).sqrt().max(1.0) * 0.5; + Some((centre, radius)) +} + +#[derive(Script, ScriptHook, Widget)] +pub struct FabricBodyView { + #[uid] + uid: WidgetUid, + #[source] + source: ScriptObjectRef, + #[walk] + walk: Walk, + #[layout] + layout: Layout, + #[redraw] + #[area] + area: Area, + #[live] + draw_bg: DrawColor, + #[live] + draw_point: DrawBodyPoint, + #[live] + draw_line: DrawFabricLine, + #[live] + draw_text: DrawText, + #[rust] + mesh: Option>, + #[rust] + posed: Option>>, + #[rust] + rings: Vec, + #[rust] + lines: Vec, + #[rust] + pose_mapping: BodyPoseMapping, + #[rust] + mesh_scale: f32, + #[rust] + centre: [f32; 3], + #[rust] + radius: f32, + #[rust(0.35)] + yaw: f64, + #[rust(-0.06)] + pitch: f64, + #[rust(1.0)] + zoom: f64, + #[rust] + pan: DVec2, + #[rust] + drag: Option, + #[rust(true)] + mirrored: bool, +} + +impl FabricBodyView { + pub fn set_body( + &mut self, + cx: &mut Cx, + mesh: Arc, + measured: &Measured, + pose_mapping: BodyPoseMapping, + ) { + self.mesh_scale = measured.scale; + self.rings = measured.rings.clone(); + self.lines = measured.lines.clone(); + self.pose_mapping = pose_mapping; + if self.posed.is_none() { + self.fit_bounds(&mesh.vertices); + self.reset_camera(); + } + self.mesh = Some(mesh); + self.redraw(cx); + } + + pub fn set_pose(&mut self, cx: &mut Cx, posed: Option>>) { + match posed { + Some(posed) + if self + .mesh + .as_ref() + .is_some_and(|mesh| mesh.vertices.len() == posed.len()) => + { + if self.posed.is_none() { + self.fit_bounds(posed.as_slice()); + self.reset_camera(); + } + self.posed = Some(posed); + } + _ => { + self.posed = None; + if let Some(fit) = self + .mesh + .as_ref() + .and_then(|mesh| bounds(&mesh.vertices, self.mesh_scale)) + { + (self.centre, self.radius) = fit; + self.reset_camera(); + } + } + } + self.redraw(cx); + } + + pub fn set_mirrored(&mut self, cx: &mut Cx, mirrored: bool) { + self.mirrored = mirrored; + self.redraw(cx); + } + + fn fit_bounds(&mut self, points: &[[f32; 3]]) { + if let Some((centre, radius)) = bounds(points, self.mesh_scale) { + self.centre = centre; + self.radius = radius; + } + } + + fn reset_camera(&mut self) { + self.yaw = 0.35; + self.pitch = -0.06; + self.zoom = 1.0; + self.pan = dvec2(0.0, 0.0); + } + + fn project(&self, point: [f32; 3], mesh_point: bool, rect: Rect) -> Option<(DVec2, f32)> { + let scale = if mesh_point { self.mesh_scale } else { 1.0 } as f64; + let x = point[0] as f64 * scale - self.centre[0] as f64; + let y = point[1] as f64 * scale - self.centre[1] as f64; + let z = point[2] as f64 * scale - self.centre[2] as f64; + let (sy, cy) = self.yaw.sin_cos(); + let (sp, cp) = self.pitch.sin_cos(); + let rx = mirror_x(cy * x + sy * z, self.mirrored); + let rz = -sy * x + cy * z; + let ry = cp * y - sp * rz; + let rz = sp * y + cp * rz; + let fov = 35.0_f64.to_radians(); + let fit_distance = self.radius as f64 / (fov * 0.5).tan() * 1.2; + let camera_z = fit_distance / self.zoom.max(0.08) - rz; + if camera_z <= 0.01 { + return None; + } + let focal = rect.size.y.max(1.0) * 0.5 / (fov * 0.5).tan(); + let centre = rect.pos + rect.size * 0.5 + self.pan; + let screen = dvec2(centre.x + focal * rx / camera_z, centre.y - focal * ry / camera_z); + Some((screen, camera_z as f32)) + } + + fn projected_polyline(&self, points: &[[f32; 3]], rect: Rect) -> Vec<(DVec2, f32)> { + points + .iter() + .filter_map(|point| self.project(*point, false, rect)) + .collect() + } +} + +impl Widget for FabricBodyView { + fn draw_walk(&mut self, cx: &mut Cx2d, _scope: &mut Scope, walk: Walk) -> DrawStep { + let rect = cx.walk_turtle(walk); + self.draw_bg.draw_abs(cx, rect); + cx.push_clip_rect(rect); + let Some(mesh) = self.mesh.as_ref() else { + self.draw_text.color = Vec4f { + x: 0.48, + y: 0.55, + z: 0.62, + w: 1.0, + }; + self.draw_text.draw_abs( + cx, + rect.pos + rect.size * 0.5 - dvec2(70.0, 5.0), + "drop a photo to start", + ); + cx.pop_clip_rect(); + cx.add_aligned_rect_area(&mut self.area, rect); + return DrawStep::done(); + }; + let posed = self + .posed + .as_deref() + .filter(|posed| posed.len() == mesh.vertices.len()) + .map(Vec::as_slice); + let display_vertices = posed.unwrap_or(mesh.vertices.as_slice()); + + let mut points: Vec<(DVec2, f32)> = display_vertices + .iter() + .filter_map(|point| self.project(*point, true, rect)) + .collect(); + points.sort_by(|a, b| b.1.total_cmp(&a.1)); + let min_depth = points.iter().map(|point| point.1).fold(f32::INFINITY, f32::min); + let max_depth = points + .iter() + .map(|point| point.1) + .fold(f32::NEG_INFINITY, f32::max); + let depth_span = (max_depth - min_depth).max(0.001); + self.draw_point.begin_many_instances(cx); + for (point, depth) in points { + self.draw_point.depth = (depth - min_depth) / depth_span; + self.draw_point.draw_abs( + cx, + Rect { + pos: point - dvec2(1.25, 1.25), + size: dvec2(2.5, 2.5), + }, + ); + } + self.draw_point.end_many_instances(cx); + + let rings: Vec<(String, Vec<(DVec2, f32)>)> = self + .rings + .iter() + .enumerate() + .map(|(ring_index, ring)| { + let points = match ( + posed, + self.pose_mapping.ring_vertices.get(ring_index), + ) { + (Some(posed), Some(indices)) if indices.len() == ring.points.len() => indices + .iter() + .filter_map(|&index| self.project(*posed.get(index)?, true, rect)) + .collect(), + _ => self.projected_polyline(&ring.points, rect), + }; + (ring.key.replace('_', " "), points) + }) + .collect(); + let lines: Vec<(DVec2, DVec2)> = self + .lines + .iter() + .enumerate() + .filter_map(|(line_index, line)| { + let (from, to, mesh_points) = match ( + posed, + self.pose_mapping.line_vertices.get(line_index), + ) { + (Some(posed), Some([from, to])) => + (*posed.get(*from)?, *posed.get(*to)?, true), + _ => (line.from, line.to, false), + }; + Some(( + self.project(from, mesh_points, rect)?.0, + self.project(to, mesh_points, rect)?.0, + )) + }) + .collect(); + + self.draw_line.begin_many_instances(cx); + self.draw_line.color = Vec4f { + x: 1.0, + y: 0.38, + z: 0.18, + w: 0.92, + }; + for (_, points) in &rings { + for pair in points.windows(2) { + self.draw_line.segment(cx, pair[0].0, pair[1].0, 1.5); + } + if let (Some(first), Some(last)) = (points.first(), points.last()) { + self.draw_line.segment(cx, last.0, first.0, 1.5); + } + } + self.draw_line.color = Vec4f { + x: 0.42, + y: 0.78, + z: 1.0, + w: 0.9, + }; + for (from, to) in lines { + self.draw_line.segment(cx, from, to, 1.25); + } + self.draw_line.end_many_instances(cx); + + self.draw_text.color = Vec4f { + x: 1.0, + y: 0.66, + z: 0.48, + w: 1.0, + }; + for (key, points) in rings { + if let Some(front) = points.iter().min_by(|a, b| a.1.total_cmp(&b.1)) { + self.draw_text + .draw_abs(cx, front.0 + dvec2(4.0, -5.0), &key); + } + } + cx.pop_clip_rect(); + cx.add_aligned_rect_area(&mut self.area, rect); + DrawStep::done() + } + + fn handle_event(&mut self, cx: &mut Cx, event: &Event, _scope: &mut Scope) { + match event.hits(cx, self.area) { + Hit::FingerDown(event) if event.device.is_primary_hit() => { + if event.tap_count >= 2 { + self.reset_camera(); + self.redraw(cx); + return; + } + self.drag = Some(BodyDrag { + from: event.abs, + yaw: self.yaw, + pitch: self.pitch, + pan: self.pan, + panning: event.modifiers.shift, + }); + } + Hit::FingerMove(event) => { + if let Some(drag) = self.drag { + let delta = event.abs - drag.from; + if drag.panning { + self.pan = drag.pan + delta; + } else { + self.yaw = drag.yaw - delta.x * 0.008; + self.pitch = (drag.pitch + delta.y * 0.007).clamp(-1.35, 1.35); + } + self.redraw(cx); + } + } + Hit::FingerUp(_) => self.drag = None, + Hit::FingerScroll(event) => { + self.zoom = (self.zoom * (-event.scroll.y * 0.004).exp()).clamp(0.15, 12.0); + self.redraw(cx); + } + Hit::FingerHoverIn(_) | Hit::FingerHoverOver(_) => { + cx.set_cursor(MouseCursor::Grab); + } + _ => {} + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn nearest_vertex_mapping_uses_scaled_rest_mesh_positions() { + let mesh = BodyMesh { + vertices: vec![[0.0, 0.0, 0.0], [4.0, 0.0, 0.0], [9.0, 0.0, 0.0]], + faces: Vec::new(), + landmarks: None, + }; + let measured = Measured { + values: makepad_fabric_measure::Measurements::sample(), + scale: 2.0, + rings: vec![Ring { + key: "test_ring", + y_cm: 0.0, + points: vec![[0.2, 0.0, 0.0], [7.5, 0.0, 0.0]], + skin_perimeter_cm: 0.0, + tape_perimeter_cm: 0.0, + }], + lines: vec![Line { + key: "test_line", + from: [7.5, 0.0, 0.0], + to: [17.0, 0.0, 0.0], + }], + }; + let mapping = map_measurements_to_vertices(&mesh, &measured); + assert_eq!(mapping.ring_vertices, vec![vec![0, 1]]); + assert_eq!(mapping.line_vertices, vec![[1, 2]]); + } + + #[test] + fn mirror_transform_only_flips_horizontal_view_axis() { + assert_eq!(mirror_x(12.5, false), 12.5); + assert_eq!(mirror_x(12.5, true), -12.5); + assert_eq!(mirror_x(-3.0, true), 3.0); + } +} diff --git a/apps/fabric/src/camera.rs b/apps/fabric/src/camera.rs new file mode 100644 index 000000000..9e9add372 --- /dev/null +++ b/apps/fabric/src/camera.rs @@ -0,0 +1,303 @@ +use makepad_widgets::{ + makepad_platform::video::{ + CameraFrameLayout, CameraFrameRef, VideoFormatId, VideoInputId, VideoInputsEvent, + VideoPixelFormat, + }, + Cx, CxMediaApi, +}; +use std::{ + cmp::Reverse, + sync::{ + atomic::{AtomicBool, AtomicU64, Ordering}, + Arc, Mutex, + }, + time::{Duration, Instant}, +}; + +pub const SEND_MAX_WIDTH: usize = 640; +const PREVIEW_MAX_WIDTH: usize = 320; +const PREVIEW_INTERVAL: Duration = Duration::from_millis(100); + +#[derive(Clone, Debug, PartialEq)] +pub struct CameraRgbFrame { + pub width: u32, + pub height: u32, + pub rgb: Vec, + pub serial: u64, +} + +#[derive(Default)] +struct PreviewSlot { + frame: Option, + updated_at: Option, +} + +#[derive(Default)] +struct CameraMailboxInner { + want: AtomicBool, + serial: AtomicU64, + frame: Mutex>, + model_size: Mutex>, + preview: Mutex, +} + +/// A one-frame handoff from the camera callback to the model worker. The +/// callback only converts a model-sized frame after the worker asks for one; +/// the smaller preview is independent and limited to ten updates per second. +#[derive(Clone, Default)] +pub struct CameraMailbox { + inner: Arc, +} + +impl CameraMailbox { + pub fn request(&self) { + if let Ok(mut frame) = self.inner.frame.lock() { + *frame = None; + } + self.inner.want.store(true, Ordering::Release); + } + + pub fn take(&self) -> Option { + self.inner.frame.lock().ok()?.take() + } + + pub fn peek_preview(&self) -> Option { + self.inner.preview.lock().ok()?.frame.clone() + } + + pub fn model_size(&self) -> Option<(u32, u32)> { + *self.inner.model_size.lock().ok()? + } + + fn capture(&self, frame: &CameraFrameRef<'_>) { + let wanted = self.inner.want.swap(false, Ordering::AcqRel); + let preview_due = self + .inner + .preview + .lock() + .ok() + .map(|slot| { + slot.updated_at + .map(|updated| updated.elapsed() >= PREVIEW_INTERVAL) + .unwrap_or(true) + }) + .unwrap_or(false); + if !wanted && !preview_due { + return; + } + + if wanted { + if let Some((rgb, width, height)) = frame_to_rgb(frame) { + let serial = self.inner.serial.fetch_add(1, Ordering::Relaxed) + 1; + if let Ok(mut size) = self.inner.model_size.lock() { + *size = Some((width, height)); + } + if let Ok(mut slot) = self.inner.frame.lock() { + *slot = Some(CameraRgbFrame { + width, + height, + rgb, + serial, + }); + } + } + } + + if preview_due { + if let Some((rgb, width, height)) = frame_to_rgb_max(frame, PREVIEW_MAX_WIDTH) { + let serial = self.inner.serial.fetch_add(1, Ordering::Relaxed) + 1; + if let Ok(mut slot) = self.inner.preview.lock() { + slot.frame = Some(CameraRgbFrame { + width, + height, + rgb, + serial, + }); + slot.updated_at = Some(Instant::now()); + } + } + } + } +} + +/// Register the capture callback. Calling this also asks the platform to +/// enumerate cameras, which produces `Event::VideoInputs` on the UI thread. +pub fn install_camera(cx: &mut Cx, mailbox: CameraMailbox) { + cx.camera_frame_input(0, move |frame| mailbox.capture(&frame)); +} + +/// Choose the first device's smallest raw-YUV format at least 640x360. +pub fn pick_camera(event: &VideoInputsEvent) -> Option<(VideoInputId, VideoFormatId)> { + let device = event.descs.first()?; + let format = device + .formats + .iter() + .filter(|format| { + format.width >= 640 + && format.height >= 360 + && matches!( + format.pixel_format, + VideoPixelFormat::NV12 | VideoPixelFormat::YUY2 + ) + }) + .min_by_key(|format| { + ( + format.width.saturating_mul(format.height), + if format.pixel_format == VideoPixelFormat::NV12 { + 0 + } else { + 1 + }, + Reverse((format.frame_rate.unwrap_or(0.0) * 1000.0) as u64), + ) + })?; + Some((device.input_id, format.format_id)) +} + +/// Convert NV12 or YUY2 to packed RGB8, using an integer sampling step so +/// the entire frame fits within [`SEND_MAX_WIDTH`]. +pub fn frame_to_rgb(frame: &CameraFrameRef<'_>) -> Option<(Vec, u32, u32)> { + frame_to_rgb_max(frame, SEND_MAX_WIDTH) +} + +fn frame_to_rgb_max( + frame: &CameraFrameRef<'_>, + max_width: usize, +) -> Option<(Vec, u32, u32)> { + let (width, height) = (frame.width, frame.height); + if width == 0 || height == 0 || max_width == 0 { + return None; + } + let step = width.div_ceil(max_width).max(1); + let out_width = width.div_ceil(step); + let out_height = height.div_ceil(step); + let mut rgb = Vec::with_capacity(out_width.checked_mul(out_height)?.checked_mul(3)?); + + for out_y in 0..out_height { + let source_y = (out_y * step).min(height - 1); + for out_x in 0..out_width { + let source_x = (out_x * step).min(width - 1); + let (y, u, v) = match frame.layout { + CameraFrameLayout::NV12 => nv12_pixel(frame, source_x, source_y)?, + CameraFrameLayout::YUY2 => yuy2_pixel(frame, source_x, source_y)?, + _ => return None, + }; + rgb.extend_from_slice(&yuv_to_rgb(y, u, v)); + } + } + + Some(( + rgb, + u32::try_from(out_width).ok()?, + u32::try_from(out_height).ok()?, + )) +} + +fn nv12_pixel(frame: &CameraFrameRef<'_>, x: usize, y: usize) -> Option<(u8, u8, u8)> { + if frame.plane_count < 2 { + return None; + } + let y_plane = frame.planes[0]; + let uv_plane = frame.planes[1]; + let y_index = y + .checked_mul(y_plane.row_stride)? + .checked_add(x.checked_mul(y_plane.pixel_stride)?)?; + let uv_index = (y / 2) + .checked_mul(uv_plane.row_stride)? + .checked_add((x / 2).checked_mul(uv_plane.pixel_stride)?)?; + Some(( + *y_plane.bytes.get(y_index)?, + *uv_plane.bytes.get(uv_index)?, + *uv_plane.bytes.get(uv_index + 1)?, + )) +} + +fn yuy2_pixel(frame: &CameraFrameRef<'_>, x: usize, y: usize) -> Option<(u8, u8, u8)> { + if frame.plane_count < 1 { + return None; + } + let plane = frame.planes[0]; + let pixel_stride = plane.pixel_stride.max(2); + let pair = y + .checked_mul(plane.row_stride)? + .checked_add((x / 2).checked_mul(pixel_stride.checked_mul(2)?)?)?; + Some(( + *plane.bytes.get(pair + (x & 1) * pixel_stride)?, + *plane.bytes.get(pair + 1)?, + *plane.bytes.get(pair + pixel_stride + 1)?, + )) +} + +/// One BT.709, video-range YUV pixel to RGB8. +pub(crate) fn yuv_to_rgb(y: u8, u: u8, v: u8) -> [u8; 3] { + let c = i32::from(y) - 16; + let d = i32::from(u) - 128; + let e = i32::from(v) - 128; + let clip = |value: i32| ((value + 128) >> 8).clamp(0, 255) as u8; + [ + clip(298 * c + 459 * e), + clip(298 * c - 55 * d - 136 * e), + clip(298 * c + 541 * d), + ] +} + +#[cfg(test)] +mod tests { + use super::*; + use makepad_widgets::makepad_platform::video::{ + CameraColorMatrix, CameraFramePlaneRef, + }; + + #[test] + fn yuv_grey_red_and_blue() { + let grey = yuv_to_rgb(126, 128, 128); + assert!(grey.iter().all(|channel| (126i16 - i16::from(*channel)).abs() <= 2)); + + let red = yuv_to_rgb(81, 90, 240); + assert!(red[0] > 220 && red[1] < 60 && red[2] < 60, "{red:?}"); + + let blue = yuv_to_rgb(41, 240, 110); + assert!(blue[2] > 220 && blue[0] < 60 && blue[1] < 60, "{blue:?}"); + } + + #[test] + fn nv12_integer_downsample_keeps_the_whole_frame() { + const WIDTH: usize = 64; + const HEIGHT: usize = 32; + let mut y_plane = vec![0u8; WIDTH * HEIGHT]; + for y in 0..HEIGHT { + for x in 0..WIDTH { + y_plane[y * WIDTH + x] = 16 + ((x + y) % 200) as u8; + } + } + let uv_plane = vec![128u8; WIDTH * HEIGHT / 2]; + let frame = CameraFrameRef { + timestamp_ns: 0, + width: WIDTH, + height: HEIGHT, + layout: CameraFrameLayout::NV12, + matrix: CameraColorMatrix::BT709, + plane_count: 2, + planes: [ + CameraFramePlaneRef { + bytes: &y_plane, + row_stride: WIDTH, + pixel_stride: 1, + }, + CameraFramePlaneRef { + bytes: &uv_plane, + row_stride: WIDTH, + pixel_stride: 2, + }, + CameraFramePlaneRef::empty(), + ], + }; + + let (rgb, width, height) = frame_to_rgb_max(&frame, 16).unwrap(); + assert_eq!((width, height), (16, 8)); + assert_eq!(rgb.len(), 16 * 8 * 3); + assert_eq!(&rgb[..3], &yuv_to_rgb(y_plane[0], 128, 128)); + let last_source = y_plane[28 * WIDTH + 60]; + assert_eq!(&rgb[rgb.len() - 3..], &yuv_to_rgb(last_source, 128, 128)); + } +} diff --git a/apps/fabric/src/install.rs b/apps/fabric/src/install.rs new file mode 100644 index 000000000..4813751d3 --- /dev/null +++ b/apps/fabric/src/install.rs @@ -0,0 +1,73 @@ +use makepad_ai_hub::{ + local::{InstallState, LocalModels}, + registry::LicenseRestriction, +}; +use makepad_ai_hub_ui::{ModelRowInstallState, ModelRowState}; + +pub const BODY_MODEL_ID: &str = "sam3dbody"; +pub const BODY_MODEL_ROLE: &str = "native-body"; + +pub fn body_model_row(models: &LocalModels) -> ModelRowState { + let spec = models.spec(BODY_MODEL_ID); + let bytes_from_spec = spec + .map(|spec| spec.files.iter().filter_map(|file| file.size).sum()) + .unwrap_or(0); + let (bytes_done, bytes_total, state) = match models.install_state(BODY_MODEL_ID) { + InstallState::NotInstalled { bytes_total } => { + (0, bytes_total.max(bytes_from_spec), ModelRowInstallState::NotInstalled) + } + InstallState::Partial { + bytes_done, + bytes_total, + } => (bytes_done, bytes_total, ModelRowInstallState::NotInstalled), + InstallState::Installed => ( + bytes_from_spec, + bytes_from_spec, + ModelRowInstallState::Installed, + ), + }; + let license = spec.and_then(|spec| spec.license.as_ref()); + ModelRowState { + model_id: BODY_MODEL_ID.to_string(), + name: "SAM 3D Body".to_string(), + bytes_total, + bytes_done, + state, + license_name: license + .map(|license| license.name.clone()) + .unwrap_or_else(|| "Licence unavailable".to_string()), + restriction: license + .map(|license| restriction_name(license.restriction).to_string()) + .unwrap_or_else(|| "restricted".to_string()), + } +} + +pub fn body_model_status(models: &LocalModels, downloading: bool) -> String { + if !models.license_acknowledged(BODY_MODEL_ID) { + return "licence not accepted".to_string(); + } + match models.install_state(BODY_MODEL_ID) { + InstallState::Installed => "installed · 2.8 GB · Metal".to_string(), + InstallState::NotInstalled { .. } => "not installed · 2.8 GB".to_string(), + InstallState::Partial { + bytes_done, + bytes_total, + } => { + let percent = bytes_done.saturating_mul(100) / bytes_total.max(1); + if downloading { + format!("downloading {percent} %") + } else { + format!("not installed · {percent} % downloaded") + } + } + } +} + +fn restriction_name(restriction: LicenseRestriction) -> &'static str { + match restriction { + LicenseRestriction::None => "none", + LicenseRestriction::NonCommercial => "non-commercial", + LicenseRestriction::Community => "community", + LicenseRestriction::Restricted => "restricted", + } +} diff --git a/apps/fabric/src/main.rs b/apps/fabric/src/main.rs new file mode 100644 index 000000000..f5d8b1ea1 --- /dev/null +++ b/apps/fabric/src/main.rs @@ -0,0 +1,1802 @@ +mod body_view; +mod camera; +mod install; +mod pattern_view; +mod pipeline; + +use body_view::FabricBodyView; +use camera::{install_camera, pick_camera, CameraMailbox}; +use install::{body_model_row, body_model_status, BODY_MODEL_ID, BODY_MODEL_ROLE}; +use makepad_ai_hub::local::{InstallState, LocalModels}; +use makepad_ai_hub_ui::{ModelInstallPanel, ModelRowInstallState}; +use makepad_fabric_draft::{ + designs, nest, to_pdf, to_svg, Design, OptionSpec, Options, PageSize, Pattern, +}; +use makepad_fabric_measure::{Measurements, MEASUREMENT_KEYS}; +use makepad_widgets::*; +use pattern_view::FabricPatternView; +use pipeline::{Pipeline, PipelineMessage}; +use std::{ + collections::VecDeque, + path::{Path, PathBuf}, + time::{Duration, Instant, SystemTime, UNIX_EPOCH}, +}; + +app_main!(App); + +script_mod! { + use mod.prelude.widgets.* + use mod.widgets.* + + let SectionTitle = Label { + width: Fill + height: 20 + draw_text +: { + color: #x8da0b5 + text_style: theme.font_bold{font_size: 9.0} + } + } + + let Hint = Label { + width: Fill + height: Fit + draw_text +: { + color: #x667789 + text_style: theme.font_regular{font_size: 9.0} + } + } + + let Status = Label { + width: Fill + height: Fit + draw_text +: { + color: #xa9b6c4 + text_style: theme.font_regular{font_size: 9.0} + } + } + + let ToolButton = Button { + height: 30 + draw_bg +: { + color: #x27313c + color_hover: #x334252 + color_down: #x1d252e + border_color: #x52677c + border_size: 1.0 + border_radius: 3.0 + } + draw_text +: { + color: #xe7edf3 + color_hover: #xffffffff + text_style: theme.font_bold{font_size: 9.5} + } + } + + let CameraImage = Image { + draw_bg +: { + bbox: uniform(vec4(-1.0, -1.0, -1.0, -1.0)) + mirror: uniform(1.0) + pixel: fn() { + let scale = self.fit_scale * self.image_scale + let pan = self.fit_pan * self.image_scale + self.image_pan + let sample_scale = vec2(scale.x * mix(1.0, -1.0, self.mirror), scale.y) + let sample_pan = vec2(mix(pan.x, 1.0 - pan.x, self.mirror), pan.y) + let color = self.get_color_scale_pan(sample_scale, sample_pan) + if self.bbox.x >= 0.0 { + let line = vec2(1.5, 1.5) / self.rect_size + let on_x = (abs(self.pos.x - self.bbox.x) < line.x || abs(self.pos.x - self.bbox.z) < line.x) + && self.pos.y >= self.bbox.y && self.pos.y <= self.bbox.w + let on_y = (abs(self.pos.y - self.bbox.y) < line.y || abs(self.pos.y - self.bbox.w) < line.y) + && self.pos.x >= self.bbox.x && self.pos.x <= self.bbox.z + if on_x || on_y { + return Pal.premul(#x54d59a) + } + } + return Pal.premul(color) + } + } + } + + let TabButton = Button { + height: 24 + padding: Inset{left: 12 right: 12 top: 4 bottom: 4} + draw_bg +: { + color: #x1b232c + color_hover: #x2a3542 + color_down: #x151b22 + border_color: #x1b232c + border_size: 1.0 + border_radius: 3.0 + } + draw_text +: { + color: #x9aa8b7 + color_hover: #xffffffff + text_style: theme.font_bold{font_size: 9.0} + } + } + mod.widgets.FabricMeasurementGridBase = #(FabricMeasurementGrid::register_widget(vm)) + mod.widgets.FabricMeasurementGrid = set_type_default() do mod.widgets.FabricMeasurementGridBase { + width: Fill + height: Fill + grid := DataGrid { + width: Fill + height: Fill + rows: 25 + cols: 2 + default_col_width: 110.0 + default_row_height: 24.0 + row_header_width: 30.0 + color_bg: #x161d25 + color_cell: #x1a222c + color_cell_alt: #x1d2631 + color_text: #xdbe4ee + color_header: #x222c38 + color_header_active: #x2f3d4d + color_header_text: #x9aa8b7 + color_selection: #x4fa3ff26 + color_selection_border: #x4fa3ff + draw_text +: {color: #xdbe4ee} + draw_text_bold +: {color: #xdbe4ee} + Editor := TextInput { + width: Fill + height: Fill + margin: 0 + padding: Inset{left: 4 right: 4 top: 4 bottom: 3} + draw_bg +: { + border_radius: 0.0 + border_size: 2.0 + border_color: #x4fa3ff + border_color_hover: #x4fa3ff + border_color_focus: #x4fa3ff + color: #x0f151c + color_hover: #x0f151c + color_focus: #x0f151c + } + draw_text +: { + text_style: theme.font_code{font_size: 9.0} + color: #xffffffff + } + } + } + } + mod.widgets.FabricOptionsListBase = #(FabricOptionsList::register_widget(vm)) + mod.widgets.FabricOptionsList = set_type_default() do mod.widgets.FabricOptionsListBase { + width: Fill + height: Fill + empty := Hint{text: "this design has no options"} + list := PortalList { + width: Fill + height: Fill + flow: Down + drag_scrolling: true + Row := View { + width: Fill + height: 46 + flow: Down + spacing: 2 + option_caption := View { + width: Fill + height: Fit + flow: Right + option_name := Label { + width: Fill + height: Fit + draw_text +: { + color: #xa9b6c4 + text_style: theme.font_regular{font_size: 8.5} + } + } + option_unit := Label { + width: Fit + height: Fit + draw_text +: { + color: #x667789 + text_style: theme.font_regular{font_size: 8.0} + } + } + } + option_slider := Slider { + width: Fill + height: 24 + min: 0.0 + max: 100.0 + default: 0.0 + precision: 1 + text: "" + } + } + } + } + + startup() do #(App::script_component(vm)) { + ui: Root { + main_window := Window { + window.title: "Fabric" + window.inner_size: vec2(1400, 900) + pass +: {clear_color: #x0b1016} + body +: { + width: Fill + height: Fill + flow: Right + spacing: 1 + padding: 0 + // Events go to children in draw order here, first come first + // served: the licence modal lives inside the LEFT column and + // must get a click before the body view under it does. + event_order: EventOrder.Down + + left_panel := SolidView { + width: 320 + height: Fill + flow: Down + spacing: 8 + padding: 14 + draw_bg +: {color: #x161d25} + + Label { + width: Fill + height: 28 + text: "FABRIC" + draw_text +: { + color: #xf0f4f8 + text_style: theme.font_bold{font_size: 15.0} + } + } + SectionTitle{text: "BODY MODEL"} + model_install := mod.widgets.ModelInstallPanel { + width: Fill + height: 142 + } + model_status := Status{text: "checking model…"} + + SectionTitle{text: "PHOTO"} + drop_zone := RoundedView { + width: Fill + height: 225 + flow: Down + spacing: 7 + padding: 10 + align: Align{x: 0.5 y: 0.5} + show_bg: true + draw_bg +: { + color: #x111820 + border_color: #x405164 + border_size: 1.0 + border_radius: 4.0 + pixel: fn() { + let p = self.pos * self.rect_size + let edge_x = min(p.x, self.rect_size.x - p.x) + let edge_y = min(p.y, self.rect_size.y - p.y) + let edge = min(edge_x, edge_y) + let along = if edge_x < edge_y p.y else p.x + if edge < self.border_size && modf(along, 10.0) < 6.0 { + return Pal.premul(self.border_color) + } + return Pal.premul(self.color) + } + } + photo_image := Image { + width: Fill + height: 163 + fit: ImageFit.Smallest + visible: false + } + live_image := CameraImage { + width: Fill + height: 163 + fit: ImageFit.Smallest + visible: false + } + drop_title := Label { + width: Fill + height: Fit + text: "drop a photo" + draw_text +: { + color: #xd1dae4 + text_style: theme.font_bold{font_size: 11.0} + } + } + drop_help := Hint { + text: "front view · tight clothes · whole body in frame" + } + photo_name := Hint{text: "JPG or PNG"} + } + + View { + width: Fill + height: 29 + flow: Right + spacing: 8 + align: Align{x: 0.0 y: 0.5} + Label { + width: 58 + height: Fit + text: "HEIGHT" + draw_text +: { + color: #x8da0b5 + text_style: theme.font_bold{font_size: 9.0} + } + } + height_input := TextInput { + width: Fill + height: 25 + empty_text: "optional" + } + Label { + width: 22 + height: Fit + text: "cm" + draw_text +: {color: #x667789} + } + } + View { + width: Fill + height: 30 + flow: Right + spacing: 8 + measure_button := ToolButton { + width: Fill + text: "MEASURE" + } + live_button := ToolButton { + width: 78 + text: "LIVE" + } + mirror_toggle := Toggle { + width: 88 + height: 30 + text: "MIRROR" + active: true + } + } + progress_status := Status{text: "drop a photo to begin"} + } + + centre_panel := SolidView { + width: Fill + height: Fill + flow: Down + draw_bg +: {color: #x11161d} + split := Splitter { + width: Fill + height: Fill + axis: SplitterAxis.Horizontal + align: SplitterAlign.Weighted(0.42) + size: 6.0 + draw_bg +: { + color_bg: #x11161d + color: #x1f2731 + color_hover: #x2f3d4d + color_drag: #x4fa3ff + } + a: View { + width: Fill + height: Fill + flow: Down + View { + width: Fill + height: 38 + flow: Right + spacing: 10 + padding: Inset{left: 14 right: 14} + align: Align{x: 0.0 y: 0.5} + SectionTitle{width: Fit text: "BODY"} + Hint{text: "drag orbit · shift-drag pan · wheel zoom · double-click reset"} + } + body_preview := mod.widgets.FabricBodyView {} + } + b: View { + width: Fill + height: Fill + flow: Right + View { + width: Fill + height: Fill + flow: Down + View { + width: Fill + height: 38 + flow: Right + spacing: 10 + padding: Inset{left: 14 right: 14} + align: Align{x: 0.0 y: 0.5} + SectionTitle{width: Fit text: "PATTERN"} + settling_tag := RoundedView { + width: Fit + height: 18 + padding: Inset{left: 7 right: 7 top: 2 bottom: 2} + visible: false + show_bg: true + draw_bg +: {color: #x493d24 border_radius: 9.0} + Label { + width: Fit + height: Fit + text: "settling…" + draw_text +: { + color: #xe8c36d + text_style: theme.font_regular{font_size: 8.0} + } + } + } + Hint{text: "drag pan · wheel zoom · cut line solid, seam line dim"} + } + pattern_preview := mod.widgets.FabricPatternView {} + } + design_column := SolidView { + width: 232 + height: Fill + flow: Down + spacing: 8 + padding: 12 + draw_bg +: {color: #x141a22} + SectionTitle{text: "DESIGN"} + design_select := DropDown { + width: Fill + height: 28 + labels: ["No designs available"] + } + design_options := mod.widgets.FabricOptionsList { + width: Fill + height: Fill + } + export_svg := ToolButton{width: Fill text: "EXPORT SVG"} + export_pdf := ToolButton{width: Fill text: "EXPORT PDF (A4)"} + app_status := Status{text: "sample measurements ready"} + } + } + } + } + right_panel := SolidView { + width: 380 + height: Fill + flow: Down + spacing: 8 + padding: 14 + draw_bg +: {color: #x161d25} + View { + width: Fill + height: 20 + flow: Right + spacing: 8 + align: Align{x: 0.0 y: 0.5} + SectionTitle{width: Fill text: "MEASUREMENTS"} + sample_tag := RoundedView { + width: Fit + height: 18 + padding: Inset{left: 7 right: 7 top: 2 bottom: 2} + show_bg: true + draw_bg +: {color: #x293440 border_radius: 9.0} + sample_tag_label := Label { + width: Fit + height: Fit + text: "sample body" + draw_text +: { + color: #x8998a8 + text_style: theme.font_regular{font_size: 8.0} + } + } + } + copy_all := TabButton{ + height: 20 + padding: Inset{left: 8 right: 8 top: 2 bottom: 2} + text: "COPY ALL" + } + } + measurement_grid := mod.widgets.FabricMeasurementGrid { + width: Fill + height: Fill + } + Hint{text: "click and drag to select · ⌘C copies as tab-separated text · double-click a value to correct it"} + } + } + } + } + } +} +#[derive(Clone, Debug, Default)] +enum MeasurementListAction { + Changed { key: &'static str, value: f32 }, + #[default] + None, +} + +/// The measurement table as a spreadsheet grid: keys down, centimetres in +/// the second column, so a selection copies as tab-separated text straight +/// into a spreadsheet. Double-click (or type on) a value to correct it. +#[derive(Script, ScriptHook, Widget)] +struct FabricMeasurementGrid { + #[source] + source: ScriptObjectRef, + #[deref] + view: View, + #[rust] + values: Measurements, + #[rust(true)] + sample: bool, + #[rust] + initialized: bool, + /// The row whose value is being edited in place. + #[rust] + editing: Option, + /// The text the editor starts with; taken on the draw that seeds it. + #[rust] + edit_seed: Option, +} + +impl FabricMeasurementGrid { + fn set_measurements(&mut self, cx: &mut Cx, values: Measurements, sample: bool) { + self.values = values; + self.sample = sample; + self.refresh_copy_provider(cx); + self.view.redraw(cx); + } + + /// Tab-separated rows for a selection; every row when nothing is selected. + fn tsv(&self, selection: Option) -> String { + let entries = self.values.entries(); + let last = entries.len() - 1; + let (r0, r1, c0, c1) = match selection { + Some(sel) => { + let (r0, r1) = sel.row_range(); + let (c0, c1) = sel.col_range(); + (r0.min(last), r1.min(last), c0.min(1), c1.min(1)) + } + None => (0, last, 0, 1), + }; + let mut out = String::new(); + for (key, value) in &entries[r0..=r1] { + let mut cols: Vec = Vec::new(); + if c0 == 0 { + cols.push(humanise_key(key)); + } + if c1 == 1 { + cols.push(format!("{value:.1}")); + } + out.push_str(&cols.join("\t")); + out.push('\n'); + } + out + } + + fn refresh_copy_provider(&self, cx: &mut Cx) { + let grid = self.view.data_grid(cx, ids!(grid)); + let text = self.tsv(grid.selection()); + grid.set_copy_provider(Box::new(move |_sel| text.clone())); + } + + fn start_edit(&mut self, cx: &mut Cx, row: usize, replace: Option) { + if row >= MEASUREMENT_KEYS.len() { + return; + } + self.editing = Some(row); + self.edit_seed = Some( + replace.unwrap_or_else(|| format!("{:.1}", self.values.entries()[row].1)), + ); + let grid = self.view.data_grid(cx, ids!(grid)); + grid.set_selection(cx, Some(GridSelection::single(row, 1))); + grid.redraw(cx); + } + + fn commit_edit(&mut self, cx: &mut Cx, row: usize, text: &str) { + self.editing = None; + self.edit_seed = None; + if let Some(&key) = MEASUREMENT_KEYS.get(row) { + if let Ok(value) = text.trim().replace(',', ".").parse::() { + if value.is_finite() && value > 0.0 && self.values.set(key, value) { + cx.widget_action( + self.widget_uid(), + MeasurementListAction::Changed { key, value }, + ); + } + } + } + self.refresh_copy_provider(cx); + self.view.data_grid(cx, ids!(grid)).redraw(cx); + } + + fn commit_current(&mut self, cx: &mut Cx) { + let Some(row) = self.editing else { return }; + let text = self + .view + .data_grid(cx, ids!(grid)) + .get_item(row, 1) + .map(|(_, widget)| widget.as_text_input().text()); + match text { + Some(text) => self.commit_edit(cx, row, &text), + None => self.cancel_edit(cx), + } + } + + fn cancel_edit(&mut self, cx: &mut Cx) { + self.editing = None; + self.edit_seed = None; + self.view.data_grid(cx, ids!(grid)).redraw(cx); + } +} + +impl Widget for FabricMeasurementGrid { + fn handle_event(&mut self, cx: &mut Cx, event: &Event, scope: &mut Scope) { + self.view.handle_event(cx, event, scope); + let Event::Actions(actions) = event else { + return; + }; + let grid = self.view.data_grid(cx, ids!(grid)); + for action in grid.actions(actions) { + match action { + DataGridAction::EditCell { row, col, replace } if col == 1 => { + self.start_edit(cx, row, replace); + } + DataGridAction::CellDoubleClicked { row, col } if col == 1 => { + self.start_edit(cx, row, None); + } + DataGridAction::CellClicked { .. } => { + if self.editing.is_some() { + self.commit_current(cx); + } + } + DataGridAction::SelectionChanged { .. } => self.refresh_copy_provider(cx), + _ => {} + } + } + for (row, _col, widget) in grid.cell_widgets_with_actions(actions) { + let input = widget.as_text_input(); + if let Some((text, _modifiers)) = input.returned(actions) { + self.commit_edit(cx, row, &text); + } else if input.escaped(actions) { + self.cancel_edit(cx); + } + } + } + + fn draw_walk(&mut self, cx: &mut Cx2d, scope: &mut Scope, walk: Walk) -> DrawStep { + while let Some(step) = self.view.draw_walk(cx, scope, walk).step() { + let grid_ref = step.as_data_grid(); + let Some(mut grid) = grid_ref.borrow_mut() else { + continue; + }; + if !self.initialized { + self.initialized = true; + grid.set_col_labels(vec!["measurement".to_string(), "cm".to_string()]); + grid.set_col_width(0, 190.0); + grid.set_col_width(1, 84.0); + } + grid.set_grid_size(MEASUREMENT_KEYS.len(), 2); + let entries = self.values.entries(); + let value_color = if self.sample { + vec4(0.55, 0.61, 0.67, 1.0) + } else { + vec4(0.92, 0.95, 0.98, 1.0) + }; + let key_color = vec4(0.66, 0.72, 0.78, 1.0); + while let Some(cell) = grid.next_cell(cx) { + let Some((key, value)) = entries.get(cell.row).copied() else { + continue; + }; + if cell.col == 1 && self.editing == Some(cell.row) { + if let Some(item) = grid.item(cx, cell.row, cell.col, id!(Editor)) { + let seed = self.edit_seed.take(); + if let Some(seed) = &seed { + item.as_text_input().set_text(cx, seed); + } + grid.draw_item(cx, &cell, &item, None); + // Focus only once the editor has a drawn area. + if seed.is_some() { + item.as_text_input().take_key_focus(cx); + } + } + continue; + } + let (text, align, color) = if cell.col == 0 { + (humanise_key(key), 0.0, key_color) + } else { + (format!("{value:.1}"), 1.0, value_color) + }; + grid.cell_text_styled( + cx, + &cell, + &text, + CellStyle { + align, + color: Some(color), + ..CellStyle::default() + }, + ); + } + } + DrawStep::done() + } +} + +#[derive(Clone, Debug, Default)] +enum OptionsListAction { + Changed { key: String, value: f64 }, + #[default] + None, +} + +#[derive(Script, ScriptHook, Widget)] +struct FabricOptionsList { + #[source] + source: ScriptObjectRef, + #[deref] + view: View, + #[rust] + specs: Vec, + #[rust] + values: Vec, +} + +impl FabricOptionsList { + fn set_specs(&mut self, cx: &mut Cx, specs: Vec, options: &Options) { + self.values = specs.iter().map(|spec| options.get(spec)).collect(); + self.specs = specs; + self.view.label(cx, ids!(empty)).set_visible(cx, self.specs.is_empty()); + self.view + .portal_list(cx, ids!(list)) + .set_visible(cx, !self.specs.is_empty()); + self.view.redraw(cx); + } +} + +impl Widget for FabricOptionsList { + fn handle_event(&mut self, cx: &mut Cx, event: &Event, scope: &mut Scope) { + self.view.handle_event(cx, event, scope); + let Event::Actions(actions) = event else { + return; + }; + let list = self.view.portal_list(cx, ids!(list)); + for (index, item) in list.items_with_actions(actions) { + let Some(spec) = self.specs.get(index) else { + continue; + }; + if let Some(value) = item.slider(cx, ids!(option_slider)).slided(actions) { + if let Some(slot) = self.values.get_mut(index) { + *slot = value; + } + cx.widget_action( + self.widget_uid(), + OptionsListAction::Changed { + key: spec.key.to_string(), + value, + }, + ); + } + } + } + + fn draw_walk(&mut self, cx: &mut Cx2d, scope: &mut Scope, walk: Walk) -> DrawStep { + while let Some(item) = self.view.draw_walk(cx, scope, walk).step() { + let Some(mut list) = item.borrow_mut::() else { + continue; + }; + list.set_item_range(cx, 0, self.specs.len()); + while let Some(index) = list.next_visible_item(cx) { + let row = list.item(cx, index, id!(Row)); + if let (Some(spec), Some(value)) = (self.specs.get(index), self.values.get(index)) { + row.label(cx, ids!(option_name)).set_text(cx, spec.label); + row.label(cx, ids!(option_unit)).set_text(cx, spec.unit); + let mut slider = row.slider(cx, ids!(option_slider)); + let min = spec.min; + let max = spec.max; + let default = spec.default; + let step = ((max - min).abs() / 100.0).max(0.01); + script_apply_eval!(cx, slider, { + min: #(min) + max: #(max) + default: #(default) + step: #(step) + }); + slider.set_value(cx, *value); + } + row.draw_all_unscoped(cx); + } + } + DrawStep::done() + } +} + +const SETTLE_AFTER: Duration = Duration::from_millis(1_500); +const SETTLE_TOLERANCE_CM: f32 = 0.5; + +#[derive(Default)] +struct MeasurementSettler { + history: VecDeque<(Duration, Measurements)>, + settled: bool, +} + +impl MeasurementSettler { + /// Returns true only on the transition into a settled state, so a stable + /// live stream drafts once instead of rebuilding the pattern every frame. + fn push(&mut self, now: Duration, measurements: Measurements) -> bool { + self.history.push_back((now, measurements)); + let Some(cutoff) = now.checked_sub(SETTLE_AFTER) else { + self.settled = false; + return false; + }; + while self.history.len() > 1 + && self + .history + .get(1) + .is_some_and(|(time, _)| *time <= cutoff) + { + self.history.pop_front(); + } + let stable = self + .history + .front() + .filter(|(time, _)| *time <= cutoff) + .is_some_and(|(_, previous)| { + measurements.entries().iter().all(|(key, value)| { + previous + .get(key) + .is_some_and(|old| (value - old).abs() <= SETTLE_TOLERANCE_CM) + }) + }); + let became_settled = stable && !self.settled; + self.settled = stable; + became_settled + } + + fn reset(&mut self) { + self.history.clear(); + self.settled = false; + } +} + +#[derive(Script, ScriptHook)] +pub struct App { + #[live] + ui: WidgetRef, + #[rust] + models: Option, + #[rust] + pipeline: Option, + #[rust] + refresh_timer: Timer, + #[rust] + photo: Option, + #[rust] + pipeline_busy: bool, + #[rust] + pipeline_started: Option, + #[rust] + camera: CameraMailbox, + #[rust] + camera_installed: bool, + #[rust] + live: bool, + #[rust(true)] + mirrored: bool, + #[rust] + live_started: Option, + #[rust] + preview_texture: Option, + #[rust] + preview_serial: u64, + #[rust] + live_bbox: Option<[f32; 4]>, + #[rust] + live_frame_size: Option<(u32, u32)>, + #[rust] + settler: MeasurementSettler, + #[rust] + measurements: Measurements, + #[rust] + has_measured_body: bool, + #[rust] + designs: Vec>, + #[rust] + design_index: usize, + #[rust] + options: Options, + #[rust] + pattern: Option, + #[rust] + model_ready: bool, + #[rust] + drag_over: bool, +} + +impl App { + fn startup(&mut self, cx: &mut Cx) { + self.pipeline = Some(Pipeline::new()); + self.refresh_timer = cx.start_interval(0.1); + self.measurements = Measurements::sample(); + self.sync_measurements(cx); + self.designs = designs(); + let labels = if self.designs.is_empty() { + vec!["No designs available".to_string()] + } else { + self.designs + .iter() + .map(|design| design.name().to_string()) + .collect() + }; + self.ui + .drop_down(cx, ids!(design_select)) + .set_labels(cx, labels); + self.configure_options(cx); + self.redraft(cx); + + match LocalModels::open() { + Ok(models) => { + let row = body_model_row(&models); + let panel = self.ui.widget(cx, ids!(model_install)); + if let Some(mut panel) = panel.borrow_mut::() { + panel.set_rows(cx, vec![row]); + } + self.models = Some(models); + self.refresh_model_ui(cx); + } + Err(error) => { + self.set_progress(cx, format!("could not open local models: {error}")); + self.ui + .label(cx, ids!(model_status)) + .set_text(cx, "model registry unavailable"); + self.ui + .button(cx, ids!(measure_button)) + .set_disabled(cx, true); + self.ui + .button(cx, ids!(live_button)) + .set_disabled(cx, true); + } + } + } + + fn sync_measurements(&self, cx: &mut Cx) { + let widget = self.ui.widget(cx, ids!(measurement_grid)); + if let Some(mut grid) = widget.borrow_mut::() { + grid.set_measurements(cx, self.measurements, !self.has_measured_body); + } + self.ui + .view(cx, ids!(sample_tag)) + .set_visible(cx, self.live || !self.has_measured_body); + self.ui + .label(cx, ids!(sample_tag_label)) + .set_text(cx, if self.live { "live body" } else { "sample body" }); + } + + fn configure_options(&mut self, cx: &mut Cx) { + self.options = Options::default(); + let specs = self + .designs + .get(self.design_index) + .map(|design| design.options()) + .unwrap_or_default(); + for spec in &specs { + self.options.0.insert(spec.key.to_string(), spec.default); + } + let widget = self.ui.widget(cx, ids!(design_options)); + if let Some(mut list) = widget.borrow_mut::() { + list.set_specs(cx, specs, &self.options); + }; + } + + fn redraft(&mut self, cx: &mut Cx) { + let Some(design) = self.designs.get(self.design_index) else { + self.pattern = None; + self.set_pattern_error(cx, "no designs are available from the draft library"); + self.set_app_status(cx, "sample body ready · waiting for a draft design"); + return; + }; + match design.draft(&self.measurements, &self.options) { + Ok(pattern) => { + self.pattern = Some(pattern.clone()); + let widget = self.ui.widget(cx, ids!(pattern_preview)); + if let Some(mut view) = widget.borrow_mut::() { + view.set_pattern(cx, pattern); + } + self.set_app_status(cx, format!("{} pattern ready", design.name())); + } + Err(error) => { + self.pattern = None; + self.set_pattern_error(cx, error.to_string()); + self.set_app_status(cx, error.to_string()); + } + } + } + + fn set_pattern_error(&self, cx: &mut Cx, error: impl Into) { + let widget = self.ui.widget(cx, ids!(pattern_preview)); + if let Some(mut view) = widget.borrow_mut::() { + view.set_error(cx, error); + }; + } + + fn set_progress(&self, cx: &mut Cx, status: impl AsRef) { + self.ui + .label(cx, ids!(progress_status)) + .set_text(cx, status.as_ref()); + } + + fn set_app_status(&self, cx: &mut Cx, status: impl AsRef) { + self.ui + .label(cx, ids!(app_status)) + .set_text(cx, status.as_ref()); + } + + fn panel_is_downloading(&self, cx: &mut Cx) -> bool { + let panel = self.ui.widget(cx, ids!(model_install)); + panel + .borrow::() + .map(|panel| { + panel.rows().iter().any(|row| { + row.model_id == BODY_MODEL_ID + && matches!(row.state, ModelRowInstallState::Downloading) + }) + }) + .unwrap_or(false) + } + + fn refresh_model_ui(&mut self, cx: &mut Cx) { + let downloading = self.panel_is_downloading(cx); + let ready = self.models.as_ref().is_some_and(|models| { + models.license_acknowledged(BODY_MODEL_ID) + && matches!(models.install_state(BODY_MODEL_ID), InstallState::Installed) + && models + .installed_path(BODY_MODEL_ID, BODY_MODEL_ROLE) + .is_some() + }); + if let Some(models) = self.models.as_ref() { + self.ui + .label(cx, ids!(model_status)) + .set_text(cx, &body_model_status(models, downloading)); + } + self.ui + .button(cx, ids!(measure_button)) + .set_disabled(cx, !ready || self.pipeline_busy || self.photo.is_none()); + self.ui + .button(cx, ids!(live_button)) + .set_disabled(cx, !self.live && (!ready || self.pipeline_busy)); + let became_ready = ready && !self.model_ready; + self.model_ready = ready; + if became_ready && self.photo.is_some() && !self.pipeline_busy { + self.start_measurement(cx); + } + } + + fn start_measurement(&mut self, cx: &mut Cx) { + if self.pipeline_busy { + return; + } + let Some(photo) = self.photo.clone() else { + self.set_progress(cx, "drop a photo first"); + return; + }; + let (weights, height_cm) = match self.pipeline_inputs(cx) { + Ok(inputs) => inputs, + Err(error) => { + self.set_progress(cx, error); + return; + } + }; + let Some(pipeline) = self.pipeline.as_ref() else { + self.set_progress(cx, "the body model worker is unavailable"); + return; + }; + match pipeline.run(photo, weights, height_cm) { + Ok(()) => { + self.pipeline_busy = true; + self.pipeline_started = Some(Instant::now()); + self.set_progress(cx, "queued…"); + self.refresh_model_ui(cx); + } + Err(error) => self.set_progress(cx, error), + } + } + + fn pipeline_inputs(&self, cx: &mut Cx) -> Result<(PathBuf, Option), String> { + let models = self + .models + .as_ref() + .ok_or_else(|| "install the body model first".to_string())?; + if !models.license_acknowledged(BODY_MODEL_ID) { + return Err("accept the body model licence first".to_string()); + } + let weights = models + .installed_path(BODY_MODEL_ID, BODY_MODEL_ROLE) + .ok_or_else(|| "install the body model first".to_string())?; + let height_text = self.ui.text_input(cx, ids!(height_input)).text(); + let height_cm = if height_text.trim().is_empty() { + None + } else { + match height_text.trim().parse::() { + Ok(value) if value.is_finite() && (80.0..=260.0).contains(&value) => Some(value), + _ => return Err("height must be 80–260 cm, or left empty".to_string()), + } + }; + Ok((weights, height_cm)) + } + + fn start_live(&mut self, cx: &mut Cx) { + if self.live || self.pipeline_busy { + return; + } + let (weights, height_cm) = match self.pipeline_inputs(cx) { + Ok(inputs) => inputs, + Err(error) => { + self.set_progress(cx, error); + return; + } + }; + use makepad_widgets::makepad_platform::permission::Permission; + cx.request_permission(Permission::Camera); + if !self.camera_installed { + install_camera(cx, self.camera.clone()); + self.camera_installed = true; + } + let Some(pipeline) = self.pipeline.as_ref() else { + self.set_progress(cx, "the body model worker is unavailable"); + return; + }; + if let Err(error) = pipeline.start_live(weights, height_cm, self.camera.clone()) { + self.set_progress(cx, error); + return; + } + + self.live = true; + self.live_started = Some(Instant::now()); + self.live_bbox = None; + self.live_frame_size = None; + self.settler.reset(); + self.set_live_ui(cx); + self.set_progress(cx, "live · looking for a camera…"); + self.refresh_model_ui(cx); + } + + fn stop_live(&mut self, cx: &mut Cx) { + if !self.live { + return; + } + cx.use_video_input(&[]); + if let Some(pipeline) = self.pipeline.as_ref() { + pipeline.stop_live(); + } + self.live = false; + self.live_started = None; + self.live_bbox = None; + self.live_frame_size = None; + self.settler.reset(); + self.set_live_ui(cx); + self.set_progress( + cx, + if self.pipeline_busy { + "live stopped · queued photo next" + } else { + "live stopped" + }, + ); + self.refresh_model_ui(cx); + } + + fn set_live_ui(&self, cx: &mut Cx) { + self.ui + .image(cx, ids!(live_image)) + .set_visible(cx, self.live); + self.ui + .image(cx, ids!(photo_image)) + .set_visible(cx, !self.live && self.photo.is_some()); + self.ui + .label(cx, ids!(drop_title)) + .set_text(cx, if self.live { "live camera" } else if self.photo.is_some() { "photo loaded" } else { "drop a photo" }); + let photo_name = if self.live { + "camera · body model runs continuously".to_string() + } else { + self.photo + .as_ref() + .and_then(|path| path.file_name()) + .map(|name| name.to_string_lossy().into_owned()) + .unwrap_or_else(|| "JPG or PNG".to_string()) + }; + self.ui + .label(cx, ids!(photo_name)) + .set_text(cx, &photo_name); + self.ui + .view(cx, ids!(settling_tag)) + .set_visible(cx, self.live && !self.settler.settled); + + let color = if self.live { + Vec4f { + x: 0.08, + y: 0.38, + z: 0.24, + w: 1.0, + } + } else { + Vec4f { + x: 0.153, + y: 0.192, + z: 0.235, + w: 1.0, + } + }; + let border = if self.live { + Vec4f { + x: 0.33, + y: 0.84, + z: 0.60, + w: 1.0, + } + } else { + Vec4f { + x: 0.322, + y: 0.404, + z: 0.486, + w: 1.0, + } + }; + let mut button = self.ui.button(cx, ids!(live_button)); + script_apply_eval!(cx, button, { + draw_bg +: { + color: #(color) + border_color: #(border) + } + }); + self.update_live_bbox(cx); + self.sync_measurements(cx); + } + + fn set_mirrored(&mut self, cx: &mut Cx, mirrored: bool) { + self.mirrored = mirrored; + self.ui + .image(cx, ids!(live_image)) + .set_uniform( + cx, + live_id!(mirror), + &[if mirrored { 1.0 } else { 0.0 }], + ); + let widget = self.ui.widget(cx, ids!(body_preview)); + if let Some(mut view) = widget.borrow_mut::() { + view.set_mirrored(cx, mirrored); + } + self.update_live_bbox(cx); + } + + fn pump_camera_preview(&mut self, cx: &mut Cx) { + if !self.live { + return; + } + let Some(frame) = self.camera.peek_preview() else { + return; + }; + if frame.serial <= self.preview_serial { + return; + } + self.preview_serial = frame.serial; + let pixels: Vec = frame + .rgb + .chunks_exact(3) + .map(|rgb| { + 0xff00_0000 | (u32::from(rgb[0]) << 16) | (u32::from(rgb[1]) << 8) | u32::from(rgb[2]) + }) + .collect(); + if let Some(texture) = self.preview_texture.as_ref() { + texture.set_data_u32(cx, frame.width as usize, frame.height as usize, pixels); + } else { + self.preview_texture = Some(Texture::new_with_format( + cx, + TextureFormat::VecBGRAu8_32 { + width: frame.width as usize, + height: frame.height as usize, + data: Some(pixels), + updated: TextureUpdated::Full, + }, + )); + } + self.ui + .image(cx, ids!(live_image)) + .set_texture(cx, self.preview_texture.clone()); + self.ui.widget(cx, ids!(live_image)).redraw(cx); + } + + fn update_live_bbox(&self, cx: &mut Cx) { + let bbox = self + .live_bbox + .zip(self.live_frame_size) + .map(|(bbox, (width, height))| { + [ + (bbox[0] / width as f32).clamp(0.0, 1.0), + (bbox[1] / height as f32).clamp(0.0, 1.0), + (bbox[2] / width as f32).clamp(0.0, 1.0), + (bbox[3] / height as f32).clamp(0.0, 1.0), + ] + }) + .unwrap_or([-1.0; 4]); + let bbox = mirror_normalized_bbox(bbox, self.mirrored); + self.ui + .image(cx, ids!(live_image)) + .set_uniform(cx, live_id!(bbox), &bbox); + self.ui.widget(cx, ids!(live_image)).redraw(cx); + } + + fn drain_pipeline(&mut self, cx: &mut Cx) { + let messages = self + .pipeline + .as_ref() + .map(Pipeline::poll) + .unwrap_or_default(); + for message in messages { + match message { + PipelineMessage::Stage(stage) => { + if self.live { + self.set_progress(cx, format!("live · {stage}")); + } else { + self.set_progress(cx, stage); + } + } + PipelineMessage::LiveFrame { + fps, + model_ms, + pose_ms, + person, + bbox, + } => { + if !self.live { + continue; + } + self.live_bbox = bbox; + self.live_frame_size = self.camera.model_size(); + self.update_live_bbox(cx); + if person { + self.set_progress( + cx, + format!( + "live · {fps:.1} fps · model {model_ms:.0} ms · pose {pose_ms:.1} ms · person" + ), + ); + } else { + self.set_progress(cx, "live · no person in frame"); + } + } + PipelineMessage::Done { + measured, + mesh, + posed, + pose_mapping, + reset_pose, + } => { + self.measurements = measured.values; + self.has_measured_body = true; + self.sync_measurements(cx); + let widget = self.ui.widget(cx, ids!(body_preview)); + if let Some(mut view) = widget.borrow_mut::() { + if reset_pose { + view.set_pose(cx, None); + } + view.set_body(cx, mesh, &measured, pose_mapping); + view.set_pose(cx, posed); + } + if self.live { + let now = self + .live_started + .map(|start| start.elapsed()) + .unwrap_or_default(); + let redraft = self.settler.push(now, self.measurements); + self.ui + .view(cx, ids!(settling_tag)) + .set_visible(cx, !self.settler.settled); + if redraft { + self.redraft(cx); + } + } else if self.pipeline_busy { + self.pipeline_busy = false; + let seconds = self + .pipeline_started + .take() + .map(|start| start.elapsed().as_secs_f32()) + .unwrap_or(0.0); + self.set_progress(cx, format!("done in {seconds:.1} s")); + self.redraft(cx); + } + } + PipelineMessage::Failed(error) => { + if self.live { + self.stop_live(cx); + } else { + self.pipeline_busy = false; + self.pipeline_started = None; + } + self.set_progress(cx, error); + } + } + } + self.refresh_model_ui(cx); + } + + fn accept_photo(&mut self, cx: &mut Cx, path: PathBuf) { + let Some(name) = path.file_name().map(|name| name.to_string_lossy().into_owned()) else { + self.set_progress(cx, "the dropped photo has no file name"); + return; + }; + self.photo = Some(path.clone()); + if !self.live { + self.ui + .label(cx, ids!(photo_name)) + .set_text(cx, &name); + self.ui + .label(cx, ids!(drop_title)) + .set_text(cx, "photo loaded"); + } + self.ui + .image(cx, ids!(photo_image)) + .set_visible(cx, !self.live); + if let Err(error) = self + .ui + .image(cx, ids!(photo_image)) + .load_image_file_by_path_async(cx, &path) + { + self.set_progress(cx, format!("could not decode {name}: {error}")); + return; + } + if self.model_ready { + self.start_measurement(cx); + } else { + self.set_progress(cx, "install the body model first"); + } + self.refresh_model_ui(cx); + } + + fn set_drop_highlight(&mut self, cx: &mut Cx, active: bool) { + if self.drag_over == active { + return; + } + self.drag_over = active; + let color = if active { + Vec4f { + x: 0.10, + y: 0.18, + z: 0.24, + w: 1.0, + } + } else { + Vec4f { + x: 0.067, + y: 0.094, + z: 0.125, + w: 1.0, + } + }; + let border = if active { + Vec4f { + x: 0.31, + y: 0.78, + z: 1.0, + w: 1.0, + } + } else { + Vec4f { + x: 0.25, + y: 0.32, + z: 0.39, + w: 1.0, + } + }; + let mut zone = self.ui.view(cx, ids!(drop_zone)); + script_apply_eval!(cx, zone, { + draw_bg +: { + color: #(color) + border_color: #(border) + } + }); + } + + fn handle_file_drop(&mut self, cx: &mut Cx, event: &Event) { + if !matches!(event, Event::Drag(_) | Event::Drop(_) | Event::DragEnd) { + return; + } + let area = self.ui.widget(cx, ids!(drop_zone)).area(); + match event.drag_hits(cx, area) { + DragHit::Drag(drag) => { + let accepts = drag.items.iter().any(accepted_photo_item); + *drag.response.lock().unwrap() = if accepts { + DragResponse::Copy + } else { + DragResponse::None + }; + self.set_drop_highlight(cx, accepts && drag.state != DragState::Out); + } + DragHit::Drop(drop) => { + self.set_drop_highlight(cx, false); + if let Some(path) = drop.items.iter().find_map(photo_item_path) { + self.accept_photo(cx, path); + } + } + DragHit::DragEnd | DragHit::NoHit => self.set_drop_highlight(cx, false), + } + } + + fn export(&mut self, cx: &mut Cx, extension: &str) { + let Some(pattern) = self.pattern.as_ref() else { + self.set_app_status(cx, "there is no drafted pattern to export"); + return; + }; + let layout = nest(pattern, 1500.0); + let seconds = SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|duration| duration.as_secs()) + .unwrap_or(0); + let directory = makepad_ai_hub::home::makepad_home().join("fabric/exports"); + if let Err(error) = std::fs::create_dir_all(&directory) { + self.set_app_status(cx, format!("could not create {}: {error}", directory.display())); + return; + } + let path = directory.join(export_file_name(&pattern.design_id, extension, seconds)); + let result = match extension { + "svg" => std::fs::write(&path, to_svg(pattern, &layout).as_bytes()), + "pdf" => std::fs::write(&path, to_pdf(pattern, &layout, PageSize::A4)), + _ => return, + }; + match result { + Ok(()) => self.set_app_status(cx, format!("exported {}", path.display())), + Err(error) => self.set_app_status( + cx, + format!("could not write {}: {error}", path.display()), + ), + } + } + + fn pump_install_panel(&mut self, cx: &mut Cx) { + let panel = self.ui.widget(cx, ids!(model_install)); + if let (Some(models), Some(mut panel)) = + (self.models.as_mut(), panel.borrow_mut::()) + { + panel.pump(cx, models); + }; + } +} + +impl MatchEvent for App { + fn handle_actions(&mut self, cx: &mut Cx, actions: &Actions) { + if self.ui.button(cx, ids!(measure_button)).clicked(actions) { + self.start_measurement(cx); + } + if self.ui.button(cx, ids!(live_button)).clicked(actions) { + if self.live { + self.stop_live(cx); + } else { + self.start_live(cx); + } + } + if let Some(mirrored) = self + .ui + .check_box(cx, ids!(mirror_toggle)) + .changed(actions) + { + self.set_mirrored(cx, mirrored); + } + if let Some(index) = self.ui.drop_down(cx, ids!(design_select)).changed(actions) { + if index < self.designs.len() { + self.design_index = index; + self.configure_options(cx); + self.redraft(cx); + } + } + if self.ui.button(cx, ids!(copy_all)).clicked(actions) { + let widget = self.ui.widget(cx, ids!(measurement_grid)); + let text = widget + .borrow::() + .map(|grid| grid.tsv(None)) + .unwrap_or_default(); + cx.copy_to_clipboard(&text); + self.set_app_status(cx, "measurements copied · paste into a spreadsheet"); + } + let measurement_uid = self.ui.widget(cx, ids!(measurement_grid)).widget_uid(); + match actions.find_widget_action_cast::(measurement_uid) { + MeasurementListAction::Changed { key, value } => { + if self.measurements.set(key, value) { + self.sync_measurements(cx); + self.redraft(cx); + } + } + MeasurementListAction::None => {} + } + let options_uid = self.ui.widget(cx, ids!(design_options)).widget_uid(); + match actions.find_widget_action_cast::(options_uid) { + OptionsListAction::Changed { key, value } => { + self.options.0.insert(key, value); + self.redraft(cx); + } + OptionsListAction::None => {} + } + if self.ui.button(cx, ids!(export_svg)).clicked(actions) { + self.export(cx, "svg"); + } + if self.ui.button(cx, ids!(export_pdf)).clicked(actions) { + self.export(cx, "pdf"); + } + } +} + +impl AppMain for App { + fn script_mod(vm: &mut ScriptVm) -> ScriptValue { + makepad_widgets::script_mod(vm); + makepad_ai_hub_ui::script_mod(vm); + crate::body_view::script_mod(vm); + crate::pattern_view::script_mod(vm); + self::script_mod(vm) + } + + fn handle_event(&mut self, cx: &mut Cx, event: &Event) { + if let Event::Startup = event { + self.startup(cx); + // The bar shows our title when the window manager hosts us. + makepad_wm_api::set_title(cx, "Fabric"); + } + // The window manager asked politely (SUPER+W): go now. + if let Event::Custom(json) = event { + if let Some(makepad_wm_api::WmEvent::CloseRequested) = makepad_wm_api::WmEvent::parse(json) { + cx.quit(); + return; + } + } + self.match_event(cx, event); + self.ui.handle_event(cx, event, &mut Scope::empty()); + self.handle_file_drop(cx, event); + self.pump_install_panel(cx); + match event { + Event::VideoInputs(inputs) if self.live => { + if let Some(input) = pick_camera(inputs) { + cx.use_video_input(&[input]); + self.set_progress(cx, "live · camera ready…"); + } else { + self.set_progress(cx, "live · no NV12/YUY2 camera at 640×360"); + } + } + Event::PermissionResult(result) + if self.live + && result.permission + == makepad_widgets::makepad_platform::permission::Permission::Camera => + { + use makepad_widgets::makepad_platform::permission::PermissionStatus; + if result.status != PermissionStatus::Granted { + let status = format!("camera permission: {:?}", result.status); + self.stop_live(cx); + self.set_progress(cx, status); + } + } + _ => {} + } + if let Event::Signal = event { + self.drain_pipeline(cx); + } + if self.refresh_timer.is_event(event).is_some() { + self.pump_camera_preview(cx); + self.refresh_model_ui(cx); + } + self.refresh_model_ui(cx); + } +} + +fn accepted_photo_item(item: &DragItem) -> bool { + photo_item_path(item).is_some() +} + +fn mirror_normalized_bbox(bbox: [f32; 4], mirrored: bool) -> [f32; 4] { + if mirrored && bbox[0] >= 0.0 { + [1.0 - bbox[2], bbox[1], 1.0 - bbox[0], bbox[3]] + } else { + bbox + } +} + +fn photo_item_path(item: &DragItem) -> Option { + let DragItem::FilePath { + path, + internal_id: None, + } = item + else { + return None; + }; + let path = Path::new(path); + let extension = path.extension()?.to_str()?.to_ascii_lowercase(); + matches!(extension.as_str(), "jpg" | "jpeg" | "png").then(|| path.to_path_buf()) +} + +pub(crate) fn humanise_key(key: &str) -> String { + let words = key.replace('_', " "); + let mut chars = words.chars(); + match chars.next() { + Some(first) => first.to_uppercase().collect::() + chars.as_str(), + None => String::new(), + } +} + +pub(crate) fn export_file_name(design_id: &str, extension: &str, unix_seconds: u64) -> String { + let stem: String = design_id + .chars() + .map(|character| { + if character.is_ascii_alphanumeric() { + character.to_ascii_lowercase() + } else { + '-' + } + }) + .collect(); + let stem = stem.trim_matches('-'); + let stem = if stem.is_empty() { "pattern" } else { stem }; + format!("{stem}-{}.{}", utc_timestamp(unix_seconds), extension) +} + +fn utc_timestamp(unix_seconds: u64) -> String { + let days = (unix_seconds / 86_400) as i64; + let seconds = unix_seconds % 86_400; + let hour = seconds / 3_600; + let minute = (seconds % 3_600) / 60; + let second = seconds % 60; + let (year, month, day) = civil_from_days(days); + format!("{year:04}{month:02}{day:02}-{hour:02}{minute:02}{second:02}") +} + +fn civil_from_days(days_since_epoch: i64) -> (i64, u64, u64) { + let z = days_since_epoch + 719_468; + let era = if z >= 0 { z } else { z - 146_096 } / 146_097; + let day_of_era = z - era * 146_097; + let year_of_era = + (day_of_era - day_of_era / 1_460 + day_of_era / 36_524 - day_of_era / 146_096) + / 365; + let mut year = year_of_era + era * 400; + let day_of_year = day_of_era - (365 * year_of_era + year_of_era / 4 - year_of_era / 100); + let month_prime = (5 * day_of_year + 2) / 153; + let day = day_of_year - (153 * month_prime + 2) / 5 + 1; + let month = month_prime + if month_prime < 10 { 3 } else { -9 }; + year += i64::from(month <= 2); + (year, month as u64, day as u64) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn humanises_measurement_keys() { + assert_eq!(humanise_key("shoulder_to_bust"), "Shoulder to bust"); + assert_eq!(humanise_key("height"), "Height"); + } + + #[test] + fn mirror_transform_flips_preview_bbox_horizontally() { + assert_eq!( + mirror_normalized_bbox([0.1, 0.2, 0.4, 0.8], true), + [0.6, 0.2, 0.9, 0.8] + ); + assert_eq!( + mirror_normalized_bbox([0.1, 0.2, 0.4, 0.8], false), + [0.1, 0.2, 0.4, 0.8] + ); + assert_eq!(mirror_normalized_bbox([-1.0; 4], true), [-1.0; 4]); + } + + #[test] + fn export_names_are_safe_and_timestamped() { + assert_eq!( + export_file_name("Classic Shirt", "svg", 0), + "classic-shirt-19700101-000000.svg" + ); + assert_eq!( + export_file_name("dress/v2", "pdf", 1_700_000_000), + "dress-v2-20231114-221320.pdf" + ); + } + + #[test] + fn stable_measurements_settle_after_one_and_a_half_seconds() { + let mut settler = MeasurementSettler::default(); + let measurements = Measurements::sample(); + assert!(!settler.push(Duration::ZERO, measurements)); + assert!(!settler.push(Duration::from_millis(750), measurements)); + assert!(settler.push(Duration::from_millis(1_500), measurements)); + assert!(settler.settled); + assert!(!settler.push(Duration::from_millis(1_750), measurements)); + } + + #[test] + fn jittering_measurements_never_settle() { + let mut settler = MeasurementSettler::default(); + for index in 0..16 { + let mut measurements = Measurements::sample(); + measurements.bust += index as f32; + assert!(!settler.push(Duration::from_millis(index * 250), measurements)); + assert!(!settler.settled); + } + } +} diff --git a/apps/fabric/src/pattern_view.rs b/apps/fabric/src/pattern_view.rs new file mode 100644 index 000000000..1fb85d902 --- /dev/null +++ b/apps/fabric/src/pattern_view.rs @@ -0,0 +1,398 @@ +use crate::body_view::DrawFabricLine; +use makepad_fabric_draft::{flatten, nest, offset, Layout as PatternLayout, Part, Pattern, Point}; +use makepad_widgets::*; + +script_mod! { + use mod.prelude.widgets_internal.* + use mod.widgets.* + + mod.widgets.FabricPatternViewBase = #(FabricPatternView::register_widget(vm)) + mod.widgets.FabricPatternView = set_type_default() do mod.widgets.FabricPatternViewBase { + width: Fill + height: Fill + draw_bg +: {color: #x0d1218} + draw_line +: {} + draw_text +: { + color: #xc5d0dc + text_style: theme.font_regular{font_size: 8.5} + } + } +} + +#[derive(Clone, Copy)] +struct PatternDrag { + from: DVec2, + pan: DVec2, +} + +#[derive(Clone, Copy, Default)] +struct Bounds { + min: DVec2, + max: DVec2, + valid: bool, +} + +impl Bounds { + fn include(&mut self, point: DVec2) { + if !self.valid { + self.min = point; + self.max = point; + self.valid = true; + } else { + self.min.x = self.min.x.min(point.x); + self.min.y = self.min.y.min(point.y); + self.max.x = self.max.x.max(point.x); + self.max.y = self.max.y.max(point.y); + } + } + + fn size(self) -> DVec2 { + let size = self.max - self.min; + dvec2(size.x.max(1.0), size.y.max(1.0)) + } + + fn centre(self) -> DVec2 { + (self.min + self.max) * 0.5 + } +} + +#[derive(Script, ScriptHook, Widget)] +pub struct FabricPatternView { + #[uid] + uid: WidgetUid, + #[source] + source: ScriptObjectRef, + #[walk] + walk: Walk, + #[layout] + layout: Layout, + #[redraw] + #[area] + area: Area, + #[live] + draw_bg: DrawColor, + #[live] + draw_line: DrawFabricLine, + #[live] + draw_text: DrawText, + #[rust] + pattern: Option, + #[rust] + nested: Option, + #[rust] + error: String, + #[rust] + bounds: Bounds, + #[rust(1.0)] + zoom: f64, + /// The view aspect the current nest was chosen for. + #[rust(1.0)] + nest_aspect: f64, + #[rust] + pan: DVec2, + #[rust] + drag: Option, +} + +impl FabricPatternView { + pub fn set_pattern(&mut self, cx: &mut Cx, pattern: Pattern) { + self.pattern = Some(pattern); + // Nested on the next draw, for the pane's shape. + self.nested = None; + self.error.clear(); + self.zoom = 1.0; + self.pan = dvec2(0.0, 0.0); + self.redraw(cx); + } + + pub fn set_error(&mut self, cx: &mut Cx, error: impl Into) { + self.pattern = None; + self.nested = None; + self.error = error.into(); + self.redraw(cx); + } + + /// Nest onto the fabric width whose finished layout has the pane's + /// aspect ratio, so the pieces fill the view instead of a tall strip. + fn ensure_nest(&mut self, rect: Rect) { + let target = (rect.size.x - 24.0).max(1.0) / (rect.size.y - 24.0).max(1.0); + if self.nested.is_some() && (self.nest_aspect / target).ln().abs() < 0.12 { + return; + } + const WIDTHS: [f64; 9] = [ + 900.0, 1200.0, 1500.0, 2000.0, 2500.0, 3000.0, 4000.0, 5000.0, 6500.0, + ]; + let best = { + let Some(pattern) = &self.pattern else { return }; + let mut best: Option<(f64, PatternLayout, Bounds)> = None; + for width in WIDTHS { + let layout = nest(pattern, width); + let bounds = pattern_bounds(pattern, &layout); + let size = bounds.size(); + let aspect = size.x.max(1.0) / size.y.max(1.0); + let score = (aspect / target).ln().abs(); + if best.as_ref().map_or(true, |(other, _, _)| score < *other) { + best = Some((score, layout, bounds)); + } + } + best + }; + if let Some((_, layout, bounds)) = best { + self.nested = Some(layout); + self.bounds = bounds; + self.nest_aspect = target; + } + } + + fn to_screen(&self, point: DVec2, rect: Rect) -> DVec2 { + let size = self.bounds.size(); + let fit = ((rect.size.x - 24.0) / size.x) + .min((rect.size.y - 24.0) / size.y) + .max(0.0001); + rect.pos + rect.size * 0.5 + + self.pan + + (point - self.bounds.centre()) * fit * self.zoom + } + + fn part_offset<'a>( + &'a self, + layout: &'a PatternLayout, + part_index: usize, + ) -> (Point, f64) { + layout + .placements + .iter() + .find(|placement| placement.part == part_index) + .map(|placement| (placement.offset, placement.rotation_deg)) + .unwrap_or((Point::default(), 0.0)) + } + + fn path_segments( + &self, + path: &makepad_fabric_draft::Path, + offset_by: Point, + rotation: f64, + rect: Rect, + ) -> Vec<(DVec2, DVec2)> { + let points = flatten(path, 1.0); + let transformed: Vec = points + .iter() + .map(|point| self.to_screen(place(*point, offset_by, rotation), rect)) + .collect(); + let mut segments: Vec<_> = transformed + .windows(2) + .map(|pair| (pair[0], pair[1])) + .collect(); + if path.closed { + if let (Some(first), Some(last)) = (transformed.first(), transformed.last()) { + segments.push((*last, *first)); + } + } + segments + } +} + +impl Widget for FabricPatternView { + fn draw_walk(&mut self, cx: &mut Cx2d, _scope: &mut Scope, walk: Walk) -> DrawStep { + let rect = cx.walk_turtle(walk); + self.draw_bg.draw_abs(cx, rect); + cx.push_clip_rect(rect); + self.ensure_nest(rect); + let (Some(pattern), Some(layout)) = (&self.pattern, &self.nested) else { + let message = if self.error.is_empty() { + "pattern preview" + } else { + &self.error + }; + self.draw_text.color = Vec4f { + x: 0.52, + y: 0.58, + z: 0.65, + w: 1.0, + }; + self.draw_text + .draw_abs(cx, rect.pos + dvec2(14.0, 16.0), message); + cx.pop_clip_rect(); + cx.add_aligned_rect_area(&mut self.area, rect); + return DrawStep::done(); + }; + + struct Stroke { + from: DVec2, + to: DVec2, + color: Vec4f, + width: f64, + } + let mut strokes = Vec::new(); + let mut labels = Vec::new(); + let cut_color = Vec4f { + x: 0.93, + y: 0.96, + z: 0.99, + w: 1.0, + }; + let seam_color = Vec4f { + x: 0.40, + y: 0.49, + z: 0.58, + w: 0.72, + }; + let mark_color = Vec4f { + x: 1.0, + y: 0.40, + z: 0.18, + w: 0.95, + }; + + for (part_index, part) in pattern.parts.iter().enumerate() { + let (part_offset, rotation) = self.part_offset(layout, part_index); + let cut_path = offset(&part.outline, part.seam_allowance_mm); + for (from, to) in self.path_segments(&cut_path, part_offset, rotation, rect) { + strokes.push(Stroke { + from, + to, + color: cut_color, + width: 1.25, + }); + } + for (from, to) in self.path_segments(&part.outline, part_offset, rotation, rect) { + strokes.push(Stroke { + from, + to, + color: seam_color, + width: 0.75, + }); + } + for path in &part.internal { + for (from, to) in self.path_segments(path, part_offset, rotation, rect) { + strokes.push(Stroke { + from, + to, + color: seam_color, + width: 0.75, + }); + } + } + for notch in &part.notches { + let at = self.to_screen(place(*notch, part_offset, rotation), rect); + strokes.push(Stroke { + from: at + dvec2(-3.0, -3.0), + to: at + dvec2(3.0, 3.0), + color: mark_color, + width: 1.0, + }); + strokes.push(Stroke { + from: at + dvec2(-3.0, 3.0), + to: at + dvec2(3.0, -3.0), + color: mark_color, + width: 1.0, + }); + } + let grain_from = self.to_screen(place(part.grainline.0, part_offset, rotation), rect); + let grain_to = self.to_screen(place(part.grainline.1, part_offset, rotation), rect); + strokes.push(Stroke { + from: grain_from, + to: grain_to, + color: mark_color, + width: 1.0, + }); + let label_at = part + .labels + .first() + .map(|label| label.at) + .unwrap_or(part.outline.start); + labels.push(( + self.to_screen(place(label_at, part_offset, rotation), rect), + format!( + "{} · {}", + part.name, + if part.on_fold { + "cut on fold".to_string() + } else { + format!("cut {}", part.cut_count) + } + ), + )); + } + + self.draw_line.begin_many_instances(cx); + for stroke in strokes { + self.draw_line.color = stroke.color; + self.draw_line + .segment(cx, stroke.from, stroke.to, stroke.width); + } + self.draw_line.end_many_instances(cx); + self.draw_text.color = Vec4f { + x: 0.78, + y: 0.84, + z: 0.90, + w: 1.0, + }; + for (at, text) in labels { + self.draw_text.draw_abs(cx, at + dvec2(4.0, -5.0), &text); + } + cx.pop_clip_rect(); + cx.add_aligned_rect_area(&mut self.area, rect); + DrawStep::done() + } + + fn handle_event(&mut self, cx: &mut Cx, event: &Event, _scope: &mut Scope) { + match event.hits(cx, self.area) { + Hit::FingerDown(event) if event.device.is_primary_hit() => { + self.drag = Some(PatternDrag { + from: event.abs, + pan: self.pan, + }); + } + Hit::FingerMove(event) => { + if let Some(drag) = self.drag { + self.pan = drag.pan + event.abs - drag.from; + self.redraw(cx); + } + } + Hit::FingerUp(_) => self.drag = None, + Hit::FingerScroll(event) => { + self.zoom = (self.zoom * (-event.scroll.y * 0.004).exp()).clamp(0.1, 30.0); + self.redraw(cx); + } + Hit::FingerHoverIn(_) | Hit::FingerHoverOver(_) => { + cx.set_cursor(MouseCursor::Grab) + } + _ => {} + } + } +} + +fn place(point: Point, offset: Point, rotation_deg: f64) -> DVec2 { + let angle = rotation_deg.to_radians(); + let (sin, cos) = angle.sin_cos(); + dvec2( + point.x * cos - point.y * sin + offset.x, + point.x * sin + point.y * cos + offset.y, + ) +} + +fn pattern_bounds(pattern: &Pattern, layout: &PatternLayout) -> Bounds { + let mut bounds = Bounds::default(); + for (part_index, part) in pattern.parts.iter().enumerate() { + let (part_offset, rotation) = layout + .placements + .iter() + .find(|placement| placement.part == part_index) + .map(|placement| (placement.offset, placement.rotation_deg)) + .unwrap_or((Point::default(), 0.0)); + include_part(&mut bounds, part, part_offset, rotation); + } + if !bounds.valid && layout.width_mm > 0.0 && layout.height_mm > 0.0 { + bounds.include(dvec2(0.0, 0.0)); + bounds.include(dvec2(layout.width_mm, layout.height_mm)); + } + bounds +} + +fn include_part(bounds: &mut Bounds, part: &Part, part_offset: Point, rotation: f64) { + let cut = offset(&part.outline, part.seam_allowance_mm); + for point in flatten(&cut, 1.0) { + bounds.include(place(point, part_offset, rotation)); + } +} diff --git a/apps/fabric/src/pipeline.rs b/apps/fabric/src/pipeline.rs new file mode 100644 index 000000000..969adfb5c --- /dev/null +++ b/apps/fabric/src/pipeline.rs @@ -0,0 +1,676 @@ +use crate::{ + body_view::{map_measurements_to_vertices, BodyPoseMapping}, + camera::CameraMailbox, +}; +use makepad_ai_body::model::BodyModel; +use makepad_fabric_measure::{measure, BodyMesh, MeasureOptions, Measured}; +use makepad_widgets::image_cache::ImageBuffer; +use makepad_widgets::makepad_platform::thread::SignalToUI; +use std::{ + collections::VecDeque, + path::{Path, PathBuf}, + sync::{ + atomic::{AtomicBool, Ordering}, + mpsc::{self, Receiver, Sender}, + Arc, + }, + thread, + time::{Duration, Instant}, +}; + +const PHOTO_CROP_SIZE: usize = 512; +const LIVE_CROP_SIZE: usize = 384; +const SHAPE_ALPHA: f32 = 0.35; +const POSE_ALPHA: f32 = 0.6; +const SHAPE_RESET_GAP: Duration = Duration::from_secs(1); +const FRAME_TIMEOUT: Duration = Duration::from_secs(2); +const FRAME_POLL: Duration = Duration::from_millis(5); + +pub enum PipelineMessage { + Stage(String), + LiveFrame { + fps: f32, + model_ms: f32, + pose_ms: f32, + person: bool, + bbox: Option<[f32; 4]>, + }, + Done { + measured: Box, + mesh: Arc, + posed: Option>>, + pose_mapping: BodyPoseMapping, + reset_pose: bool, + }, + Failed(String), +} + +struct RunRequest { + photo: PathBuf, + weights: PathBuf, + height_cm: Option, +} + +struct LiveRequest { + weights: PathBuf, + height_cm: Option, + mailbox: CameraMailbox, +} + +enum WorkerRequest { + Photo(RunRequest), + Live(LiveRequest), +} + +pub struct Pipeline { + request_tx: Sender, + message_rx: Receiver, + live_stop: Arc, +} + +impl Pipeline { + pub fn new() -> Self { + let (request_tx, request_rx) = mpsc::channel(); + let (message_tx, message_rx) = mpsc::channel(); + let live_stop = Arc::new(AtomicBool::new(true)); + let worker_stop = live_stop.clone(); + thread::Builder::new() + .name("fabric-body-pipeline".to_string()) + .spawn(move || worker(request_rx, message_tx, worker_stop)) + .expect("spawn fabric body worker"); + Self { + request_tx, + message_rx, + live_stop, + } + } + + pub fn run( + &self, + photo: PathBuf, + weights: PathBuf, + height_cm: Option, + ) -> Result<(), String> { + self.request_tx + .send(WorkerRequest::Photo(RunRequest { + photo, + weights, + height_cm, + })) + .map_err(|_| "the body model worker stopped".to_string()) + } + + pub fn start_live( + &self, + weights: PathBuf, + height_cm: Option, + mailbox: CameraMailbox, + ) -> Result<(), String> { + self.live_stop.store(false, Ordering::Release); + if self + .request_tx + .send(WorkerRequest::Live(LiveRequest { + weights, + height_cm, + mailbox, + })) + .is_err() + { + self.live_stop.store(true, Ordering::Release); + return Err("the body model worker stopped".to_string()); + } + Ok(()) + } + + pub fn stop_live(&self) { + self.live_stop.store(true, Ordering::Release); + } + + pub fn poll(&self) -> Vec { + self.message_rx.try_iter().collect() + } +} + +fn emit(sender: &Sender, message: PipelineMessage) -> bool { + if sender.send(message).is_err() { + return false; + } + SignalToUI::set_ui_signal(); + true +} + +fn worker( + requests: Receiver, + messages: Sender, + live_stop: Arc, +) { + let mut loaded: Option<(PathBuf, BodyModel)> = None; + while let Ok(request) = requests.recv() { + let result = match request { + WorkerRequest::Photo(request) => run_one(&request, &messages, &mut loaded), + WorkerRequest::Live(request) => { + run_live(&request, &messages, &mut loaded, &live_stop) + } + }; + if let Err(error) = result { + if !emit(&messages, PipelineMessage::Failed(error)) { + return; + } + } + } +} + +fn run_one( + request: &RunRequest, + messages: &Sender, + loaded: &mut Option<(PathBuf, BodyModel)>, +) -> Result<(), String> { + emit(messages, PipelineMessage::Stage("decoding photo…".to_string())); + let (rgb, width, height) = decode_rgb(&request.photo)?; + + ensure_model(&request.weights, messages, loaded)?; + + emit(messages, PipelineMessage::Stage("inferring…".to_string())); + let model = &mut loaded.as_mut().expect("model was loaded above").1; + model + .set_crop_size(PHOTO_CROP_SIZE) + .map_err(|error| format!("could not configure the body model: {error}"))?; + let packet = model + .infer(&rgb, width, height, None) + .map_err(|error| format!("body inference failed: {error}"))?; + let person = packet + .people + .first() + .ok_or_else(|| "no person found".to_string())?; + let posed = Arc::new(posed_vertices( + model, + &person.shape, + &person.expr, + person.mhr, + person.global_rot, + )?); + let vertices = model.rig().rest_vertices(&person.shape, &person.expr); + let face_indices = model + .weights + .i64_shaped("head_pose.faces", &[36_874, 3]) + .map_err(|error| format!("could not read the body mesh faces: {error}"))?; + let mesh = Arc::new(body_mesh_from_flat(&vertices, &face_indices)?); + + emit(messages, PipelineMessage::Stage("measuring…".to_string())); + let measured = measure( + &mesh, + &MeasureOptions { + height_cm: request.height_cm, + }, + ) + .map_err(|error| error.to_string())?; + let pose_mapping = map_measurements_to_vertices(&mesh, &measured); + emit( + messages, + PipelineMessage::Done { + measured: Box::new(measured), + mesh, + posed: Some(posed), + pose_mapping, + reset_pose: true, + }, + ); + Ok(()) +} + +fn ensure_model( + weights: &Path, + messages: &Sender, + loaded: &mut Option<(PathBuf, BodyModel)>, +) -> Result<(), String> { + if loaded + .as_ref() + .map(|(path, _)| path != weights) + .unwrap_or(true) + { + emit( + messages, + PipelineMessage::Stage("loading model 2.8 GB…".to_string()), + ); + let model = BodyModel::load(weights) + .map_err(|error| format!("could not load the body model: {error}"))?; + *loaded = Some((weights.to_path_buf(), model)); + } + Ok(()) +} + +fn run_live( + request: &LiveRequest, + messages: &Sender, + loaded: &mut Option<(PathBuf, BodyModel)>, + stop: &AtomicBool, +) -> Result<(), String> { + if stop.load(Ordering::Acquire) { + return Ok(()); + } + ensure_model(&request.weights, messages, loaded)?; + let model = &mut loaded.as_mut().expect("model was loaded above").1; + model + .set_crop_size(LIVE_CROP_SIZE) + .map_err(|error| format!("could not configure live body inference: {error}"))?; + let face_indices = model + .weights + .i64_shaped("head_pose.faces", &[36_874, 3]) + .map_err(|error| format!("could not read the body mesh faces: {error}"))?; + + let started = Instant::now(); + let mut previous_bbox = None; + let mut smoother = ShapeSmoother::default(); + let mut pose_smoother = PoseSmoother::default(); + let mut fps = FpsCounter::default(); + let mut reset_pose = true; + + while !stop.load(Ordering::Acquire) { + request.mailbox.request(); + let wait_started = Instant::now(); + let frame = loop { + if stop.load(Ordering::Acquire) { + return Ok(()); + } + if let Some(frame) = request.mailbox.take() { + break Some(frame); + } + if wait_started.elapsed() >= FRAME_TIMEOUT { + break None; + } + thread::sleep(FRAME_POLL); + }; + let Some(frame) = frame else { + if !emit( + messages, + PipelineMessage::Stage("no camera frames".to_string()), + ) { + return Ok(()); + } + continue; + }; + + let model_started = Instant::now(); + let packet = model + .infer(&frame.rgb, frame.width, frame.height, previous_bbox) + .map_err(|error| format!("live body inference failed: {error}"))?; + let model_ms = model_started.elapsed().as_secs_f32() * 1000.0; + let now = started.elapsed(); + let person = packet.people.first(); + let bbox = person.map(|person| person.bbox); + let mut pose_ms = 0.0; + let mut posed = None; + let smoothed_shape = if let Some(person) = person { + previous_bbox = Some(expand_bbox( + person.bbox, + frame.width, + frame.height, + 0.15, + )); + let (mhr, global_rot) = pose_smoother.observe(person.mhr, person.global_rot); + let pose_started = Instant::now(); + posed = Some(Arc::new(posed_vertices( + model, + &person.shape, + &person.expr, + mhr, + global_rot, + )?)); + pose_ms = pose_started.elapsed().as_secs_f32() * 1000.0; + Some(smoother.observe_person(person.shape, now)) + } else { + previous_bbox = None; + smoother.observe_miss(now); + pose_smoother.reset(); + None + }; + let current_fps = fps.tick(now); + if !emit( + messages, + PipelineMessage::LiveFrame { + fps: current_fps, + model_ms, + pose_ms, + person: person.is_some(), + bbox, + }, + ) { + return Ok(()); + } + if stop.load(Ordering::Acquire) { + return Ok(()); + } + + let Some(shape) = smoothed_shape else { + continue; + }; + let expression = [0.0f32; 72]; + let vertices = model.rig().rest_vertices(&shape, &expression); + let mesh = Arc::new(body_mesh_from_flat(&vertices, &face_indices)?); + let measured = measure( + &mesh, + &MeasureOptions { + height_cm: request.height_cm, + }, + ) + .map_err(|error| error.to_string())?; + let pose_mapping = map_measurements_to_vertices(&mesh, &measured); + if !emit( + messages, + PipelineMessage::Done { + measured: Box::new(measured), + mesh, + posed, + pose_mapping, + reset_pose, + }, + ) { + return Ok(()); + } + reset_pose = false; + } + Ok(()) +} + +#[derive(Default)] +struct PoseSmoother { + mhr: Option<[f32; 204]>, + global_rot: Option<[f32; 3]>, +} + +impl PoseSmoother { + fn observe( + &mut self, + mhr: [f32; 204], + global_rot: [f32; 3], + ) -> ([f32; 204], [f32; 3]) { + let global_rot = match self.global_rot { + Some(previous) => std::array::from_fn(|index| { + previous[index] + POSE_ALPHA * (global_rot[index] - previous[index]) + }), + None => global_rot, + }; + let mut mhr = match self.mhr { + Some(previous) => std::array::from_fn(|index| { + previous[index] + POSE_ALPHA * (mhr[index] - previous[index]) + }), + None => mhr, + }; + // The packet's 204 values are exactly the rig's [pose 136 | scales 68] + // input. Global rotation is pose slots 3..6; keep the separately + // smoothed copy authoritative before MhrRig::forward pads to 249. + mhr[3..6].copy_from_slice(&global_rot); + self.mhr = Some(mhr); + self.global_rot = Some(global_rot); + (mhr, global_rot) + } + + fn reset(&mut self) { + self.mhr = None; + self.global_rot = None; + } +} + +#[derive(Default)] +struct ShapeSmoother { + shape: Option<[f32; 45]>, + last_person: Option, +} + +impl ShapeSmoother { + fn observe_person(&mut self, shape: [f32; 45], now: Duration) -> [f32; 45] { + if self + .last_person + .is_some_and(|last| now.saturating_sub(last) > SHAPE_RESET_GAP) + { + self.shape = None; + } + self.last_person = Some(now); + let smoothed = match self.shape { + Some(previous) => std::array::from_fn(|index| { + previous[index] + SHAPE_ALPHA * (shape[index] - previous[index]) + }), + None => shape, + }; + self.shape = Some(smoothed); + smoothed + } + + fn observe_miss(&mut self, now: Duration) { + if self + .last_person + .is_some_and(|last| now.saturating_sub(last) > SHAPE_RESET_GAP) + { + self.shape = None; + self.last_person = None; + } + } +} + +#[derive(Default)] +struct FpsCounter { + samples: VecDeque, +} + +impl FpsCounter { + fn tick(&mut self, now: Duration) -> f32 { + self.samples.push_back(now); + while self.samples.len() > 10 { + self.samples.pop_front(); + } + let Some(first) = self.samples.front().copied() else { + return 0.0; + }; + let seconds = now.saturating_sub(first).as_secs_f32(); + if seconds <= f32::EPSILON { + 0.0 + } else { + (self.samples.len().saturating_sub(1)) as f32 / seconds + } + } +} + +pub(crate) fn expand_bbox( + bbox: [f32; 4], + width: u32, + height: u32, + amount: f32, +) -> [f32; 4] { + let box_width = (bbox[2] - bbox[0]).max(0.0); + let box_height = (bbox[3] - bbox[1]).max(0.0); + [ + (bbox[0] - box_width * amount).clamp(0.0, width as f32), + (bbox[1] - box_height * amount).clamp(0.0, height as f32), + (bbox[2] + box_width * amount).clamp(0.0, width as f32), + (bbox[3] + box_height * amount).clamp(0.0, height as f32), + ] +} + +fn posed_vertices( + model: &BodyModel, + shape: &[f32; 45], + expression: &[f32; 72], + mut mhr: [f32; 204], + global_rot: [f32; 3], +) -> Result, String> { + // BodyPerson::mhr is already model_params(): pose 0..136 followed by + // scales 136..204. The root translation remains zero and the model's + // global rotation occupies 3..6. forward() pads the 45 identity slots + // to the rig's 249-wide internal vector, then returns rig-space cm. + mhr[3..6].copy_from_slice(&global_rot); + let output = model.rig().forward(shape, &mhr, expression, true); + vertices_from_flat(&output.verts) +} + +fn decode_rgb(path: &Path) -> Result<(Vec, u32, u32), String> { + let bytes = std::fs::read(path) + .map_err(|error| format!("could not read {}: {error}", path.display()))?; + let extension = path + .extension() + .and_then(|extension| extension.to_str()) + .unwrap_or_default() + .to_ascii_lowercase(); + let image = match extension.as_str() { + "jpg" | "jpeg" => ImageBuffer::from_jpg(&bytes), + "png" => ImageBuffer::from_png(&bytes), + _ => return Err("choose a JPG or PNG photo".to_string()), + } + .map_err(|error| format!("could not decode {}: {error}", path.display()))?; + let pixel_count = image + .width + .checked_mul(image.height) + .ok_or_else(|| "photo dimensions are too large".to_string())?; + if image.data.len() < pixel_count { + return Err("decoded photo has too few pixels".to_string()); + } + let width = u32::try_from(image.width).map_err(|_| "photo is too wide".to_string())?; + let height = u32::try_from(image.height).map_err(|_| "photo is too tall".to_string())?; + Ok((argb_to_rgb(&image.data[..pixel_count]), width, height)) +} + +pub(crate) fn argb_to_rgb(pixels: &[u32]) -> Vec { + let mut rgb = Vec::with_capacity(pixels.len() * 3); + for pixel in pixels { + rgb.push((pixel >> 16) as u8); + rgb.push((pixel >> 8) as u8); + rgb.push(*pixel as u8); + } + rgb +} + +pub(crate) fn faces_i64_to_u32( + values: &[i64], + vertex_count: usize, +) -> Result, String> { + if values.len() % 3 != 0 { + return Err("body face index buffer is not made of triangles".to_string()); + } + values + .chunks_exact(3) + .enumerate() + .map(|(face_index, triangle)| { + let mut face = [0; 3]; + for corner in 0..3 { + let index = usize::try_from(triangle[corner]).map_err(|_| { + format!("body face {face_index} contains a negative vertex index") + })?; + if index >= vertex_count { + return Err(format!( + "body face {face_index} references vertex {index}, but there are {vertex_count} vertices" + )); + } + face[corner] = u32::try_from(index) + .map_err(|_| format!("body vertex index {index} exceeds u32"))?; + } + Ok(face) + }) + .collect() +} + +pub(crate) fn body_mesh_from_flat( + vertices: &[f32], + face_indices: &[i64], +) -> Result { + let vertices = vertices_from_flat(vertices)?; + let faces = faces_i64_to_u32(face_indices, vertices.len())?; + Ok(BodyMesh { + vertices, + faces, + landmarks: None, + }) +} + +fn vertices_from_flat(vertices: &[f32]) -> Result, String> { + if vertices.is_empty() || vertices.len() % 3 != 0 { + return Err("body vertex buffer is empty or malformed".to_string()); + } + Ok(vertices + .chunks_exact(3) + .map(|point| [point[0], point[1], point[2]]) + .collect()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn converts_aarrggbb_to_rgb() { + assert_eq!( + argb_to_rgb(&[0xff_12_34_56, 0x00_ab_cd_ef]), + vec![0x12, 0x34, 0x56, 0xab, 0xcd, 0xef] + ); + } + + #[test] + fn builds_a_tiny_checked_mesh() { + let mesh = body_mesh_from_flat( + &[0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0], + &[0, 1, 2], + ) + .unwrap(); + assert_eq!(mesh.vertices.len(), 3); + assert_eq!(mesh.faces, vec![[0, 1, 2]]); + assert!(faces_i64_to_u32(&[0, 1, 3], 3).is_err()); + assert!(faces_i64_to_u32(&[-1, 1, 2], 3).is_err()); + } + + #[test] + fn live_shape_ema_converges_and_resets_after_a_gap() { + let mut smoother = ShapeSmoother::default(); + assert_eq!( + smoother.observe_person([0.0; 45], Duration::ZERO), + [0.0; 45] + ); + let first = smoother.observe_person([10.0; 45], Duration::from_millis(250)); + assert!((first[0] - 3.5).abs() < 0.0001); + let second = smoother.observe_person([10.0; 45], Duration::from_millis(500)); + assert!((second[0] - 5.775).abs() < 0.0001); + + smoother.observe_miss(Duration::from_millis(1_501)); + let reset = smoother.observe_person([8.0; 45], Duration::from_millis(1_750)); + assert_eq!(reset, [8.0; 45]); + } + + #[test] + fn live_pose_ema_smooths_parameters_and_keeps_global_slots_in_sync() { + let mut smoother = PoseSmoother::default(); + let mut initial_mhr = [0.0; 204]; + initial_mhr[20] = 2.0; + let (initial, initial_rot) = smoother.observe(initial_mhr, [1.0, 2.0, 3.0]); + assert_eq!(initial[3..6], initial_rot); + + let mut next_mhr = [10.0; 204]; + next_mhr[20] = 12.0; + let (smoothed, smoothed_rot) = smoother.observe(next_mhr, [3.0, 4.0, 5.0]); + assert!((smoothed[20] - 8.0).abs() < 0.0001); + assert_eq!(smoothed_rot, [2.2, 3.2, 4.2]); + assert_eq!(smoothed[3..6], smoothed_rot); + + smoother.reset(); + let (reset, _) = smoother.observe([7.0; 204], [0.5, 1.0, 1.5]); + assert_eq!(reset[20], 7.0); + } + + #[test] + fn bbox_expansion_is_fifteen_percent_and_clamped() { + let expanded = expand_bbox([100.0, 50.0, 300.0, 250.0], 640, 360, 0.15); + for (actual, expected) in expanded.into_iter().zip([70.0, 20.0, 330.0, 280.0]) { + assert!((actual - expected).abs() < 0.0001, "{expanded:?}"); + } + assert_eq!( + expand_bbox([5.0, 10.0, 635.0, 350.0], 640, 360, 0.15), + [0.0, 0.0, 640.0, 360.0] + ); + } + + #[test] + fn fps_counter_tracks_recent_frame_cadence() { + let mut counter = FpsCounter::default(); + assert_eq!(counter.tick(Duration::ZERO), 0.0); + for quarter in 1..=4 { + let fps = counter.tick(Duration::from_millis(quarter * 250)); + assert!((fps - 4.0).abs() < 0.0001, "{fps}"); + } + } +} diff --git a/apps/files/Cargo.toml b/apps/files/Cargo.toml new file mode 100644 index 000000000..07ff16163 --- /dev/null +++ b/apps/files/Cargo.toml @@ -0,0 +1,25 @@ +[package] +name = "makepad-files" +version = "0.1.0" +edition = "2021" + +[[bin]] +name = "files" +path = "src/main.rs" + +[features] +default = ["chat"] +chat = ["dep:makepad-ai-hub"] +demo = [] + +[dependencies] +makepad-widgets = { path = "../../widgets" } +makepad-wm-theme = { path = "../../libs/wm_theme" } +makepad-wm-api = { path = "../../libs/wm_api" } +# The AI services wire: bounded read and confirmed mutation tools on the bus +# (hosted port, id-correlated runner — see ai_service.rs). +makepad-ai-services = { path = "../../libs/ai/services" } +makepad-strict-json = { path = "../../libs/strict_json" } +# The ask panel's local model: an in-process Qwen GGUF on makepad-ggml, loaded +# only when the panel is first opened. +makepad-ai-hub = { path = "../../libs/ai/hub", default-features = false, features = ["llm"], optional = true } diff --git a/apps/files/build.rs b/apps/files/build.rs new file mode 100644 index 000000000..6b130ff49 --- /dev/null +++ b/apps/files/build.rs @@ -0,0 +1,64 @@ +#[path = "../../libs/ai/models/paint/src/png.rs"] +#[allow(dead_code)] +mod png; + +use std::{env, fs, path::Path}; + +const WIDTH: u32 = 256; +const HEIGHT: u32 = 128; + +const PALETTES: [(&str, [u8; 3], [u8; 3], [u8; 3]); 6] = [ + ("aurora-vignette.png", [28, 31, 78], [69, 175, 170], [227, 151, 232]), + ("canyon-vignette.png", [86, 35, 38], [221, 119, 70], [255, 211, 128]), + ("lagoon-vignette.png", [12, 55, 77], [28, 154, 166], [151, 232, 207]), + ("meadow-vignette.png", [36, 68, 45], [124, 167, 79], [234, 213, 126]), + ("twilight-vignette.png", [36, 25, 65], [103, 71, 141], [238, 144, 116]), + ("cinema-still.png", [19, 24, 39], [56, 75, 105], [235, 176, 91]), +]; + +fn mix(a: u8, b: u8, amount: u32) -> u32 { + (u32::from(a) * (255 - amount) + u32::from(b) * amount) / 255 +} + +fn picture(top: [u8; 3], bottom: [u8; 3], accent: [u8; 3], seed: u32) -> Vec { + let mut pixels = Vec::with_capacity((WIDTH * HEIGHT * 3) as usize); + let glow_x = 36 + seed * 31; + let glow_y = 22 + (seed * 17) % 54; + for y in 0..HEIGHT { + for x in 0..WIDTH { + let blend = (y * 190 / (HEIGHT - 1) + x * 65 / (WIDTH - 1)).min(255); + let edge_x = (x as i32 * 2 - (WIDTH - 1) as i32).unsigned_abs(); + let edge_y = (y as i32 * 2 - (HEIGHT - 1) as i32).unsigned_abs(); + let vignette = edge_x * edge_x * 18 / ((WIDTH - 1) * (WIDTH - 1)) + + edge_y * edge_y * 24 / ((HEIGHT - 1) * (HEIGHT - 1)); + let glow_dx = (x as i32 - glow_x as i32).unsigned_abs(); + let glow_dy = (y as i32 - glow_y as i32).unsigned_abs(); + let glow = 72u32.saturating_sub(glow_dx * glow_dx / 360 + glow_dy * glow_dy / 90); + let texture = ((x * 17 + y * 31 + seed * 43) & 7) as i32 - 3; + + for channel in 0..3 { + let base = mix(top[channel], bottom[channel], blend); + let lit = (base * (72 - glow) + u32::from(accent[channel]) * glow) / 72; + pixels.push((lit as i32 - vignette as i32 + texture).clamp(0, 255) as u8); + } + } + } + pixels +} + +fn write_if_changed(path: &Path, bytes: &[u8]) { + if fs::read(path).ok().as_deref() != Some(bytes) { + fs::write(path, bytes).expect("write generated files demo picture"); + } +} + +fn main() { + println!("cargo:rerun-if-changed=../../libs/ai/models/paint/src/png.rs"); + println!("cargo:rerun-if-changed=build.rs"); + let out_dir = env::var_os("OUT_DIR").expect("Cargo did not set OUT_DIR"); + for (seed, (name, top, bottom, accent)) in PALETTES.iter().copied().enumerate() { + let pixels = picture(top, bottom, accent, seed as u32); + let encoded = png::encode_png(WIDTH, HEIGHT, png::PngColor::Rgb, &pixels); + write_if_changed(&Path::new(&out_dir).join(name), &encoded); + } +} diff --git a/apps/files/demos/amusement-ride.jpg b/apps/files/demos/amusement-ride.jpg new file mode 100644 index 000000000..a4c55fe62 Binary files /dev/null and b/apps/files/demos/amusement-ride.jpg differ diff --git a/apps/files/demos/royal-esplanade-panorama.jpg b/apps/files/demos/royal-esplanade-panorama.jpg new file mode 100644 index 000000000..41e66b6c9 Binary files /dev/null and b/apps/files/demos/royal-esplanade-panorama.jpg differ diff --git a/apps/files/demos/rubber-duck-illustration.png b/apps/files/demos/rubber-duck-illustration.png new file mode 100644 index 000000000..21c64d294 Binary files /dev/null and b/apps/files/demos/rubber-duck-illustration.png differ diff --git a/apps/mpfiles/resources/icons/archive.svg b/apps/files/resources/icons/archive.svg similarity index 100% rename from apps/mpfiles/resources/icons/archive.svg rename to apps/files/resources/icons/archive.svg diff --git a/apps/mpfiles/resources/icons/audio.svg b/apps/files/resources/icons/audio.svg similarity index 100% rename from apps/mpfiles/resources/icons/audio.svg rename to apps/files/resources/icons/audio.svg diff --git a/apps/mpfiles/resources/icons/back.svg b/apps/files/resources/icons/back.svg similarity index 100% rename from apps/mpfiles/resources/icons/back.svg rename to apps/files/resources/icons/back.svg diff --git a/apps/mpfiles/resources/icons/bookmark.svg b/apps/files/resources/icons/bookmark.svg similarity index 100% rename from apps/mpfiles/resources/icons/bookmark.svg rename to apps/files/resources/icons/bookmark.svg diff --git a/apps/mpfiles/resources/icons/chat.svg b/apps/files/resources/icons/chat.svg similarity index 100% rename from apps/mpfiles/resources/icons/chat.svg rename to apps/files/resources/icons/chat.svg diff --git a/apps/mpfiles/resources/icons/check.svg b/apps/files/resources/icons/check.svg similarity index 100% rename from apps/mpfiles/resources/icons/check.svg rename to apps/files/resources/icons/check.svg diff --git a/apps/mpfiles/resources/icons/clock.svg b/apps/files/resources/icons/clock.svg similarity index 100% rename from apps/mpfiles/resources/icons/clock.svg rename to apps/files/resources/icons/clock.svg diff --git a/apps/mpfiles/resources/icons/close.svg b/apps/files/resources/icons/close.svg similarity index 100% rename from apps/mpfiles/resources/icons/close.svg rename to apps/files/resources/icons/close.svg diff --git a/apps/mpfiles/resources/icons/code.svg b/apps/files/resources/icons/code.svg similarity index 100% rename from apps/mpfiles/resources/icons/code.svg rename to apps/files/resources/icons/code.svg diff --git a/apps/mpfiles/resources/icons/compact.svg b/apps/files/resources/icons/compact.svg similarity index 100% rename from apps/mpfiles/resources/icons/compact.svg rename to apps/files/resources/icons/compact.svg diff --git a/apps/mpfiles/resources/icons/delete-forever.svg b/apps/files/resources/icons/delete-forever.svg similarity index 100% rename from apps/mpfiles/resources/icons/delete-forever.svg rename to apps/files/resources/icons/delete-forever.svg diff --git a/apps/mpfiles/resources/icons/eye.svg b/apps/files/resources/icons/eye.svg similarity index 100% rename from apps/mpfiles/resources/icons/eye.svg rename to apps/files/resources/icons/eye.svg diff --git a/apps/mpfiles/resources/icons/file.svg b/apps/files/resources/icons/file.svg similarity index 100% rename from apps/mpfiles/resources/icons/file.svg rename to apps/files/resources/icons/file.svg diff --git a/apps/mpfiles/resources/icons/filter.svg b/apps/files/resources/icons/filter.svg similarity index 100% rename from apps/mpfiles/resources/icons/filter.svg rename to apps/files/resources/icons/filter.svg diff --git a/apps/mpfiles/resources/icons/folder.svg b/apps/files/resources/icons/folder.svg similarity index 100% rename from apps/mpfiles/resources/icons/folder.svg rename to apps/files/resources/icons/folder.svg diff --git a/apps/mpfiles/resources/icons/forward.svg b/apps/files/resources/icons/forward.svg similarity index 100% rename from apps/mpfiles/resources/icons/forward.svg rename to apps/files/resources/icons/forward.svg diff --git a/apps/mpfiles/resources/icons/grid.svg b/apps/files/resources/icons/grid.svg similarity index 100% rename from apps/mpfiles/resources/icons/grid.svg rename to apps/files/resources/icons/grid.svg diff --git a/apps/mpfiles/resources/icons/home.svg b/apps/files/resources/icons/home.svg similarity index 100% rename from apps/mpfiles/resources/icons/home.svg rename to apps/files/resources/icons/home.svg diff --git a/apps/mpfiles/resources/icons/image.svg b/apps/files/resources/icons/image.svg similarity index 100% rename from apps/mpfiles/resources/icons/image.svg rename to apps/files/resources/icons/image.svg diff --git a/apps/mpfiles/resources/icons/info.svg b/apps/files/resources/icons/info.svg similarity index 100% rename from apps/mpfiles/resources/icons/info.svg rename to apps/files/resources/icons/info.svg diff --git a/apps/mpfiles/resources/icons/list.svg b/apps/files/resources/icons/list.svg similarity index 100% rename from apps/mpfiles/resources/icons/list.svg rename to apps/files/resources/icons/list.svg diff --git a/apps/mpfiles/resources/icons/menu-dots.svg b/apps/files/resources/icons/menu-dots.svg similarity index 100% rename from apps/mpfiles/resources/icons/menu-dots.svg rename to apps/files/resources/icons/menu-dots.svg diff --git a/apps/mpfiles/resources/icons/network.svg b/apps/files/resources/icons/network.svg similarity index 100% rename from apps/mpfiles/resources/icons/network.svg rename to apps/files/resources/icons/network.svg diff --git a/apps/mpfiles/resources/icons/newfolder.svg b/apps/files/resources/icons/newfolder.svg similarity index 100% rename from apps/mpfiles/resources/icons/newfolder.svg rename to apps/files/resources/icons/newfolder.svg diff --git a/apps/mpfiles/resources/icons/pdf.svg b/apps/files/resources/icons/pdf.svg similarity index 100% rename from apps/mpfiles/resources/icons/pdf.svg rename to apps/files/resources/icons/pdf.svg diff --git a/apps/mpfiles/resources/icons/reload.svg b/apps/files/resources/icons/reload.svg similarity index 100% rename from apps/mpfiles/resources/icons/reload.svg rename to apps/files/resources/icons/reload.svg diff --git a/apps/mpfiles/resources/icons/search.svg b/apps/files/resources/icons/search.svg similarity index 100% rename from apps/mpfiles/resources/icons/search.svg rename to apps/files/resources/icons/search.svg diff --git a/apps/mpfiles/resources/icons/star.svg b/apps/files/resources/icons/star.svg similarity index 100% rename from apps/mpfiles/resources/icons/star.svg rename to apps/files/resources/icons/star.svg diff --git a/apps/mpfiles/resources/icons/terminal.svg b/apps/files/resources/icons/terminal.svg similarity index 100% rename from apps/mpfiles/resources/icons/terminal.svg rename to apps/files/resources/icons/terminal.svg diff --git a/apps/mpfiles/resources/icons/text.svg b/apps/files/resources/icons/text.svg similarity index 100% rename from apps/mpfiles/resources/icons/text.svg rename to apps/files/resources/icons/text.svg diff --git a/apps/mpfiles/resources/icons/trash.svg b/apps/files/resources/icons/trash.svg similarity index 100% rename from apps/mpfiles/resources/icons/trash.svg rename to apps/files/resources/icons/trash.svg diff --git a/apps/mpfiles/resources/icons/treemap.svg b/apps/files/resources/icons/treemap.svg similarity index 100% rename from apps/mpfiles/resources/icons/treemap.svg rename to apps/files/resources/icons/treemap.svg diff --git a/apps/mpfiles/resources/icons/treemap25.svg b/apps/files/resources/icons/treemap25.svg similarity index 100% rename from apps/mpfiles/resources/icons/treemap25.svg rename to apps/files/resources/icons/treemap25.svg diff --git a/apps/mpfiles/resources/icons/treemap3d.svg b/apps/files/resources/icons/treemap3d.svg similarity index 100% rename from apps/mpfiles/resources/icons/treemap3d.svg rename to apps/files/resources/icons/treemap3d.svg diff --git a/apps/mpfiles/resources/icons/twist-down.svg b/apps/files/resources/icons/twist-down.svg similarity index 100% rename from apps/mpfiles/resources/icons/twist-down.svg rename to apps/files/resources/icons/twist-down.svg diff --git a/apps/mpfiles/resources/icons/twist-right.svg b/apps/files/resources/icons/twist-right.svg similarity index 100% rename from apps/mpfiles/resources/icons/twist-right.svg rename to apps/files/resources/icons/twist-right.svg diff --git a/apps/mpfiles/resources/icons/video.svg b/apps/files/resources/icons/video.svg similarity index 100% rename from apps/mpfiles/resources/icons/video.svg rename to apps/files/resources/icons/video.svg diff --git a/apps/files/src/ai_service.rs b/apps/files/src/ai_service.rs new file mode 100644 index 000000000..2ed56031d --- /dev/null +++ b/apps/files/src/ai_service.rs @@ -0,0 +1,250 @@ +//! The file browser on the desktop's AI bus. +//! +//! Hosted by the window manager, the app opens one [`AiServicePort`] with +//! the manifest from `chat_tools::service_manifest` and answers the calls +//! that come back through it. The tools are the same seven the app's own +//! panel has; what is new is how they run: every call carries the +//! engine's `call_id`, the answer carries it back, and the person (or the +//! router) can give up on a call mid-walk. The old panel's runner is +//! order-only and cannot do either, so this one sits beside it rather +//! than inside it, and the two never share a job. +//! +//! One worker thread, one job at a time. A job in flight is cancelled by +//! a flag the walk checks on every entry; a job still queued behind it is +//! cancelled before it starts. Progress from the walk and the finished +//! results come back on channels the UI drains on its signal. + +use std::{ + collections::HashMap, + path::PathBuf, + sync::{ + atomic::{AtomicBool, Ordering}, + mpsc::{channel, Receiver, Sender}, + Arc, + }, +}; + +use makepad_ai_services::wire::{ServiceCall, ToolResult}; +use makepad_strict_json as json; +use makepad_widgets::makepad_platform::thread::{SignalToUI, ThreadOptions, ThreadSpawner}; + +use crate::chat_tools::{run_with, ToolJob, ToolOutcome}; + +#[cfg(test)] +use std::thread; + +/// One call on its way to the worker. +struct ServiceJob { + call_id: String, + job: ToolJob, + cancel: Arc, +} + +/// What the worker sends back, in the order it happened. +pub enum ServiceReply { + Result { result: ToolResult, mutated: bool }, + Progress { call_id: String, note: String, permille: u16 }, +} + +/// The bus's tool worker: correlated by call id, cancellable. +pub struct ServiceRunner { + jobs: Sender, + replies: Receiver, + /// The cancel flag of every call not yet answered. + live: HashMap>, +} + +impl ServiceRunner { + pub fn new(spawner: &ThreadSpawner) -> Self { + let (jobs, job_rx) = channel::(); + let (reply_tx, replies) = channel::(); + if let Ok(handle) = spawner.spawn_worker( + ThreadOptions { name: Some("files-ai-service".into()), ..Default::default() }, + move || { + while let Ok(ServiceJob { call_id, job, cancel }) = job_rx.recv() { + let (result, mutated) = if cancel.load(Ordering::Relaxed) { + // Given up on while it waited its turn: never started. + (ToolResult::cancelled(&call_id), false) + } else { + let progress_tx = reply_tx.clone(); + let progress_id = call_id.clone(); + let progress = move |permille: u16| { + let _ = progress_tx.send(ServiceReply::Progress { + call_id: progress_id.clone(), + note: "measuring…".to_string(), + permille, + }); + SignalToUI::set_ui_signal(); + }; + let outcome = run_with(&job, &cancel, &progress); + if cancel.load(Ordering::Relaxed) { + // The walk stopped early on the flag: what it has + // is a floor nobody asked for any more. + (ToolResult::cancelled(&call_id), false) + } else { + let mutated = outcome.mutated; + (result_for(&call_id, outcome), mutated) + } + }; + if reply_tx.send(ServiceReply::Result { result, mutated }).is_err() { + return; + } + SignalToUI::set_ui_signal(); + } + }, + ) { + handle.detach(); + } + Self { jobs, replies, live: HashMap::new() } + } + + /// Queue one call. `cwd` is the folder the person is looking at — what + /// a relative path is read from — and `home` the jail. + pub fn submit(&mut self, call: &ServiceCall, cwd: PathBuf, home: PathBuf) { + let cancel = Arc::new(AtomicBool::new(false)); + self.live.insert(call.call_id.clone(), cancel.clone()); + let job = ToolJob { + name: call.tool.clone(), + args: flat_args(&call.args), + cwd, + home, + }; + let _ = self.jobs.send(ServiceJob { call_id: call.call_id.clone(), job, cancel }); + } + + /// Give up on one call, running or queued. Unknown ids are nothing. + pub fn cancel(&mut self, call_id: &str) { + if let Some(flag) = self.live.get(call_id) { + flag.store(true, Ordering::Relaxed); + } + } + + /// Everything the worker sent since the last drain. A result retires + /// its call's flag. + pub fn drain(&mut self) -> Vec { + let replies: Vec = self.replies.try_iter().collect(); + for reply in &replies { + if let ServiceReply::Result { result, .. } = reply { + self.live.remove(&result.call_id); + } + } + replies + } +} + +/// A tool's outcome as the wire says it. The tools already sort their own +/// failures into "refused" (the jail, an unknown name) and "could not" +/// (the disk said no); the wire keeps that distinction. +fn result_for(call_id: &str, outcome: ToolOutcome) -> ToolResult { + if !outcome.is_error { + return ToolResult::ok(call_id, outcome.text, outcome.note); + } + if outcome.note.starts_with("refused") || outcome.note.starts_with("unknown tool") { + ToolResult::refused(call_id, outcome.text) + } else { + ToolResult::failed(call_id, outcome.text) + } +} + +/// The call's JSON argument object as the tools read it: one string per +/// key. Numbers and booleans become their text; nested values are dropped, +/// since no tool here takes one. +pub fn flat_args(args: &str) -> Vec<(String, String)> { + let Ok(json::Value::Obj(fields)) = json::parse(args.as_bytes()) else { + return Vec::new(); + }; + fields + .into_iter() + .filter_map(|(key, value)| { + let text = match value { + json::Value::Str(s) => s, + json::Value::Int(i) => i.to_string(), + json::Value::F64(f) => f.to_string(), + json::Value::Bool(b) => b.to_string(), + _ => return None, + }; + Some((key, text)) + }) + .collect() +} + +#[cfg(test)] +// Native worker tests use std deadlines while exercising blocking threads. +#[allow(clippy::disallowed_types, clippy::disallowed_methods)] +mod tests { + use super::*; + use makepad_ai_services::wire::ToolOutcome as Outcome; + use std::time::{Duration, Instant}; + + fn call(id: &str, tool: &str, args: &str) -> ServiceCall { + ServiceCall { call_id: id.into(), tool: tool.into(), args: args.into() } + } + + fn wait_for(runner: &mut ServiceRunner, n: usize) -> Vec { + let deadline = Instant::now() + Duration::from_secs(20); + let mut out = Vec::new(); + while out.len() < n && Instant::now() < deadline { + for reply in runner.drain() { + if let ServiceReply::Result { result: r, .. } = reply { + out.push(r); + } + } + thread::sleep(Duration::from_millis(5)); + } + out + } + + #[test] + fn json_args_become_the_tools_flat_pairs() { + let args = flat_args(r#"{"path":"~/Downloads","top":3,"deep":{"x":1},"quick":true}"#); + assert_eq!( + args, + vec![ + ("path".to_string(), "~/Downloads".to_string()), + ("top".to_string(), "3".to_string()), + ("quick".to_string(), "true".to_string()), + ] + ); + assert!(flat_args("not json").is_empty()); + assert!(flat_args("[1,2]").is_empty()); + } + + #[test] + fn results_carry_their_own_call_ids_and_a_queued_cancel_never_runs() { + let home = crate::model::home_dir(); + let spawner = makepad_widgets::Cx::new(Box::new(|_, _| {})).thread_spawner(); + let mut runner = ServiceRunner::new(&spawner); + // An unknown tool is refused; a jail escape is refused; both keep + // their ids whatever order the worker answers in. + runner.submit(&call("a", "stat", r#"{"path":"/etc/passwd"}"#), home.clone(), home.clone()); + runner.submit(&call("b", "rm_rf", r#"{"path":"~"}"#), home.clone(), home.clone()); + runner.submit(&call("c", "stat", r#"{"path":"~"}"#), home.clone(), home.clone()); + // Cancelled while it is still queued: it must come back cancelled + // without the tool ever running. + runner.cancel("c"); + assert_eq!(runner.live.len(), 3); + let results = wait_for(&mut runner, 3); + assert_eq!(results.len(), 3); + let by_id = |id: &str| results.iter().find(|r| r.call_id == id).unwrap(); + assert_eq!(by_id("a").outcome, Outcome::Refused); + assert!(by_id("a").text.contains("refused")); + assert_eq!(by_id("b").outcome, Outcome::Refused); + assert!(by_id("b").text.contains("no tool called")); + assert_eq!(by_id("c").outcome, Outcome::Cancelled); + assert_eq!(runner.live.len(), 0); + // An unknown id is nothing. + runner.cancel("zzz"); + } + + #[test] + fn a_folder_walk_stops_on_the_flag() { + let cancel = AtomicBool::new(true); + let started = Instant::now(); + let (bytes, files, complete) = crate::chat_tools::measure_for_test( + std::path::Path::new(env!("CARGO_MANIFEST_DIR")), + &cancel, + ); + assert_eq!((bytes, files, complete), (0, 0, false)); + assert!(started.elapsed() < Duration::from_secs(1)); + } +} diff --git a/apps/mpfiles/src/bookmarks.rs b/apps/files/src/bookmarks.rs similarity index 94% rename from apps/mpfiles/src/bookmarks.rs rename to apps/files/src/bookmarks.rs index 46a43c2e5..433f839a8 100644 --- a/apps/mpfiles/src/bookmarks.rs +++ b/apps/files/src/bookmarks.rs @@ -1,6 +1,6 @@ //! The sidebar's bookmarks: folders the user keeps, in their own section. //! -//! Bookmarks are the one piece of mpfiles state that outlives the process, so +//! Bookmarks are the one piece of files state that outlives the process, so //! the format is the one a person can fix in an editor when it goes wrong: one //! absolute path per line, in the order the sidebar shows them. That is also //! what GNOME Files stores (`~/.config/gtk-3.0/bookmarks`), minus the URI @@ -18,9 +18,9 @@ use std::{ /// last slot would be saved and never shown, which is worse than refusing it. pub const MAX_BOOKMARKS: usize = 12; -/// The bookmarks file for a given home directory. -pub fn config_path(home: &Path) -> PathBuf { - home.join(".config").join("mpfiles").join("bookmarks") +/// The bookmarks file for a given Makepad home directory. +pub fn config_path(makepad_home: &Path) -> PathBuf { + makepad_home.join("files/bookmarks") } /// The bookmark list, in sidebar order, and where it is persisted. @@ -160,7 +160,7 @@ mod tests { #[test] fn survives_a_round_trip_through_a_real_file() { - let home = std::env::temp_dir().join("mpfiles-test-bookmarks"); + let home = std::env::temp_dir().join("files-test-bookmarks"); let _ = fs::remove_dir_all(&home); fs::create_dir_all(&home).unwrap(); diff --git a/apps/mpfiles/src/chat_agent.rs b/apps/files/src/chat_agent.rs similarity index 94% rename from apps/mpfiles/src/chat_agent.rs rename to apps/files/src/chat_agent.rs index 911e06423..ca3125c1a 100644 --- a/apps/mpfiles/src/chat_agent.rs +++ b/apps/files/src/chat_agent.rs @@ -17,7 +17,7 @@ pub use makepad_ai_hub::local_llm::ChatEvent; /// Where the weights live, relative to the checkout this was built from. pub const MODEL_FILE: &str = "local/models/Qwen3.5-9B-UD-Q4_K_XL.gguf"; /// The environment variable that overrides it. -pub const MODEL_ENV: &str = "MPFILES_CHAT_MODEL"; +pub const MODEL_ENV: &str = "MAKEPAD_FILES_CHAT_MODEL"; pub struct ChatAgent { session: HubChatSession, @@ -57,7 +57,7 @@ impl ChatAgent { /// Where the weights are, or `None` when this machine has none. /// -/// `MPFILES_CHAT_MODEL` wins; otherwise the file is looked for relative to the +/// `MAKEPAD_FILES_CHAT_MODEL` wins; otherwise the file is looked for relative to the /// working directory, then up from the binary (which finds `target/release` /// runs from anywhere), then in the checkout this binary was compiled in. pub fn model_path() -> Option { diff --git a/apps/mpfiles/src/chat_panel.rs b/apps/files/src/chat_panel.rs similarity index 100% rename from apps/mpfiles/src/chat_panel.rs rename to apps/files/src/chat_panel.rs diff --git a/apps/files/src/chat_tools.rs b/apps/files/src/chat_tools.rs new file mode 100644 index 000000000..f52df2519 --- /dev/null +++ b/apps/files/src/chat_tools.rs @@ -0,0 +1,1019 @@ +//! What the chat panel's model is allowed to read and change. +//! +//! Four bounded read tools inspect paths; three mutation tools make a folder, +//! rename one item, or move one item to the platform Trash. On the desktop bus +//! those three wait for confirmation because their manifest risk is +//! `Destructive`. There is +//! no permanent delete or shell, and [`run`] is a closed match over the seven +//! names. +//! +//! Every path the model names goes through [`resolve`] first. It expands `~`, +//! folds `.` and `..` away *lexically* (so `~/../../etc` is refused before the +//! disk is touched at all), then canonicalises — which is what resolves any +//! symlink — and refuses anything that does not land inside the user's home. +//! A tool can therefore be handed any string at all and still only ever read +//! something the person running the app could already open in the browser. +//! +//! The tools run on a worker thread, never the UI's: measuring a folder is a +//! disk walk, and a file browser that stops painting because a chat is +//! counting bytes would be worse than one with no chat. Two callers share +//! them — the app's own panel through [`ToolRunner`] (in call order), and +//! the desktop's assistant through the bus runner in `ai_service.rs`, which +//! correlates by call id and can give up on a walk half-way ([`run_with`]). + +use std::{ + path::{Component, Path, PathBuf}, + sync::atomic::{AtomicBool, Ordering}, +}; +#[cfg(feature = "chat")] +use std::sync::mpsc::{channel, Receiver, Sender}; + +#[cfg(feature = "chat")] +use makepad_ai_hub::local_llm::ToolSpec; +use makepad_ai_services::wire::{Risk, ServiceManifest, ToolDef}; + +use crate::{ + model::{self, FileEntry}, + vfs::vfs, +}; + +/// The most entries one `list_dir` ever returns. A folder with ten thousand +/// files in it answers the question "what is in here" with the first two +/// hundred and a count, not with ten thousand lines of context. +const LIST_LIMIT: usize = 200; +/// The most bytes `read_file` will ever hand back. +const READ_LIMIT: usize = 16 * 1024; +/// The default, and the ceiling, for `treemap_summary`'s child count. +const SUMMARY_TOP: usize = 12; +/// How long one `treemap_summary` may spend walking before it answers with +/// what it has and says the numbers are a floor. +const MEASURE_BUDGET_SECS: f64 = 4.0; +/// How deep that walk goes, and how many entries it will look at. +const MEASURE_DEPTH: usize = 10; +const MEASURE_ENTRIES: usize = 400_000; + +/// The one table every description of the tools is built from: the old +/// panel's `ToolSpec`s and the desktop bus's manifest read the SAME name, +/// sentence and schema, so the two can never drift apart. Every schema is +/// an argument object (`"type":"object"`), which the wire insists on. +const TOOL_TABLE: [(&str, &str, &str, Risk); 7] = [ + ( + "list_dir", + "List what is directly inside a folder: each entry's name, whether it is a folder, its kind and its size. Bounded to the first 200 entries. Use this before saying anything about what a folder contains.", + r#"{"type":"object","properties":{"path":{"type":"string","description":"folder path; ~ means the home folder, and a relative path is read from the folder the user is in"}},"required":["path"]}"#, + Risk::Read, + ), + ( + "read_file", + "Read the beginning of a text file (at most 16 kB). Binary files are refused with a note of what they are instead. Use this to answer questions about what a file actually says.", + r#"{"type":"object","properties":{"path":{"type":"string"},"max_bytes":{"type":"integer","description":"how much to read, up to 16384"}},"required":["path"]}"#, + Risk::Read, + ), + ( + "stat", + "One path's kind, size and modification time. Cheap — use it when you only need to know what something is.", + r#"{"type":"object","properties":{"path":{"type":"string"}},"required":["path"]}"#, + Risk::Read, + ), + ( + "treemap_summary", + "Where a folder's bytes actually are: its heaviest direct children with their recursive sizes and file counts. This is what the treemap draws. Use it for 'what is taking up the space' questions.", + r#"{"type":"object","properties":{"path":{"type":"string"},"top":{"type":"integer","description":"how many children to list, up to 12"}},"required":["path"]}"#, + Risk::Read, + ), + ( + "mkdir", + "Create a folder inside the home-folder jail. Refuses an existing path.", + r#"{"type":"object","properties":{"path":{"type":"string","description":"new folder path"}},"required":["path"]}"#, + Risk::Destructive, + ), + ( + "rename", + "Rename one file or folder inside the home-folder jail. The new name must be a bare name and must not already exist.", + r#"{"type":"object","properties":{"path":{"type":"string"},"new_name":{"type":"string","description":"bare new name, with no path separators"}},"required":["path","new_name"]}"#, + Risk::Destructive, + ), + ( + "trash", + "Move one file or folder inside the home-folder jail to the platform Trash. It is never permanently deleted.", + r#"{"type":"object","properties":{"path":{"type":"string"}},"required":["path"]}"#, + Risk::Destructive, + ), +]; + +/// The tools, exactly as the panel's own model is told about them. +#[cfg(feature = "chat")] +pub fn tools() -> Vec { + TOOL_TABLE + .iter() + .map(|(name, description, schema, _)| ToolSpec::new(*name, *description, *schema)) + .collect() +} + +/// The same seven tools as the desktop assistant learns them over the bus +/// (`ai_service.rs`). Mutations are `Destructive`, so the router confirms +/// them before a call reaches this app. +pub fn service_manifest() -> ServiceManifest { + let mut manifest = ServiceManifest::new( + "files", + "Files", + "The file browser. Its tools list folders, read text, inspect metadata, measure folder sizes, create folders, rename items, and move items to Trash. Paths may be absolute, `~` for the home folder, or relative to the folder the person is looking at; anything outside the home is refused. Mutations require confirmation.", + ); + for (name, description, schema, risk) in TOOL_TABLE { + manifest = manifest.with_tool(ToolDef::new(name, description, schema, risk)); + } + manifest +} + +/// One tool call, as it goes to the worker. +pub struct ToolJob { + pub name: String, + pub args: Vec<(String, String)>, + /// The folder the user is looking at: what a relative path is read from. + pub cwd: PathBuf, + pub home: PathBuf, +} + +/// One tool call, as it comes back. +pub struct ToolOutcome { + /// The dim line the transcript shows — "looked at ~/local/maps — 12 entries". + pub note: String, + /// What the model is told. + pub text: String, + pub is_error: bool, + /// A successful filesystem mutation asks the UI to relist its folder. + pub mutated: bool, +} + +/// The tool worker: one thread, one job at a time, results in call order. +#[cfg(feature = "chat")] +pub struct ToolRunner { + jobs: Sender, + results: Receiver, +} + +#[cfg(feature = "chat")] +impl ToolRunner { + pub fn new(spawner: &makepad_widgets::makepad_platform::thread::ThreadSpawner) -> Self { + let (jobs, job_rx) = channel::(); + let (result_tx, results) = channel(); + if let Ok(handle) = spawner.spawn_worker( + makepad_widgets::makepad_platform::thread::ThreadOptions { + name: Some("files-chat-tools".into()), + ..Default::default() + }, + move || { + while let Ok(job) = job_rx.recv() { + if result_tx.send(run(&job)).is_err() { + return; + } + makepad_widgets::makepad_platform::thread::SignalToUI::set_ui_signal(); + } + }, + ) { + handle.detach(); + } + Self { jobs, results } + } + + pub fn submit(&self, job: ToolJob) { + let _ = self.jobs.send(job); + } + + pub fn drain(&self) -> Vec { + self.results.try_iter().collect() + } +} + +/// Run one tool. The whole of what the model can do to a filesystem. +#[cfg(any(test, feature = "chat"))] +pub fn run(job: &ToolJob) -> ToolOutcome { + run_with(job, &AtomicBool::new(false), &|_| {}) +} + +/// [`run`] for a caller that may give up on the call: `cancel` set from any +/// thread makes the folder walk return at once (the result is then a floor, +/// and the bus runner reports the call as cancelled rather than as an +/// answer), and `progress` hears a permille as `treemap_summary` finishes +/// each direct child — the only tool slow enough to be worth watching. +pub fn run_with(job: &ToolJob, cancel: &AtomicBool, progress: &dyn Fn(u16)) -> ToolOutcome { + run_with_vfs(job, vfs().as_ref(), cancel, progress) +} + +fn run_with_vfs( + job: &ToolJob, + fs: &dyn crate::vfs::Vfs, + cancel: &AtomicBool, + progress: &dyn Fn(u16), +) -> ToolOutcome { + let raw = arg(&job.args, "path"); + let resolved = resolve_with(raw, &job.home, &job.cwd, fs); + let path = match resolved { + Ok(path) => path, + Err(error) => { + return ToolOutcome { + note: format!("refused {}", short(Path::new(raw), &job.home)), + text: error, + is_error: true, + mutated: false, + } + } + }; + let shown = short(&path, &job.home); + match job.name.as_str() { + "list_dir" => finish(list_dir(fs, &path), format!("looked at {shown}"), shown), + "read_file" => { + let max = number(arg(&job.args, "max_bytes")).unwrap_or(READ_LIMIT); + finish(read_file(fs, &path, max), format!("read {shown}"), shown) + } + "stat" => finish(stat(fs, &path), format!("checked {shown}"), shown), + "treemap_summary" => { + let top = number(arg(&job.args, "top")) + .unwrap_or(SUMMARY_TOP) + .clamp(1, SUMMARY_TOP); + finish(summary(fs, &path, top, cancel, progress), format!("measured {shown}"), shown) + } + "mkdir" => mkdir(fs, &path, &shown), + "rename" => rename(fs, &path, arg(&job.args, "new_name"), &job.home, &shown), + "trash" => trash(fs, &path, &job.home, &shown), + other => ToolOutcome { + note: format!("unknown tool {other}"), + text: format!("there is no tool called {other}"), + is_error: true, + mutated: false, + }, + } +} + +/// A tool's result plus the one-line note the transcript shows. The note gets +/// the tool's own tail ("— 12 entries") when it succeeded. +fn finish(result: Result<(String, String), String>, verb: String, shown: String) -> ToolOutcome { + match result { + Ok((tail, text)) => ToolOutcome { + note: if tail.is_empty() { + verb + } else { + format!("{verb} — {tail}") + }, + text, + is_error: false, + mutated: false, + }, + Err(error) => ToolOutcome { + note: format!("could not read {shown}"), + text: error, + is_error: true, + mutated: false, + }, + } +} + +fn refused(shown: &str, text: impl Into) -> ToolOutcome { + ToolOutcome { + note: format!("refused {shown}"), + text: text.into(), + is_error: true, + mutated: false, + } +} + +fn mutation(result: Result, note: String, failed_note: String) -> ToolOutcome { + match result { + Ok(text) => ToolOutcome { note, text, is_error: false, mutated: true }, + Err(text) => ToolOutcome { + note: failed_note, + text, + is_error: true, + mutated: false, + }, + } +} + +// ------------------------------------------------------------- the sandbox + +/// The path the model named, as a real path inside the user's home — or an +/// explanation of why it is not going to get one. +pub fn resolve(raw: &str, home: &Path, cwd: &Path) -> Result { + resolve_with(raw, home, cwd, vfs().as_ref()) +} + +fn resolve_with(raw: &str, home: &Path, cwd: &Path, fs: &dyn crate::vfs::Vfs) -> Result { + let wanted = expand(raw, home, cwd); + // Lexically first, so a path that walks out of the home is refused without + // the disk being touched at all. + if !within(&wanted, home) { + return Err(format!( + "refused: {} is outside {} — this assistant only looks inside the home folder", + wanted.display(), + home.display() + )); + } + // Then for real: canonicalising is what follows a symlink, and a link out + // of the home is exactly the case the lexical check cannot see. + let real = match fs.canonicalize(&wanted) { + Ok(real) => real, + // A mutation may name a leaf that does not exist yet. Resolve the + // nearest existing ancestor so a symlink cannot smuggle that leaf + // outside the jail, then put the missing suffix back. + Err(_) if fs.is_demo() => wanted.clone(), + Err(first_error) => { + let mut ancestor = wanted.clone(); + let mut suffix = Vec::new(); + let canonical = loop { + let Some(name) = ancestor.file_name().map(|name| name.to_os_string()) else { + return Err(format!("{}: {first_error}", wanted.display())); + }; + suffix.push(name); + if !ancestor.pop() { + return Err(format!("{}: {first_error}", wanted.display())); + } + if let Ok(real) = fs.canonicalize(&ancestor) { + break real; + } + }; + suffix.into_iter().rev().fold(canonical, |path, name| path.join(name)) + } + }; + let real_home = fs.canonicalize(home).unwrap_or_else(|_| home.to_path_buf()); + if !within(&real, &real_home) { + return Err(format!( + "refused: {} leads outside {} — this assistant only looks inside the home folder", + wanted.display(), + home.display() + )); + } + Ok(real) +} + +/// `~`, relative paths and `.`/`..` folded away, without touching the disk. +pub fn expand(raw: &str, home: &Path, cwd: &Path) -> PathBuf { + let raw = raw.trim().trim_matches('"'); + let joined = if raw.is_empty() || raw == "." { + cwd.to_path_buf() + } else if raw == "~" { + home.to_path_buf() + } else if let Some(rest) = raw.strip_prefix("~/") { + home.join(rest) + } else { + let path = Path::new(raw); + if path.is_absolute() { + path.to_path_buf() + } else { + cwd.join(path) + } + }; + normalize(&joined) +} + +/// `.` and `..` resolved textually. `..` past the root stays at the root, +/// which is what every filesystem does and what keeps the check below honest. +fn normalize(path: &Path) -> PathBuf { + let mut out = PathBuf::new(); + for part in path.components() { + match part { + Component::CurDir => {} + Component::ParentDir => { + if !out.pop() { + // Nothing above the root: keep it, so the result stays + // absolute and the containment check still means something. + out.push(Component::RootDir.as_os_str()); + } + } + other => out.push(other.as_os_str()), + } + } + if out.as_os_str().is_empty() { + out.push(Component::RootDir.as_os_str()); + } + out +} + +/// Is `path` the home folder, or something inside it? +pub fn within(path: &Path, home: &Path) -> bool { + path == home || path.starts_with(home) +} + +/// `~/rest` for anything under the home, the full path otherwise. +pub fn short(path: &Path, home: &Path) -> String { + match path.strip_prefix(home) { + Ok(rest) if rest.as_os_str().is_empty() => "~".to_string(), + Ok(rest) => format!("~/{}", rest.display()), + Err(_) => path.display().to_string(), + } +} + +/// One argument by name; empty when the call did not give it. +fn arg<'a>(args: &'a [(String, String)], key: &str) -> &'a str { + args.iter() + .find(|(k, _)| k == key) + .map(|(_, v)| v.as_str()) + .unwrap_or("") +} + +/// The bus runner's test seam: one bounded walk with a cancel flag. +#[cfg(test)] +pub fn measure_for_test(path: &Path, cancel: &AtomicBool) -> (u64, u32, bool) { + let mut budget = MEASURE_ENTRIES; + measure( + vfs().as_ref(), + path, + makepad_widgets::Cx::monotonic_now() + MEASURE_BUDGET_SECS, + &mut budget, + 0, + cancel, + ) +} + +fn number(text: &str) -> Option { + text.trim().parse::().ok() +} + +// ---------------------------------------------------------------- the tools + +fn mkdir(fs: &dyn crate::vfs::Vfs, path: &Path, shown: &str) -> ToolOutcome { + if fs.exists(path) { + return refused(shown, format!("refused: {shown} already exists")); + } + mutation( + fs.mkdir(path).map(|()| format!("created folder {shown}")), + format!("created {shown}"), + format!("could not create {shown}"), + ) +} + +fn rename( + fs: &dyn crate::vfs::Vfs, + path: &Path, + new_name: &str, + home: &Path, + shown: &str, +) -> ToolOutcome { + if let Some(problem) = crate::rename::name_error(new_name) { + return refused(shown, format!("refused: {problem}")); + } + if new_name.contains('\\') { + return refused(shown, "refused: a name cannot contain a path separator"); + } + if path == home { + return refused(shown, "refused: the home folder itself cannot be renamed"); + } + if !fs.exists(path) { + return refused(shown, format!("refused: there is nothing at {shown}")); + } + let Some(parent) = path.parent() else { + return refused(shown, format!("refused: {shown} has no parent folder")); + }; + let target = parent.join(new_name); + if !within(&target, home) { + return refused(shown, "refused: the renamed path would leave the home folder"); + } + if fs.exists(&target) { + return refused( + shown, + format!("refused: {} already exists", short(&target, home)), + ); + } + let target_shown = short(&target, home); + mutation( + fs.rename(path, &target) + .map(|()| format!("renamed {shown} to {target_shown}")), + format!("renamed {shown} to {new_name}"), + format!("could not rename {shown}"), + ) +} + +fn trash(fs: &dyn crate::vfs::Vfs, path: &Path, home: &Path, shown: &str) -> ToolOutcome { + if path == home { + return refused(shown, "refused: the home folder itself cannot be trashed"); + } + if !fs.exists(path) { + return refused(shown, format!("refused: there is nothing at {shown}")); + } + let trash_dir = crate::ops::trash_dir(home); + let trash_dir = match resolve_with(&trash_dir.display().to_string(), home, home, fs) { + Ok(path) => path, + Err(error) => return refused(shown, error), + }; + if path == trash_dir { + return refused(shown, "refused: the Trash folder itself cannot be trashed"); + } + if !fs.exists(&trash_dir) { + if let Err(error) = fs.mkdir(&trash_dir) { + return mutation( + Err(error), + String::new(), + format!("could not reach Trash for {shown}"), + ); + } + } else if !fs.is_dir(&trash_dir) { + return refused(shown, "refused: the Trash path is not a folder"); + } + let Some(name) = path.file_name().and_then(|name| name.to_str()) else { + return refused(shown, format!("refused: {shown} has no movable name")); + }; + let target = unique_destination(fs, &trash_dir, name); + mutation( + fs.rename(path, &target) + .map(|()| format!("moved {shown} to Trash as {}", target.file_name().unwrap().to_string_lossy())), + format!("trashed {shown}"), + format!("could not trash {shown}"), + ) +} + +/// Collision handling identical to the operations engine: suffix before an +/// extension, or at the end for an extensionless name and a dotfile. +fn unique_destination(fs: &dyn crate::vfs::Vfs, dir: &Path, name: &str) -> PathBuf { + let candidate = dir.join(name); + if !fs.exists(&candidate) { + return candidate; + } + let path = Path::new(name); + let (stem, ext) = match (path.file_stem(), path.extension()) { + (Some(stem), Some(ext)) => ( + stem.to_string_lossy().into_owned(), + ext.to_string_lossy().into_owned(), + ), + _ => (name.to_string(), String::new()), + }; + let mut n = 2u64; + loop { + let candidate_name = if ext.is_empty() { + format!("{name} ({n})") + } else { + format!("{stem} ({n}).{ext}") + }; + let candidate = dir.join(candidate_name); + if !fs.exists(&candidate) { + return candidate; + } + n += 1; + } +} + +fn list_dir(fs: &dyn crate::vfs::Vfs, path: &Path) -> Result<(String, String), String> { + if !fs.is_dir(path) { + return Err(format!("{} is not a folder", path.display())); + } + let entries = fs.read_dir(path, false)?; + let total = entries.len(); + let mut out = format!("{} — {total} entries", path.display()); + if total > LIST_LIMIT { + out.push_str(&format!(" (first {LIST_LIMIT} shown)")); + } + out.push('\n'); + for entry in entries.iter().take(LIST_LIMIT) { + out.push_str(&format!( + "{} {:<10} {}\n", + if entry.is_dir { "dir " } else { "file" }, + entry.size_text(), + entry.name, + )); + } + Ok((format!("{total} entries"), out)) +} + +fn read_file(fs: &dyn crate::vfs::Vfs, path: &Path, max_bytes: usize) -> Result<(String, String), String> { + if fs.is_dir(path) { + return Err(format!( + "{} is a folder — use list_dir on it", + path.display() + )); + } + let size = fs.stat(path)?.size; + let data = fs.read_bytes(path, max_bytes.clamp(1, READ_LIMIT))?; + let kind = model::kind_for(path, false); + let looked_at = data.len().min(4096); + if data[..looked_at].contains(&0) { + return Ok(( + "binary".to_string(), + format!( + "{} is a {} of {} — not text, so there is nothing to read out of it here", + path.display(), + kind.label().to_lowercase(), + model::format_size(size, false), + ), + )); + } + let cut = data.len(); + let text = match std::str::from_utf8(&data[..cut]) { + Ok(text) => text.to_string(), + Err(error) if error.valid_up_to() > cut / 2 => { + String::from_utf8_lossy(&data[..error.valid_up_to()]).into_owned() + } + Err(_) => { + return Ok(( + "binary".to_string(), + format!( + "{} is a {} of {} — not text", + path.display(), + kind.label().to_lowercase(), + model::format_size(size, false), + ), + )) + } + }; + let mut out = format!( + "{} — {}{}\n", + path.display(), + model::format_size(size, false), + if (cut as u64) < size { + format!(", first {} shown", model::format_size(cut as u64, false)) + } else { + String::new() + }, + ); + out.push_str(&text); + Ok((model::format_size(cut as u64, false), out)) +} + +fn stat(fs: &dyn crate::vfs::Vfs, path: &Path) -> Result<(String, String), String> { + let entry = entry_for(fs, path)?; + let mut out = format!( + "{}\nkind: {}\nsize: {}\nmodified: {}", + path.display(), + entry.kind_text(), + if entry.is_dir { + entry.size_text() + } else { + model::format_size(entry.size, false) + }, + entry.modified_text(), + ); + if !entry.permissions.is_empty() { + out.push_str(&format!("\npermissions: {}", entry.permissions)); + } + Ok((entry.kind_text().to_lowercase(), out)) +} + +/// The entry for one path: straight off the disk when there is one, out of the +/// parent's listing otherwise (which is the only way the demo can answer). +fn entry_for(fs: &dyn crate::vfs::Vfs, path: &Path) -> Result { + if let Ok(entry) = fs.stat(path) { + return Ok(entry); + } + let parent = path + .parent() + .ok_or_else(|| format!("{} has no parent to look in", path.display()))?; + fs + .read_dir(parent, true)? + .into_iter() + .find(|e| e.path == path) + .ok_or_else(|| format!("there is nothing at {}", path.display())) +} + +fn summary( + fs: &dyn crate::vfs::Vfs, + path: &Path, + top: usize, + cancel: &AtomicBool, + progress: &dyn Fn(u16), +) -> Result<(String, String), String> { + if !fs.is_dir(path) { + // A file has no children; saying so beats an empty table. + return stat(fs, path); + } + let entries = fs.read_dir(path, false)?; + let deadline = makepad_widgets::Cx::monotonic_now() + MEASURE_BUDGET_SECS; + let mut budget = MEASURE_ENTRIES; + let mut measured: Vec<(String, u64, u32, bool)> = Vec::new(); + let mut complete = true; + for (index, entry) in entries.iter().enumerate() { + if entry.is_dir { + let (bytes, files, done) = measure(fs, &entry.path, deadline, &mut budget, 0, cancel); + complete &= done; + measured.push((entry.name.clone(), bytes, files, done)); + } else { + measured.push((entry.name.clone(), entry.size, 1, true)); + } + if cancel.load(Ordering::Relaxed) { + // The caller gave up: what is measured so far is a floor, said so + // below; the runner turns the whole answer into "cancelled". + complete = false; + break; + } + progress(((index + 1) * 1000 / entries.len().max(1)) as u16); + } + let total: u64 = measured.iter().map(|m| m.1).sum(); + let files: u32 = measured.iter().map(|m| m.2).sum(); + measured.sort_by(|a, b| b.1.cmp(&a.1)); + let shown = measured.len().min(top); + let mut out = format!( + "{} — {} in {} files across {} entries{}\n", + path.display(), + model::format_size(total, false), + files, + entries.len(), + if complete { + "" + } else { + " (the walk was cut short, so the sizes are a floor)" + }, + ); + for (name, bytes, count, done) in measured.iter().take(shown) { + out.push_str(&format!( + "{:>10}{} {:>5.1}% {} ({} files)\n", + model::format_size(*bytes, false), + if *done { " " } else { "+" }, + *bytes as f64 * 100.0 / total.max(1) as f64, + name, + count, + )); + } + if measured.len() > shown { + out.push_str(&format!("…and {} smaller\n", measured.len() - shown)); + } + Ok((format!("{} entries", entries.len()), out)) +} + +/// A folder's recursive bytes and file count, bounded by a deadline, an entry +/// budget and a depth. Returns false when it ran out of one of them — a number +/// that stopped early is a floor, and the caller says so rather than passing +/// it off as the answer. +fn measure( + fs: &dyn crate::vfs::Vfs, + path: &Path, + deadline: f64, + budget: &mut usize, + depth: usize, + cancel: &AtomicBool, +) -> (u64, u32, bool) { + if depth >= MEASURE_DEPTH + || *budget == 0 + || makepad_widgets::Cx::monotonic_now() >= deadline + || cancel.load(Ordering::Relaxed) + { + return (0, 0, false); + } + // Never walk through a link: the tree below it is somebody else's, and it + // can lead straight back to where we started. + if fs.is_symlink(path).unwrap_or(false) { + return (0, 0, true); + } + if model::skip_for_scan(path, &fs.home()) { + return (0, 0, true); + } + let Ok(entries) = fs.read_dir(path, true) else { + return (0, 0, true); + }; + let mut bytes = 0u64; + let mut files = 0u32; + let mut complete = true; + for entry in entries { + *budget = budget.saturating_sub(1); + if entry.is_dir { + let (child_bytes, child_files, done) = + measure(fs, &entry.path, deadline, budget, depth + 1, cancel); + bytes += child_bytes; + files += child_files; + complete &= done; + } else { + bytes += entry.size; + files += 1; + } + if *budget == 0 + || makepad_widgets::Cx::monotonic_now() >= deadline + || cancel.load(Ordering::Relaxed) + { + return (bytes, files, false); + } + } + (bytes, files, complete) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::vfs::Vfs; + + fn home() -> PathBuf { + PathBuf::from("/Users/someone") + } + + #[test] + fn a_tilde_path_lands_in_the_home() { + let cwd = home().join("Documents"); + assert_eq!( + expand("~/Documents/notes", &home(), &cwd), + home().join("Documents/notes") + ); + assert_eq!(expand("~", &home(), &cwd), home()); + // Nothing at all means "where the user is". + assert_eq!(expand("", &home(), &cwd), cwd); + } + + #[test] + fn a_relative_path_is_read_from_the_folder_the_user_is_in() { + let cwd = home().join("Pictures"); + assert_eq!(expand("holiday", &home(), &cwd), cwd.join("holiday")); + assert_eq!(expand("./holiday/..", &home(), &cwd), cwd); + } + + #[test] + fn dot_dot_is_folded_away_before_anything_is_read() { + let cwd = home().join("Documents"); + assert_eq!(expand("~/a/../b", &home(), &cwd), home().join("b")); + assert_eq!(expand("../Pictures", &home(), &cwd), home().join("Pictures")); + // Past the root it stops at the root rather than going negative. + assert_eq!(expand("/../../..", &home(), &cwd), PathBuf::from("/")); + } + + #[test] + fn paths_outside_the_home_are_refused() { + let cwd = home(); + for escape in [ + "/etc/passwd", + "~/../../etc/passwd", + "../../../etc", + "/Users/someone_else/Documents", + "/", + ] { + let error = resolve(escape, &home(), &cwd) + .expect_err(&format!("{escape} should have been refused")); + assert!( + error.contains("refused"), + "{escape} gave the wrong reason: {error}" + ); + } + } + + #[test] + fn a_sibling_whose_name_starts_with_the_home_is_not_inside_it() { + // The string "/Users/someone-backup" starts with "/Users/someone", + // and a prefix test on strings rather than components would let it in. + assert!(!within(Path::new("/Users/someone-backup/x"), &home())); + assert!(within(Path::new("/Users/someone/x"), &home())); + assert!(within(&home(), &home())); + } + + #[test] + fn the_home_itself_resolves() { + // Uses the real home, because resolve() canonicalises. + let real_home = model::home_dir(); + let resolved = resolve("~", &real_home, &real_home); + assert!(resolved.is_ok(), "{resolved:?}"); + } + + #[cfg(feature = "chat")] + #[test] + fn every_tool_has_a_schema_and_a_safe_name() { + let tools = tools(); + assert_eq!(tools.len(), 7); + for tool in &tools { + assert!(tool + .name + .chars() + .all(|c| c.is_ascii_lowercase() || c == '_')); + assert!(tool.parameters.starts_with('{')); + assert!(tool.parameters.contains("\"properties\"")); + assert!(!tool.description.is_empty()); + } + assert!(tools.iter().any(|tool| tool.name == "mkdir")); + assert!(tools.iter().any(|tool| tool.name == "rename")); + assert!(tools.iter().any(|tool| tool.name == "trash")); + } + + #[test] + fn the_bus_manifest_has_the_table_and_validates() { + let manifest = service_manifest(); + assert_eq!(manifest.id, "files"); + manifest.validate().expect("a manifest the wire accepts"); + assert_eq!(manifest.tools.len(), TOOL_TABLE.len()); + for ((name, description, schema, risk), tool) in TOOL_TABLE.iter().zip(&manifest.tools) { + assert_eq!(tool.name, *name); + assert_eq!(tool.description, *description); + assert_eq!(tool.parameters, *schema); + assert_eq!(tool.risk, *risk); + assert!(schema.contains(r#""type":"object""#), "{name}: an argument object"); + } + for name in ["list_dir", "read_file", "stat", "treemap_summary"] { + assert_eq!(manifest.tool(name).unwrap().risk, Risk::Read); + } + for name in ["mkdir", "rename", "trash"] { + assert_eq!(manifest.tool(name).unwrap().risk, Risk::Destructive); + } + } + + fn demo_job(home: &Path, name: &str, args: &[(&str, &str)]) -> ToolJob { + ToolJob { + name: name.to_string(), + args: args + .iter() + .map(|(key, value)| ((*key).to_string(), (*value).to_string())) + .collect(), + cwd: home.join("Documents"), + home: home.to_path_buf(), + } + } + + #[test] + fn mkdir_rename_and_trash_mutate_the_demo_vfs() { + let fs = crate::demo::DemoVfs::new(); + let home = fs.home(); + let created = home.join("Documents/assistant-created"); + let renamed = home.join("Documents/assistant-renamed"); + + let outcome = run_with_vfs( + &demo_job(&home, "mkdir", &[("path", "assistant-created")]), + &fs, + &AtomicBool::new(false), + &|_| {}, + ); + assert!(!outcome.is_error, "{}", outcome.text); + assert!(outcome.mutated); + assert!(fs.is_dir(&created)); + + let outcome = run_with_vfs( + &demo_job( + &home, + "rename", + &[("path", "assistant-created"), ("new_name", "assistant-renamed")], + ), + &fs, + &AtomicBool::new(false), + &|_| {}, + ); + assert!(!outcome.is_error, "{}", outcome.text); + assert!(!fs.exists(&created)); + assert!(fs.is_dir(&renamed)); + + let outcome = run_with_vfs( + &demo_job(&home, "trash", &[("path", "assistant-renamed")]), + &fs, + &AtomicBool::new(false), + &|_| {}, + ); + assert!(!outcome.is_error, "{}", outcome.text); + assert!(!fs.exists(&renamed)); + assert!(fs.is_dir(&crate::ops::trash_dir(&home).join("assistant-renamed"))); + } + + #[test] + fn every_mutation_refuses_a_jail_escape() { + let fs = crate::demo::DemoVfs::new(); + let home = fs.home(); + for (tool, args) in [ + ("mkdir", vec![("path", "/Outside/new")]), + ("rename", vec![("path", "/Outside/item"), ("new_name", "new")]), + ("trash", vec![("path", "/Outside/item")]), + ] { + let outcome = run_with_vfs( + &demo_job(&home, tool, &args), + &fs, + &AtomicBool::new(false), + &|_| {}, + ); + assert!(outcome.is_error, "{tool}"); + assert!(outcome.text.contains("refused"), "{tool}: {}", outcome.text); + assert!(!outcome.mutated, "{tool}"); + } + } + + #[test] + fn a_cancelled_walk_answers_at_once_and_says_it_is_a_floor() { + // Cancel before the first child: the walk returns immediately and + // the summary carries the floor marker. + let home = model::home_dir(); + let cancel = AtomicBool::new(true); + let job = ToolJob { + name: "treemap_summary".to_string(), + args: vec![("path".to_string(), "~".to_string())], + cwd: home.clone(), + home, + }; + let started = makepad_widgets::Cx::monotonic_now(); + let outcome = run_with(&job, &cancel, &|_| {}); + assert!( + makepad_widgets::Cx::monotonic_now() - started < 2.0, + "the flag must be honoured at once" + ); + assert!(!outcome.is_error, "{}", outcome.text); + assert!(outcome.text.contains("cut short"), "{}", outcome.text); + } + + #[test] + fn an_unknown_tool_is_an_error_not_a_panic() { + let home = model::home_dir(); + let outcome = run(&ToolJob { + name: "rm_rf".to_string(), + args: vec![("path".to_string(), "~".to_string())], + cwd: home.clone(), + home, + }); + assert!(outcome.is_error); + assert!(outcome.text.contains("no tool called")); + } + + #[test] + fn a_refused_path_never_reaches_a_tool() { + let home = model::home_dir(); + let outcome = run(&ToolJob { + name: "read_file".to_string(), + args: vec![("path".to_string(), "/etc/passwd".to_string())], + cwd: home.clone(), + home, + }); + assert!(outcome.is_error); + assert!(outcome.text.contains("refused")); + assert!(!outcome.text.contains("root:")); + } +} diff --git a/apps/mpfiles/src/contents.rs b/apps/files/src/contents.rs similarity index 100% rename from apps/mpfiles/src/contents.rs rename to apps/files/src/contents.rs diff --git a/apps/files/src/demo.rs b/apps/files/src/demo.rs new file mode 100644 index 000000000..2f0919db4 --- /dev/null +++ b/apps/files/src/demo.rs @@ -0,0 +1,1674 @@ +//! A closed, deterministic fake filesystem for the native and web demos. +//! +//! `--demo` (or `MAKEPAD_FILES_DEMO=1`, see [`crate::vfs::demo_requested`]) points +//! the browser at [`DemoVfs`] instead of the real disk, so a recording can +//! show `files` doing real work — thumbnails, Space preview, rename, copy, +//! the treemap, undo — without a single one of the user's own files ever +//! appearing on screen. Every operation genuinely mutates the in-memory tree; +//! thumbnails are supplied separately from embedded, repo-owned images. +//! +//! The tree is built once, deterministically — a seeded PRNG, never the +//! clock — so two runs (and two recordings) show byte-identical sizes and +//! dates. Everything after that lives behind a [`Mutex`], because the +//! [`Vfs`] trait hands out `&self`: an in-memory filesystem still needs +//! interior mutability to survive a rename. + +use std::{ + path::{Path, PathBuf}, + sync::{atomic::AtomicBool, atomic::Ordering, Mutex}, +}; + +use crate::{ + model::{self, FileEntry, SortSpec}, + ops::{OpKind, OpRequest, Undo}, + treemap::{Node, ScanProgress}, + vfs::{outcome_message, OpOutcome, Vfs, VfsError}, +}; + +/// The demo's home. Rooted somewhere that cannot be mistaken for a real +/// path and reads cleanly in the breadcrumb — `/Demo`, `/Demo/Documents`, +/// and so on. +const VIRTUAL_HOME: &str = "/Demo"; + +/// Where a trashed demo file goes; a plain hidden folder under the virtual +/// home, exactly the way `~/.Trash` sits under a real one. +const TRASH_NAME: &str = ".Trash"; + +/// The anchor "now" every seeded date is measured back from. A fixed +/// constant, not [`std::time::SystemTime::now`] — that is what keeps the +/// tree byte-identical across runs instead of drifting a little further +/// from "today" every time someone records a demo. (2026-08-27 00:00:00 +/// UTC, chosen simply because it postdates every asset this module reads.) +const DEMO_NOW_SECS: u64 = 1_787_788_800; + +/// Modified times are spread somewhere in this window before [`DEMO_NOW_SECS`]. +const TWO_YEARS_SECS: u64 = 63_072_000; + +/// A file's created time sits at most this far before its modified time. +const THIRTY_DAYS_SECS: u64 = 2_592_000; + +/// The PRNG's seed. Any nonzero constant works; this one has no meaning +/// beyond "not zero, not a round number that looks like a bug". +const SEED: u64 = 0x9E37_79B9_7F4A_7C15; + +// --------------------------------------------------------------------- +// A tiny, deterministic PRNG +// --------------------------------------------------------------------- + +/// xorshift64* — plenty of spread for sizes and dates, and small enough not +/// to be worth a `rand` dependency for a module whose only requirement is +/// "the same numbers every time". +struct Rng(u64); + +impl Rng { + /// `seed` is forced odd: xorshift's state never leaves zero once it + /// gets there, so a zero (or even, which can shift down to zero) seed + /// would make every "random" number the same number. + fn new(seed: u64) -> Self { + Rng(seed | 1) + } + + fn next_u64(&mut self) -> u64 { + let mut x = self.0; + x ^= x << 13; + x ^= x >> 7; + x ^= x << 17; + self.0 = x; + x + } + + /// A value in `[lo, hi)`. + fn range(&mut self, lo: u64, hi: u64) -> u64 { + lo + self.next_u64() % (hi - lo) + } + + /// A bounded Pareto variate. Most results sit near `minimum`, while a + /// power-law tail supplies progressively rarer large files up to `maximum`. + fn pareto(&mut self, minimum: u64, maximum: u64, shape: f64) -> u64 { + debug_assert!(minimum > 0 && minimum < maximum && shape > 0.0); + let unit = (self.next_u64() >> 11) as f64 / (1u64 << 53) as f64; + let truncated_tail = 1.0 - (minimum as f64 / maximum as f64).powf(shape); + let sample = minimum as f64 / (1.0 - unit * truncated_tail).powf(1.0 / shape); + (sample as u64).clamp(minimum, maximum) + } +} + +/// A modified/created pair somewhere in the last two years, never in the +/// future and never zero (zero reads as "unknown" everywhere this app +/// formats a timestamp, which a seeded file must never claim to be). +fn seeded_age(rng: &mut Rng) -> (u64, u64) { + let modified = DEMO_NOW_SECS - rng.range(0, TWO_YEARS_SECS); + let created = modified + .saturating_sub(rng.range(0, THIRTY_DAYS_SECS)) + .max(DEMO_NOW_SECS - TWO_YEARS_SECS); + (modified, created) +} + +// --------------------------------------------------------------------- +// The tree +// --------------------------------------------------------------------- + +/// One node in the closed tree. Paths are rebuilt while walking so moving a +/// subtree never requires rewriting thousands of descendants. +#[derive(Clone, Debug, PartialEq, Eq)] +struct VNode { + name: String, + is_dir: bool, + /// A file's own size; always `0` for a folder — a folder's size is the + /// fold of its children, computed by whoever needs it, exactly the way + /// [`FileEntry::size`] is `0` for a directory too. + size: u64, + modified_secs: u64, + created_secs: u64, + children: Vec, +} + +fn folder_at(name: impl Into, modified_secs: u64, created_secs: u64, children: Vec) -> VNode { + VNode { + name: name.into(), + is_dir: true, + size: 0, + modified_secs, + created_secs, + children, + } +} + +fn file_at(name: impl Into, size: u64, modified_secs: u64, created_secs: u64) -> VNode { + VNode { + name: name.into(), + is_dir: false, + size, + modified_secs, + created_secs, + children: Vec::new(), + } +} + +#[derive(Clone, Copy, PartialEq, Eq)] +enum Profile { + Pictures, + Projects, + Library, + Mail, + Music, + Documents, + Videos, + Downloads, + Desktop, + Network, + Trash, +} + +impl Profile { + fn folder_name(self, index: usize) -> String { + match self { + Profile::Pictures => format!("archive-{index:04}"), + Profile::Projects => format!("package-{index:04}"), + Profile::Library => format!("cache-shard-{index:04}"), + Profile::Mail => format!("mailbox-{index:04}"), + Profile::Music => format!("Artist {index:04}"), + Profile::Documents => format!("Archive-{index:04}"), + Profile::Videos => format!("Clips-{index:03}"), + Profile::Downloads => format!("download-batch-{index:03}"), + Profile::Desktop => format!("Workspace {index:03}"), + Profile::Network => format!("shared-{index:03}"), + Profile::Trash => format!("deleted-{index:03}"), + } + } + + fn file_name(self, index: usize) -> String { + match self { + Profile::Pictures => { + if index % 17 == 0 { format!("DSC_{index:05}.ARW") } else { format!("IMG_{index:05}.jpg") } + } + Profile::Projects => match index % 8 { + 0 => format!("module_{index:05}.rs"), + 1 => format!("index_{index:05}.js"), + 2 => format!("package-{index:05}.json"), + 3 => format!("types_{index:05}.ts"), + 4 => format!("README-{index:05}.md"), + 5 => format!("Cargo-{index:05}.toml"), + 6 => format!("shader_{index:05}.wgsl"), + _ => format!("config_{index:05}.json"), + }, + Profile::Library => format!("blob-{index:06}.cache"), + Profile::Mail => format!("message-{index:06}.eml"), + Profile::Music => { + if index % 23 == 0 { format!("{index:02} Lossless.flac") } else { format!("{index:02} Track.mp3") } + } + Profile::Documents => match index % 8 { + 0 => format!("report-{index:05}.pdf"), + 1 => format!("letter-{index:05}.docx"), + 2 => format!("ledger-{index:05}.xlsx"), + 3 => format!("export-{index:05}.csv"), + 4 => format!("slides-{index:05}.pptx"), + 5 => format!("receipt-{index:05}.pdf"), + 6 => format!("notes-{index:05}.md"), + _ => format!("outline-{index:05}.txt"), + }, + Profile::Videos => { + if index % 2 == 0 { format!("clip-{index:04}.mp4") } else { format!("camera-{index:04}.mkv") } + } + Profile::Downloads => match index % 8 { + 0 => format!("installer-{index:04}.dmg"), + 1 => format!("linux-{index:04}.iso"), + 2 => format!("assets-{index:04}.zip"), + 3 => format!("setup-{index:04}.pkg"), + 4 => format!("partial-{index:04}.download"), + 5 => format!("screenshot-{index:04}.png"), + 6 => format!("manual-{index:04}.pdf"), + _ => format!("export-{index:04}.csv"), + }, + Profile::Desktop => format!("desktop-note-{index:04}.md"), + Profile::Network => format!("team-file-{index:04}.pdf"), + Profile::Trash => if index % 2 == 0 { format!("old-export-{index:03}.zip") } else { format!("recording-{index:03}.mkv") }, + } + } + + fn size(self, index: usize, rng: &mut Rng) -> u64 { + const MIB: u64 = 1024 * 1024; + const GIB: u64 = 1024 * MIB; + match self { + Profile::Pictures if index % 17 == 0 => rng.pareto(8 * MIB, 40 * MIB, 1.15), + Profile::Pictures => rng.pareto(512 * 1024, 12 * MIB, 1.1), + Profile::Projects if index % 97 == 0 => rng.pareto(MIB, 3 * MIB, 1.2), + Profile::Projects => rng.pareto(1024, 256 * 1024, 1.1), + Profile::Library => rng.pareto(128 * 1024, 12 * MIB, 1.05), + Profile::Mail => rng.pareto(4 * 1024, 180 * 1024, 1.1), + Profile::Music if index % 23 == 0 => rng.pareto(25 * MIB, 60 * MIB, 1.2), + Profile::Music => rng.pareto(3 * MIB, 12 * MIB, 1.1), + Profile::Documents => rng.pareto(8 * 1024, 18 * MIB, 0.9), + Profile::Videos if index < 5 => rng.range(2 * GIB, 6 * GIB), + Profile::Videos if index < 85 => rng.range(100 * MIB, 900 * MIB), + Profile::Videos => rng.pareto(8 * MIB, 100 * MIB, 1.1), + Profile::Downloads if index < 40 => rng.range(80 * MIB, 4 * GIB), + Profile::Downloads => rng.pareto(64 * 1024, 160 * MIB, 1.0), + Profile::Desktop => rng.pareto(1024, 20 * MIB, 0.9), + Profile::Network => rng.pareto(32 * 1024, 50 * MIB, 0.9), + Profile::Trash => rng.pareto(MIB, 3 * GIB, 0.85), + } + } +} + +/// Depth is measured from the generated category root. The public tree adds +/// `/Demo` above it and files add one leaf level, so nine here means an +/// inclusive whole-tree maximum of eleven (`/Demo` is depth zero). +const MAX_CATEGORY_FOLDER_DEPTH: usize = 9; + +struct TempFolder { + name: String, + depth: usize, + modified_secs: u64, + created_secs: u64, + children: Vec, + files: Vec, +} + +fn add_folder(folders: &mut Vec, parent: usize, name: String, rng: &mut Rng) -> usize { + let (modified_secs, created_secs) = seeded_age(rng); + let index = folders.len(); + let depth = folders[parent].depth + 1; + folders.push(TempFolder { name, depth, modified_secs, created_secs, children: Vec::new(), files: Vec::new() }); + folders[parent].children.push(index); + index +} + +fn add_path(folders: &mut Vec, path: &[&str], rng: &mut Rng) -> usize { + let mut parent = 0; + for name in path { + let found = folders[parent] + .children + .iter() + .copied() + .find(|&child| folders[child].name == *name); + parent = found.unwrap_or_else(|| add_folder(folders, parent, (*name).to_string(), rng)); + } + parent +} + +fn materialize(index: usize, folders: &mut [Option]) -> VNode { + let temp = folders[index].take().expect("folder is materialized once"); + let mut children = Vec::with_capacity(temp.children.len() + temp.files.len()); + for child in temp.children { + children.push(materialize(child, folders)); + } + children.extend(temp.files); + folder_at(temp.name, temp.modified_secs, temp.created_secs, children) +} + +/// Build one bushy category with bounded listings. `folder_count` includes +/// the category root; the few supplied paths create the intentionally deep +/// branches before the remaining folders are spread four-wide. +fn build_category( + name: &str, + folder_count: usize, + file_count: usize, + max_depth: usize, + profile: Profile, + special_paths: &[&[&str]], + rng: &mut Rng, +) -> VNode { + let (modified_secs, created_secs) = seeded_age(rng); + let mut folders = vec![TempFolder { + name: name.to_string(), + depth: 0, + modified_secs, + created_secs, + children: Vec::new(), + files: Vec::new(), + }]; + for path in special_paths { + assert!(path.len() <= MAX_CATEGORY_FOLDER_DEPTH, "special demo path exceeds the depth bound"); + add_path(&mut folders, path, rng); + } + if profile == Profile::Pictures { + const TRIPS: [&str; 4] = ["Lisbon", "Kyoto", "Reykjavik", "Dolomites"]; + for year in 2023..=2026 { + let year = year.to_string(); + for month in 1..=12 { + for trip in 0..4 { + let trip = format!("{month:02}-{}-{trip}", TRIPS[(month + trip) % TRIPS.len()]); + add_path(&mut folders, &[year.as_str(), trip.as_str()], rng); + } + } + } + } + if profile == Profile::Projects { + let node_modules = add_path(&mut folders, &["web-dashboard", "node_modules"], rng); + for package in 0..120 { + add_folder(&mut folders, node_modules, format!("dependency-{package:03}"), rng); + } + } + + let max_depth = max_depth.min(MAX_CATEGORY_FOLDER_DEPTH); + let mut parent = 0usize; + while folders.len() < folder_count { + while folders[parent].depth >= max_depth || folders[parent].children.len() >= 4 { + parent += 1; + } + let index = folders.len(); + add_folder(&mut folders, parent, profile.folder_name(index), rng); + } + + let base = file_count / folders.len(); + let extra = file_count % folders.len(); + let mut file_index = 0usize; + for folder_index in 0..folders.len() { + let count = base + usize::from(folder_index < extra); + let folder_modified = folders[folder_index].modified_secs; + folders[folder_index].files.reserve(count); + for _ in 0..count { + let modified_secs = (folder_modified + rng.range(0, 7 * 86_400)).min(DEMO_NOW_SECS); + let created_secs = modified_secs + .saturating_sub(rng.range(0, THIRTY_DAYS_SECS)) + .max(DEMO_NOW_SECS - TWO_YEARS_SECS); + folders[folder_index].files.push(file_at( + profile.file_name(file_index), + profile.size(file_index, rng), + modified_secs, + created_secs, + )); + file_index += 1; + } + } + let mut folders: Vec> = folders.into_iter().map(Some).collect(); + let mut root = materialize(0, &mut folders); + if profile == Profile::Projects { + let node_modules = descendant_mut(&mut root, &["web-dashboard", "node_modules"]) + .expect("the web project has node_modules"); + let target = 950 * 1024 * 1024; + let current = sum_bytes_unchecked(node_modules); + if current < target { + let (modified, created) = seeded_age(rng); + node_modules.children.push(file_at(".vite-dependency-cache.bin", target - current, modified, created)); + } + } + root +} + +fn descendant_mut<'a>(mut node: &'a mut VNode, path: &[&str]) -> Option<&'a mut VNode> { + for name in path { + node = node.children.iter_mut().find(|child| child.is_dir && child.name == *name)?; + } + Some(node) +} + +fn push_featured_file(folder: &mut VNode, name: &str, size: u64, rng: &mut Rng) { + let (modified, created) = seeded_age(rng); + folder.children.push(file_at(name, size, modified, created)); +} + +/// 38,000 files in 2,026 folders. The category ratios keep ordinary listings +/// near twenty entries while a scan of Home sees the whole varied tree. +fn build_root_with_seed(seed: u64) -> VNode { + let mut rng = Rng::new(seed); + let mut pictures = build_category( + "Pictures", 350, 8_000, 6, Profile::Pictures, + &[&["2024", "07-Lisbon"], &["2025", "11-Kyoto"], &["wallpapers"], &["screenshots", "2026", "08"]], + &mut rng, + ); + push_featured_file(&mut pictures, "wallpaper-sunrise.jpg", 6 * 1024 * 1024, &mut rng); + + let mut projects = build_category( + "Projects", 650, 12_500, 10, Profile::Projects, + &[ + &["atlas", "crates", "render", "src", "passes", "shadow", "cascade", "partition", "cache"], + &["web-dashboard", "node_modules", "@makepad", "renderer", "node_modules", "tiny-color"], + &[ + "orbit", "src", "platform", "web", "runtime", "renderer", "cache", "shaders", + "compiled", + ], + ], + &mut rng, + ); + push_featured_file(&mut projects, "README.md", 6_000, &mut rng); + + let library = build_category( + "Library", 400, 7_000, 8, Profile::Library, + &[ + &[ + "Caches", + "com.makepad.studio", + "versions", + "v12", + "data", + "blobs", + "segments", + "compiled", + "chunks", + ], + &["Application Support", "Browser", "CacheStorage"], + ], + &mut rng, + ); + let mail = build_category( + "Mail", + 200, + 3_500, + 9, + Profile::Mail, + &[&[ + "Accounts", + "Personal", + "Archive", + "2025", + "Receipts", + "Travel", + "Thread Data", + "Attachments", + "Inline", + ]], + &mut rng, + ); + let music = build_category("Music", 180, 3_200, 5, Profile::Music, &[&["Aurora Lines", "Midnight Hours"], &["Northbound", "Lossless Sessions"]], &mut rng); + let mut documents = build_category( + "Documents", 100, 1_600, 6, Profile::Documents, + &[&["Archive", "2024", "Taxes"], &["Scanned Receipts", "2025", "Q4"]], + &mut rng, + ); + push_featured_file(&mut documents, "notes.md", 4_096, &mut rng); + push_featured_file(&mut documents, "budget.csv", 18_432, &mut rng); + push_featured_file(&mut documents, "contacts.csv", 12_288, &mut rng); + let videos = build_category("Videos", 25, 500, 4, Profile::Videos, &[&["Camera Uploads", "2025"], &["Edits", "Final"]], &mut rng); + let downloads = build_category("Downloads", 50, 1_000, 4, Profile::Downloads, &[&["Installers"], &["Unsorted"]], &mut rng); + let desktop = build_category("Desktop", 30, 400, 4, Profile::Desktop, &[&["Current Work"]], &mut rng); + let network = build_category("Network", 30, 250, 4, Profile::Network, &[&["shared", "Design Team"], &["shared", "Engineering"]], &mut rng); + let trash = build_category(TRASH_NAME, 10, 50, 3, Profile::Trash, &[&["Old Downloads"]], &mut rng); + let (modified, created) = seeded_age(&mut rng); + folder_at( + "Demo", + modified, + created, + vec![desktop, documents, downloads, library, mail, music, network, pictures, projects, videos, trash], + ) +} + +fn build_root() -> VNode { + build_root_with_seed(SEED) +} + +// --------------------------------------------------------------------- +// Tree lookups and edits +// --------------------------------------------------------------------- + +/// The node at `path`, or `None` when `path` is not under [`VIRTUAL_HOME`] +/// or does not exist in the tree — the same "just doesn't resolve" outcome +/// either way, since nothing this module does treats them differently. +fn resolve<'a>(root: &'a VNode, path: &Path) -> Option<&'a VNode> { + let home = Path::new(VIRTUAL_HOME); + if path == home { + return Some(root); + } + let rel = path.strip_prefix(home).ok()?; + let mut node = root; + for component in rel.components() { + let std::path::Component::Normal(part) = component else { return None }; + let name = part.to_string_lossy(); + node = node.children.iter().find(|c| c.name == name)?; + } + Some(node) +} + +/// The mutable twin of [`resolve`]. +fn resolve_mut<'a>(root: &'a mut VNode, path: &Path) -> Option<&'a mut VNode> { + let home = Path::new(VIRTUAL_HOME); + if path == home { + return Some(root); + } + let rel = path.strip_prefix(home).ok()?; + let mut node = root; + for component in rel.components() { + let std::path::Component::Normal(part) = component else { return None }; + let name = part.to_string_lossy().into_owned(); + node = node.children.iter_mut().find(|c| c.name == name)?; + } + Some(node) +} + +/// `path` split into its parent folder and its own name — `None` for a +/// path with neither (the root, or something not path-shaped at all). +fn split_path(path: &Path) -> Option<(PathBuf, String)> { + let parent = path.parent()?.to_path_buf(); + let name = path.file_name()?.to_string_lossy().into_owned(); + Some((parent, name)) +} + +/// Remove and return the child named `name`, or `None` when there is no +/// such child. +fn take_child(parent: &mut VNode, name: &str) -> Option { + let index = parent.children.iter().position(|c| c.name == name)?; + Some(parent.children.remove(index)) +} + +/// Byte total of a subtree: a file's own size, or the recursive fold of a +/// folder's children — never the folder's own (always-zero) `size` field. +/// Bails out with whatever it has already added up once `cancel` is +/// raised, matching [`crate::ops::total_bytes`]'s contract. +fn sum_bytes(node: &VNode, cancel: &AtomicBool) -> u64 { + if !node.is_dir { + return node.size; + } + let mut total = 0u64; + for child in &node.children { + if cancel.load(Ordering::SeqCst) { + break; + } + total += sum_bytes(child, cancel); + } + total +} + +fn sum_bytes_unchecked(node: &VNode) -> u64 { + if node.is_dir { + node.children.iter().map(sum_bytes_unchecked).sum() + } else { + node.size + } +} + +fn file_entry(path: PathBuf, node: &VNode) -> FileEntry { + FileEntry { + kind: model::kind_for(&path, node.is_dir), + name: node.name.clone(), + is_dir: node.is_dir, + size: sum_bytes_unchecked(node), + modified_secs: node.modified_secs, + created_secs: node.created_secs, + permissions: if node.is_dir { "rwxr-xr-x".to_string() } else { "rw-r--r--".to_string() }, + child_count: node.is_dir.then(|| node.children.len() as u32), + path, + } +} + +/// `name` split the way [`unique_name`] needs it: a dotfile or an +/// extensionless name reports no extension, which is the signal to put the +/// disambiguating suffix at the very end instead of splicing it into the +/// name's only dot. Mirrors `ops::split_stem_ext` exactly (that one works +/// against the disk, this one against the tree — see the module doc +/// comment on why `ops.rs` isn't reused here). +fn split_stem_ext(name: &str) -> (String, String) { + let path = Path::new(name); + match (path.file_stem(), path.extension()) { + (Some(stem), Some(ext)) => (stem.to_string_lossy().into_owned(), ext.to_string_lossy().into_owned()), + _ => (name.to_string(), String::new()), + } +} + +/// A name for `name` that does not collide with any of `siblings`: "report +/// (2).txt", then "report (3).txt", exactly the way [`crate::ops::unique_path`] +/// disambiguates a real copy on disk — just checked against a folder's +/// children instead of `Path::exists`. +fn unique_name(siblings: &[VNode], name: &str) -> String { + if !siblings.iter().any(|c| c.name == name) { + return name.to_string(); + } + let (stem, ext) = split_stem_ext(name); + let mut n: u64 = 2; + loop { + let candidate = if ext.is_empty() { format!("{name} ({n})") } else { format!("{stem} ({n}).{ext}") }; + if !siblings.iter().any(|c| c.name == candidate) { + return candidate; + } + n += 1; + } +} + +/// Refuses a copy/move whose destination is one of the sources or sits +/// inside one of them — mirrors `ops::refuse_into_self`'s rule, just +/// without needing `canonicalize` (there are no symlinks, and no two +/// virtual paths ever alias the same node). +fn refuse_into_self(sources: &[PathBuf], dest_dir: &Path) -> Option { + for source in sources { + if dest_dir == source.as_path() || dest_dir.starts_with(source) { + return Some(format!("Can't copy or move \"{}\" into itself", model::display_name(source))); + } + } + None +} + +fn reserved_trash_path() -> PathBuf { + Path::new(VIRTUAL_HOME).join(TRASH_NAME) +} + +fn reject_reserved_trash_source(request: &OpRequest) -> Result<(), String> { + let trash = reserved_trash_path(); + if request.sources.iter().any(|source| source == &trash) { + let action = match request.kind { + OpKind::Rename => "rename", + OpKind::Move => "move", + OpKind::Trash => "trash", + OpKind::Delete => "delete", + _ => "modify", + }; + return Err(format!("Can't {action} the reserved demo Trash folder")); + } + Ok(()) +} + +// --------------------------------------------------------------------- +// Operations +// --------------------------------------------------------------------- + +fn perform_rename(tree: &mut VNode, request: &OpRequest) -> Result { + let old_path = request.sources.first().ok_or_else(|| "Rename needs a source".to_string())?; + let new_name = request.new_name.as_deref().ok_or_else(|| "Rename needs a new name".to_string())?; + let old_name = old_path + .file_name() + .map(|n| n.to_string_lossy().into_owned()) + .ok_or_else(|| format!("Can't rename {}", old_path.display()))?; + let new_path = request.dest_dir.join(new_name); + + let parent = resolve_mut(tree, &request.dest_dir) + .ok_or_else(|| format!("No such folder: {}", request.dest_dir.display()))?; + if &new_path != old_path && parent.children.iter().any(|c| c.name == new_name) { + return Err(format!("\"{new_name}\" already exists")); + } + let node = parent + .children + .iter_mut() + .find(|c| c.name == old_name) + .ok_or_else(|| format!("No such file: {}", old_path.display()))?; + node.name = new_name.to_string(); + + Ok(OpOutcome { + message: format!("Renamed to \"{new_name}\""), + undo: Some(Undo::Moved { pairs: vec![(old_path.clone(), new_path.clone())] }), + touched: vec![new_path], + }) +} + +fn perform_new_folder(tree: &mut VNode, request: &OpRequest) -> Result { + let dest = resolve_mut(tree, &request.dest_dir) + .ok_or_else(|| format!("No such folder: {}", request.dest_dir.display()))?; + if !dest.is_dir { + return Err(format!("{} is not a folder", request.dest_dir.display())); + } + let requested = request.new_name.as_deref().unwrap_or("New Folder"); + let name = unique_name(&dest.children, requested); + dest.children.push(folder_at(name.clone(), DEMO_NOW_SECS, DEMO_NOW_SECS, Vec::new())); + let path = request.dest_dir.join(&name); + + Ok(OpOutcome { + message: outcome_message(OpKind::NewFolder, 1, &path), + undo: Some(Undo::Created { paths: vec![path.clone()] }), + touched: vec![path], + }) +} + +fn perform_copy(tree: &mut VNode, request: &OpRequest) -> Result { + if let Some(message) = refuse_into_self(&request.sources, &request.dest_dir) { + return Err(message); + } + let mut touched = Vec::new(); + for source in &request.sources { + let (parent_path, name) = split_path(source).ok_or_else(|| format!("Can't copy {}", source.display()))?; + let cloned: VNode = { + let parent = resolve(tree, &parent_path).ok_or_else(|| format!("No such folder: {}", parent_path.display()))?; + parent + .children + .iter() + .find(|c| c.name == name) + .ok_or_else(|| format!("No such file: {}", source.display()))? + .clone() + }; + let dest = resolve_mut(tree, &request.dest_dir) + .ok_or_else(|| format!("No such folder: {}", request.dest_dir.display()))?; + let unique = unique_name(&dest.children, &name); + let mut item = cloned; + item.name = unique.clone(); + dest.children.push(item); + touched.push(request.dest_dir.join(&unique)); + } + + Ok(OpOutcome { + message: outcome_message(OpKind::Copy, touched.len(), &request.dest_dir), + undo: Some(Undo::Created { paths: touched.clone() }), + touched, + }) +} + +fn perform_move(tree: &mut VNode, request: &OpRequest) -> Result { + if let Some(message) = refuse_into_self(&request.sources, &request.dest_dir) { + return Err(message); + } + { + let dest = resolve(tree, &request.dest_dir) + .ok_or_else(|| format!("No such folder: {}", request.dest_dir.display()))?; + if !dest.is_dir { + return Err(format!("{} is not a folder", request.dest_dir.display())); + } + } + + let mut moved_pairs = Vec::new(); + let mut touched = Vec::new(); + let mut skipped = 0usize; + for source in &request.sources { + // A cut-and-paste back onto the folder it came from is a no-op, + // not a move that happens to land where it started — same rule as + // `ops::already_there`. + if source.parent() == Some(request.dest_dir.as_path()) { + skipped += 1; + touched.push(source.clone()); + continue; + } + let (parent_path, name) = split_path(source).ok_or_else(|| format!("Can't move {}", source.display()))?; + let node = { + let parent = resolve_mut(tree, &parent_path) + .ok_or_else(|| format!("No such folder: {}", parent_path.display()))?; + take_child(parent, &name).ok_or_else(|| format!("No such file: {}", source.display()))? + }; + let dest = resolve_mut(tree, &request.dest_dir) + .ok_or_else(|| format!("No such folder: {}", request.dest_dir.display()))?; + let unique = unique_name(&dest.children, &name); + let mut item = node; + item.name = unique.clone(); + dest.children.push(item); + let target = request.dest_dir.join(&unique); + moved_pairs.push((source.clone(), target.clone())); + touched.push(target); + } + + if moved_pairs.is_empty() && skipped > 0 { + return Ok(OpOutcome { message: "Nothing to move — already there".to_string(), undo: None, touched }); + } + let message = if skipped > 0 { + format!("Moved {} item(s) ({} already there)", moved_pairs.len(), skipped) + } else { + outcome_message(OpKind::Move, moved_pairs.len(), &request.dest_dir) + }; + Ok(OpOutcome { message, undo: Some(Undo::Moved { pairs: moved_pairs }), touched }) +} + +fn perform_trash(tree: &mut VNode, request: &OpRequest) -> Result { + let trash_path = reserved_trash_path(); + let trash = resolve(tree, &trash_path).ok_or_else(|| "The demo Trash folder is missing".to_string())?; + if !trash.is_dir { + return Err("The demo Trash path is not a folder".to_string()); + } + let mut pairs = Vec::new(); + let mut touched = Vec::new(); + for source in &request.sources { + let (parent_path, name) = split_path(source).ok_or_else(|| format!("Can't trash {}", source.display()))?; + let node = { + let parent = resolve_mut(tree, &parent_path) + .ok_or_else(|| format!("No such folder: {}", parent_path.display()))?; + take_child(parent, &name).ok_or_else(|| format!("No such file: {}", source.display()))? + }; + let dest = resolve_mut(tree, &trash_path).ok_or_else(|| "The demo Trash folder is missing".to_string())?; + let unique = unique_name(&dest.children, &name); + let mut item = node; + item.name = unique.clone(); + dest.children.push(item); + let target = trash_path.join(&unique); + pairs.push((source.clone(), target.clone())); + touched.push(target); + } + + Ok(OpOutcome { + message: outcome_message(OpKind::Trash, pairs.len(), &trash_path), + undo: Some(Undo::Moved { pairs }), + touched, + }) +} + +/// Erases every source outright — no undo, no trash behind it, per +/// `OpKind::Delete`'s contract. +fn perform_delete(tree: &mut VNode, request: &OpRequest) -> Result { + let mut removed = 0usize; + for source in &request.sources { + let (parent_path, name) = split_path(source).ok_or_else(|| format!("Can't delete {}", source.display()))?; + let parent = + resolve_mut(tree, &parent_path).ok_or_else(|| format!("No such folder: {}", parent_path.display()))?; + take_child(parent, &name).ok_or_else(|| format!("No such file: {}", source.display()))?; + removed += 1; + } + Ok(OpOutcome { + message: format!("Deleted {removed} item{} permanently", if removed == 1 { "" } else { "s" }), + undo: None, + touched: Vec::new(), + }) +} + +fn undo_moved(tree: &mut VNode, pairs: &[(PathBuf, PathBuf)]) -> Result { + let mut restored = Vec::new(); + for (from, to) in pairs { + let (to_parent, to_name) = split_path(to).ok_or_else(|| format!("Can't undo move of {}", to.display()))?; + let node = { + let parent = + resolve_mut(tree, &to_parent).ok_or_else(|| format!("No such folder: {}", to_parent.display()))?; + take_child(parent, &to_name).ok_or_else(|| format!("Nothing to undo at {}", to.display()))? + }; + let (from_parent, from_name) = split_path(from).ok_or_else(|| format!("Can't undo move to {}", from.display()))?; + let dest = resolve_mut(tree, &from_parent) + .ok_or_else(|| format!("No such folder: {}", from_parent.display()))?; + let mut item = node; + item.name = from_name; + dest.children.push(item); + restored.push(from.clone()); + } + Ok(OpOutcome { message: format!("Undid move of {} item(s)", restored.len()), undo: None, touched: restored }) +} + +fn undo_created(tree: &mut VNode, paths: &[PathBuf]) -> Result { + let mut removed = Vec::new(); + for path in paths { + let (parent_path, name) = split_path(path).ok_or_else(|| format!("Can't undo creation of {}", path.display()))?; + let parent = + resolve_mut(tree, &parent_path).ok_or_else(|| format!("No such folder: {}", parent_path.display()))?; + take_child(parent, &name).ok_or_else(|| format!("Nothing to undo at {}", path.display()))?; + removed.push(path.clone()); + } + Ok(OpOutcome { message: format!("Undid creation of {} item(s)", removed.len()), undo: None, touched: removed }) +} + +// --------------------------------------------------------------------- +// Scanning, for the treemap +// --------------------------------------------------------------------- + +/// Entries visited between [`ScanProgress`] reports — the in-memory +/// equivalent of `treemap::PROGRESS_STRIDE`. The tree is tiny compared to a +/// real disk, so this mostly just guarantees the final report; it exists +/// so the demo's `scan` still honours the "bounded rate" half of the +/// contract rather than assuming a small tree makes it moot. +const SCAN_PROGRESS_STRIDE: u64 = 64; + +fn scan_vnode( + node: &VNode, + path: &Path, + cancel: &AtomicBool, + progress: &dyn Fn(ScanProgress), + total: &mut ScanProgress, + since_report: &mut u64, + skip: &dyn Fn(&Path) -> bool, +) -> Option { + if cancel.load(Ordering::Relaxed) { + return None; + } + let kind = model::kind_for(path, node.is_dir) as u8; + let result = if node.is_dir { + let mut children = Vec::with_capacity(node.children.len()); + let mut size = 0u64; + for child in &node.children { + let child_path = path.join(&child.name); + if child.is_dir && skip(&child_path) { + continue; + } + let child_node = scan_vnode(child, &child_path, cancel, progress, total, since_report, skip)?; + size += child_node.size; + children.push(child_node); + } + Node { + files: children.iter().map(|c| c.files).sum(), + modified: children.iter().map(|c| c.modified).max().unwrap_or(0), + name: node.name.clone(), + is_dir: true, + done: true, + denied: false, + size, + kind, + children, + } + } else { + total.files += 1; + total.bytes += node.size; + Node::file_at(node.name.clone(), kind, node.size, (node.modified_secs / 60) as u32) + }; + // Reported at most once every `SCAN_PROGRESS_STRIDE` nodes (folders and + // files both count), the same bounded-rate rule `treemap::scan` keeps — + // a demo tree is small enough that this rarely fires before the final + // report `Vfs::scan` sends once the whole walk is done. + *since_report += 1; + if *since_report >= SCAN_PROGRESS_STRIDE { + *since_report = 0; + progress(*total); + } + Some(result) +} + +// --------------------------------------------------------------------- +// The Vfs +// --------------------------------------------------------------------- + +/// The demo filesystem is entirely memory-backed and never resolves a +/// virtual path against the host. +pub struct DemoVfs { + root: Mutex, +} + +impl DemoVfs { + /// Build the full seeded tree immediately so every later operation is an + /// in-memory lookup. + pub fn new() -> Self { + DemoVfs { root: Mutex::new(build_root()) } + } + + fn scan_with_skip( + &self, + root: &Path, + cancel: &AtomicBool, + progress: &dyn Fn(ScanProgress), + skip: &dyn Fn(&Path) -> bool, + ) -> Option { + if cancel.load(Ordering::Relaxed) { + return None; + } + let tree = self.root.lock().unwrap(); + let node = resolve(&tree, root)?; + let mut total = ScanProgress::default(); + let mut since_report = 0u64; + let result = scan_vnode(node, root, cancel, progress, &mut total, &mut since_report, skip)?; + progress(total); + Some(result) + } +} + +impl Default for DemoVfs { + fn default() -> Self { + Self::new() + } +} + +impl Vfs for DemoVfs { + fn home(&self) -> PathBuf { + PathBuf::from(VIRTUAL_HOME) + } + + fn now_secs(&self) -> u64 { + DEMO_NOW_SECS + } + + fn read_dir(&self, path: &Path, show_hidden: bool) -> Result, String> { + let tree = self.root.lock().unwrap(); + let node = resolve(&tree, path).ok_or_else(|| format!("No such folder: {}", path.display()))?; + if !node.is_dir { + return Err(format!("{} is not a folder", path.display())); + } + let mut entries = Vec::new(); + for child in &node.children { + // Same rule as `model::read_directory`: a name starting with a + // dot (here, exactly `.Trash`) is hidden unless asked for. + if !show_hidden && child.name.starts_with('.') { + continue; + } + let child_path = path.join(&child.name); + entries.push(file_entry(child_path, child)); + } + let mut order: Vec = (0..entries.len()).collect(); + model::sort_indices(&entries, &mut order, SortSpec::default()); + Ok(order.into_iter().map(|i| entries[i].clone()).collect()) + } + + fn is_dir(&self, path: &Path) -> bool { + let tree = self.root.lock().unwrap(); + resolve(&tree, path).is_some_and(|n| n.is_dir) + } + + fn mkdir(&self, path: &Path) -> Result<(), String> { + let home = Path::new(VIRTUAL_HOME); + let rel = path + .strip_prefix(home) + .map_err(|_| format!("{} is outside the demo home", path.display()))?; + let mut tree = self.root.lock().unwrap(); + let mut node = &mut *tree; + for component in rel.components() { + let std::path::Component::Normal(part) = component else { + return Err(format!("Invalid folder path: {}", path.display())); + }; + let name = part.to_string_lossy().into_owned(); + let index = match node.children.iter().position(|child| child.name == name) { + Some(index) => index, + None => { + node.children.push(folder_at( + name, + DEMO_NOW_SECS, + DEMO_NOW_SECS, + Vec::new(), + )); + node.children.len() - 1 + } + }; + node = &mut node.children[index]; + if !node.is_dir { + return Err(format!("{} is not a folder", path.display())); + } + } + Ok(()) + } + + fn rename(&self, source: &Path, target: &Path) -> Result<(), String> { + if source == reserved_trash_path() { + return Err("Can't move the reserved demo Trash folder".to_string()); + } + let (source_parent, source_name) = + split_path(source).ok_or_else(|| format!("Can't move {}", source.display()))?; + let (target_parent, target_name) = + split_path(target).ok_or_else(|| format!("Can't move to {}", target.display()))?; + let mut tree = self.root.lock().unwrap(); + let destination = resolve(&tree, &target_parent) + .ok_or_else(|| format!("No such folder: {}", target_parent.display()))?; + if !destination.is_dir { + return Err(format!("{} is not a folder", target_parent.display())); + } + if destination.children.iter().any(|child| child.name == target_name) { + return Err(format!("{} already exists", target.display())); + } + let mut moved = { + let parent = resolve_mut(&mut tree, &source_parent) + .ok_or_else(|| format!("No such folder: {}", source_parent.display()))?; + take_child(parent, &source_name) + .ok_or_else(|| format!("No such file: {}", source.display()))? + }; + moved.name = target_name; + resolve_mut(&mut tree, &target_parent) + .expect("destination was validated under the same lock") + .children + .push(moved); + Ok(()) + } + + fn canonicalize(&self, path: &Path) -> Result { + let tree = self.root.lock().unwrap(); + resolve(&tree, path) + .map(|_| path.to_path_buf()) + .ok_or_else(|| VfsError::Io(format!("No such file: {}", path.display()))) + } + + fn is_symlink(&self, path: &Path) -> Result { + let tree = self.root.lock().unwrap(); + resolve(&tree, path) + .map(|_| false) + .ok_or_else(|| VfsError::Io(format!("No such file: {}", path.display()))) + } + + fn stat(&self, path: &Path) -> Result { + let tree = self.root.lock().unwrap(); + let node = resolve(&tree, path).ok_or_else(|| format!("No such file: {}", path.display()))?; + Ok(file_entry(path.to_path_buf(), node)) + } + + fn read_bytes(&self, path: &Path, max: usize) -> Result, String> { + let tree = self.root.lock().unwrap(); + let node = resolve(&tree, path).ok_or_else(|| format!("No such file: {}", path.display()))?; + if node.is_dir { + return Err(format!("{} is a folder", path.display())); + } + let ext = path.extension().and_then(|e| e.to_str()).unwrap_or("").to_ascii_lowercase(); + if !matches!(ext.as_str(), "txt" | "md" | "rs" | "toml" | "json" | "csv" | "ts" | "js" | "wgsl" | "eml") { + return Err(format!("{} has no text content in the demo", path.display())); + } + let name = node.name.as_str(); + let text = match ext.as_str() { + "json" => format!("{{\n \"file\": \"{name}\",\n \"source\": \"files demo\",\n \"generated\": true\n}}\n"), + "csv" => format!("file,kind,status\n{name},synthetic,ready\nsummary.csv,demo,closed filesystem\n"), + "rs" => format!("// Synthetic preview for {name}\npub fn demo_file() -> &'static str {{\n \"files demo\"\n}}\n"), + "md" => format!("# {name}\n\nThis is synthetic content from the closed files demo filesystem.\n\nNo host files were read.\n"), + _ => format!("Synthetic preview for {name}\nGenerated by the closed files demo filesystem.\nNo host files were read.\n"), + }; + let mut bytes = text.into_bytes(); + bytes.truncate(max); + Ok(bytes) + } + + fn total_bytes(&self, path: &Path, cancel: &AtomicBool) -> u64 { + let tree = self.root.lock().unwrap(); + resolve(&tree, path).map(|node| sum_bytes(node, cancel)).unwrap_or(0) + } + + fn scan(&self, root: &Path, cancel: &AtomicBool, progress: &dyn Fn(ScanProgress)) -> Option { + let home = self.home(); + self.scan_with_skip(root, cancel, progress, &|path| model::skip_for_scan(path, &home)) + } + + fn perform(&self, request: &OpRequest) -> Result { + let mut tree = self.root.lock().unwrap(); + if matches!(request.kind, OpKind::Rename | OpKind::Move | OpKind::Trash | OpKind::Delete) { + reject_reserved_trash_source(request)?; + } + match request.kind { + OpKind::Rename => perform_rename(&mut tree, request), + OpKind::NewFolder => perform_new_folder(&mut tree, request), + OpKind::Copy => perform_copy(&mut tree, request), + OpKind::Move => perform_move(&mut tree, request), + OpKind::Trash => perform_trash(&mut tree, request), + OpKind::Delete => perform_delete(&mut tree, request), + } + } + + fn perform_undo(&self, undo: &Undo) -> Result { + let mut tree = self.root.lock().unwrap(); + match undo { + Undo::Moved { pairs } => undo_moved(&mut tree, pairs), + Undo::Created { paths } => undo_created(&mut tree, paths), + } + } + + fn is_instant(&self) -> bool { + true + } +} + +#[cfg(test)] +mod tests { + use super::*; + use makepad_widgets::makepad_platform::thread::TaskPool; + use makepad_widgets::Cx; + use std::collections::HashSet; + + /// `DemoVfs::scan_stream` uses the `Vfs` default impl, which never + /// touches the pool — a throwaway one just satisfies the signature. + fn test_pool() -> TaskPool { + Cx::new(Box::new(|_, _| {})).task_pool() + } + + fn listing(vfs: &DemoVfs, path: &str) -> Vec { + vfs.read_dir(Path::new(path), true).unwrap() + } + + fn top_level_folders() -> [&'static str; 10] { + ["Desktop", "Documents", "Downloads", "Library", "Mail", "Music", "Network", "Pictures", "Projects", "Videos"] + } + + fn full_scan(vfs: &DemoVfs) -> Node { + vfs.scan_with_skip(Path::new(VIRTUAL_HOME), &AtomicBool::new(false), &|_| {}, &|_| false) + .expect("full demo scan should complete") + } + + #[test] + fn the_tree_is_deterministic() { + assert_eq!(build_root_with_seed(SEED), build_root_with_seed(SEED)); + } + + #[test] + fn fixed_seed_pareto_sizes_have_a_power_law_tail() { + let mut rng = Rng::new(SEED); + let minimum = 1024; + let mut sizes: Vec = (0..10_000) + .map(|_| rng.pareto(minimum, 1024 * 1024, 1.1)) + .collect(); + let tail_counts: Vec = [2, 4, 8, 16] + .into_iter() + .map(|multiple| sizes.iter().filter(|&&size| size >= minimum * multiple).count()) + .collect(); + assert!(tail_counts.windows(2).all(|pair| pair[0] > pair[1]), "tail buckets: {tail_counts:?}"); + assert!((4_000..=5_200).contains(&tail_counts[0]), "tail buckets: {tail_counts:?}"); + assert!((1_700..=2_700).contains(&tail_counts[1]), "tail buckets: {tail_counts:?}"); + assert!((750..=1_350).contains(&tail_counts[2]), "tail buckets: {tail_counts:?}"); + assert!((300..=700).contains(&tail_counts[3]), "tail buckets: {tail_counts:?}"); + + sizes.sort_unstable(); + assert!((1_700..=2_100).contains(&sizes[5_000]), "median: {}", sizes[5_000]); + assert!((6_500..=10_000).contains(&sizes[9_000]), "p90: {}", sizes[9_000]); + assert!((40_000..=100_000).contains(&sizes[9_900]), "p99: {}", sizes[9_900]); + } + + #[test] + fn no_timestamp_is_zero_or_in_the_future() { + let vfs = DemoVfs::new(); + let mut stack = vec![PathBuf::from(VIRTUAL_HOME)]; + let mut files_checked = 0; + while let Some(dir) = stack.pop() { + for entry in listing(&vfs, dir.to_str().unwrap()) { + if entry.is_dir { + stack.push(entry.path.clone()); + continue; + } + files_checked += 1; + assert_ne!(entry.modified_secs, 0, "{} has no modified time", entry.path.display()); + assert_ne!(entry.created_secs, 0, "{} has no created time", entry.path.display()); + assert!(entry.modified_secs <= DEMO_NOW_SECS, "{} is modified in the future", entry.path.display()); + assert!(entry.created_secs <= DEMO_NOW_SECS, "{} is created in the future", entry.path.display()); + assert!(entry.modified_secs >= DEMO_NOW_SECS - TWO_YEARS_SECS, "{} is older than the demo window", entry.path.display()); + assert!(entry.created_secs >= DEMO_NOW_SECS - TWO_YEARS_SECS, "{} was created before the demo window", entry.path.display()); + } + } + assert!((30_000..=45_000).contains(&files_checked), "file count out of range: {files_checked}"); + } + + #[test] + fn read_dir_sorts_folders_first_then_by_name() { + let vfs = DemoVfs::new(); + let entries = listing(&vfs, VIRTUAL_HOME); + let first_file = entries.iter().position(|e| !e.is_dir); + let last_folder = entries.iter().rposition(|e| e.is_dir); + if let (Some(first_file), Some(last_folder)) = (first_file, last_folder) { + assert!(last_folder < first_file, "a folder sorted after a file"); + } + let names: Vec<&str> = entries.iter().filter(|e| e.is_dir).map(|e| e.name.as_str()).collect(); + let mut sorted = names.clone(); + sorted.sort_by_key(|n| n.to_lowercase()); + assert_eq!(names, sorted); + } + + #[test] + fn rename_works_collides_and_undoes() { + let vfs = DemoVfs::new(); + let dest_dir = PathBuf::from(VIRTUAL_HOME).join("Documents"); + let old_path = dest_dir.join("notes.md"); + + let outcome = vfs + .perform(&OpRequest { + id: 1, + kind: OpKind::Rename, + sources: vec![old_path.clone()], + dest_dir: dest_dir.clone(), + new_name: Some("journal.md".to_string()), + home: vfs.home(), + }) + .unwrap(); + let new_path = dest_dir.join("journal.md"); + assert_eq!(outcome.touched, vec![new_path.clone()]); + assert!(vfs.read_dir(&dest_dir, true).unwrap().iter().any(|e| e.name == "journal.md")); + assert!(!vfs.read_dir(&dest_dir, true).unwrap().iter().any(|e| e.name == "notes.md")); + + // Renaming onto an existing sibling is refused. + let collide = vfs.perform(&OpRequest { + id: 2, + kind: OpKind::Rename, + sources: vec![new_path.clone()], + dest_dir: dest_dir.clone(), + new_name: Some("budget.csv".to_string()), + home: vfs.home(), + }); + assert!(collide.is_err()); + + let Some(Undo::Moved { pairs }) = outcome.undo else { panic!("expected a Moved undo") }; + let undo_outcome = vfs.perform_undo(&Undo::Moved { pairs }).unwrap(); + assert_eq!(undo_outcome.touched, vec![old_path.clone()]); + assert!(vfs.read_dir(&dest_dir, true).unwrap().iter().any(|e| e.name == "notes.md")); + } + + #[test] + fn copy_into_the_same_folder_gets_a_suffix_and_undoes() { + let vfs = DemoVfs::new(); + let dir = PathBuf::from(VIRTUAL_HOME).join("Documents"); + let source = dir.join("notes.md"); + + let outcome = vfs + .perform(&OpRequest { + id: 1, + kind: OpKind::Copy, + sources: vec![source.clone()], + dest_dir: dir.clone(), + new_name: None, + home: vfs.home(), + }) + .unwrap(); + let copy_path = dir.join("notes (2).md"); + assert_eq!(outcome.touched, vec![copy_path.clone()]); + assert!(vfs.read_dir(&dir, true).unwrap().iter().any(|e| e.name == "notes.md"), "the original must survive its own copy"); + assert!(vfs.read_dir(&dir, true).unwrap().iter().any(|e| e.name == "notes (2).md")); + + let Some(Undo::Created { paths }) = outcome.undo else { panic!("expected a Created undo") }; + vfs.perform_undo(&Undo::Created { paths }).unwrap(); + assert!(!vfs.read_dir(&dir, true).unwrap().iter().any(|e| e.name == "notes (2).md")); + assert!(vfs.read_dir(&dir, true).unwrap().iter().any(|e| e.name == "notes.md")); + } + + #[test] + fn trash_moves_out_and_undo_restores_it() { + let vfs = DemoVfs::new(); + let dir = PathBuf::from(VIRTUAL_HOME).join("Documents"); + let source = dir.join("budget.csv"); + + let outcome = vfs + .perform(&OpRequest { + id: 1, + kind: OpKind::Trash, + sources: vec![source.clone()], + dest_dir: dir.clone(), + new_name: None, + home: vfs.home(), + }) + .unwrap(); + assert!(!vfs.read_dir(&dir, true).unwrap().iter().any(|e| e.name == "budget.csv")); + let trash_path = PathBuf::from(VIRTUAL_HOME).join(".Trash").join("budget.csv"); + assert_eq!(outcome.touched, vec![trash_path.clone()]); + assert!(vfs.read_dir(&PathBuf::from(VIRTUAL_HOME).join(".Trash"), true).unwrap().iter().any(|e| e.name == "budget.csv")); + + let Some(Undo::Moved { pairs }) = outcome.undo else { panic!("expected a Moved undo") }; + vfs.perform_undo(&Undo::Moved { pairs }).unwrap(); + assert!(vfs.read_dir(&dir, true).unwrap().iter().any(|e| e.name == "budget.csv"), "undo must put it back in the same folder"); + } + + #[test] + fn delete_removes_permanently_with_no_undo() { + let vfs = DemoVfs::new(); + let dir = PathBuf::from(VIRTUAL_HOME).join("Documents"); + let source = dir.join("contacts.csv"); + + let outcome = vfs + .perform(&OpRequest { + id: 1, + kind: OpKind::Delete, + sources: vec![source.clone()], + dest_dir: dir.clone(), + new_name: None, + home: vfs.home(), + }) + .unwrap(); + assert!(outcome.undo.is_none()); + assert!(outcome.touched.is_empty()); + assert!(!vfs.read_dir(&dir, true).unwrap().iter().any(|e| e.name == "contacts.csv")); + } + + #[test] + fn reserved_trash_root_rejects_every_destructive_operation() { + let vfs = DemoVfs::new(); + let trash = reserved_trash_path(); + for kind in [OpKind::Rename, OpKind::Move, OpKind::Trash, OpKind::Delete] { + let result = vfs.perform(&OpRequest { + id: 1, + kind, + sources: vec![trash.clone()], + dest_dir: PathBuf::from(VIRTUAL_HOME).join("Documents"), + new_name: (kind == OpKind::Rename).then(|| "Old Trash".to_string()), + home: vfs.home(), + }); + let error = match result { + Ok(_) => panic!("reserved Trash operation must fail"), + Err(error) => error, + }; + assert!(error.contains("reserved demo Trash"), "{kind:?}: {error}"); + assert!(vfs.stat(&trash).unwrap().is_dir, "{kind:?} removed the Trash root"); + } + } + + #[test] + fn total_bytes_matches_the_scans_own_size() { + let vfs = DemoVfs::new(); + let home = PathBuf::from(VIRTUAL_HOME); + let cancel = AtomicBool::new(false); + let total = vfs.total_bytes(&home, &cancel); + assert!(total > 0); + + let scanned = full_scan(&vfs); + assert_eq!(scanned.size, total); + + // And the scan's own count should agree with a manual walk. + let mut stack = vec![home.clone()]; + let mut file_total = 0u64; + while let Some(dir) = stack.pop() { + for entry in vfs.read_dir(&dir, true).unwrap() { + if entry.is_dir { + stack.push(entry.path.clone()); + } else { + file_total += entry.size; + } + } + } + assert_eq!(file_total, total); + } + + #[test] + fn rescan_stream_matches_the_initial_demo_scan_totals() { + let vfs = DemoVfs::new(); + let initial = vfs + .scan(&vfs.home(), &AtomicBool::new(false), &|_| {}) + .expect("initial demo scan should complete"); + let rescanned = Mutex::new(Node::dir(initial.name.clone(), initial.kind)); + let pool = test_pool(); + assert!(vfs.scan_stream(&vfs.home(), &AtomicBool::new(false), &|step| { + assert!(rescanned.lock().unwrap().apply(step)); + }, &pool)); + let rescanned = rescanned.into_inner().unwrap(); + assert_eq!((rescanned.size, rescanned.files), (initial.size, initial.files)); + } + + #[test] + fn scan_scope_excludes_library_and_trash_beneath_the_demo_home() { + let vfs = DemoVfs::new(); + let full = full_scan(&vfs); + let home = vfs.home(); + let scoped = vfs + .scan_with_skip(&home, &AtomicBool::new(false), &|_| {}, &|path| { + model::home_scan_exclusion(path, &home) + }) + .unwrap(); + assert!(full.child_named("Library").is_some()); + assert!(full.child_named(TRASH_NAME).is_some()); + assert!(scoped.child_named("Library").is_none()); + assert!(scoped.child_named(TRASH_NAME).is_none()); + assert!(scoped.files < full.files); + assert!(scoped.size < full.size); + } + + fn scan_stats(node: &Node, depth: usize, stats: &mut (usize, usize, usize, usize)) -> u64 { + stats.2 = stats.2.max(depth); + if node.is_dir { + stats.1 += 1; + let sum: u64 = node.children.iter().map(|child| scan_stats(child, depth + 1, stats)).sum(); + assert_eq!(node.size, sum, "folder {} has an inconsistent recursive size", node.name); + sum + } else { + stats.0 += 1; + if node.size >= 2 * 1024 * 1024 * 1024 { + stats.3 += 1; + } + node.size + } + } + + #[test] + fn generator_shape_sizes_and_timings_meet_the_demo_contract() { + let generated_at = makepad_widgets::Cx::monotonic_now(); + let vfs = DemoVfs::new(); + let generation = makepad_widgets::Cx::monotonic_now() - generated_at; + let scan_at = makepad_widgets::Cx::monotonic_now(); + let node = full_scan(&vfs); + let scan = makepad_widgets::Cx::monotonic_now() - scan_at; + let mut stats = (0usize, 0usize, 0usize, 0usize); + let total = scan_stats(&node, 0, &mut stats); + eprintln!("files demo generator: {generation:.3}s; full inline scan: {scan:.3}s"); + assert_eq!(stats.0, 38_006, "file count"); + assert_eq!(stats.1, 2_026, "folder count"); + assert!((9..=11).contains(&stats.2), "max depth (root is zero and leaf files count): {}", stats.2); + assert!(stats.3 >= 3, "only {} files are at least 2 GiB", stats.3); + assert!(total >= 150 * 1024 * 1024 * 1024, "tree is only {total} bytes"); + assert!(generation < 0.150, "generation took {generation:.3}s"); + assert!(scan < 0.100, "inline scan took {scan:.3}s"); + } + + #[test] + fn stat_and_synthetic_markdown_reads_work() { + let vfs = DemoVfs::new(); + let path = Path::new("/Demo/Documents/notes.md"); + let entry = vfs.stat(path).unwrap(); + assert_eq!(entry.path, path); + assert_eq!(entry.name, "notes.md"); + assert!(!entry.is_dir); + let bytes = vfs.read_bytes(path, 4096).unwrap(); + let text = String::from_utf8(bytes).unwrap(); + assert!(text.contains("notes.md")); + assert!(text.contains("closed files demo filesystem")); + } + + #[test] + fn home_is_demo_and_operations_are_instant() { + let vfs = DemoVfs::new(); + assert_eq!(vfs.home(), PathBuf::from("/Demo")); + assert!(vfs.is_instant()); + assert!(vfs.is_demo()); + } + + #[test] + fn every_expected_top_level_folder_is_present_and_non_empty() { + let vfs = DemoVfs::new(); + let root = listing(&vfs, VIRTUAL_HOME); + for name in top_level_folders() { + let entry = root.iter().find(|e| e.name == name).unwrap_or_else(|| panic!("missing top-level folder {name}")); + assert!(entry.is_dir); + let children = listing(&vfs, &format!("{VIRTUAL_HOME}/{name}")); + assert!(!children.is_empty(), "{name} has no contents"); + } + // The trash exists (it showed up in `root`, which asked to see + // hidden entries too) but is hidden from a normal listing. + assert!(root.iter().any(|e| e.name == ".Trash"), "the trash folder should still exist when hidden entries are shown"); + assert!(vfs.read_dir(Path::new(VIRTUAL_HOME), false).unwrap().iter().all(|e| !e.name.starts_with('.'))); + } + + #[test] + fn sibling_names_are_unique_and_child_counts_match_listings() { + let vfs = DemoVfs::new(); + let mut stack = vec![PathBuf::from(VIRTUAL_HOME)]; + while let Some(dir) = stack.pop() { + let entries = vfs.read_dir(&dir, true).unwrap(); + let mut names = HashSet::with_capacity(entries.len()); + for entry in entries { + assert!(names.insert(entry.name.clone()), "duplicate sibling {} in {}", entry.name, dir.display()); + if entry.is_dir { + let actual = vfs.read_dir(&entry.path, true).unwrap().len() as u32; + assert_eq!(entry.child_count, Some(actual), "child count for {}", entry.path.display()); + stack.push(entry.path); + } else { + assert_eq!(entry.child_count, None, "file child count for {}", entry.path.display()); + } + } + } + } + + #[test] + fn network_has_shared_design_and_engineering_teams() { + let vfs = DemoVfs::new(); + let shared = Path::new(VIRTUAL_HOME).join("Network").join("shared"); + let entries = vfs.read_dir(&shared, true).unwrap(); + for team in ["Design Team", "Engineering"] { + let entry = entries.iter().find(|entry| entry.name == team).unwrap_or_else(|| panic!("missing Network/{team}")); + assert!(entry.is_dir); + assert!(!vfs.read_dir(&entry.path, true).unwrap().is_empty(), "Network/{team} is empty"); + } + } + + struct SpyDemoVfs(DemoVfs); + + impl SpyDemoVfs { + fn dispatch_listing(&self) -> Result, String> { + panic!("demo listing was dispatched to a thread") + } + + fn dispatch_scan(&self) -> bool { + panic!("demo scan was dispatched to a thread") + } + + fn dispatch_thumbnail(&self) -> Option { + panic!("demo thumbnail was dispatched to a thread") + } + + fn dispatch_preview(&self) -> Option { + panic!("demo preview reached an external dispatcher") + } + + fn dispatch_operation(&self) -> Result { + panic!("demo operation was dispatched to a thread") + } + } + + impl Vfs for SpyDemoVfs { + fn now_secs(&self) -> u64 { + DEMO_NOW_SECS + } + fn home(&self) -> PathBuf { + self.0.home() + } + + fn read_dir(&self, path: &Path, show_hidden: bool) -> Result, String> { + self.0.read_dir(path, show_hidden) + } + + fn stat(&self, path: &Path) -> Result { + self.0.stat(path) + } + + fn read_bytes(&self, path: &Path, max: usize) -> Result, String> { + self.0.read_bytes(path, max) + } + + fn is_dir(&self, path: &Path) -> bool { + self.0.is_dir(path) + } + + fn mkdir(&self, path: &Path) -> Result<(), String> { + self.0.mkdir(path) + } + + fn rename(&self, source: &Path, target: &Path) -> Result<(), String> { + self.0.rename(source, target) + } + + fn native_path(&self, _path: &Path) -> Result { + Err(VfsError::Unavailable("native filesystem path")) + } + + fn total_bytes(&self, path: &Path, cancel: &AtomicBool) -> u64 { + self.0.total_bytes(path, cancel) + } + + fn scan(&self, root: &Path, cancel: &AtomicBool, progress: &dyn Fn(ScanProgress)) -> Option { + self.0.scan(root, cancel, progress) + } + + fn perform(&self, request: &OpRequest) -> Result { + self.0.perform(request) + } + + fn perform_undo(&self, undo: &Undo) -> Result { + self.0.perform_undo(undo) + } + + fn is_instant(&self) -> bool { + true + } + } + + #[test] + fn demo_routes_never_resolve_host_paths_or_dispatch_threads() { + let spy = SpyDemoVfs(DemoVfs::new()); + let home = spy.home(); + + let listing = if spy.is_instant() { + spy.read_dir(&home, false) + } else { + spy.dispatch_listing() + } + .unwrap(); + assert!(!listing.is_empty()); + + let scan_ok = if spy.is_instant() { + spy.scan_stream(&home, &AtomicBool::new(false), &|_| {}, &test_pool()) + } else { + spy.dispatch_scan() + }; + assert!(scan_ok); + + let picture = home.join("Pictures/wallpaper-sunrise.jpg"); + let thumb = if spy.is_instant() { + crate::thumbs::decode_thumb_from(&spy, &picture) + } else { + spy.dispatch_thumbnail() + }; + assert!(thumb.is_some()); + + let preview = if spy.is_demo() { + crate::preview::demo_preview(&spy) + } else { + spy.dispatch_preview() + }; + assert!(matches!(preview, Some(crate::preview::Preview::NoViewer(_)))); + + let request = OpRequest { + id: 1, + kind: OpKind::NewFolder, + sources: Vec::new(), + dest_dir: home.clone(), + new_name: Some("Inline Operation".to_string()), + home: home.clone(), + }; + let outcome = if spy.is_instant() { + spy.perform(&request) + } else { + spy.dispatch_operation() + } + .unwrap(); + assert_eq!(outcome.touched, vec![home.join("Inline Operation")]); + } +} diff --git a/apps/mpfiles/src/main.rs b/apps/files/src/main.rs similarity index 94% rename from apps/mpfiles/src/main.rs rename to apps/files/src/main.rs index f6b4cdfb4..bc03be53f 100644 --- a/apps/mpfiles/src/main.rs +++ b/apps/files/src/main.rs @@ -1,10 +1,10 @@ -//! mpfiles — the file browser of the mp* desktop. +//! files — the file browser of the Makepad desktop. //! //! A GNOME-Files-shaped browser: tabs, a places-and-bookmarks sidebar, an //! editable breadcrumb path bar, and four views over one folder (icons with //! real thumbnails, a sortable DataGrid list with expandable folders, a //! compact list, and a treemap of where the bytes actually are). Space quick- -//! looks the selection the way macOS does; inside mpwm the compositor hosts +//! looks the selection the way macOS does; inside wm the compositor hosts //! that popup for us. //! //! Everything here is the shell — the entry model lives in `model`, the views @@ -18,7 +18,6 @@ use makepad_widgets::*; use std::{ path::{Path, PathBuf}, - process::Command, sync::{ atomic::{AtomicBool, Ordering}, mpsc::{self, Receiver, Sender}, @@ -27,9 +26,15 @@ use std::{ thread, }; +#[cfg(not(target_arch = "wasm32"))] +use std::process::Command; + mod bookmarks; +#[cfg(feature = "chat")] mod chat_agent; +#[cfg(feature = "chat")] mod chat_panel; +mod ai_service; mod chat_tools; mod contents; mod demo; @@ -47,9 +52,6 @@ mod vfs; use crate::{ bookmarks::Bookmarks, - chat_agent::{ChatAgent, ChatEvent}, - chat_panel::{ChatState, ChatVoice}, - chat_tools::{ToolJob, ToolRunner}, contents::{FileContents, FileContentsAction, ViewMode, DEFAULT_ZOOM, ZOOM_LEVELS}, model::{display_name, trash_dir, FileEntry}, menu::{MenuAction, MenuRow}, @@ -61,6 +63,25 @@ use crate::{ vfs::vfs, }; +#[cfg(feature = "chat")] +use crate::{ + chat_agent::{ChatAgent, ChatEvent}, + chat_panel::{ChatState, ChatVoice}, + chat_tools::{ToolJob, ToolRunner}, +}; +use crate::ai_service::{ServiceReply, ServiceRunner}; +use makepad_ai_services::port::{AiServicePort, PortEvent}; + +#[cfg(not(feature = "chat"))] +mod no_chat { + use makepad_widgets::*; + + script_mod! { + use mod.prelude.widgets.* + mod.widgets.MpfChatPanel = View{visible: false width: 0 height: 0} + } +} + app_main!(App); script_mod! { @@ -636,6 +657,7 @@ script_mod! { } } chat_button := ToolButton{ + visible: #(cfg!(feature = "chat")) Icon{ icon_walk: Walk{width: 15 height: 15} draw_icon +: { @@ -1449,6 +1471,7 @@ enum FocusTarget { Path, Search, Batch, + #[cfg(feature = "chat")] Chat, Filter, } @@ -1556,12 +1579,12 @@ const CTX_ROW_H: f64 = 28.0; const CTX_SEP_H: f64 = 9.0; const CTX_PANEL_W: f64 = 268.0; -/// The warm-pool dormancy state machine (see `mp_wm_api::warm_start` / -/// `WmEvent::Adopted`). mpwm pre-spawns hidden warm instances of this app -/// (`MPWM_WARM_START=1`); a cached file browser must not scan a directory or +/// The warm-pool dormancy state machine (see `makepad_wm_api::warm_start` / +/// `WmEvent::Adopted`). wm pre-spawns hidden warm instances of this app +/// (`MAKEPAD_WM_WARM_START=1`); a cached file browser must not scan a directory or /// decode thumbnails for a window nobody is looking at. A warm instance /// starts `Dormant` — no initial directory scan — and wakes exactly once: -/// either mpwm adopts it into a real tile (`WmEvent::Adopted` on the studio +/// either wm adopts it into a real tile (`WmEvent::Adopted` on the studio /// `Custom` channel), or, defensively, a human touches the window directly /// (a key or pointer/touch event, in case an `Adopted` message is ever /// lost). A non-warm instance is never dormant. @@ -1577,7 +1600,7 @@ pub enum Dormancy { } impl Dormancy { - /// `warm` is `mp_wm_api::warm_start()`, read once at startup. + /// `warm` is `makepad_wm_api::warm_start()`, read once at startup. pub fn start(warm: bool) -> Self { if warm { Dormancy::Dormant } else { Dormancy::Active } } @@ -1738,41 +1761,66 @@ pub struct App { /// The ask-about-these-files panel. Everything below it stays `None` until /// the panel is opened for the first time: a file browser must not load /// nine billion parameters for a panel nobody asked for. + #[cfg(feature = "chat")] #[rust] chat_open: bool, + #[cfg(feature = "chat")] #[rust] chat: ChatState, + #[cfg(feature = "chat")] #[rust] agent: Option, + #[cfg(feature = "chat")] #[rust] tool_runner: Option, /// True between sending a question and the answer being finished. + #[cfg(feature = "chat")] #[rust] chat_busy: bool, + #[cfg(feature = "chat")] #[rust] chat_ready: bool, /// How many tool results the model is still owed for this turn, and the /// ones that have come back so far — they go over in call order, together. + #[cfg(feature = "chat")] #[rust] chat_awaiting_tools: usize, + #[cfg(feature = "chat")] #[rust] chat_tool_replies: Vec, /// Tool rounds spent on the current question, so a model that decides to /// keep looking forever is stopped rather than left running. + #[cfg(feature = "chat")] #[rust] chat_tool_rounds: usize, /// The status line under the panel header. + #[cfg(feature = "chat")] #[rust] chat_status: String, /// The last "about:" chip and map-strip hint that were pushed into the UI, /// so the per-signal refresh only touches a widget when something changed. + #[cfg(feature = "chat")] #[rust] chat_about: String, #[rust] map_tools_note: String, + + // ------------------------------------------------------------- AI bus + /// The app's service on the desktop's AI bus: open while the window + /// manager hosts this process, `None` standalone (see ai_service.rs). + #[rust] + ai_port: Option, + /// The bus's tool worker, made on the first call. + #[rust] + ai_runner: Option, + /// The context line last sent over the bus, so a selection that did not + /// change sends nothing. + #[rust] + ai_context: String, } /// One finished tool call, waiting for its turn-mates. +#[cfg(feature = "chat")] pub struct ToolReply { text: String, is_error: bool, @@ -1887,6 +1935,7 @@ impl App { .ui .view(cx, ids!(batch_find)) .text_input(cx, ids!(field_input)), + #[cfg(feature = "chat")] FocusTarget::Chat => self.ui.text_input(cx, ids!(chat_input)), FocusTarget::Filter => self.ui.text_input(cx, ids!(filter_query)), }; @@ -1980,6 +2029,12 @@ impl App { } } let dir = path.clone(); + if vfs().is_instant() { + let result = vfs().read_dir(&path, show_hidden); + let _ = sender.send(DirectoryResult { dir, request_id, parent: None, result }); + self.drain_directory_results(cx); + return; + } thread::spawn(move || { let result = vfs().read_dir(&path, show_hidden); let sent = sender.send(DirectoryResult { @@ -2002,7 +2057,12 @@ impl App { let show_hidden = self.show_hidden; let dir = self.current_dir(); let request_id = self.request_id; - let _ = cx; + if vfs().is_instant() { + let result = vfs().read_dir(&folder, show_hidden); + let _ = sender.send(DirectoryResult { dir, request_id, parent: Some(folder), result }); + self.drain_directory_results(cx); + return; + } thread::spawn(move || { let result = vfs().read_dir(&folder, show_hidden); let sent = sender.send(DirectoryResult { @@ -2144,6 +2204,7 @@ impl App { fn place_path(&self, name: &str) -> PathBuf { match name { "home" | "recent" | "starred" => self.home.clone(), + "network" if vfs().is_demo() => self.home.join("Network"), "network" => PathBuf::from("/"), "trash" => trash_dir(&self.home), folder => self.home.join(folder), @@ -2424,7 +2485,7 @@ impl App { // WM's last PreviewShown/PreviewHidden is the answer; standalone, the // answer is whether the child we spawned is still alive — and nothing // told us when it exited, so ask now rather than trust what was true. - if !mp_wm_api::hosted(cx) { + if !makepad_wm_api::hosted(cx) { self.preview.poll(); } let open = self.preview.showing().is_some() || self.preview.hosted_showing().is_some(); @@ -2533,10 +2594,140 @@ impl App { } } + // --------------------------------------------------------------- AI bus + + /// Open the service toward the window manager. Nothing standalone (the + /// port says no); nothing twice. + fn open_ai_port(&mut self, cx: &mut Cx) { + if self.ai_port.is_some() { + return; + } + self.ai_port = AiServicePort::hosted(cx, chat_tools::service_manifest()); + if self.ai_port.is_some() { + log!("files: AI service opened toward the window manager"); + } + } + + fn on_ai_port_events(&mut self, cx: &mut Cx, events: Vec) { + for event in events { + match event { + PortEvent::Registered(endpoint) => { + log!("files: AI service registered as {}", endpoint.as_str()); + // A (re)registration starts the context afresh. + self.ai_context.clear(); + self.refresh_ai_context(cx); + } + PortEvent::Call(call) => { + let (cwd, home) = (self.current_dir(), self.home.clone()); + let spawner = cx.thread_spawner(); + self.ai_runner + .get_or_insert_with(|| ServiceRunner::new(&spawner)) + .submit(&call, cwd, home); + } + PortEvent::Cancel { call_id } => { + if let Some(runner) = self.ai_runner.as_mut() { + runner.cancel(&call_id); + } + } + PortEvent::Subscribe { .. } | PortEvent::Unsubscribe { .. } => {} + PortEvent::ChatOpen { open } => { + // The desktop's pane is the chat now: the app's own panel + // steps aside (Cmd+K brings it back on purpose). + #[cfg(feature = "chat")] + if open && self.chat_open { + self.toggle_chat(cx); + } + #[cfg(not(feature = "chat"))] + let _ = open; + } + } + } + } + + /// The worker's answers and progress go back over the port. + fn drain_ai_replies(&mut self, cx: &mut Cx) { + let Some(runner) = self.ai_runner.as_mut() else { + return; + }; + let replies = runner.drain(); + let Some(port) = self.ai_port.as_ref() else { + return; + }; + let mut refresh = false; + for reply in replies { + match reply { + ServiceReply::Result { result, mutated } => { + port.reply(result); + refresh |= mutated; + } + ServiceReply::Progress { + call_id, + note, + permille, + } => port.progress(&call_id, ¬e, permille), + } + } + if refresh { + self.request_directory(cx); + } + } + + /// What the assistant is told about where the person is — the folder, + /// the view and the selection — whenever that changes. This is how + /// "my downloads" means the folder on screen. + fn refresh_ai_context(&mut self, cx: &mut Cx) { + if self.ai_port.is_none() { + return; + } + let text = self.ai_context_line(cx); + if text == self.ai_context { + return; + } + self.ai_context = text.clone(); + if let Some(port) = self.ai_port.as_ref() { + port.set_context(&text); + } + } + + fn ai_context_line(&mut self, cx: &mut Cx) -> String { + let Some(tab) = self.tabs.get(self.tab) else { + return String::new(); + }; + let mode = tab.mode; + let dir = self.current_dir(); + let mut out = format!( + "The person is looking at {} in the {} view.", + chat_tools::short(&dir, &self.home), + mode.label(), + ); + let selected = self + .with_contents(cx, |contents, _| contents.selected_entries()) + .unwrap_or_default(); + if selected.is_empty() { + out.push_str(" Nothing is selected."); + } else { + out.push_str(&format!(" Selected ({}):", selected.len())); + for entry in selected.iter().take(12) { + out.push_str(&format!( + " {} ({}, {});", + entry.name, + entry.kind_text(), + entry.size_text() + )); + } + if selected.len() > 12 { + out.push_str(&format!(" …and {} more", selected.len() - 12)); + } + } + out + } + /// The window manager's side of the conversation. - fn handle_wm_event(&mut self, cx: &mut Cx, event: &mp_wm_api::WmEvent) { - if matches!(event, mp_wm_api::WmEvent::Adopted) { + fn handle_wm_event(&mut self, cx: &mut Cx, event: &makepad_wm_api::WmEvent) { + if matches!(event, makepad_wm_api::WmEvent::Adopted) { self.wake(cx); + // Adopted into a real tile: now it is a running Files. + self.open_ai_port(cx); } if !self.preview.on_wm_event(event) { return; @@ -2962,7 +3153,7 @@ impl App { self.set_prop(cx, ids!(prop_kind), &kind); // The date says when; the age says whether that is recent, which is // the question anyone actually has about a file. - let now = model::now_secs(); + let now = vfs::now_secs(); let age = entry .as_ref() .filter(|e| e.modified_secs > 0) @@ -2984,9 +3175,9 @@ impl App { cx, ids!(prop_opens), if is_dir { - "mpfiles" + "files" } else { - mp_wm_api::viewer_for(&path) + makepad_wm_api::viewer_for(&path) }, ); if !is_dir { @@ -3011,7 +3202,12 @@ impl App { }; let cancel = Arc::new(AtomicBool::new(false)); self.size_cancel = Some(cancel.clone()); - let _ = cx; + if vfs().is_instant() { + let bytes = vfs().total_bytes(&path, &cancel); + let _ = sender.send(SizeResult { path, bytes }); + self.drain_sizes(cx); + return; + } thread::spawn(move || { let bytes = vfs().total_bytes(&path, &cancel); if cancel.load(Ordering::Relaxed) { @@ -3465,22 +3661,33 @@ impl App { // ------------------------------------------------------------ terminal fn open_terminal(&mut self, cx: &mut Cx) { + if vfs().is_demo() { + self.status(cx, "Terminal is not in this demo"); + return; + } let dir = self.current_dir(); - let request = mp_wm_api::WmRequest::Launch { + let request = makepad_wm_api::WmRequest::Launch { app: "terminal".to_string(), args: vec!["--cwd".to_string(), dir.display().to_string()], }; - if mp_wm_api::send(cx, &request) { + if makepad_wm_api::send(cx, &request) { self.status(cx, &format!("Opening a terminal in {}", dir.display())); return; } - let Some(bin) = preview::sibling_bin("mpterm") else { - self.status(cx, "mpterm is not built — nothing to open a terminal with"); - return; - }; - match Command::new(&bin).arg("--cwd").arg(&dir).spawn() { - Ok(_) => self.status(cx, &format!("Opening a terminal in {}", dir.display())), - Err(error) => self.status(cx, &format!("Could not start mpterm: {error}")), + #[cfg(not(target_arch = "wasm32"))] + { + let Some(bin) = preview::sibling_bin("terminal") else { + self.status(cx, "terminal is not built — nothing to open a terminal with"); + return; + }; + match Command::new(&bin).arg("--cwd").arg(&dir).spawn() { + Ok(_) => self.status(cx, &format!("Opening a terminal in {}", dir.display())), + Err(error) => self.status(cx, &format!("Could not start terminal: {error}")), + } + } + #[cfg(target_arch = "wasm32")] + { + self.status(cx, "Terminal is not in this demo"); } } @@ -3520,7 +3727,7 @@ impl App { entry.kind.label(), model::format_size(entry.size, false), entry.modified_text(), - mp_wm_api::viewer_for(&entry.path), + makepad_wm_api::viewer_for(&entry.path), ) }; self.status(cx, &text); @@ -3552,6 +3759,7 @@ impl App { } // Only while the caret is actually in the ask field: Escape on the // map still means "zoom back out", panel or no panel. + #[cfg(feature = "chat")] if self.chat_open && self.chat_is_typing(cx) { return self.toggle_chat(cx); } @@ -3616,6 +3824,7 @@ impl App { if command { match event.key_code { + #[cfg(feature = "chat")] KeyCode::KeyK => return self.toggle_chat(cx), KeyCode::KeyT if !shift => return self.new_tab(cx), KeyCode::KeyW => return self.close_tab(cx), @@ -3897,24 +4106,7 @@ struct MapJob { /// The mode as `755`, next to the `rwx` letters the listing already shows. fn octal_mode(path: &Path) -> String { - // A virtual file has no inode to ask, and inventing one would be a number - // that means nothing. - if vfs::is_demo() { - return "—".to_string(); - } - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - match std::fs::metadata(path) { - Ok(meta) => format!("{:o}", meta.permissions().mode() & 0o7777), - Err(_) => "—".to_string(), - } - } - #[cfg(not(unix))] - { - let _ = path; - "—".to_string() - } + vfs().unix_mode(path).map(|mode| format!("{mode:o}")).unwrap_or_else(|_| "—".to_string()) } impl App { @@ -3979,7 +4171,7 @@ impl App { /// arriving after `Adopted` already woke it never rescans. fn wake(&mut self, cx: &mut Cx) { if self.dormancy.wake() { - log!("mpfiles: warm instance woken, scanning now"); + log!("files: warm instance woken, scanning now"); self.enter_tab(cx); } } @@ -3990,7 +4182,12 @@ impl App { /// starts the model loading — until then this app has no idea a language /// model exists, which is the only way a file browser is allowed to have /// one. Cmd+K, or the speech-bubble button in the toolbar. + #[cfg(feature = "chat")] fn toggle_chat(&mut self, cx: &mut Cx) { + if vfs().is_demo() { + self.status(cx, "Chat is not in this demo"); + return; + } let open = !self.chat_open; self.chat_open = open; self.ui.widget(cx, ids!(chat_panel)).set_visible(cx, open); @@ -4006,6 +4203,7 @@ impl App { /// Load the model, once. A machine without the weights on it says so and /// carries on being a file browser. + #[cfg(feature = "chat")] fn start_chat(&mut self, cx: &mut Cx) { let Some(model) = chat_agent::model_path() else { self.chat.push( @@ -4025,7 +4223,7 @@ impl App { CHAT_SYSTEM_PROMPT.to_string(), chat_tools::tools(), )); - self.tool_runner = Some(ToolRunner::new()); + self.tool_runner = Some(ToolRunner::new(&cx.thread_spawner())); self.chat.push( ChatVoice::Info, format!("Loading {}…", display_name(&model)), @@ -4034,6 +4232,7 @@ impl App { self.redraw_chat(cx); } + #[cfg(feature = "chat")] fn set_chat_status(&mut self, cx: &mut Cx, text: &str) { if self.chat_status == text { return; @@ -4042,6 +4241,7 @@ impl App { self.ui.label(cx, ids!(chat_status)).set_text(cx, text); } + #[cfg(feature = "chat")] fn redraw_chat(&mut self, cx: &mut Cx) { let list = self.ui.portal_list(cx, ids!(chat_list)); list.set_tail_range(true); @@ -4051,6 +4251,7 @@ impl App { /// Where the user is, as the model reads it: the folder, the view, and /// what is picked. This rides in front of every question and never appears /// in the transcript — "what is this?" is the whole of what was asked. + #[cfg(feature = "chat")] fn chat_where(&mut self, cx: &mut Cx) -> String { let mode = self.tabs[self.tab].mode; let dir = self.current_dir(); @@ -4099,8 +4300,10 @@ impl App { /// button states. Called from `report`, so it follows every selection /// change — and only touches a widget when its text actually changed. fn refresh_chat(&mut self, cx: &mut Cx) { + self.refresh_ai_context(cx); let mode = self.tabs[self.tab].mode; let picked = self.chat_subject(cx); + #[cfg(feature = "chat")] if self.chat_open { let about = match &picked { Some(path) => format!("about: {}", describe_path(path)), @@ -4147,6 +4350,7 @@ impl App { /// Is the caret in the ask field? The panel stays open while its answer is /// read, so "open" cannot be what decides whether a key is text. + #[cfg(feature = "chat")] fn chat_is_typing(&mut self, cx: &mut Cx) -> bool { if !self.chat_open { return false; @@ -4155,6 +4359,11 @@ impl App { !area.is_empty() && cx.has_key_focus(area) } + #[cfg(not(feature = "chat"))] + fn chat_is_typing(&mut self, _cx: &mut Cx) -> bool { + false + } + /// Whether the caret is in the filter sidebar's query field. fn filter_is_typing(&mut self, cx: &mut Cx) -> bool { let area = self.ui.text_input(cx, ids!(filter_query)).area(); @@ -4174,6 +4383,7 @@ impl App { .map(|entry| entry.path) } + #[cfg(feature = "chat")] fn send_chat(&mut self, cx: &mut Cx) { let field = self.ui.text_input(cx, ids!(chat_input)); let text = field.text().trim().to_string(); @@ -4213,6 +4423,7 @@ impl App { self.redraw_chat(cx); } + #[cfg(feature = "chat")] fn stop_chat(&mut self, cx: &mut Cx) { if !self.chat_busy { return; @@ -4231,6 +4442,7 @@ impl App { } /// Swap the Ask button for Stop while a turn is running. + #[cfg(feature = "chat")] fn set_chat_running(&mut self, cx: &mut Cx, running: bool) { self.ui .widget(cx, ids!(chat_send)) @@ -4239,6 +4451,7 @@ impl App { } /// Everything the model and the tool worker have said since the last frame. + #[cfg(feature = "chat")] fn drain_chat(&mut self, cx: &mut Cx) { let events = match &self.agent { Some(agent) => agent.poll(), @@ -4251,7 +4464,9 @@ impl App { Some(runner) => runner.drain(), None => Vec::new(), }; + let mut refresh = false; for reply in replies { + refresh |= reply.mutated; self.chat.push( ChatVoice::Tool, if reply.is_error { @@ -4278,8 +4493,12 @@ impl App { } self.redraw_chat(cx); } + if refresh { + self.request_directory(cx); + } } + #[cfg(feature = "chat")] fn on_chat_event(&mut self, cx: &mut Cx, event: ChatEvent) { match event { ChatEvent::Loading { phase, fraction } => { @@ -4681,19 +4900,18 @@ fn slider_bytes(value: f64) -> Option { /// Now, in whole minutes since the epoch — the clock the age filter runs on. fn now_minutes() -> u32 { - std::time::SystemTime::now() - .duration_since(std::time::UNIX_EPOCH) - .map(|d| (d.as_secs() / 60).min(u32::MAX as u64) as u32) - .unwrap_or(0) + (vfs::now_secs() / 60).min(u32::MAX as u64) as u32 } /// How many times the model may go round the look-then-think loop for one /// question before it has to answer with what it has. +#[cfg(feature = "chat")] const MAX_TOOL_ROUNDS: usize = 6; /// One path, as a sentence: what it is and how big. Reads off the disk, so it /// is the truth at the moment it is asked rather than whatever a listing /// remembered. +#[cfg(feature = "chat")] fn describe_path(path: &Path) -> String { match model::entry_at(path) { Some(entry) => format!( @@ -4707,8 +4925,9 @@ fn describe_path(path: &Path) -> String { } /// What the model is told it is, once, in front of everything else. +#[cfg(feature = "chat")] const CHAT_SYSTEM_PROMPT: &str = "\ -You are the assistant inside mpfiles, a file browser. You answer questions \ +You are the assistant inside files, a file browser. You answer questions \ about the files the person is looking at right now. Every question arrives behind a [where the user is] block: the folder they \ @@ -4731,7 +4950,12 @@ impl MatchEvent for App { fn handle_startup(&mut self, cx: &mut Cx) { // Checked once: a warm-pool instance stays dormant until // `WmEvent::Adopted` or a real input wakes it (see `Dormancy`). - self.dormancy = Dormancy::start(mp_wm_api::warm_start()); + self.dormancy = Dormancy::start(makepad_wm_api::warm_start()); + // The feature build and the native switch install the same closed + // filesystem before preferences or paths can consult a backend. + if vfs::demo_requested() { + vfs::install(Arc::new(demo::DemoVfs::new())); + } // The scan-scope checkbox shows the saved choice from the first // frame; checked means the system folders stay out. self.ui @@ -4742,35 +4966,41 @@ impl MatchEvent for App { self.projection = match model::pref_get("projection").as_deref() { Some("ortho") => MapProjection::Ortho, Some("persp") => MapProjection::Persp, - _ => MapProjection::Flat, + _ => MapProjection::default(), }; self.filter_popup_open = model::pref_get("filter_side").as_deref() == Some("1"); self.style_projection_buttons(cx); - // `--demo` browses a home that does not exist, so a screen recording - // can show every feature of this app without showing anybody's disk. - // It is chosen before anything reads a path, and never afterwards. - if vfs::demo_requested() { - vfs::install(Arc::new(demo::DemoVfs::new())); - } let (sender, receiver) = mpsc::channel(); self.sender = Some(sender); self.receiver = Some(receiver); let (size_sender, size_receiver) = mpsc::channel(); self.size_sender = Some(size_sender); self.size_receiver = Some(size_receiver); - self.ops = Some(Ops::new(Box::new(SignalToUI::set_ui_signal))); + self.ops = if vfs().is_instant() { + None + } else { + Some(Ops::new(Box::new(SignalToUI::set_ui_signal), &cx.thread_spawner())) + }; self.home = vfs().home(); + // The desktop's assistant hears about this instance now — unless it + // is a warm-pool standby, which waits for `Adopted` so a dormant + // process never shows up as a running Files. + if !makepad_wm_api::warm_start() { + self.open_ai_port(cx); + } // The demo must not write to the real home, so its bookmarks live and // die with the window. self.bookmarks = if vfs::is_demo() { Bookmarks::in_memory(Vec::new()) } else { - Bookmarks::load(&self.home) + Bookmarks::load(&crate::model::makepad_home()) }; if vfs::is_demo() { // Say so where it cannot be missed: a recording of the demo must // never be mistaken for a recording of somebody's files. self.ui.label(cx, ids!(files_title)).set_text(cx, "Files · Demo"); + #[cfg(feature = "chat")] + self.ui.widget(cx, ids!(chat_button)).set_visible(cx, false); } let palette = Palette::shared(); let colors = contents::Colors { @@ -4781,19 +5011,21 @@ impl MatchEvent for App { contents.set_colors(cx, colors); contents.set_zoom(cx, DEFAULT_ZOOM); }); - // An explicit folder argument wins over Home; mpwm passes none. + // An explicit folder argument wins over Home; wm passes none. let start = std::env::args() .skip(1) .find(|a| !a.starts_with('-')) .map(PathBuf::from) .filter(|p| vfs().is_dir(p)) .unwrap_or_else(|| self.home.clone()); - self.tabs = vec![Tab::new(start, ViewMode::Icons)]; + // The first tab opens on the space view: where the bytes are is the + // question a file manager gets asked first. + self.tabs = vec![Tab::new(start, ViewMode::Treemap)]; self.tab = 0; // Warm and still dormant: no disk scan and no thumbnails until // `wake` runs it — see `Dormancy`. if self.dormancy.is_dormant() { - log!("mpfiles: warm-start dormant, deferring the initial scan"); + log!("files: warm-start dormant, deferring the initial scan"); } else { self.enter_tab(cx); } @@ -4850,24 +5082,27 @@ impl MatchEvent for App { } // ---- the ask panel - if self.ui.view(cx, ids!(chat_button)).finger_down(actions).is_some() { - self.toggle_chat(cx); - } - if self.chat_open { - if self.ui.view(cx, ids!(chat_close)).finger_down(actions).is_some() { + #[cfg(feature = "chat")] + { + if self.ui.view(cx, ids!(chat_button)).finger_down(actions).is_some() { self.toggle_chat(cx); - return; } - if self.ui.view(cx, ids!(chat_stop)).finger_down(actions).is_some() { - self.stop_chat(cx); - return; - } - let field = self.ui.text_input(cx, ids!(chat_input)); - let returned = field.returned(actions).is_some(); - drop(field); - if returned || self.ui.view(cx, ids!(chat_send)).finger_down(actions).is_some() { - self.send_chat(cx); - return; + if self.chat_open { + if self.ui.view(cx, ids!(chat_close)).finger_down(actions).is_some() { + self.toggle_chat(cx); + return; + } + if self.ui.view(cx, ids!(chat_stop)).finger_down(actions).is_some() { + self.stop_chat(cx); + return; + } + let field = self.ui.text_input(cx, ids!(chat_input)); + let returned = field.returned(actions).is_some(); + drop(field); + if returned || self.ui.view(cx, ids!(chat_send)).finger_down(actions).is_some() { + self.send_chat(cx); + return; + } } } self.handle_map_tool_actions(cx, actions); @@ -5073,13 +5308,18 @@ impl AppMain for App { crate::makepad_widgets::script_mod(vm); // The WM's theme, first into the stock widgets and then into `mod.mpf` // for our own chrome — both before anything reads a color. - mp_theme::apply(vm); + if !vfs::demo_requested() { + makepad_wm_theme::apply(vm); + } Palette::shared().publish(vm); crate::theme::script_mod(vm); crate::thumbs::script_mod(vm); crate::treemap_view::script_mod(vm); crate::contents::script_mod(vm); + #[cfg(feature = "chat")] crate::chat_panel::script_mod(vm); + #[cfg(not(feature = "chat"))] + crate::no_chat::script_mod(vm); self::script_mod(vm) } @@ -5125,14 +5365,24 @@ impl AppMain for App { self.refresh_filter_popup(cx); } } + #[cfg(feature = "chat")] self.drain_chat(cx); + self.drain_ai_replies(cx); self.preview.poll(); } if let Event::Custom(json) = event { - if let Some(wm) = mp_wm_api::WmEvent::parse(json) { + if let Some(wm) = makepad_wm_api::WmEvent::parse(json) { self.handle_wm_event(cx, &wm); } } + // The bus's frames ride the same channel under their own envelope. + let port_events = match self.ai_port.as_mut() { + Some(port) => port.handle_event(cx, event), + None => Vec::new(), + }; + if !port_events.is_empty() { + self.on_ai_port_events(cx, port_events); + } if self.focus_next.is_event(event).is_some() { self.apply_focus(cx); } @@ -5141,8 +5391,10 @@ impl AppMain for App { } // The transcript draws from the chat state, so it rides down the tree // as the scope — every other widget in this window ignores it. - self.ui - .handle_event(cx, event, &mut Scope::with_data(&mut self.chat)); + #[cfg(feature = "chat")] + self.ui.handle_event(cx, event, &mut Scope::with_data(&mut self.chat)); + #[cfg(not(feature = "chat"))] + self.ui.handle_event(cx, event, &mut Scope::empty()); } } diff --git a/apps/mpfiles/src/menu.rs b/apps/files/src/menu.rs similarity index 96% rename from apps/mpfiles/src/menu.rs rename to apps/files/src/menu.rs index 158b3171e..f0583f935 100644 --- a/apps/mpfiles/src/menu.rs +++ b/apps/files/src/menu.rs @@ -186,12 +186,12 @@ pub fn empty_menu(mode: ViewMode, clipboard: usize, show_hidden: bool) -> Vec bool) -> Vec<(String, String)> { let mut out: Vec<(String, String)> = Vec::new(); - let primary = mp_wm_api::viewer_for(path); + let primary = makepad_wm_api::viewer_for(path); if available(primary) { out.push((primary.to_string(), format!("Open with {primary}"))); } - if primary != "mpterm" && available("mpterm") { - out.push(("mpterm".to_string(), "Open in the terminal pager".to_string())); + if primary != "terminal" && available("terminal") { + out.push(("terminal".to_string(), "Open in the terminal pager".to_string())); } // The desktop's own opener always exists; it is the honest last resort. out.push(( @@ -283,8 +283,8 @@ mod tests { let all = |_: &str| true; let picture = Path::new("/a/x.png"); let offered = open_with_apps(picture, &all); - assert_eq!(offered[0].0, "mpimage"); - assert_eq!(offered[1].0, "mpterm"); + assert_eq!(offered[0].0, "image"); + assert_eq!(offered[1].0, "terminal"); // The desktop opener is the last resort and has no binary of its own. assert!(offered.last().unwrap().0.is_empty()); // With nothing built, only the desktop opener is left. @@ -294,7 +294,7 @@ mod tests { .into_iter() .map(|(id, _)| id) .collect(); - assert_eq!(ids, ["mpterm", ""]); + assert_eq!(ids, ["terminal", ""]); assert!(offered.len() <= MAX_APPS); } diff --git a/apps/mpfiles/src/model.rs b/apps/files/src/model.rs similarity index 89% rename from apps/mpfiles/src/model.rs rename to apps/files/src/model.rs index 46d22d974..37967b9f8 100644 --- a/apps/mpfiles/src/model.rs +++ b/apps/files/src/model.rs @@ -6,7 +6,6 @@ use std::{ fs, path::{Path, PathBuf}, - time::{Duration, SystemTime}, }; /// What a file *is*, as far as the browser is concerned: it picks the icon, @@ -90,7 +89,7 @@ const VIDEO_EXTS: &[&str] = &[ ]; /// The videos the platform decoder demuxes, i.e. the ones that can get a real -/// first-frame thumbnail and an `mpvideo` association. The rest of +/// first-frame thumbnail and an `video` association. The rest of /// [`VIDEO_EXTS`] still reads as a video, it just gets the film-strip icon and /// the desktop's own opener. pub const PLAYABLE_VIDEO_EXTS: &[&str] = &["mp4", "mov", "m4v", "webm", "mkv", "avi"]; @@ -100,7 +99,7 @@ const ARCHIVE_EXTS: &[&str] = &[ "whl", "deb", "rpm", ]; -// There is no association table here: `mp_wm_api::viewer_for` is the one the +// There is no association table here: `makepad_wm_api::viewer_for` is the one the // window manager and the browser share. /// Lowercased extension of `path`, or "" when it has none. @@ -405,14 +404,14 @@ pub fn sort_indices(entries: &[FileEntry], order: &mut [usize], sort: SortSpec) /// it is what they already threw away, and counting it would double every /// number the moment they trashed something. /// -/// `MPFILES_SCAN_ALL=1` turns the whole rule off for anyone who wants the +/// `MAKEPAD_FILES_SCAN_ALL=1` turns the whole rule off for anyone who wants the /// literal truth about their home directory and does not mind the dialogs. const HOME_SKIP: [&str; 2] = ["Library", ".Trash"]; /// Whether the size map measures the system folders too. Off by default — /// the map skips ~/Library and ~/.Trash so macOS never storms the user with /// permission dialogs — and flipped by the "ignore system" checkbox on the -/// map's tool strip. `MPFILES_SCAN_ALL=1` or a saved preference turns it on +/// map's tool strip. `MAKEPAD_FILES_SCAN_ALL=1` or a saved preference turns it on /// at startup; every change is written back so the choice survives launches. pub fn scan_all() -> bool { *scan_all_flag().lock().unwrap_or_else(|e| e.into_inner()) @@ -427,7 +426,7 @@ pub fn set_scan_all(on: bool) { fn scan_all_flag() -> &'static std::sync::Mutex { static FLAG: std::sync::OnceLock> = std::sync::OnceLock::new(); FLAG.get_or_init(|| { - if std::env::var_os("MPFILES_SCAN_ALL").is_some_and(|v| v != "0") { + if std::env::var_os("MAKEPAD_FILES_SCAN_ALL").is_some_and(|v| v != "0") { return std::sync::Mutex::new(true); } std::sync::Mutex::new(pref_get("scan_all").as_deref() == Some("1")) @@ -436,12 +435,21 @@ fn scan_all_flag() -> &'static std::sync::Mutex { /// Where the little `key=value` preference file lives. fn prefs_path() -> PathBuf { - home_dir().join(".config").join("mpfiles").join("prefs") + makepad_home().join("files/prefs") +} + +fn memory_prefs() -> &'static std::sync::Mutex { + static PREFS: std::sync::OnceLock> = std::sync::OnceLock::new(); + PREFS.get_or_init(|| std::sync::Mutex::new(String::new())) } /// One saved preference, by key. The file is `key=value` lines, nothing /// more; a missing file is simply no preferences. pub fn pref_get(key: &str) -> Option { + if cfg!(test) || cfg!(feature = "demo") || crate::vfs::is_demo() { + let text = memory_prefs().lock().unwrap_or_else(|e| e.into_inner()); + return pref_find(&text, key); + } let text = std::fs::read_to_string(prefs_path()).ok()?; pref_find(&text, key) } @@ -449,6 +457,11 @@ pub fn pref_get(key: &str) -> Option { /// Save one preference, leaving every other key exactly as it was — the /// file is shared by whatever small choices the app remembers. pub fn pref_set(key: &str, value: &str) { + if cfg!(test) || cfg!(feature = "demo") || crate::vfs::is_demo() { + let mut text = memory_prefs().lock().unwrap_or_else(|e| e.into_inner()); + *text = pref_replace(&text, key, value); + return; + } let path = prefs_path(); if let Some(dir) = path.parent() { let _ = std::fs::create_dir_all(dir); @@ -493,11 +506,11 @@ fn pref_replace(old: &str, key: &str, value: &str) -> String { /// Only ever consulted for directories, and only for the ones directly under /// the user's home — a `Library` folder inside a project is a project's /// library and gets measured like anything else. -pub fn skip_for_scan(path: &Path) -> bool { - if scan_all() { - return false; - } - let home = home_dir(); +pub fn skip_for_scan(path: &Path, home: &Path) -> bool { + !scan_all() && home_scan_exclusion(path, home) +} + +pub(crate) fn home_scan_exclusion(path: &Path, home: &Path) -> bool { let Some(parent) = path.parent() else { return false; }; @@ -523,13 +536,13 @@ pub fn scan_exclusions() -> Option { /// folder the browser is listing, and a context menu on a file three folders /// down has to describe that file, not fail to find a row for it. /// -/// `None` when there is nothing there, or when the browser is on the demo -/// filesystem — a virtual path has no `std::fs` entry to read, and inventing -/// one would let an operation run against a file that does not exist. +/// `None` when the active filesystem has nothing there. Both real metadata +/// and virtual metadata arrive through `Vfs::stat`. pub fn entry_at(path: &Path) -> Option { - if crate::vfs::is_demo() { - return None; - } + crate::vfs::vfs().stat(path).ok() +} + +pub(crate) fn real_entry_at(path: &Path) -> Option { let metadata = fs::metadata(path).ok()?; let is_dir = metadata.is_dir(); Some(FileEntry { @@ -537,8 +550,8 @@ pub fn entry_at(path: &Path) -> Option { kind: kind_for(path, is_dir), is_dir, size: if is_dir { 0 } else { metadata.len() }, - modified_secs: epoch_secs(metadata.modified().ok()), - created_secs: epoch_secs(metadata.created().ok()), + modified_secs: modified_secs(&metadata), + created_secs: created_secs(&metadata), permissions: permissions_text(&metadata), child_count: is_dir.then(|| count_children(path)).flatten(), path: path.to_path_buf(), @@ -567,8 +580,8 @@ pub fn read_directory(path: &Path, show_hidden: bool) -> Result, name, is_dir, size: if is_dir { 0 } else { metadata.len() }, - modified_secs: epoch_secs(metadata.modified().ok()), - created_secs: epoch_secs(metadata.created().ok()), + modified_secs: modified_secs(&metadata), + created_secs: created_secs(&metadata), permissions: permissions_text(&metadata), // One extra `read_dir` per folder, on this worker thread — never // on the UI thread, and never past the cap. @@ -581,8 +594,16 @@ pub fn read_directory(path: &Path, show_hidden: bool) -> Result, Ok(order.into_iter().map(|i| entries[i].clone()).collect()) } -fn epoch_secs(time: Option) -> u64 { - time.and_then(|t| t.duration_since(SystemTime::UNIX_EPOCH).ok()) +fn modified_secs(metadata: &fs::Metadata) -> u64 { + metadata.modified().ok() + .and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok()) + .map(|d| d.as_secs()) + .unwrap_or(0) +} + +fn created_secs(metadata: &fs::Metadata) -> u64 { + metadata.created().ok() + .and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok()) .map(|d| d.as_secs()) .unwrap_or(0) } @@ -647,11 +668,8 @@ pub fn format_size(bytes: u64, is_dir: bool) -> String { } } -pub fn now_secs() -> u64 { - SystemTime::now() - .duration_since(SystemTime::UNIX_EPOCH) - .unwrap_or(Duration::ZERO) - .as_secs() +pub fn real_now_secs() -> u64 { + makepad_widgets::Cx::time_now().max(0.0) as u64 } /// The machine's UTC offset in seconds, read once. The platform has no @@ -660,19 +678,20 @@ pub fn now_secs() -> u64 { pub fn local_utc_offset_secs() -> i64 { static OFFSET: std::sync::OnceLock = std::sync::OnceLock::new(); *OFFSET.get_or_init(|| { - #[cfg(any(target_os = "macos", target_os = "linux"))] + #[cfg(all(not(target_arch = "wasm32"), any(target_os = "macos", target_os = "linux")))] { let Ok(out) = std::process::Command::new("date").arg("+%z").output() else { return 0; }; return parse_utc_offset(String::from_utf8_lossy(&out.stdout).trim()); } - #[cfg(not(any(target_os = "macos", target_os = "linux")))] + #[cfg(any(target_arch = "wasm32", not(any(target_os = "macos", target_os = "linux"))))] 0 }) } /// `+0200` / `-0730` -> seconds east of UTC. +#[cfg(not(target_arch = "wasm32"))] fn parse_utc_offset(text: &str) -> i64 { let bytes = text.as_bytes(); if bytes.len() < 5 || (bytes[0] != b'+' && bytes[0] != b'-') { @@ -754,8 +773,8 @@ pub fn format_age(secs: u64, now: u64) -> String { /// The first `lines` lines of a text file, for the in-app quick look. pub fn read_head(path: &Path, lines: usize, max_bytes: usize) -> Result { - let data = fs::read(path).map_err(|e| format!("{}", e))?; - let cut = data.len().min(max_bytes); + let data = crate::vfs::vfs().read_bytes(path, max_bytes)?; + let cut = data.len(); // Never split a UTF-8 sequence: back off to the last boundary in the cut. let text = match std::str::from_utf8(&data[..cut]) { Ok(text) => text.to_string(), @@ -827,17 +846,17 @@ mod tests { // protected folder: those folders are never entered at all. #[test] fn the_map_leaves_apples_folders_alone_and_touches_nothing_else() { - let home = home_dir(); - assert!(skip_for_scan(&home.join("Library"))); - assert!(skip_for_scan(&home.join(".Trash"))); + let home = Path::new("/active-home"); + assert!(home_scan_exclusion(&home.join("Library"), home)); + assert!(home_scan_exclusion(&home.join(".Trash"), home)); // The user's own files, which is the entire point. - assert!(!skip_for_scan(&home.join("Documents"))); - assert!(!skip_for_scan(&home.join("Pictures"))); - assert!(!skip_for_scan(&home.join("Downloads"))); + assert!(!home_scan_exclusion(&home.join("Documents"), home)); + assert!(!home_scan_exclusion(&home.join("Pictures"), home)); + assert!(!home_scan_exclusion(&home.join("Downloads"), home)); // Only *directly* under home. A project's own `Library` folder is the // project's, and gets measured like anything else in it. - assert!(!skip_for_scan(&home.join("code/thing/Library"))); - assert!(!skip_for_scan(Path::new("/tmp/Library"))); + assert!(!home_scan_exclusion(&home.join("code/thing/Library"), home)); + assert!(!home_scan_exclusion(Path::new("/tmp/Library"), home)); // Whatever it leaves out, it says so. assert!(scan_exclusions().is_some()); } @@ -1025,7 +1044,7 @@ mod tests { #[test] fn reads_a_head_of_lines() { - let dir = std::env::temp_dir().join("mpfiles-test-head"); + let dir = std::env::temp_dir().join("files-test-head"); fs::create_dir_all(&dir).unwrap(); let file = dir.join("head.txt"); fs::write(&file, "one\ntwo\nthree\nfour\n").unwrap(); @@ -1034,3 +1053,24 @@ mod tests { fs::remove_file(&file).ok(); } } + +/// The makepad home directory (`MAKEPAD_HOME`, else the user home; a temp dir as a +/// last resort) — the same rule the AI hub uses, kept local so demo builds without the +/// chat feature do not link the hub for a path. +// The shared per-user home for Makepad AI state. +pub fn makepad_home() -> PathBuf { + if let Some(home) = std::env::var_os("MAKEPAD_HOME") { + return PathBuf::from(home); + } + // USERPROFILE on Windows, HOME elsewhere; temp dir as a last resort. The web has no + // environment and no temp dir (std's temp_dir panics there): its home is a fixed + // virtual path that only ever names browser-storage keys. + #[cfg(target_arch = "wasm32")] + return PathBuf::from("/.makepad"); + #[cfg(not(target_arch = "wasm32"))] + std::env::var_os("USERPROFILE") + .or_else(|| std::env::var_os("HOME")) + .map(PathBuf::from) + .unwrap_or_else(std::env::temp_dir) + .join(".makepad") +} diff --git a/apps/mpfiles/src/ops.rs b/apps/files/src/ops.rs similarity index 97% rename from apps/mpfiles/src/ops.rs rename to apps/files/src/ops.rs index 42cebe7a0..06c7f28b2 100644 --- a/apps/mpfiles/src/ops.rs +++ b/apps/files/src/ops.rs @@ -18,10 +18,16 @@ use std::{ atomic::{AtomicBool, AtomicUsize, Ordering}, mpsc, Arc, Mutex, }, - thread, - time::{Duration, Instant}, }; +use makepad_widgets::makepad_platform::thread::{ThreadOptions, ThreadSpawner}; +use makepad_widgets::Cx; + +#[cfg(test)] +use std::thread; +#[cfg(test)] +use std::time::Duration; + // --------------------------------------------------------------------- // Vocabulary // --------------------------------------------------------------------- @@ -169,7 +175,7 @@ pub enum OpUpdate { /// very end instead. pub fn unique_path(dir: &Path, name: &str) -> PathBuf { let candidate = dir.join(name); - if !candidate.exists() { + if !crate::vfs::vfs().exists(&candidate) { return candidate; } let (stem, ext) = split_stem_ext(name); @@ -181,7 +187,7 @@ pub fn unique_path(dir: &Path, name: &str) -> PathBuf { format!("{stem} ({n}).{ext}") }; let candidate = dir.join(&candidate_name); - if !candidate.exists() { + if !crate::vfs::vfs().exists(&candidate) { return candidate; } n += 1; @@ -408,7 +414,7 @@ struct Progress { total: u64, done: Cell, bytes_since_emit: Cell, - last_emit: Cell, + last_emit: Cell, current: RefCell, updates: Arc>>, notify: Arc, @@ -422,7 +428,7 @@ impl Progress { total, done: Cell::new(0), bytes_since_emit: Cell::new(0), - last_emit: Cell::new(Instant::now()), + last_emit: Cell::new(Cx::monotonic_now()), current: RefCell::new(String::new()), updates, notify, @@ -443,9 +449,10 @@ impl Progress { let done = self.done.get() + delta; self.done.set(done); let since = self.bytes_since_emit.get() + delta; - if since >= 1_000_000 || self.last_emit.get().elapsed() >= Duration::from_millis(32) { + let now = Cx::monotonic_now(); + if since >= 1_000_000 || now - self.last_emit.get() >= 0.032 { self.bytes_since_emit.set(0); - self.last_emit.set(Instant::now()); + self.last_emit.set(now); let update = OpUpdate::Progress { id: self.id, kind: self.kind, @@ -495,8 +502,9 @@ pub struct Ops { impl Ops { /// `notify` is called (from the worker thread) whenever an update is /// queued, so the UI can wake itself. Pass a closure that raises the - /// framework's UI signal. - pub fn new(notify: Box) -> Self { + /// framework's UI signal. `spawner` creates the one dedicated worker + /// thread this engine runs on for the life of the app. + pub fn new(notify: Box, spawner: &ThreadSpawner) -> Self { let (request_tx, request_rx) = mpsc::channel::(); let updates: Arc>> = Arc::new(Mutex::new(VecDeque::new())); let cancel_flags: Arc>>> = Arc::new(Mutex::new(HashMap::new())); @@ -507,9 +515,14 @@ impl Ops { let worker_cancel_flags = cancel_flags.clone(); let worker_busy_count = busy_count.clone(); let worker_notify = notify.clone(); - thread::spawn(move || { - worker_loop(request_rx, worker_updates, worker_cancel_flags, worker_busy_count, worker_notify); - }); + if let Ok(handle) = spawner.spawn_worker( + ThreadOptions { name: Some("files-ops".into()), ..Default::default() }, + move || { + worker_loop(request_rx, worker_updates, worker_cancel_flags, worker_busy_count, worker_notify); + }, + ) { + handle.detach(); + } Ops { request_tx, updates, cancel_flags, busy_count } } @@ -557,9 +570,11 @@ impl Ops { } } +#[cfg(test)] impl Default for Ops { fn default() -> Self { - Ops::new(Box::new(|| {})) + let spawner = Cx::new(Box::new(|_, _| {})).thread_spawner(); + Ops::new(Box::new(|| {}), &spawner) } } @@ -1036,7 +1051,7 @@ fn run_undo_created( // there is no dedicated `OpKind` for "delete" to report instead. let total = paths.len() as u64; let mut done = 0u64; - let mut last_emit = Instant::now(); + let mut last_emit = Cx::monotonic_now(); let mut removed = Vec::new(); let mut cancelled = false; let mut failure = None; @@ -1049,8 +1064,9 @@ fn run_undo_created( Ok(()) => { removed.push(path.clone()); done += 1; - if done == total || last_emit.elapsed() >= Duration::from_millis(32) { - last_emit = Instant::now(); + let now = Cx::monotonic_now(); + if done == total || now - last_emit >= 0.032 { + last_emit = now; push_update( updates, notify, @@ -1148,7 +1164,7 @@ mod tests { use std::sync::atomic::AtomicU64; static COUNTER: AtomicU64 = AtomicU64::new(0); let n = COUNTER.fetch_add(1, Ordering::SeqCst); - let dir = std::env::temp_dir().join(format!("mpfiles-ops-test-{tag}-{}-{n}", std::process::id())); + let dir = std::env::temp_dir().join(format!("files-ops-test-{tag}-{}-{n}", std::process::id())); fs::create_dir_all(&dir).unwrap(); dir } @@ -1161,7 +1177,7 @@ mod tests { /// by `timeout` so a bug in the engine fails the test instead of /// hanging the suite. fn wait_for_done(ops: &Ops, id: u64, timeout: Duration) -> OpUpdate { - let start = Instant::now(); + let start = Cx::monotonic_now(); loop { for update in ops.drain() { let is_match = match &update { @@ -1172,7 +1188,7 @@ mod tests { return update; } } - if start.elapsed() > timeout { + if Cx::monotonic_now() - start > timeout.as_secs_f64() { panic!("timed out waiting for update {id}"); } thread::sleep(Duration::from_millis(5)); diff --git a/apps/mpfiles/src/preview.rs b/apps/files/src/preview.rs similarity index 64% rename from apps/mpfiles/src/preview.rs rename to apps/files/src/preview.rs index 4496ba01e..7350cbe51 100644 --- a/apps/mpfiles/src/preview.rs +++ b/apps/files/src/preview.rs @@ -1,17 +1,17 @@ -//! Opening and previewing files, through `mp_wm_api`. +//! Opening and previewing files, through `makepad_wm_api`. //! //! Which app answers is never decided here, and there is deliberately no -//! association table in this crate: `mp_wm_api::viewer_for` is the one the -//! compositor and the browser share (pictures → mpimage, video → mpvideo, -//! csv/tsv → mpsheets, pdf → mppdf, html → mpbrowser, everything else → -//! mpterm's `--preview` pager). A file type that opens in the wrong app is +//! association table in this crate: `makepad_wm_api::viewer_for` is the one the +//! compositor and the browser share (pictures → image, video → video, +//! csv/tsv → sheets, pdf → pdf, html → browser, everything else → +//! terminal's `--preview` pager). A file type that opens in the wrong app is //! fixed there, never here. //! -//! Hosted as an mpwm tile, an app never spawns anything: it asks, and the +//! Hosted as an wm tile, an app never spawns anything: it asks, and the //! compositor floats the viewer over the desk (Quick Look) or opens it as a //! tile. Standalone the same call spawns the sibling binary — except for the -//! preview, which mpfiles spawns itself so that Space and Escape can take the -//! popup away again; `mp_wm_api::preview`'s child is detached and could not be +//! preview, which files spawns itself so that Space and Escape can take the +//! popup away again; `makepad_wm_api::preview`'s child is detached and could not be //! dismissed. //! //! Whether a Quick Look panel is open is **never** this app's own belief: @@ -21,15 +21,16 @@ //! the next Space then silently "closes" a panel that is already gone. use makepad_widgets::*; -use mp_wm_api::{viewer_for, WmEvent, WmRequest}; +use makepad_wm_api::{viewer_for, WmEvent, WmRequest}; -use std::{ - path::{Path, PathBuf}, - process::{Child, Command}, -}; +use std::path::{Path, PathBuf}; -/// Resolve a sibling binary of the running executable, the way mpwm resolves +#[cfg(not(target_arch = "wasm32"))] +use std::process::{Child, Command}; + +/// Resolve a sibling binary of the running executable, the way wm resolves /// its clients. +#[cfg(not(target_arch = "wasm32"))] pub fn sibling_bin(bin: &str) -> Option { let exe = std::env::current_exe().ok()?; let mut path = exe.parent()?.join(bin); @@ -39,6 +40,11 @@ pub fn sibling_bin(bin: &str) -> Option { path.exists().then_some(path) } +#[cfg(target_arch = "wasm32")] +pub fn sibling_bin(_bin: &str) -> Option { + None +} + /// What came of a Quick Look request. pub enum Preview { /// A viewer window is showing the file; the status line to say so. @@ -47,10 +53,15 @@ pub enum Preview { NoViewer(String), } +pub(crate) fn demo_preview(fs: &dyn crate::vfs::Vfs) -> Option { + fs.is_demo().then(|| Preview::NoViewer("External previews are not in this demo".to_string())) +} + /// The one preview this window has open, if any. #[derive(Default)] pub struct PreviewHost { /// Only set when *we* spawned it; hosted, the compositor owns the float. + #[cfg(not(target_arch = "wasm32"))] child: Option, path: Option, /// What the window manager says its Quick Look panel is showing. Only the @@ -95,25 +106,34 @@ impl PreviewHost { /// panel is open, which is what makes it safe to call on every selection /// change — that is how arrow keys dial through previews. pub fn retarget(&mut self, cx: &Cx, path: &Path) -> bool { - if self.hosted.is_none() || path.is_dir() { + let fs = crate::vfs::vfs(); + if self.hosted.is_none() || fs.is_dir(path) { return false; } - mp_wm_api::preview(cx, &crate::vfs::vfs().real_path(path)) + let Ok(native) = fs.native_path(path) else { + return false; + }; + makepad_wm_api::preview(cx, &native) } - /// Quick Look `path` in its associated viewer. What the viewer is handed - /// is the real file behind the name — identical on a real disk, and the - /// backing asset in the demo. + /// Quick Look `path` in its associated viewer. External viewers are a + /// RealVfs-only integration; demo files fall back to the in-app preview. pub fn open(&mut self, cx: &Cx, path: &Path) -> Preview { let name = crate::model::display_name(path); + let fs = crate::vfs::vfs(); + if let Some(preview) = demo_preview(fs.as_ref()) { + return preview; + } let app = viewer_for(path); - let real = crate::vfs::vfs().real_path(path); + let Ok(real) = fs.native_path(path) else { + return Preview::NoViewer(format!("External previews are unavailable for {name}")); + }; let path = real.as_path(); - if mp_wm_api::hosted(cx) { + if makepad_wm_api::hosted(cx) { // No `close` first: the WM keeps the viewer warm and retargets it, // so hiding the panel a frame before showing it again would only // make it blink. The panel's state arrives as `PreviewShown`. - if mp_wm_api::preview(cx, path) { + if makepad_wm_api::preview(cx, path) { return Preview::Shown(format!( "Previewing {} in {} — arrow keys dial through, Space or Esc closes", name, app @@ -125,13 +145,21 @@ impl PreviewHost { let Some(bin) = sibling_bin(app) else { return Preview::NoViewer(format!("{} is not built — no preview for {}", app, name)); }; - match Command::new(&bin).arg("--preview").arg(path).spawn() { - Ok(child) => { - self.child = Some(child); - self.path = Some(path.to_path_buf()); - Preview::Shown(format!("Previewing {} in {} — Space or Esc to close", name, app)) + #[cfg(not(target_arch = "wasm32"))] + { + match Command::new(&bin).arg("--preview").arg(path).spawn() { + Ok(child) => { + self.child = Some(child); + self.path = Some(path.to_path_buf()); + Preview::Shown(format!("Previewing {} in {} — Space or Esc to close", name, app)) + } + Err(error) => Preview::NoViewer(format!("Could not preview {}: {}", name, error)), } - Err(error) => Preview::NoViewer(format!("Could not preview {}: {}", name, error)), + } + #[cfg(target_arch = "wasm32")] + { + let _ = (bin, path, app); + Preview::NoViewer("External previews are not in this demo".to_string()) } } @@ -139,11 +167,14 @@ impl PreviewHost { /// `PreviewHidden` says it is, never because we assumed so. pub fn close(&mut self, cx: &Cx) { if self.hosted.is_some() { - mp_wm_api::send(cx, &WmRequest::PreviewClose); + makepad_wm_api::send(cx, &WmRequest::PreviewClose); } - if let Some(mut child) = self.child.take() { - let _ = child.kill(); - let _ = child.wait(); + #[cfg(not(target_arch = "wasm32"))] + { + if let Some(mut child) = self.child.take() { + let _ = child.kill(); + let _ = child.wait(); + } } self.path = None; } @@ -153,10 +184,13 @@ impl PreviewHost { /// decision that depends on the answer — not only when a signal happens to /// arrive. pub fn poll(&mut self) { - if let Some(child) = self.child.as_mut() { - if matches!(child.try_wait(), Ok(Some(_))) { - self.child = None; - self.path = None; + #[cfg(not(target_arch = "wasm32"))] + { + if let Some(child) = self.child.as_mut() { + if matches!(child.try_wait(), Ok(Some(_))) { + self.child = None; + self.path = None; + } } } } @@ -167,9 +201,11 @@ impl PreviewHost { pub fn open_file(cx: &Cx, path: &Path) -> String { let name = crate::model::display_name(path); let app = viewer_for(path); - let real = crate::vfs::vfs().real_path(path); + let Ok(real) = crate::vfs::vfs().native_path(path) else { + return format!("Opening {name} is not available on this filesystem"); + }; let path = real.as_path(); - if mp_wm_api::open(cx, path) { + if makepad_wm_api::open(cx, path) { return format!("Opening {} in {}", name, app); } match os_open(path) { @@ -183,7 +219,9 @@ pub fn open_file(cx: &Cx, path: &Path) -> String { /// of ours claims the file. pub fn open_file_with(cx: &Cx, path: &Path, app: &str) -> String { let name = crate::model::display_name(path); - let real = crate::vfs::vfs().real_path(path); + let Ok(real) = crate::vfs::vfs().native_path(path) else { + return format!("Open With for {name} is not available on this filesystem"); + }; let path = real.as_path(); if app.is_empty() { return match os_open(path) { @@ -191,30 +229,39 @@ pub fn open_file_with(cx: &Cx, path: &Path, app: &str) -> String { Err(error) => format!("Could not open {name}: {error}"), }; } - if mp_wm_api::hosted(cx) { - let request = mp_wm_api::WmRequest::Open { + if makepad_wm_api::hosted(cx) { + let request = makepad_wm_api::WmRequest::Open { app: Some(app.to_string()), path: path.display().to_string(), }; - if mp_wm_api::send(cx, &request) { + if makepad_wm_api::send(cx, &request) { return format!("Opening {name} in {app}"); } } let Some(bin) = sibling_bin(app) else { return format!("{app} is not built"); }; - match Command::new(&bin).arg(path).spawn() { - Ok(_) => format!("Opening {name} in {app}"), - Err(error) => format!("Could not start {app}: {error}"), + #[cfg(not(target_arch = "wasm32"))] + { + match Command::new(&bin).arg(path).spawn() { + Ok(_) => format!("Opening {name} in {app}"), + Err(error) => format!("Could not start {app}: {error}"), + } + } + #[cfg(target_arch = "wasm32")] + { + let _ = (bin, path); + format!("Open With for {name} is not in this demo") } } /// True when a sibling app of this name can actually be run — what the Open /// With submenu offers is only ever what exists. pub fn app_available(cx: &Cx, app: &str) -> bool { - mp_wm_api::hosted(cx) || sibling_bin(app).is_some() + !crate::vfs::vfs().is_demo() && (makepad_wm_api::hosted(cx) || sibling_bin(app).is_some()) } +#[cfg(not(target_arch = "wasm32"))] fn os_open(path: &Path) -> std::io::Result<()> { #[cfg(target_os = "macos")] { @@ -234,30 +281,38 @@ fn os_open(path: &Path) -> std::io::Result<()> { } } +#[cfg(target_arch = "wasm32")] +fn os_open(_path: &Path) -> std::io::Result<()> { + Err(std::io::Error::new( + std::io::ErrorKind::Unsupported, + "opening files is not in this demo", + )) +} + #[cfg(test)] mod tests { use super::*; - use mp_wm_api::WmRequest; + use makepad_wm_api::WmRequest; #[test] fn the_shared_table_picks_the_viewer() { - // mpfiles keeps no association table of its own: every kind it shows - // is routed by mp_wm_api, and the kinds it thumbnails are exactly the + // files keeps no association table of its own: every kind it shows + // is routed by makepad_wm_api, and the kinds it thumbnails are exactly the // ones the picture and video viewers claim. for ext in crate::model::IMAGE_EXTS { - assert_eq!(viewer_for(&PathBuf::from(format!("/a/x.{ext}"))), "mpimage"); + assert_eq!(viewer_for(&PathBuf::from(format!("/a/x.{ext}"))), "image"); } for ext in crate::model::PLAYABLE_VIDEO_EXTS { - assert_eq!(viewer_for(&PathBuf::from(format!("/a/x.{ext}"))), "mpvideo"); + assert_eq!(viewer_for(&PathBuf::from(format!("/a/x.{ext}"))), "video"); } // Text and code fall to the terminal's pager, not to nothing. - assert_eq!(viewer_for(Path::new("/a/m.rs")), "mpterm"); - assert_eq!(viewer_for(Path::new("/a/n.txt")), "mpterm"); + assert_eq!(viewer_for(Path::new("/a/m.rs")), "terminal"); + assert_eq!(viewer_for(Path::new("/a/n.txt")), "terminal"); // Every type this browser names in its Kind column has an owner, and // none of them is decided in this crate. - assert_eq!(viewer_for(Path::new("/a/d.pdf")), "mppdf"); - assert_eq!(viewer_for(Path::new("/a/t.csv")), "mpsheets"); - assert_eq!(viewer_for(Path::new("/a/p.html")), "mpbrowser"); + assert_eq!(viewer_for(Path::new("/a/d.pdf")), "pdf"); + assert_eq!(viewer_for(Path::new("/a/t.csv")), "sheets"); + assert_eq!(viewer_for(Path::new("/a/p.html")), "browser"); } #[test] diff --git a/apps/mpfiles/src/rename.rs b/apps/files/src/rename.rs similarity index 100% rename from apps/mpfiles/src/rename.rs rename to apps/files/src/rename.rs diff --git a/apps/mpfiles/src/sizecache.rs b/apps/files/src/sizecache.rs similarity index 98% rename from apps/mpfiles/src/sizecache.rs rename to apps/files/src/sizecache.rs index 502c62182..b01bf66d0 100644 --- a/apps/mpfiles/src/sizecache.rs +++ b/apps/files/src/sizecache.rs @@ -24,7 +24,6 @@ use std::{ fs, io::{Read, Write}, path::{Path, PathBuf}, - time::{SystemTime, UNIX_EPOCH}, }; use crate::treemap::Node; @@ -61,10 +60,7 @@ pub struct Cached { /// Seconds since the epoch, or 0 when the clock cannot say. pub fn now() -> u64 { - SystemTime::now() - .duration_since(UNIX_EPOCH) - .map(|d| d.as_secs()) - .unwrap_or(0) + crate::vfs::now_secs() } /// "2h ago", "just now" — how a person reads an age. @@ -91,8 +87,7 @@ pub fn age_text(scanned_at: u64) -> String { /// itself: a path can be longer than a filename may be, and can hold every /// character a filename may not. fn cache_path(root: &Path) -> Option { - let home = std::env::var_os("HOME")?; - let dir = PathBuf::from(home).join(".config/mpfiles/sizemaps"); + let dir = crate::model::makepad_home().join("files/sizemaps"); // The scan scope is part of the map's identity: a tree measured with the // system folders in it must never be served as the excluded one, or the // other way round. Both scopes keep their own file, so flipping the diff --git a/apps/mpfiles/src/theme.rs b/apps/files/src/theme.rs similarity index 94% rename from apps/mpfiles/src/theme.rs rename to apps/files/src/theme.rs index 919ce1218..381c449dd 100644 --- a/apps/mpfiles/src/theme.rs +++ b/apps/files/src/theme.rs @@ -1,6 +1,6 @@ -//! The palette. mpwm exports its active `theme.splash` as MPWM_THEME_SPLASH; -//! `mp_theme` line-scans it and retints the stock widgets, and this module -//! publishes the same colors as `mod.mpf.*` so mpfiles' own chrome — which is +//! The palette. wm exports its active `theme.splash` as MAKEPAD_WM_THEME_SPLASH; +//! `makepad_wm_theme` line-scans it and retints the stock widgets, and this module +//! publishes the same colors as `mod.mpf.*` so files' own chrome — which is //! all custom views — follows the desktop theme too. Standalone runs get //! Tokyo Night, so the app is dark and square either way. @@ -64,7 +64,7 @@ impl Default for Palette { } impl Palette { - /// The fallback theme, matching mpwm's default. + /// The fallback theme, matching wm's default. pub fn tokyo_night() -> Self { Self::derive( "#7aa2f7", "#1a1b26", "#16161e", "#24283b", "#a9b1d6", "#c0caf5", "#565f89", "#414868", @@ -79,9 +79,12 @@ impl Palette { PALETTE.get_or_init(Palette::load) } - /// The palette mpwm exported for this process, or Tokyo Night. + /// The palette wm exported for this process, or Tokyo Night. pub fn load() -> Self { - let Some(p) = mp_theme::current() else { + if crate::vfs::demo_requested() { + return Self::tokyo_night(); + } + let Some(p) = makepad_wm_theme::current() else { return Self::tokyo_night(); }; Self::derive( @@ -182,7 +185,7 @@ impl Palette { ); vm.eval(ScriptMod { cargo_manifest_path: env!("CARGO_MANIFEST_DIR").to_string(), - module_path: "mpfiles_palette".to_string(), + module_path: "files_palette".to_string(), file: "palette.splash".to_string(), line: 0, column: 0, @@ -190,7 +193,7 @@ impl Palette { values: vec![], }); for e in vm.take_errors() { - log!("mpfiles palette: {}", e); + log!("files palette: {}", e); } } } diff --git a/apps/mpfiles/src/thumbs.rs b/apps/files/src/thumbs.rs similarity index 64% rename from apps/mpfiles/src/thumbs.rs rename to apps/files/src/thumbs.rs index 0d223ffe8..bc59c729f 100644 --- a/apps/mpfiles/src/thumbs.rs +++ b/apps/files/src/thumbs.rs @@ -1,14 +1,14 @@ //! Thumbnails and type icons — one widget draws both. //! -//! Pictures get a real thumbnail: the file is read and decoded on a worker -//! thread (never the UI thread), box-filtered down to at most [`THUMB_PX`] on -//! its long edge, and handed back as BGRA pixels the UI turns into a texture. +//! Pictures get a real thumbnail: native files are decoded on workers; demo +//! paths select from a tiny embedded pool and decode inline. Both are +//! box-filtered down to at most [`THUMB_PX`] on their long edge and handed +//! back as BGRA pixels the UI turns into a texture. //! Decoded thumbs live in a bounded LRU so browsing a 20k-file photo folder //! costs a fixed amount of GPU memory. //! -//! Playable video gets the same treatment through the platform's standalone -//! file decoder: its first frame is the thumbnail. Videos the decoder does not -//! demux keep the film-strip icon. +//! Native video uses the platform decoder's first frame. Demo video uses one +//! embedded still and never opens a demuxer or decoder. //! //! Everything else gets its kind's SVG, drawn by the same `Image` widget — //! which is why [`MpfThumb`] exists: it remembers what it is already showing, @@ -16,7 +16,8 @@ //! texture (and, through `Image::set_texture`'s redraw, spin the frame clock). use makepad_widgets::*; -use makepad_widgets::makepad_platform::thread::SignalToUI; +use makepad_widgets::makepad_platform::thread::{SignalToUI, ThreadOptions}; +#[cfg(not(target_arch = "wasm32"))] use makepad_widgets::makepad_platform::video_file::VideoFileDecoder; use std::{ @@ -26,7 +27,6 @@ use std::{ mpsc::{channel, Receiver, Sender}, Arc, OnceLock, }, - thread, }; use crate::model::FileKind; @@ -78,12 +78,15 @@ struct CacheSlot { /// The thumbnail cache: request pictures, drain finished decodes, look them up. pub struct Thumbs { + done_tx: Sender, senders: Vec>, results: Receiver, slots: HashMap, inflight: HashMap, tick: u64, next_worker: usize, + instant: bool, + started: bool, } impl Default for Thumbs { @@ -95,36 +98,57 @@ impl Default for Thumbs { impl Thumbs { pub fn new() -> Self { let (done_tx, results) = channel::(); - let mut senders = Vec::with_capacity(WORKERS); - for _ in 0..WORKERS { - let (tx, rx) = channel::(); - let done = done_tx.clone(); - // A dedicated channel per worker (instead of one shared, mutex-guarded - // receiver) keeps a blocking `recv` from serializing the pool. - thread::spawn(move || { - while let Ok(path) = rx.recv() { - let pixels = decode_thumb(&path); - if done.send(ThumbDone { path, pixels }).is_err() { - return; - } - SignalToUI::set_ui_signal(); - } - }); - senders.push(tx); - } Self { - senders, + done_tx, + senders: Vec::new(), results, slots: HashMap::new(), inflight: HashMap::new(), tick: 0, next_worker: 0, + instant: false, + started: false, + } + } + + fn ensure_started(&mut self, cx: &Cx) { + if self.started { + return; + } + self.started = true; + self.instant = crate::vfs::vfs().is_instant(); + if self.instant { + return; + } + let spawner = cx.thread_spawner(); + self.senders.reserve(WORKERS); + for index in 0..WORKERS { + let (tx, rx) = channel::(); + let done = self.done_tx.clone(); + // A dedicated channel per worker (instead of one shared, mutex-guarded + // receiver) keeps a blocking `recv` from serializing the pool. + if let Ok(handle) = spawner.spawn_worker( + ThreadOptions { name: Some(format!("files-thumb-{index}").into()), ..Default::default() }, + move || { + while let Ok(path) = rx.recv() { + let pixels = decode_thumb(&path); + if done.send(ThumbDone { path, pixels }).is_err() { + return; + } + SignalToUI::set_ui_signal(); + } + }, + ) { + handle.detach(); + } + self.senders.push(tx); } } /// The texture for `path` if it is decoded; queues a decode if it is not. /// Returns `None` while the decode is pending or after it failed. - pub fn get_or_request(&mut self, path: &Path) -> Option { + pub fn get_or_request(&mut self, cx: &mut Cx, path: &Path) -> Option { + self.ensure_started(cx); self.tick += 1; let tick = self.tick; if let Some(slot) = self.slots.get_mut(path) { @@ -135,6 +159,12 @@ impl Thumbs { return None; } self.inflight.insert(path.to_path_buf(), ()); + if self.instant { + let item = ThumbDone { path: path.to_path_buf(), pixels: decode_thumb(path) }; + self.finish(cx, item); + self.evict(); + return self.slots.get(path).and_then(|slot| slot.texture.clone()); + } let worker = self.next_worker % self.senders.len(); self.next_worker = self.next_worker.wrapping_add(1); let _ = self.senders[worker].send(path.to_path_buf()); @@ -149,26 +179,30 @@ impl Thumbs { return false; } for item in done { - self.inflight.remove(&item.path); - let texture = item.pixels.map(|p| { - Texture::new_with_format( - cx, - TextureFormat::VecBGRAu8_32 { - width: p.width, - height: p.height, - data: Some(p.data), - updated: TextureUpdated::Full, - }, - ) - }); - self.tick += 1; - let tick = self.tick; - self.slots.insert(item.path, CacheSlot { texture, tick }); + self.finish(cx, item); } self.evict(); true } + fn finish(&mut self, cx: &mut Cx, item: ThumbDone) { + self.inflight.remove(&item.path); + let texture = item.pixels.map(|p| { + Texture::new_with_format( + cx, + TextureFormat::VecBGRAu8_32 { + width: p.width, + height: p.height, + data: Some(p.data), + updated: TextureUpdated::Full, + }, + ) + }); + self.tick += 1; + let tick = self.tick; + self.slots.insert(item.path, CacheSlot { texture, tick }); + } + /// Drop the least recently looked-at slots down to the cap. fn evict(&mut self) { while self.slots.len() > THUMB_CACHE_CAP { @@ -190,30 +224,90 @@ impl Thumbs { } } -/// Read, decode and downscale one file's picture. Runs on a worker thread. -/// -/// The *kind* comes from the name the browser shows, and the *bytes* from -/// whatever file actually backs it — the two are the same thing on a real -/// disk and deliberately different in the demo, which is what lets a made-up -/// photo have a real thumbnail. -fn decode_thumb(path: &Path) -> Option { - let real = crate::vfs::vfs().real_path(path); - if crate::model::is_playable_video(path) { - // A video is never read whole — the decoder demuxes to the first - // frame — so the picture-sized file cap does not apply here. - return decode_video_thumb(&real); +trait ThumbSource { + fn decode(&self, path: &Path) -> Option; +} + +struct NativeThumbSource<'a>(&'a dyn crate::vfs::Vfs); + +impl ThumbSource for NativeThumbSource<'_> { + fn decode(&self, path: &Path) -> Option { + if crate::model::is_playable_video(path) { + #[cfg(not(target_arch = "wasm32"))] + return decode_video_thumb(&self.0.native_path(path).ok()?); + #[cfg(target_arch = "wasm32")] + return None; + } + let entry = self.0.stat(path).ok()?; + if entry.is_dir || entry.size > THUMB_MAX_FILE_BYTES { + return None; + } + let data = self.0.read_bytes(path, THUMB_MAX_FILE_BYTES as usize + 1).ok()?; + if data.len() as u64 > THUMB_MAX_FILE_BYTES { + return None; + } + decode_image_bytes(&data) } - let meta = std::fs::metadata(&real).ok()?; - if meta.len() > THUMB_MAX_FILE_BYTES { - return None; +} + +struct DemoThumbSource; + +impl DemoThumbSource { + const IMAGES: [&'static [u8]; 8] = [ + include_bytes!("../demos/rubber-duck-illustration.png"), + include_bytes!("../demos/amusement-ride.jpg"), + include_bytes!("../demos/royal-esplanade-panorama.jpg"), + include_bytes!(concat!(env!("OUT_DIR"), "/aurora-vignette.png")), + include_bytes!(concat!(env!("OUT_DIR"), "/canyon-vignette.png")), + include_bytes!(concat!(env!("OUT_DIR"), "/lagoon-vignette.png")), + include_bytes!(concat!(env!("OUT_DIR"), "/meadow-vignette.png")), + include_bytes!(concat!(env!("OUT_DIR"), "/twilight-vignette.png")), + ]; + const VIDEO_STILL: &'static [u8] = include_bytes!(concat!(env!("OUT_DIR"), "/cinema-still.png")); + + fn bytes(path: &Path) -> &'static [u8] { + if crate::model::is_playable_video(path) { + return Self::VIDEO_STILL; + } + let hash = path + .as_os_str() + .to_string_lossy() + .bytes() + .fold(0xcbf2_9ce4_8422_2325u64, |hash, byte| { + (hash ^ byte as u64).wrapping_mul(0x1000_0000_01b3) + }); + Self::IMAGES[hash as usize % Self::IMAGES.len()] } - let data = std::fs::read(&real).ok()?; - let image = decode_image_from_data(&data).ok()?; +} + +impl ThumbSource for DemoThumbSource { + fn decode(&self, path: &Path) -> Option { + decode_image_bytes(Self::bytes(path)) + } +} + +fn decode_image_bytes(data: &[u8]) -> Option { + let image = decode_image_from_data(data).ok()?; Some(downscale(image.width, image.height, &image.data)) } +/// Dispatch at the filesystem seam: a demo path never becomes a host path, +/// and a video in the closed demo is just one embedded still. +fn decode_thumb(path: &Path) -> Option { + decode_thumb_from(crate::vfs::vfs().as_ref(), path) +} + +pub(crate) fn decode_thumb_from(fs: &dyn crate::vfs::Vfs, path: &Path) -> Option { + if fs.is_demo() { + DemoThumbSource.decode(path) + } else { + NativeThumbSource(fs).decode(path) + } +} + /// The first frame of a video, through the platform's hardware file decoder — /// the same seam the importer's video probe uses, minus its crate. +#[cfg(not(target_arch = "wasm32"))] fn decode_video_thumb(path: &Path) -> Option { let mut decoder = VideoFileDecoder::open(path.to_str()?).ok()?; let frame = decoder.next_frame().ok()??; @@ -377,7 +471,7 @@ pub fn fill_thumb(cx: &mut Cx, slot: &WidgetRef, entry: &crate::model::FileEntry return; }; if crate::model::is_thumbnailable(&entry.path) { - if let Some(texture) = thumbs.get_or_request(&entry.path) { + if let Some(texture) = thumbs.get_or_request(cx, &entry.path) { thumb.show_thumb(cx, &entry.path, texture); return; } @@ -429,4 +523,27 @@ mod tests { // load be skipped on repopulate. assert!(Arc::ptr_eq(&kind_svg(FileKind::Audio), &kind_svg(FileKind::Audio))); } + + #[test] + fn demo_source_pool_is_distinct_decodable_and_uses_one_video_still() { + let mut seen = std::collections::HashSet::new(); + let mut decoded = std::collections::HashSet::new(); + assert_eq!(DemoThumbSource::IMAGES.len() + 1, 9); + for (slot, bytes) in DemoThumbSource::IMAGES.iter().copied().enumerate() { + assert!(seen.insert(bytes), "demo picture slots contain duplicate bytes at slot {slot}"); + let image = decode_image_from_data(bytes).unwrap_or_else(|_| panic!("demo picture slot {slot} did not decode")); + assert!(decoded.insert(image.data), "demo picture slots decode to duplicate pixels at slot {slot}"); + } + assert!(seen.insert(DemoThumbSource::VIDEO_STILL), "the designated video still duplicates a picture slot"); + let video = decode_image_from_data(DemoThumbSource::VIDEO_STILL).expect("the designated video still did not decode"); + assert!(decoded.insert(video.data), "the designated video still decodes to duplicate picture pixels"); + + for path in [Path::new("/Demo/Videos/clip-0001.mp4"), Path::new("/Demo/Videos/camera-0003.mkv")] { + assert_eq!(DemoThumbSource::bytes(path), DemoThumbSource::VIDEO_STILL); + assert!(DemoThumbSource.decode(path).is_some()); + } + let total_bytes = DemoThumbSource::IMAGES.iter().map(|bytes| bytes.len()).sum::() + + DemoThumbSource::VIDEO_STILL.len(); + assert!(total_bytes < 1_500_000, "embedded demo picture pool is {total_bytes} bytes"); + } } diff --git a/apps/mpfiles/src/treemap.rs b/apps/files/src/treemap.rs similarity index 96% rename from apps/mpfiles/src/treemap.rs rename to apps/files/src/treemap.rs index 5c58412b6..03b49952a 100644 --- a/apps/mpfiles/src/treemap.rs +++ b/apps/files/src/treemap.rs @@ -29,10 +29,12 @@ use std::{ atomic::{AtomicBool, AtomicU32, Ordering}, Condvar, Mutex, }, - thread, - time::{Duration, Instant}, + time::Duration, }; +use makepad_widgets::makepad_platform::thread::{Lane, TaskPool}; +use makepad_widgets::Cx; + /// A rectangle in treemap space. Plain `f64` so this module stays free of /// any UI vector type — the view converts to its own types at the boundary. #[derive(Clone, Copy, Debug, PartialEq, Default)] @@ -452,6 +454,7 @@ fn read_listing( rules: &ScanRules, device: Option, growth: &mut Growth, + pool: Option<&TaskPool>, ) -> Listing { let read_dir = match fs::read_dir(dir) { Ok(read_dir) => read_dir, @@ -497,11 +500,18 @@ fn read_listing( // it to matter they are made to wait in parallel rather than in turn — // a folder with a quarter of a million files is otherwise one thread at // disk latency while five others have nothing to do. - if found.len() >= STAT_PARALLEL_MIN { + if let (true, Some(pool)) = (found.len() >= STAT_PARALLEL_MIN, pool) { + // Independent, so a folder big enough for it to matter waits for its + // `lstat`s in parallel rather than in turn — the caller-helping + // `fan_out` replacement for the `std::thread::scope` this used to be. let chunk = found.len().div_ceil(STAT_THREADS); - thread::scope(|scope| { - for slice in found.chunks_mut(chunk) { - scope.spawn(move || stat_all(slice, device)); + let slices: Vec> = found.chunks_mut(chunk).map(Some).collect(); + let count = slices.len(); + let slices = Mutex::new(slices); + pool.fan_out(Lane::Heavy, count, |index| { + let slice = slices.lock().unwrap_or_else(|e| e.into_inner())[index].take(); + if let Some(slice) = slice { + stat_all(slice, device); } }); } else { @@ -550,10 +560,10 @@ struct Found { keep: bool, } -/// A SystemTime as whole minutes since the epoch, saturating; 0 for a time -/// the filesystem would not say. -fn minutes_since_epoch(time: std::io::Result) -> u32 { - time.ok() +/// A file mtime as whole minutes since the epoch, saturating; 0 when the +/// filesystem would not say. +fn modified_minutes(metadata: &fs::Metadata) -> u32 { + metadata.modified().ok() .and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok()) .map(|d| (d.as_secs() / 60).min(u32::MAX as u64) as u32) .unwrap_or(0) @@ -574,7 +584,7 @@ fn stat_all(slice: &mut [Found], device: Option) { item.size = meta.len(); // Free with the stat already in hand — this is what lets the map // answer "show me only what's new". - item.modified = minutes_since_epoch(meta.modified()); + item.modified = modified_minutes(&meta); } } } @@ -605,8 +615,8 @@ struct Growth<'a> { at: Vec, size: u64, files: u32, - due: Instant, - pace_due: Instant, + due: f64, + pace_due: f64, } impl<'a> Growth<'a> { @@ -617,8 +627,8 @@ impl<'a> Growth<'a> { at: Vec::new(), size: 0, files: 0, - due: Instant::now() + GROW_EVERY, - pace_due: Instant::now(), + due: Cx::monotonic_now() + GROW_EVERY.as_secs_f64(), + pace_due: Cx::monotonic_now(), } } @@ -635,9 +645,9 @@ impl<'a> Growth<'a> { /// second in a build tree and the number on screen does not need to. fn pace(&mut self) { let folders_left = self.open.load(Ordering::Relaxed); - let now = Instant::now(); + let now = Cx::monotonic_now(); if now >= self.pace_due || folders_left == 0 { - self.pace_due = now + GROW_EVERY; + self.pace_due = now + GROW_EVERY.as_secs_f64(); (self.sink)(ScanStep::Pace { folders_left }); } } @@ -645,11 +655,11 @@ impl<'a> Growth<'a> { fn add(&mut self, size: u64) { self.size += size; self.files += 1; - let now = Instant::now(); + let now = Cx::monotonic_now(); if now < self.due { return; } - self.due = now + GROW_EVERY; + self.due = now + GROW_EVERY.as_secs_f64(); // The queue depth rides along on the same clock, so it keeps moving // even while this thread is stuck inside one huge directory. self.pace(); @@ -716,6 +726,7 @@ pub fn scan_stream( rules: &ScanRules, cancel: &AtomicBool, sink: &(dyn Fn(ScanStep) + Sync), + pool: &TaskPool, ) -> bool { if cancel.load(Ordering::Relaxed) { return false; @@ -730,19 +741,16 @@ pub fn scan_stream( }); let wake = Condvar::new(); let open = AtomicU32::new(1); - thread::scope(|scope| { - for _ in 0..SCAN_THREADS { - let queue = &queue; - let wake = &wake; - let open = &open; - scope.spawn(move || { - let mut growth = Growth::new(sink, open); - while let Some(job) = take(queue, wake, cancel) { - let children = run_job(job, rules, device, cancel, sink, &mut growth); - finish(queue, wake, children, open); - growth.pace(); - } - }); + // The walk's own worker loop, run on the pool AND the calling thread — + // the caller-helping `fan_out` replacement for the `std::thread::scope` + // this used to be. `scan_stream` only ever runs as a Heavy pool job + // itself (see `treemap_view`), never on the UI thread. + pool.fan_out(Lane::Heavy, SCAN_THREADS, |_index| { + let mut growth = Growth::new(sink, &open); + while let Some(job) = take(&queue, &wake, cancel) { + let children = run_job(job, rules, device, cancel, sink, &mut growth, pool); + finish(&queue, &wake, children, &open); + growth.pace(); } }); !cancel.load(Ordering::Relaxed) @@ -797,6 +805,7 @@ fn run_job( cancel: &AtomicBool, sink: &(dyn Fn(ScanStep) + Sync), growth: &mut Growth, + pool: &TaskPool, ) -> Vec { if cancel.load(Ordering::Relaxed) { return Vec::new(); @@ -805,7 +814,7 @@ fn run_job( // read, which on a folder with a quarter of a million files is most of // the time this job takes. growth.start(&job.at); - let listing = read_listing(&job.path, rules, device, growth); + let listing = read_listing(&job.path, rules, device, growth, Some(pool)); growth.start(&[]); sink(ScanStep::Opened { at: job.at.clone(), @@ -881,7 +890,9 @@ fn scan_blocking( return None; } let idle = AtomicU32::new(0); - let listing = read_listing(dir, rules, device, &mut Growth::new(&|_| {}, &idle)); + // The simple blocking form has no pool of its own to fan a big + // directory's `lstat`s out on; it stats serially. + let listing = read_listing(dir, rules, device, &mut Growth::new(&|_| {}, &idle), None); let denied = listing.denied; let mut children = Vec::with_capacity(listing.entries.len()); for entry in listing.entries { @@ -1844,6 +1855,16 @@ mod tests { use super::*; use std::sync::Mutex; + /// `scan_stream` fans out on its pool and helps with the work itself, so + /// it must never run on the thread that owns the pool (`fan_out` asserts + /// against that). Give it a pool built here and a dedicated thread to + /// call from, same as production code does with a Heavy pool job. + fn with_pool(f: impl FnOnce(&TaskPool) -> R + Send) -> R { + let cx = Cx::new(Box::new(|_, _| {})); + let pool = cx.task_pool(); + std::thread::scope(|scope| scope.spawn(|| f(&pool)).join().unwrap()) + } + fn leaf(name: &str, size: u64) -> Node { Node::file(name.to_string(), 0, size) } @@ -1980,7 +2001,7 @@ mod tests { // The cost canary the bundle floor used to be for: a quarter-million // files in one folder must cost the layout what its pixels cost, not // what its listing costs. Run by hand with - // `cargo test -p mpfiles --release -- --ignored --nocapture`. + // `cargo test -p files --release -- --ignored --nocapture`. #[test] #[ignore] fn packing_cost_canary_200k() { @@ -2000,7 +2021,7 @@ mod tests { // Near: 64x in, anchored inside the crowd so its files fill the panel. let near = Rect { x: -20_000.0, y: -20_000.0, w: 1200.0 * 64.0, h: 800.0 * 64.0 }; for (name, area) in [("far", far), ("near", near)] { - let t = std::time::Instant::now(); + let t = Cx::monotonic_now(); let mut cells = 0usize; const RUNS: u32 = 20; for _ in 0..RUNS { @@ -2008,7 +2029,7 @@ mod tests { } println!( "200k-folder {name}: {:.2}ms per layout, {cells} cells", - t.elapsed().as_secs_f64() * 1000.0 / RUNS as f64 + (Cx::monotonic_now() - t) * 1000.0 / RUNS as f64 ); } } @@ -2433,7 +2454,7 @@ mod tests { fn temp_root(tag: &str) -> PathBuf { let root = std::env::temp_dir().join(format!( - "mpfiles-treemap-{tag}-{}-{:?}", + "files-treemap-{tag}-{}-{:?}", std::process::id(), std::thread::current().id() )); @@ -2524,8 +2545,10 @@ mod tests { let cancel = AtomicBool::new(false); let steps = Mutex::new(Vec::new()); - let ok = scan_stream(&root, &open_rules(), &cancel, &|step| { - steps.lock().unwrap().push(step); + let ok = with_pool(|pool| { + scan_stream(&root, &open_rules(), &cancel, &|step| { + steps.lock().unwrap().push(step); + }, pool) }); assert!(ok); @@ -2565,8 +2588,10 @@ mod tests { let cancel = AtomicBool::new(false); let steps = Mutex::new(Vec::new()); - assert!(scan_stream(&root, &open_rules(), &cancel, &|step| { - steps.lock().unwrap().push(step); + assert!(with_pool(|pool| { + scan_stream(&root, &open_rules(), &cancel, &|step| { + steps.lock().unwrap().push(step); + }, pool) })); let mut opened_ats: Vec> = Vec::new(); @@ -2594,14 +2619,16 @@ mod tests { let cancel = AtomicBool::new(false); let first = Mutex::new(None); - scan_stream(&root, &open_rules(), &cancel, &|step| { - let mut slot = first.lock().unwrap(); - if slot.is_none() { - *slot = Some(match step { - ScanStep::Opened { at, children, .. } => (at, children.len()), - other => panic!("first step was {other:?}, not the root listing"), - }); - } + with_pool(|pool| { + scan_stream(&root, &open_rules(), &cancel, &|step| { + let mut slot = first.lock().unwrap(); + if slot.is_none() { + *slot = Some(match step { + ScanStep::Opened { at, children, .. } => (at, children.len()), + other => panic!("first step was {other:?}, not the root listing"), + }); + } + }, pool) }); let (at, count) = first.into_inner().unwrap().expect("no steps at all"); assert!(at.is_empty()); @@ -2615,7 +2642,7 @@ mod tests { let root = temp_root("stream-cancel"); sample_tree(&root); let cancel = AtomicBool::new(true); - assert!(!scan_stream(&root, &open_rules(), &cancel, &|_| {})); + assert!(!with_pool(|pool| scan_stream(&root, &open_rules(), &cancel, &|_| {}, pool))); fs::remove_dir_all(&root).ok(); } @@ -2733,8 +2760,10 @@ mod tests { fs::write(root.join("sub/a.txt"), b"aa").unwrap(); let cancel = AtomicBool::new(false); let steps = Mutex::new(Vec::new()); - assert!(scan_stream(&root, &open_rules(), &cancel, &|step| { - steps.lock().unwrap().push(step); + assert!(with_pool(|pool| { + scan_stream(&root, &open_rules(), &cancel, &|step| { + steps.lock().unwrap().push(step); + }, pool) })); let mut tree = Node::dir("root".into(), 0); for step in steps.into_inner().unwrap() { @@ -2743,7 +2772,7 @@ mod tests { // Written moments ago: the minutes-since-epoch must be recent and // must have reached the root through the roll-up. assert!(tree.modified > 0); - let now = super::minutes_since_epoch(Ok(std::time::SystemTime::now())); + let now = (Cx::time_now().max(0.0) as u64 / 60).min(u32::MAX as u64) as u32; assert!(now - tree.modified < 5, "root mtime {} vs now {}", tree.modified, now); fs::remove_dir_all(&root).ok(); } diff --git a/apps/mpfiles/src/treemap_view.rs b/apps/files/src/treemap_view.rs similarity index 89% rename from apps/mpfiles/src/treemap_view.rs rename to apps/files/src/treemap_view.rs index 3a207d2a3..8ca580557 100644 --- a/apps/mpfiles/src/treemap_view.rs +++ b/apps/files/src/treemap_view.rs @@ -19,7 +19,7 @@ //! deeper. The layout itself is pure arithmetic and runs inline, throttled //! while a scan is still feeding it. -use makepad_widgets::makepad_platform::thread::SignalToUI; +use makepad_widgets::makepad_platform::thread::{Lane, SignalToUI}; use makepad_widgets::*; use std::{ @@ -30,8 +30,7 @@ use std::{ mpsc::{channel, Receiver, Sender}, Arc, Mutex, }, - thread, - time::{Duration, Instant}, + time::Duration, }; use crate::{ @@ -72,6 +71,13 @@ const RISE: f64 = 11.0; /// same points. Large on purpose: the 3d mode is the ortho map breathing, /// not a flyover. const PERSP_EYE: f64 = 1500.0; +/// Faces are clipped this far in front of the perspective eye before their +/// homogeneous `w` is divided through. Keeping the plane comfortably away +/// from zero also bounds the screen coordinates of an intersection. +const PERSP_NEAR: f64 = 8.0; +/// The perspective eye is raised far enough along its view axis that its +/// vertical height clears the tallest tile top by at least this much. +const CAMERA_SURFACE_CLEARANCE: f64 = 24.0; /// Where the orbit starts and where Esc returns it: enough tilt that height /// reads immediately, nowhere near enough to hide the map behind itself. const DEFAULT_PITCH: f64 = 0.66; @@ -100,13 +106,11 @@ script_mod! { * dense map readable: adjacent tiles of the same hue are separated by * their own shading even where there is no room for a border line. * - * The geometry is a free QUAD, not a rect: the orbit camera hands four - * projected screen corners per instance (c0 top-left, c1 top-right, c2 - * bottom-right, c3 bottom-left) and the vertex stage interpolates them - * bilinearly, so one shared instance batch draws the flat map, the tilted - * plates and the prism walls alike. Because the corners are free, the - * usual vertex-clamp scissor would deform the shape — clipping happens in - * the fragment against the same draw_clip instead. */ + * The geometry is a free QUAD, not a rect: an uncut face supplies its four + * projected corners, while a near-clipped face is triangle-fanned through + * degenerate quads. One shared instance batch still draws the flat map, + * tilted plates and prism walls alike. The panel scissor happens in the + * fragment because clamping these free vertices would deform the shape. */ set_type_default() do #(DrawMapTile::script_shader(vm)) { ..mod.draw.DrawQuad /** the tile's own colour */ @@ -125,12 +129,13 @@ script_mod! { mix(self.c3, self.c2, self.geom.pos.x) self.geom.pos.y ) - self.pos = self.geom.pos - self.scr = p - self.qsize = vec2( - max(length(self.c1 - self.c0), 1.0) - max(length(self.c3 - self.c0), 1.0) + self.pos = mix( + mix(self.u0, self.u1, self.geom.pos.x) + mix(self.u3, self.u2, self.geom.pos.x) + self.geom.pos.y ) + self.scr = p + self.qsize = self.face_size let ps = p + self.draw_list.view_shift self.world = self.draw_list.view_transform * vec4( ps.x @@ -208,6 +213,19 @@ pub struct DrawMapTile { c2: Vec2f, #[live] c3: Vec2f, + /// UVs are explicit because a near-clipped polygon is emitted as a fan + /// of degenerate quads (one real triangle each). + #[live] + u0: Vec2f, + #[live] + u1: Vec2f, + #[live] + u2: Vec2f, + #[live] + u3: Vec2f, + /// Approximate full-face size used by the cushion and border shader. + #[live] + face_size: Vec2f, } /// What a press on the map means to the folder view around it. @@ -348,7 +366,7 @@ pub struct TreemapView { #[rust] stale: bool, #[rust] - last_layout: Option, + last_layout: Option, #[rust] frame: NextFrame, @@ -444,7 +462,7 @@ pub struct TreemapView { zoom_glide: Option, /// Q/E's glide: the yaw the orbit is headed for, and the last tick. #[rust] - yaw_glide: Option<(f64, Instant)>, + yaw_glide: Option<(f64, f64)>, /// The filter tween: where each surviving path was, the cells that are /// leaving (with the rect they were last seen at), and when it started. @@ -453,7 +471,7 @@ pub struct TreemapView { #[rust] tween_leavers: Vec<(Cell, MapRect, f64)>, #[rust] - tween_start: Option, + tween_start: Option, /// A snapshot of the map as it looks right now, taken when the filter /// changes, consumed by the next relayout to aim the tween. #[rust] @@ -530,7 +548,7 @@ struct CrumbHit { struct ZoomGlide { target: f64, anchor: DVec2, - last: Instant, + last: f64, } /// How fast a glide closes on its target: the ease-out's time constant. @@ -578,27 +596,35 @@ struct Cam { sin_pitch: f64, cos_pitch: f64, persp: bool, + /// Distance from the pivot to the perspective eye along the view axis. + eye: f64, } impl Cam { - /// The layout point `p` at elevation `z`, on screen. - fn project(&self, p: DVec2, z: f64) -> DVec2 { + /// Transform a map point into eye-relative view coordinates. `w` is the + /// positive distance in front of the eye, i.e. `-view_z`; clipping at + /// `view_z = -PERSP_NEAR` is therefore `w >= PERSP_NEAR`. + fn view(&self, p: DVec2, z: f64, uv: DVec2) -> FaceVertex { let dx = p.x - self.pivot.x; let dy = p.y - self.pivot.y; let xr = dx * self.cos_yaw - dy * self.sin_yaw; let yr = dx * self.sin_yaw + dy * self.cos_yaw; - let vx = xr; - let vy = yr * self.cos_pitch - z * self.sin_pitch; - if !self.persp { - return dvec2(self.pivot.x + vx, self.pivot.y + vy); - } + let view = dvec2(xr, yr * self.cos_pitch - z * self.sin_pitch); let depth = yr * self.sin_pitch + z * self.cos_pitch; - let s = (PERSP_EYE / (PERSP_EYE - depth)).clamp(0.5, 2.5); - dvec2(self.pivot.x + vx * s, self.pivot.y + vy * s) + FaceVertex { + view, + w: if self.persp { self.eye - depth } else { 1.0 }, + uv, + } + } + + fn project_view(&self, vertex: FaceVertex) -> DVec2 { + let scale = if self.persp { self.eye / vertex.w } else { 1.0 }; + self.pivot + vertex.view * scale } /// The ground point (z = 0) that projects to screen point `s` — the - /// exact inverse of [`Cam::project`], for both projections. + /// exact inverse of the camera projection, for both projections. fn unproject_ground(&self, s: DVec2) -> DVec2 { self.unproject_at(s, 0.0) } @@ -622,15 +648,14 @@ impl Cam { } else { // vy·s = sy with s = E/(E − yr·sinφ − z·cosφ) and // vy = yr·cosφ − z·sinφ is linear in yr once multiplied out. - let denom = PERSP_EYE * self.cos_pitch + sy * self.sin_pitch; + let denom = self.eye * self.cos_pitch + sy * self.sin_pitch; yr = if denom.abs() < 1e-6 { 0.0 } else { - (sy * PERSP_EYE - z * (sy * self.cos_pitch - PERSP_EYE * self.sin_pitch)) + (sy * self.eye - z * (sy * self.cos_pitch - self.eye * self.sin_pitch)) / denom }; - let sc = (PERSP_EYE / (PERSP_EYE - yr * self.sin_pitch - z * self.cos_pitch)) - .clamp(0.5, 2.5); + let sc = self.eye / (self.eye - yr * self.sin_pitch - z * self.cos_pitch); xr = sx / sc; } let dx = xr * self.cos_yaw + yr * self.sin_yaw; @@ -639,42 +664,114 @@ impl Cam { } } -/// One projected face: four screen corners, top-left first, clockwise. -#[derive(Clone, Copy)] -struct Quad { - p: [DVec2; 4], +#[derive(Clone, Copy, Default)] +struct FaceVertex { + /// View-space x/y before the perspective divide. + view: DVec2, + /// Positive homogeneous clip coordinate (`-view_z`). + w: f64, + /// Coordinate on the original, unclipped quad. + uv: DVec2, } -impl Quad { - fn of_rect(cam: &Cam, r: &MapRect, z: f64) -> Quad { - Quad { - p: [ - cam.project(dvec2(r.x, r.y), z), - cam.project(dvec2(r.x + r.w, r.y), z), - cam.project(dvec2(r.x + r.w, r.y + r.h), z), - cam.project(dvec2(r.x, r.y + r.h), z), +#[derive(Clone, Copy, Default)] +struct ProjectedVertex { + screen: DVec2, + w: f64, + uv: DVec2, +} + +/// A projected convex face after view-space near-plane clipping. A quad cut +/// by one plane has at most five vertices. +struct ProjectedFace { + vertices: [ProjectedVertex; 5], + len: usize, + clipped: bool, + size: DVec2, +} + +impl ProjectedFace { + fn of_rect(cam: &Cam, r: &MapRect, z: f64) -> Option { + Self::from_corners( + cam, + [ + (dvec2(r.x, r.y), z), + (dvec2(r.x + r.w, r.y), z), + (dvec2(r.x + r.w, r.y + r.h), z), + (dvec2(r.x, r.y + r.h), z), ], + ) + } + + fn from_corners(cam: &Cam, corners: [(DVec2, f64); 4]) -> Option { + let uvs = [ + dvec2(0.0, 0.0), + dvec2(1.0, 0.0), + dvec2(1.0, 1.0), + dvec2(0.0, 1.0), + ]; + let input = [ + cam.view(corners[0].0, corners[0].1, uvs[0]), + cam.view(corners[1].0, corners[1].1, uvs[1]), + cam.view(corners[2].0, corners[2].1, uvs[2]), + cam.view(corners[3].0, corners[3].1, uvs[3]), + ]; + let clipped = cam.persp && input.iter().any(|v| v.w < PERSP_NEAR); + let mut view_vertices = [FaceVertex::default(); 5]; + let len = if clipped { + clip_near(&input, &mut view_vertices) + } else { + view_vertices[..4].copy_from_slice(&input); + 4 + }; + if len < 3 { + return None; } + + let mut vertices = [ProjectedVertex::default(); 5]; + for (out, vertex) in vertices[..len].iter_mut().zip(&view_vertices[..len]) { + let screen = cam.project_view(*vertex); + if vertex.w <= 0.0 || !screen.x.is_finite() || !screen.y.is_finite() { + return None; + } + *out = ProjectedVertex { screen, w: vertex.w, uv: vertex.uv }; + } + let mut face = Self { + vertices, + len, + clipped, + size: DVec2::default(), + }; + let bounds = face.bounds(); + face.size = if !clipped && len == 4 { + dvec2( + (vertices[1].screen - vertices[0].screen).length().max(1.0), + (vertices[3].screen - vertices[0].screen).length().max(1.0), + ) + } else { + dvec2(bounds.size.x.max(1.0), bounds.size.y.max(1.0)) + }; + Some(face) } fn bounds(&self) -> Rect { - let mut min = self.p[0]; - let mut max = self.p[0]; - for p in &self.p[1..] { - min.x = min.x.min(p.x); - min.y = min.y.min(p.y); - max.x = max.x.max(p.x); - max.y = max.y.max(p.y); + let mut min = self.vertices[0].screen; + let mut max = min; + for vertex in &self.vertices[1..self.len] { + min.x = min.x.min(vertex.screen.x); + min.y = min.y.min(vertex.screen.y); + max.x = max.x.max(vertex.screen.x); + max.y = max.y.max(vertex.screen.y); } Rect { pos: min, size: max - min } } - /// Whether `at` is inside this (convex) face, either winding. + /// Whether `at` is inside this convex face, either winding. fn contains(&self, at: DVec2) -> bool { let mut sign = 0.0f64; - for i in 0..4 { - let a = self.p[i]; - let b = self.p[(i + 1) % 4]; + for i in 0..self.len { + let a = self.vertices[i].screen; + let b = self.vertices[(i + 1) % self.len].screen; let cross = (b.x - a.x) * (at.y - a.y) - (b.y - a.y) * (at.x - a.x); if cross.abs() < 1e-9 { continue; @@ -689,14 +786,62 @@ impl Quad { } } +/// Sutherland-Hodgman clipping against `view_z = -PERSP_NEAR`, expressed as +/// the equivalent positive-w half-space. +fn clip_near(input: &[FaceVertex; 4], output: &mut [FaceVertex; 5]) -> usize { + let mut len = 0; + let mut previous = input[3]; + let mut previous_inside = previous.w >= PERSP_NEAR; + for ¤t in input { + let current_inside = current.w >= PERSP_NEAR; + if previous_inside != current_inside { + let t = (PERSP_NEAR - previous.w) / (current.w - previous.w); + output[len] = FaceVertex { + view: previous.view + (current.view - previous.view) * t, + w: PERSP_NEAR, + uv: previous.uv + (current.uv - previous.uv) * t, + }; + len += 1; + } + if current_inside { + output[len] = current; + len += 1; + } + previous = current; + previous_inside = current_inside; + } + len +} + +/// One projected face: four screen corners, top-left first, clockwise. +#[derive(Clone, Copy)] +struct Quad { + p: [DVec2; 4], +} + +impl Quad { + fn bounds(&self) -> Rect { + let mut min = self.p[0]; + let mut max = self.p[0]; + for p in &self.p[1..] { + min.x = min.x.min(p.x); + min.y = min.y.min(p.y); + max.x = max.x.max(p.x); + max.y = max.y.max(p.y); + } + Rect { pos: min, size: max - min } + } +} + /// How the map is projected onto the panel. #[derive(Clone, Copy, Debug, Default, PartialEq)] pub enum MapProjection { /// The flat map — exactly the 2D treemap. - #[default] Flat, /// 2.5D: every cell extrudes straight up by its nesting depth, showing a - /// darker riser below its plate. Deep tangles read as towers. + /// darker riser below its plate. Deep tangles read as towers. The default: + /// the depth of a tree is the first thing the map should show. + #[default] Ortho, /// The same prisms through a gentle straight-down perspective: higher /// plates swell and lean away from the middle of the panel. @@ -772,7 +917,7 @@ impl TreemapView { if root.as_os_str().is_empty() { return; } - crate::sizecache::forget(&root); + let _ = crate::vfs::vfs().forget_scan_cache(&root); self.begin(cx, &root, true); } @@ -833,12 +978,15 @@ impl TreemapView { return; }; let root = self.root.clone(); - thread::spawn(move || { + let instant = crate::vfs::vfs().is_instant(); + let pool = cx.task_pool(); + let scan_pool = pool.clone(); + let scan = move || { // The four scan threads all report through here, so the channel // and the signal clock live behind one lock. Waking the UI is the // expensive half and is what gets rate-limited; the steps // themselves queue as fast as the disk produces them. - let gate = Mutex::new(Instant::now()); + let gate = Mutex::new(Cx::monotonic_now()); let sink = |step: ScanStep| { if sender .send(ScanMessage { @@ -851,19 +999,19 @@ impl TreemapView { return; } let mut due = gate.lock().unwrap_or_else(|e| e.into_inner()); - let now = Instant::now(); + let now = Cx::monotonic_now(); if now >= *due { - *due = now + SIGNAL_EVERY; + *due = now + SIGNAL_EVERY.as_secs_f64(); SignalToUI::set_ui_signal(); } }; // The saved map first, and off the UI thread: decoding a home // directory's worth of tree is a tenth of a second of work that // has no business happening between two frames. - let cached = if fresh || crate::vfs::is_demo() { + let cached = if fresh { None } else { - crate::sizecache::load(&root) + crate::vfs::vfs().load_scan_cache(&root).ok().flatten() }; if let Some(cached) = cached { let _ = sender.send(ScanMessage { @@ -884,14 +1032,29 @@ impl TreemapView { SignalToUI::set_ui_signal(); return; } - let ok = crate::vfs::vfs().scan_stream(&root, &cancel, &sink); + let ok = crate::vfs::vfs().scan_stream(&root, &cancel, &sink, &scan_pool); let _ = sender.send(ScanMessage { generation, step: None, finished: Some(if ok { Outcome::Scanned } else { Outcome::Failed }), }); SignalToUI::set_ui_signal(); - }); + }; + if instant { + scan(); + self.drain(cx); + } else { + match pool.submit(Lane::Heavy, scan) { + Ok(handle) => handle.detach(), + Err(_) => { + // The pool refused the job (closed or saturated): don't + // spin the "Scanning…" state forever with nothing behind it. + self.scanning = false; + self.cancel = None; + self.error = Some("background scan unavailable".to_string()); + } + } + } self.redraw(cx); } @@ -978,7 +1141,7 @@ impl TreemapView { match outcome { Outcome::Scanned => { self.scanned_at = crate::sizecache::now(); - self.save_cache(); + self.save_cache(cx); } Outcome::Loaded { scanned_at } => self.scanned_at = scanned_at, Outcome::Failed => { @@ -994,7 +1157,7 @@ impl TreemapView { } // While the walk is running the tree changes far faster than the // picture needs to; a finished scan always redraws at once. - if finished || self.layout_is_due() { + if finished || self.layout_is_due(cx.seconds_since_app_start()) { self.redraw(cx); } else { // Nothing gets lost: the trailing update is picked up on the next @@ -1009,12 +1172,12 @@ impl TreemapView { /// second; once nothing is feeding it any more there is nothing to /// throttle, and a map still showing a mid-scan snapshot after the walk /// has finished would be quietly, plausibly wrong. - fn layout_is_due(&self) -> bool { + fn layout_is_due(&self, now: f64) -> bool { if !self.scanning { return true; } match self.last_layout { - Some(at) => at.elapsed() >= RELAYOUT_EVERY, + Some(at) => now - at >= RELAYOUT_EVERY.as_secs_f64(), None => true, } } @@ -1253,7 +1416,7 @@ impl TreemapView { // Calm unless the tree itself changed underneath the gesture — // a camera settle re-derives the same picture at more detail. self.tween_calm = !self.stale; - self.tween_capture = Some(self.visual_snapshot()); + self.tween_capture = Some(self.visual_snapshot(cx.seconds_since_app_start())); } self.stale = true; self.last_layout = None; @@ -1263,12 +1426,12 @@ impl TreemapView { /// Mid-gesture, whether the coarse layout refresh may run: something to /// refresh — the layout spent, or the tree changed under the scan — and /// the cadence has passed. - fn motion_refresh_due(&self) -> bool { + fn motion_refresh_due(&self, now: f64) -> bool { if !self.layout_spent() && !self.stale { return false; } match self.last_layout { - Some(at) => at.elapsed() >= MOTION_RELAYOUT, + Some(at) => now - at >= MOTION_RELAYOUT.as_secs_f64(), None => true, } } @@ -1347,6 +1510,13 @@ impl TreemapView { MapProjection::Flat => (0.0, 0.0), _ => (self.yaw, self.pitch), }; + let max_surface = self + .cells + .iter() + .map(|cell| self.elev(cell.depth)) + .chain(self.tween_from.values().map(|from| self.elev_f(from.depth))) + .chain(self.tween_leavers.iter().map(|(_, _, depth)| self.elev_f(*depth))) + .fold(0.0f64, f64::max); Cam { pivot: dvec2( body.pos.x + body.size.x * 0.5, @@ -1357,6 +1527,11 @@ impl TreemapView { sin_pitch: pitch.sin(), cos_pitch: pitch.cos(), persp: self.projection == MapProjection::Persp, + eye: if self.projection == MapProjection::Persp { + constrained_eye(pitch, max_surface) + } else { + PERSP_EYE + }, } } @@ -1398,7 +1573,7 @@ impl TreemapView { } if dpitch == 0.0 { let base = self.yaw_glide.map_or(self.yaw, |(target, _)| target); - self.yaw_glide = Some((wrap_angle(base + dyaw), Instant::now())); + self.yaw_glide = Some((wrap_angle(base + dyaw), cx.seconds_since_app_start())); self.frame = cx.new_next_frame(); return; } @@ -1409,8 +1584,8 @@ impl TreemapView { /// keep the frame clock alive until both arrive. fn step_glides(&mut self, cx: &mut Cx) { if let Some(mut glide) = self.zoom_glide.take() { - let now = Instant::now(); - let dt = now.duration_since(glide.last).as_secs_f64().min(0.1); + let now = cx.seconds_since_app_start(); + let dt = (now - glide.last).clamp(0.0, 0.1); glide.last = now; let current = self.cam_scale.max(1.0); // Zoom lives in ratio space: equal glide time closes an equal @@ -1431,8 +1606,8 @@ impl TreemapView { } } if let Some((target, last)) = self.yaw_glide.take() { - let now = Instant::now(); - let dt = now.duration_since(last).as_secs_f64().min(0.1); + let now = cx.seconds_since_app_start(); + let dt = (now - last).clamp(0.0, 0.1); let remaining = wrap_angle(target - self.yaw); if remaining.abs() < 0.002 { self.set_orbit(cx, target, self.pitch); @@ -1470,7 +1645,7 @@ impl TreemapView { } // Aim the tween from wherever things visually are right now — a // slider mid-drag retargets smoothly instead of jumping. - self.tween_capture = Some(self.visual_snapshot()); + self.tween_capture = Some(self.visual_snapshot(cx.seconds_since_app_start())); self.tween_calm = false; self.filter = filter; self.stale = true; @@ -1496,9 +1671,9 @@ impl TreemapView { } /// Eased tween progress, or None when nothing is morphing. - fn tween_t(&self) -> Option { + fn tween_t(&self, now: f64) -> Option { let start = self.tween_start?; - let t = start.elapsed().as_secs_f64() / TWEEN.as_secs_f64(); + let t = (now - start).max(0.0) / TWEEN.as_secs_f64(); if t >= 1.0 { return None; } @@ -1510,8 +1685,8 @@ impl TreemapView { /// mid-tween and mid-gesture alike, and its fractional depth — plus the /// leavers still fading out. Remapped through the live camera, so a /// tween aimed from here starts exactly where the eye left off. - fn visual_snapshot(&self) -> Vec<(Cell, MapRect, f64)> { - let t = self.tween_t(); + fn visual_snapshot(&self, now: f64) -> Vec<(Cell, MapRect, f64)> { + let t = self.tween_t(now); let (rk, rb) = self.cam_remap(); let mut out: Vec<(Cell, MapRect, f64)> = Vec::with_capacity(self.cells.len()); for cell in &self.cells { @@ -1584,15 +1759,21 @@ impl TreemapView { /// Write the finished tree out for next time. Encoding walks the whole /// tree so it happens here, where the tree is; the file write is somebody /// else's problem, on a thread nobody is waiting for. - fn save_cache(&self) { - if crate::vfs::is_demo() { + fn save_cache(&self, cx: &Cx) { + if crate::vfs::vfs().is_demo() { return; } let Some(bytes) = crate::sizecache::encode(&self.root, &self.tree, self.scanned_at) else { return; }; let root = self.root.clone(); - thread::spawn(move || crate::sizecache::store(&root, &bytes)); + if crate::vfs::vfs().is_instant() { + let _ = crate::vfs::vfs().store_scan_cache(&root, &bytes); + } else if let Ok(handle) = cx.task_pool().submit(Lane::Light, move || { + let _ = crate::vfs::vfs().store_scan_cache(&root, &bytes); + }) { + handle.detach(); + } } /// `path` as the chain of names between the mapped folder and it. @@ -1711,11 +1892,11 @@ impl TreemapView { self.totals_dirty = true; self.tree_rev = self.tree_rev.wrapping_add(1); self.last_layout = None; - self.save_cache(); + self.save_cache(cx); self.redraw(cx); } - fn relayout(&mut self, rect: Rect) { + fn relayout(&mut self, rect: Rect, now: f64) { let base = self.focus_path(); // The region the *outgoing* layout covered, before it is replaced — // the line between a camera reveal and data actually appearing. @@ -1891,7 +2072,7 @@ impl TreemapView { // there to the layout just built. if let Some(snapshot) = self.tween_capture.take() { let calm = std::mem::take(&mut self.tween_calm); - if calm && self.tween_t().is_none() { + if calm && self.tween_t(now).is_none() { // A camera-asked settle with nothing already morphing runs // no animation at all. With zoom-invariant packing a // survivor's fresh rect IS its remapped old rect, and the @@ -1968,7 +2149,7 @@ impl TreemapView { }) .collect(); } - self.tween_start = Some(Instant::now()); + self.tween_start = Some(now); } } self.laid_out = rect; @@ -1980,7 +2161,7 @@ impl TreemapView { self.layout_yaw = self.yaw; self.layout_pitch = self.pitch; self.stale = false; - self.last_layout = Some(Instant::now()); + self.last_layout = Some(now); // The cell list is new, so the hovered index means nothing any more. self.hover = None; // The selection is a path, not an index, so it survives — but its @@ -2014,12 +2195,12 @@ impl TreemapView { let cell = &self.cells[index]; let rect = remap_rect(&cell.rect, rk, rb); let z = self.elev(cell.depth); - if Quad::of_rect(&cam, &rect, z).contains(pos) { + if ProjectedFace::of_rect(&cam, &rect, z).is_some_and(|face| face.contains(pos)) { return Some(index); } if z > 0.0 { for wall in wall_quads(&cam, &rect, z, rise.min(z)).into_iter().flatten() { - if wall.quad.contains(pos) { + if wall.face.contains(pos) { return Some(index); } } @@ -2076,7 +2257,7 @@ impl TreemapView { (scale_rgb(base, depth_shade), 0.62) } - fn draw_map(&mut self, cx: &mut Cx2d, palette: &Palette, clip: Rect) -> Vec