Compare commits

..

No commits in common. "fe5b75d92dd54ae791cce83cfa2b683fa5094709" and "435135ed50801f01205e9d907a4a34b9c4a5558b" have entirely different histories.

1547 changed files with 62525 additions and 385852 deletions

124
AGENTS.md
View file

@ -7,11 +7,6 @@
> studio websocket bridge for all agent work. Full spec: [App Remote Control](#app-remote-control---remote).
## Execution Policy
- **Designs stay local.** Design documents, plans, and reports are local files
(`local/agent_state/<topic>/DESIGN.md`) that lanes read from disk. Never
publish them to the web (no Artifacts, no hosted pages); summarize in the
terminal instead.
- Launch UI programs as standalone release binaries from this checkout. Do
not use the Studio remote bridge, `ObserveMount`, `RunItem`, or any
`cargo-makepad studio` websocket client.
@ -36,56 +31,8 @@
can be run directly in the shell.
- A standalone app's built-in screenshot/capture hook is valid for visual
inspection.
- **System-level screenshots are FORBIDDEN.** Never run `screencapture`,
`CGWindowListCreateImage`/`CGDisplayCreateImage` scripts, `xcap`,
`import`, `scrot`, `grim`, `xwd`, PowerShell/Win32 screen grabs, or any
other OS screen capture — not of the display, not of a window, not
"just the caption". The user's screen is private. The only image of a
running app you may ever take is the app's own `--remote` grab (`/g`,
`/gq`, `/tweak/grab`), which renders the app's own drawable and nothing
else. If something only shows in the OS layer (native caption buttons,
other apps, the desktop), ask the user for a screenshot instead of
taking one.
- When adding a new example crate, update both the Cargo workspace and
`makepad.splash`.
- **Zero locking on the UI thread, one mechanism everywhere.** The UI
thread never takes a `Mutex`/`RwLock`/`Condvar` that another thread can
hold, and never blocks on a channel. UI → workers/audio is commands over
a channel (bounded, non-blocking send; a full queue is reported and
retried next frame). Workers/audio → UI is snapshots over atomics, a
triple buffer, or a channel read with `try_recv`. Large payloads (PCM,
stems, grids, images) travel as `Arc` through the channel; a replaced
payload is handed back so the UI thread does the drop, never a realtime
thread. A realtime callback (audio) owns its state, never takes a lock
the UI can hold, never allocates on the hot path. This is ONE code path
for native and wasm — no `cfg` fork where desktop keeps shared mutexes.
On wasm both the browser UI thread and the AudioWorklet thread abort on
`Atomics.wait`, and a spinning fallback against a busy audio callback is
a 100 % CPU feedback loop that kills audio and frame rate together (DJ
web, 2026-09-03). `lock_from_ui` is only acceptable on state provably
touched by the UI thread alone.
- **Standard operating flow — who does what.** The main session (Fable)
designs, briefs, manages and reviews; it does not write the code itself
except one-line fixes. **Codex writes the code**: every implementation lane
is a Codex lane with a precise brief (observations, files, rules, the
verification commands) launched through `local/tools/delegate` /
`local/agent_state/webdemos/tasks/queue.sh` and landed through
`local/tools/integrate`. **Grok does the tests and the token-heavy work**:
test suites, audits, surveys, log reading, conflict resolution passes,
reviews of large diffs (`delegate grok` / `research-grok` / `review-grok`).
A Fable subagent is the exception, only for a design-level change the
other two cannot carry (a new platform mechanism), and it stops as soon as
the API is fixed so Codex can do the conversions. Keep at most six lanes
per provider; land everything through the integrator; the user tries the
result — no routine captures.
- **No temporary threads — use the pool.** Never spawn a thread for one
job (`std::thread::spawn` is unsupported on wasm anyway; the platform
spawner works everywhere). Background work goes to the platform thread
pool (`cx.thread_spawner()` / the pool `TaskHandle` API) or to a
long-lived worker created once at start-up and fed over a channel. On
the web a Web Worker takes hundreds of milliseconds to come up, so a
per-job thread is a stall; on desktop it is still churn. One mechanism
on both targets.
## Standalone Launch
1. `cargo build --release -p <package>` from this checkout.
@ -97,22 +44,6 @@
## App Remote Control (`--remote`)
> **Focus law.** A `--remote` app opens its window VISIBLE BUT UNFOCUSED and
> stays that way: it never activates, never becomes key, and bridge clicks
> never raise it. The user keeps typing wherever they were. Everything the
> bridge does (grabs, `/m`, `/k`, `/t`, `/snap`) works without focus because
> input is injected through the app's event loop, not the OS. Do not work
> around this (`MAKEPAD_FOCUS=1` exists only for a run the user asks to see
> in front); `MAKEPAD_NO_FOCUS=1` gives a non-remote launch the same manners.
> **Who may open a visible window.** Subagent/lane verification runs HIDDEN:
> launch with `MAKEPAD_HIDE_WINDOWS=1 <bin> --remote` — the window never
> appears, grabs (`/g`), `/snap`, `/m`, `/k`, `/t` all still work offscreen.
> Only the integrating session opens the one visible, unfocused window the
> user watches; several look-alike windows on screen made the user "go
> insane" (2026-08-26).
Any makepad app launched with `--remote` runs a localhost HTTP server inside
the process and prints one line before the UI appears:
@ -180,8 +111,6 @@ this pattern as an executable end-to-end test across three example apps.
`GET /gq` (or `/close` each window, then `/quit`). Never leave test windows
on the user's screen, and never `pkill` when the protocol is available.
- **Never touch an instance the user is running.** Launch your own.
- **`/g` is the only camera.** No OS-level screen capture of any kind (see
Execution Policy) — the remote grab is what you get.
- **A vanished window or app with `[makepad-remote] user closed …` in the log
means the human dismissed it — it was in their way.** Do **not** treat that
as a crash and do **not** relaunch it. The app prints
@ -222,59 +151,6 @@ this pattern as an executable end-to-end test across three example apps.
- **Cost when idle is zero.** The event loop only upshifts its paint clock
while a remote request is in flight.
### The TWEAKER (`/tweak/*`) — design feedback and live styling
Every `--remote` app carries a design-feedback overlay (plan of record:
repo-root `tweaker.md`; implementation: `widgets/src/tweaker.rs`). Off it
costs nothing. On, the person (or you) points at the UI: pointer events over
the window body are swallowed before widget dispatch — **clicking a Button in
tweak mode outlines it and never fires it** — and the window grows a property
sidebar next to the (compressed) app UI. Shift+F10 toggles it in-app; every edit,
theirs or yours, lands in one shared diff log.
| Route | Answer | Notes |
|---|---|---|
| `/tweak` `?on=1\|0&annotate=1\|0` | `{"on":1,"annotate":0}` | toggle the overlay / the freehand draw mode (Alt-drag draws too) |
| `/tweak/state` | `{"on":1,"sel":{path,ty,r,band},"props":[{n,v,set}],"hover":…,"diff":[…],"ann":[…]}` | the STRUCTURE feedback: pinned selection, its real reflected properties (`set:1` = explicitly applied), the edit log, annotation strokes with the widget paths they touch |
| `/tweak/apply` (POST) | `{"ok":1,"path":…,"changed":[{path,prop,old,new}]}` | body `{"path":"a.b.c","splash":"{padding: Inset{left: 20}}"}` or the one-property shorthand `{"path":…,"prop":"draw_bg.border_radius","value":"8"}`. Evaluates the chunk onto that ONE instance through the ordinary apply machinery (`+:` merge rules intact) and triggers a full relayout. Answers after the next drawn frame |
| `/tweak/diff` | `{"diff":[{path,prop,old,new}…]}` | the raw edit log, in order |
| `/tweak/clear` | `{"ok":1}` | reset diff + annotations |
| `/tweak/final` | `{"final":[…coalesced…],"ann":[…],"drew":0\|1,"png":path?}` | **read this when tweaking is done**: per (path, prop) only the original and final value, churn collapsed. When the user drew, `png` is the composited screenshot — look at it, the strokes mean something |
| `/tweak/grab` | like `/g` | the overlay (outlines, strokes, sidebar) draws in the window's own pass, so any grab is already composited |
`local/tools/tweak` wraps all of this:
`tweak PORT on`, `tweak PORT state`, `tweak PORT apply PATH PROP VALUE`,
`tweak PORT splash PATH 'CHUNK'`, `tweak PORT final`, …
**How to listen.** Sidebar edits push to you: each one emits a marked
`TWEAK sidebar <path> <prop> <old> -> <new>` line into the app log — the
`/log` tail is your ear; you never poll `/tweak/state` for changes. Talk back
on `/tweak/apply` (values or whole shader chunks) to the same selected
instance.
**Write-back (you do this part — the overlay never writes source).** When the
session is done, take `/tweak/final` and edit the splash source:
1. Resolve each entry's widget path to its DSL site: the dotted path mirrors
the `script_mod!` tree (`/d` shows the same ids). `-` segments are
anonymous containers — skip them when searching the source.
2. Write each property at the **most specific existing site** — the widget's
own `name := Type{…}` block if it has one; create one only when none
exists.
3. Respect the merge law: a property inside a typed sub-struct goes through
`+:` (`draw_bg +: { border_radius: 8 }`), never a replacing
`draw_bg: {…}`. Plain walk/layout values (`padding`, `margin`, `width`)
are set directly (`padding: Inset{left: 20}`).
4. Values come back in source spelling (`#rrggbbaa` colors, plain numbers) —
paste them as-is. Mind the Rust-tokenizer hex-`e` trap in `script_mod!`:
`#1e1e2e` must be written `#x1e1e2e`.
5. Rebuild and relaunch; verify the value survived with `/tweak/state` or
`/snap` before calling it done.
Reflection truth: `props` come from the widget's live Rust fields plus the
type's DSL-declared shader inputs (`instance()`/`uniform()`), so the list is
what the widget actually exposes — there is no synthetic schema to drift.
### Studio remote bridge (the older path)
The studio (`studio/desktop` + `studio/hub`) drives a hosted app over a

View file

@ -1,20 +1,17 @@
workspace.members = [
# === app ===
"apps/browser",
"apps/terminal",
"apps/wm",
"apps/aichat",
"apps/mpbrowser",
"apps/mpterm",
"apps/mpwm",
"apps/finance",
"apps/sheets",
"apps/photos",
"libs/wm_theme",
"libs/wm_api",
"libs/app_module",
"apps/task",
"apps/image",
"apps/video",
"apps/pdf",
"apps/files",
"apps/mpsheets",
"libs/mp_theme",
"libs/mp_wm_api",
"apps/mptask",
"apps/mpimage",
"apps/mpvideo",
"apps/mppdf",
"apps/mpfiles",
"apps/route",
# === arcade (game.md) — networked AI game sandbox ===
"apps/arcade",
@ -30,10 +27,6 @@ workspace.members = [
"libs/game/assets",
"apps/asset-ui",
"apps/asset-server",
"apps/flow-server",
"apps/flow-ui",
"libs/flowgraph",
"libs/media_view",
"apps/ai-hub",
"apps/vj",
"apps/mixer",
@ -47,8 +40,6 @@ workspace.members = [
"libs/asset/store",
"libs/asset/chat",
"libs/chat_ui",
"libs/ai/hub_ui",
"libs/ai/services",
"libs/asset/annotate",
"libs/render",
"libs/raytrace",
@ -59,8 +50,6 @@ workspace.members = [
"libs/sim/math",
"libs/show_control",
"libs/asset/creator",
"libs/bounded_http",
"libs/flow",
"libs/strict_json",
# === remesh (FaithC port) / xatlas (jpcy port) ===
"libs/remesh",
@ -105,9 +94,6 @@ workspace.members = [
"examples/render_to_texture",
# === digital-fabrication product ===
"apps/fab",
"apps/fabric",
"libs/fabric/measure",
"libs/fabric/draft",
# === xr app ===
"xr",
# === studio ===
@ -120,8 +106,6 @@ workspace.members = [
"libs/score_layout",
"libs/score_play",
"libs/score_render",
"libs/score_view",
"libs/score_view/tests/embed_app",
"libs/score_ai",
"libs/score_import",
"libs/score_pdf",
@ -130,8 +114,6 @@ workspace.members = [
"libs/midi_file",
"libs/soundfont",
"libs/piano_model",
"libs/drumkit",
"libs/drumkit_phys",
"libs/musicxml",
# === own MP3 / Ogg Vorbis decoders ===
"libs/audio_decode",
@ -141,7 +123,6 @@ workspace.members = [
"libs/audio_lyrics",
# === stems + lyrics side-channel bake/publish (asset-ui + VJ) ===
"libs/audio_sidechannels",
"libs/vj_analysis",
"libs/teamtalk",
# === pictures of audio (spectrogram, wave strip, composite) ===
"libs/audio_picture",
@ -154,23 +135,16 @@ workspace.members = [
"libs/frametween",
# === archive.org search + download content input (VJ / asset-ui) ===
"libs/archive_org",
# === image tile wall: HEVC tape atlases + baker CLI + TileGrid widget
# (the engine extracted from the Source Library picture wall) ===
"libs/image_tiles",
"examples/image_tiles",
# === mp4 sample index for range-streaming playback ===
"libs/mp4_index",
# === necessary tools ===
"platform/video",
"tools/cargo_makepad",
"tools/makepad_loader",
# === OSM PBF -> tile archive + nav artifact bake passes (CLI + in-app) ===
"libs/map_build",
"tools/map_tiles",
"tools/map_bake",
"tools/remote",
"tools/dj_pack",
"libs/system_speech",
# === tests ===
"platform/script/test",
]
@ -189,6 +163,7 @@ workspace.exclude = [
"libs/diffusion",
# the AI model workspace (loader + cuda/metal stores + model crates) — aiarch.md
"libs/ai",
"libs/voice",
"widgets/test",
"libs/stitch",
"libs/wasm_bridge/test",
@ -305,9 +280,6 @@ strip = true
inherits = "release"
debug = true
[profile.test.package.makepad-stitch]
opt-level = 1
#[profile.dev.package.makepad-live-tokenizer]
#opt-level = 3
#[profile.dev.package.makepad-live-compiler]

View file

@ -14,7 +14,7 @@ makepad-ai-hub = { path = "../../libs/ai/hub", default-features = false }
# Feature passthrough: the headless node exposes exactly the library's
# feature set, so box launch scripts keep their --features flags working.
[features]
default = ["flux", "paint", "paint-cuda", "llm", "tts", "indextts", "video", "interpolate", "audio", "mesh", "matte-native", "depth-native", "segment-native", "body-native", "upscale-native", "motion-native", "rig-native", "splat-native", "stems-native"]
default = ["flux", "paint", "paint-cuda", "llm", "tts", "indextts", "video", "interpolate", "audio", "mesh", "matte-native", "depth-native", "segment-native", "upscale-native", "motion-native", "rig-native", "splat-native"]
python-backends = ["makepad-ai-hub/python-backends"]
flux = ["makepad-ai-hub/flux"]
paint = ["makepad-ai-hub/paint"]
@ -29,9 +29,7 @@ mesh = ["makepad-ai-hub/mesh"]
matte-native = ["makepad-ai-hub/matte-native"]
depth-native = ["makepad-ai-hub/depth-native"]
segment-native = ["makepad-ai-hub/segment-native"]
body-native = ["makepad-ai-hub/body-native"]
upscale-native = ["makepad-ai-hub/upscale-native"]
motion-native = ["makepad-ai-hub/motion-native"]
splat-native = ["makepad-ai-hub/splat-native"]
rig-native = ["makepad-ai-hub/rig-native"]
stems-native = ["makepad-ai-hub/stems-native"]

View file

@ -36,14 +36,10 @@ fn run() -> Result<(), AssetAiError> {
let mut cache_dir: Option<PathBuf> = None;
let mut registry_path: Option<PathBuf> = None;
let mut machine = false;
let mut activity_probe = None;
let mut args = std::env::args().skip(1);
while let Some(arg) = args.next() {
match arg.as_str() {
"--activity-probe" => {
activity_probe = Some(args.next().ok_or_else(|| AssetAiError::Io("--activity-probe needs seconds (1..3600)".into()))?.parse::<u64>().map_err(|_| AssetAiError::Io("invalid activity probe seconds".into()))?);
}
"--port" => {
let value = args
.next()
@ -82,7 +78,7 @@ fn run() -> Result<(), AssetAiError> {
}
"--help" | "-h" => {
println!(
"{SERVICE_NAME} {SERVICE_VERSION}\nusage: {SERVICE_NAME} [--port N] [--host ADDR] [--fleet NAME] [--cache-dir PATH] [--registry PATH] [--machine] [--activity-probe SECONDS]"
"{SERVICE_NAME} {SERVICE_VERSION}\nusage: {SERVICE_NAME} [--port N] [--host ADDR] [--fleet NAME] [--cache-dir PATH] [--registry PATH] [--machine]"
);
return Ok(());
}
@ -92,10 +88,6 @@ fn run() -> Result<(), AssetAiError> {
}
}
if let Some(seconds) = activity_probe {
return makepad_ai_hub::activity::run_probe(seconds);
}
let port = match port {
Some(port) => port,
None => match std::env::var("MAKEPAD_ASSET_AI_PORT") {

View file

@ -1,46 +0,0 @@
# aichat — the assistant, as an app.
#
# One conversation that drives every application through the AI services
# layer. It is hosted three ways with one code base: under the window
# manager as a special child seated in the pane slot (this binary,
# `--stdin-loop`), standalone as its own window (this binary), and inside
# any app's Window as the F10 overlay or inside the mobile/web superbuild
# (this crate's lib, `makepad_aichat::script_mod` + `AiChatPanel{}`).
#
# The panel widget OWNS the engine: the service registry, the model, the
# transcript. Hosts only feed it links (in-process) or bus frames (the
# window manager's studio Custom frames) and give it a place to draw.
#
# Plan of record: repo-root aicontrol.md.
[package]
name = "makepad-aichat"
version = "0.1.0"
edition = "2021"
default-run = "aichat"
[lib]
name = "makepad_aichat"
path = "src/lib.rs"
[[bin]]
name = "aichat"
path = "src/main.rs"
[features]
default = ["engine"]
# The real models (the hub's local runtime, the cloud providers). Off for a
# build without a model runtime — the web page answers with NoModel and the
# tool console still works.
engine = ["makepad-ai-services/engine", "dep:makepad-asset-creator"]
[dependencies]
makepad-widgets = { path = "../../widgets" }
makepad-wm-theme = { path = "../../libs/wm_theme" }
makepad-wm-api = { path = "../../libs/wm_api" }
makepad-ai-services = { path = "../../libs/ai/services", default-features = false }
makepad-strict-json = { path = "../../libs/strict_json" }
# The generative pipelines behind the assistant's own `gen` service
# (src/gen.rs): the creator runner picks a fleet node and brings the
# picture back. Native only, with the engine.
makepad-asset-creator = { path = "../../libs/asset/creator", optional = true }

View file

@ -1,213 +0,0 @@
//! The client half of the window manager's service bus.
//!
//! Under the WM the other apps are not in this process. The WM forwards
//! their up-frames to the aichat child as studio `Custom` frames, each
//! stamped with the endpoint the WM issued to the sender, and forwards
//! the aichat child's down-frames (which name their target endpoint) back
//! to the right client. This adapter turns those frames into ordinary
//! [`ServiceLink`]s in the panel's registry, so the engine never knows
//! whether a service is a channel away or a process away.
//!
//! One link per endpoint. A `Register` from an endpoint the registry does
//! not know creates the link and registers it under the WM's endpoint id
//! (`register_as`); a later `Register` from the same endpoint is just the
//! manifest going down the existing link, where the registry answers it.
//! The WM tells us about a dead client by sending `Unregister` on its
//! behalf. Everything the registry sends down a bus link is drained here
//! and put on the wire to the WM.
use makepad_ai_services::engine::ServiceRegistry;
use makepad_ai_services::port::{ServiceLink, ServiceLinkHost};
use makepad_ai_services::wire::*;
use makepad_widgets::makepad_platform::studio::AppToStudio;
use makepad_widgets::*;
use std::collections::HashMap;
#[derive(Default)]
pub struct ServiceBus {
hosts: HashMap<EndpointId, ServiceLinkHost>,
}
impl ServiceBus {
/// An up-frame from the WM. `None` for frames that are not the bus's.
pub fn on_custom(&mut self, registry: &ServiceRegistry, json: &str) -> bool {
let Some(frame) = HostedUp::parse(json) else { return false };
let Some(from) = frame.from.clone() else { return true };
match (&frame.msg, self.hosts.get(&from)) {
(ServiceUp::Register { manifest, .. }, None) => {
let (link, host) = ServiceLink::pair(manifest.clone());
if registry.register_as(link, from.clone(), "", None).is_ok() {
let _ = host.up.send(frame);
self.hosts.insert(from, host);
}
}
(_, Some(host)) => {
let _ = host.up.send(frame);
}
(_, None) => {}
}
true
}
/// Put every frame the registry sent down a bus link on the wire.
/// Links whose registry side is gone are dropped.
pub fn relay_down(&mut self, registry: &ServiceRegistry) {
for frame in self.drain_down(registry) {
Cx::send_studio_message(AppToStudio::Custom(frame.to_json()));
}
}
fn drain_down(&mut self, registry: &ServiceRegistry) -> Vec<HostedDown> {
let mut gone: Vec<EndpointId> = Vec::new();
let mut frames = Vec::new();
for (endpoint, host) in &self.hosts {
loop {
match host.down.try_recv() {
Ok(mut frame) => {
// `Registered` travels without a target; the WM
// routes it by the endpoint we register as.
if frame.to.is_none() {
frame.to = Some(endpoint.clone());
}
frames.push(frame);
}
Err(std::sync::mpsc::TryRecvError::Empty) => break,
Err(std::sync::mpsc::TryRecvError::Disconnected) => {
gone.push(endpoint.clone());
break;
}
}
}
}
for endpoint in gone {
self.hosts.remove(&endpoint);
registry.unregister(&endpoint);
}
frames
}
pub fn len(&self) -> usize {
self.hosts.len()
}
pub fn is_empty(&self) -> bool {
self.hosts.is_empty()
}
}
#[cfg(test)]
mod tests {
use super::*;
use makepad_ai_services::engine::{EngineCore, NoModelWithReason, RegistryUp};
#[test]
fn hosted_subscriptions_and_messages_cross_the_adapter_unchanged() {
let registry = ServiceRegistry::new();
let mut bus = ServiceBus::default();
let endpoint = EndpointId("w4".into());
let manifest = ServiceManifest::new("flow", "Flow", "A flow.")
.with_topic(TopicDef::new("run", "Run events."));
let register = HostedUp {
from: Some(endpoint.clone()),
msg: ServiceUp::Register { manifest, port_tag: 4 },
};
assert!(bus.on_custom(&registry, &register.to_json()));
registry.pump();
let host = bus.hosts.get(&endpoint).unwrap();
let _registered = host.down.try_recv().unwrap();
assert!(registry.send(
&endpoint,
ServiceDown::Subscribe {
sub_id: "s1".into(),
topic: "run".into(),
filter: None,
},
));
assert!(matches!(
host.down.try_recv().unwrap().msg,
ServiceDown::Subscribe { sub_id, topic, filter: None }
if sub_id == "s1" && topic == "run"
));
let message = HostedUp {
from: Some(endpoint.clone()),
msg: ServiceUp::Message {
sub_id: "s1".into(),
topic: "run".into(),
text: "finished".into(),
data: None,
final_: true,
},
};
assert!(bus.on_custom(&registry, &message.to_json()));
assert!(matches!(
registry.pump().as_slice(),
[RegistryUp::Message { endpoint: from, sub_id, message }]
if from == &endpoint && sub_id == "s1" && message.final_
));
}
#[test]
fn engine_shutdown_is_flushed_through_the_hosted_service_bus() {
let registry = ServiceRegistry::new();
let mut bus = ServiceBus::default();
let endpoint = EndpointId("w4".into());
let manifest = ServiceManifest::new("flow", "Flow", "A flow.")
.with_tool(ToolDef::new(
"watch",
"Watch a run.",
r#"{"type":"object","properties":{}}"#,
Risk::Read,
))
.with_topic(TopicDef::new("run", "Run events."));
assert!(bus.on_custom(
&registry,
&HostedUp {
from: Some(endpoint.clone()),
msg: ServiceUp::Register { manifest, port_tag: 4 },
}
.to_json(),
));
let mut engine = EngineCore::new(
registry.clone(),
Box::new(NoModelWithReason::new("not used by the tool console")),
None,
0x44,
);
engine.send("/flow.watch {}", 0.0);
let call_id = bus
.drain_down(&registry)
.into_iter()
.find_map(|frame| match frame.msg {
ServiceDown::Call(call) => Some(call.call_id),
_ => None,
})
.expect("the hosted service receives the call");
assert!(bus.on_custom(
&registry,
&HostedUp {
from: Some(endpoint.clone()),
msg: ServiceUp::Result(
ToolResult::ok(call_id, "watching", "")
.with_subscription(SubscriptionRequest::new("run")),
),
}
.to_json(),
));
engine.pump(0.1);
let sub_id = bus
.drain_down(&registry)
.into_iter()
.find_map(|frame| match frame.msg {
ServiceDown::Subscribe { sub_id, .. } => Some(sub_id),
_ => None,
})
.expect("the hosted service receives the subscription");
assert_eq!(sub_id, "l44-s1");
engine.shutdown();
assert!(matches!(
bus.drain_down(&registry).as_slice(),
[HostedDown { to: Some(to), msg: ServiceDown::Unsubscribe { sub_id: ended } }]
if to == &endpoint && ended == &sub_id
));
}
}

View file

@ -1,297 +0,0 @@
//! The assistant's own generative service: `gen.image{prompt}`.
//!
//! The hub's pipelines are not an app, so no app registers them; the
//! panel registers this service in-process beside whatever apps join.
//! An `image` call runs the creator pipeline on a worker thread (the
//! runner is blocking: node pick over the LAN fleet, request, poll,
//! fetch), streams the node's progress into the card, writes the picture
//! under the makepad home's `gen` folder and answers with the path — the
//! model then hands that path to `photos.add`, which puts it on the wall.
//! Nothing goes through the asset store. Without the `engine` feature (the
//! web page) the service still exists and says it cannot.
use makepad_ai_services::engine::ServiceRegistry;
use makepad_ai_services::port::{AiServicePort, PortEvent};
use makepad_ai_services::wire::{Risk, ServiceCall, ServiceManifest, ToolDef, ToolResult};
use makepad_widgets::*;
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{Receiver, TryRecvError};
use std::sync::Arc;
/// The service id on the bus.
pub const SERVICE_ID: &str = "gen";
pub fn manifest() -> ServiceManifest {
ServiceManifest::new(
SERVICE_ID,
"Generate",
"The machine's generative pipelines, on the fleet's GPU nodes: a \
picture from a prompt. The picture is saved on this machine under \
the makepad home's gen folder and the answer carries its path. To \
SHOW it, call photos.add with that path (launch Photos first with \
os.launch if it is not running) the wall then glides onto it. A \
generation takes half a minute on a warm node, longer when a node \
must load the model.",
)
.with_tool(ToolDef::new(
"image",
"Generate one picture from a text prompt on a fleet image node; saves it under the makepad home's gen folder and returns the path.",
r#"{"type":"object","properties":{"prompt":{"type":"string","description":"what the picture shows, in plain words"},"width":{"type":"integer","description":"pixels, optional (default 1024)"},"height":{"type":"integer","description":"pixels, optional (default 1024)"}},"required":["prompt"]}"#,
Risk::Act,
))
}
/// What the worker reports back.
enum GenMsg {
Progress(String, u16),
Done(Result<GenDone, String>),
}
struct GenDone {
path: PathBuf,
node: String,
}
struct Job {
call_id: String,
cancel: Arc<AtomicBool>,
rx: Receiver<GenMsg>,
}
/// The in-process port plus the jobs in flight.
pub struct GenService {
port: AiServicePort,
jobs: Vec<Job>,
}
impl GenService {
/// Open the service and register it in the panel's registry. `None`
/// only when the manifest does not validate (a programming error).
pub fn open(registry: &ServiceRegistry) -> Option<GenService> {
let (port, link) = AiServicePort::in_process(manifest()).ok()?;
registry.register(link, "built in", None).ok()?;
Some(GenService { port, jobs: Vec::new() })
}
/// Drain the port and the workers; called on every panel event.
pub fn handle_event(&mut self, cx: &mut Cx, event: &Event) {
for ev in self.port.handle_event(cx, event) {
match ev {
PortEvent::Call(call) => self.start(call),
PortEvent::Cancel { call_id } => {
if let Some(job) = self.jobs.iter().find(|j| j.call_id == call_id) {
job.cancel.store(true, Ordering::Relaxed);
}
}
PortEvent::Registered(_)
| PortEvent::ChatOpen { .. }
| PortEvent::Subscribe { .. }
| PortEvent::Unsubscribe { .. } => {}
}
}
self.poll();
}
fn start(&mut self, call: ServiceCall) {
let id = call.call_id.clone();
if call.tool != "image" {
self.port.reply(ToolResult::refused(&id, format!("gen has no tool `{}`; it has image", call.tool)));
return;
}
let args = match parse_image_args(&call.args) {
Ok(a) => a,
Err(why) => {
self.port.reply(ToolResult::refused(&id, why));
return;
}
};
#[cfg(not(feature = "engine"))]
{
let _ = args;
self.port.reply(ToolResult::unavailable(&id, "this build has no pipeline runtime; pictures need the native app"));
}
#[cfg(feature = "engine")]
{
use makepad_widgets::makepad_platform::thread::SignalToUI;
let cancel = Arc::new(AtomicBool::new(false));
let (tx, rx) = std::sync::mpsc::channel();
let worker_cancel = cancel.clone();
let spawned = std::thread::Builder::new().name("gen-image".into()).spawn(move || {
let progress_tx = tx.clone();
let mut progress = |note: &str, permille: u16| {
let _ = progress_tx.send(GenMsg::Progress(note.to_string(), permille));
SignalToUI::set_ui_signal();
};
let result = run_image(&args, &worker_cancel, &mut progress);
let _ = tx.send(GenMsg::Done(result));
SignalToUI::set_ui_signal();
});
match spawned {
Ok(_) => self.jobs.push(Job { call_id: id, cancel, rx }),
Err(e) => self.port.reply(ToolResult::failed(&id, format!("could not start the generation: {e}"))),
}
}
}
fn poll(&mut self) {
let mut done: Vec<usize> = Vec::new();
for (i, job) in self.jobs.iter().enumerate() {
loop {
match job.rx.try_recv() {
Ok(GenMsg::Progress(note, permille)) => self.port.progress(&job.call_id, &note, permille),
Ok(GenMsg::Done(Ok(out))) => {
let path = out.path.to_string_lossy().to_string();
self.port.reply(
ToolResult::ok(
&job.call_id,
format!("saved {path} (made on {}). Show it on the wall with photos.add {{\"path\":\"{path}\"}}.", out.node),
"saved",
)
.with_data(format!("{{\"path\":{},\"node\":{}}}", json_string(&path), json_string(&out.node))),
);
done.push(i);
break;
}
Ok(GenMsg::Done(Err(e))) => {
let result = if e == "cancelled" { ToolResult::cancelled(&job.call_id) } else { ToolResult::failed(&job.call_id, e) };
self.port.reply(result);
done.push(i);
break;
}
Err(TryRecvError::Empty) => break,
Err(TryRecvError::Disconnected) => {
self.port.reply(ToolResult::failed(&job.call_id, "the generation worker died"));
done.push(i);
break;
}
}
}
}
for i in done.into_iter().rev() {
self.jobs.remove(i);
}
}
}
/// The arguments of one `image` call, checked.
#[derive(Clone, Debug, PartialEq)]
pub struct ImageArgs {
pub prompt: String,
pub width: u32,
pub height: u32,
}
/// Sizes are clamped to what the image nodes serve; a prompt is required.
pub fn parse_image_args(args: &str) -> Result<ImageArgs, String> {
use makepad_strict_json as json;
let fields = match json::parse(args.as_bytes()) {
Ok(json::Value::Obj(fields)) => fields,
_ => return Err("image needs a JSON object with a `prompt`".to_string()),
};
let get = |key: &str| fields.iter().find(|(k, _)| k == key).map(|(_, v)| v.clone());
let prompt = get("prompt").and_then(|v| v.as_str().map(|s| s.trim().to_string())).unwrap_or_default();
if prompt.is_empty() {
return Err("image needs a `prompt`".to_string());
}
if prompt.len() > 4000 {
return Err("the prompt is longer than 4000 bytes".to_string());
}
let dim = |key: &str| -> Result<u32, String> {
match get(key) {
None => Ok(1024),
Some(json::Value::Int(i)) if (256..=2048).contains(&i) => Ok(i as u32),
Some(_) => Err(format!("`{key}` must be an integer from 256 to 2048")),
}
};
Ok(ImageArgs { prompt, width: dim("width")?, height: dim("height")? })
}
/// A short file-name-safe slug of the prompt.
pub fn slug(prompt: &str) -> String {
let mut out = String::new();
for ch in prompt.chars().flat_map(|c| c.to_lowercase()) {
if ch.is_ascii_alphanumeric() {
out.push(ch);
} else if !out.ends_with('-') && !out.is_empty() {
out.push('-');
}
if out.len() >= 40 {
break;
}
}
let out = out.trim_matches('-').to_string();
if out.is_empty() { "picture".to_string() } else { out }
}
/// The file extension for a content type the nodes produce.
pub fn extension_for(content_type: &str) -> &'static str {
match content_type.split(';').next().unwrap_or("").trim() {
"image/jpeg" => "jpg",
"image/webp" => "webp",
"image/gif" => "gif",
_ => "png",
}
}
fn json_string(s: &str) -> String {
makepad_strict_json::s(s.to_string()).to_json()
}
#[cfg(feature = "engine")]
fn run_image(args: &ImageArgs, cancel: &Arc<AtomicBool>, progress: &mut dyn FnMut(&str, u16)) -> Result<GenDone, String> {
use makepad_asset_creator::makepad_ai_hub::home::makepad_home;
use makepad_asset_creator::makepad_strict_json as json;
use makepad_asset_creator::runner::generate_bytes;
let body = json::obj(vec![
("prompt", json::s(args.prompt.clone())),
("width", json::Value::Int(args.width as i64)),
("height", json::Value::Int(args.height as i64)),
]);
let seed = (Cx::time_now().max(0.0) * 1_000_000_000.0) as u64;
progress("finding an image node", 0);
let generated = generate_bytes("image.generate", &body, seed, cancel, progress)?;
let artifact = generated.artifact.ok_or("the node returned no picture")?;
let dir = makepad_home().join("gen");
std::fs::create_dir_all(&dir).map_err(|e| format!("cannot make {}: {e}", dir.display()))?;
let stamp = Cx::time_now().max(0.0) as u64;
let path = dir.join(format!("{stamp}-{}.{}", slug(&args.prompt), extension_for(&artifact.content_type)));
std::fs::write(&path, &artifact.bytes).map_err(|e| format!("cannot write {}: {e}", path.display()))?;
Ok(GenDone { path, node: generated.node })
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_manifest_validates_and_the_tool_acts() {
let m = manifest();
m.validate().expect("a manifest the wire accepts");
assert_eq!(m.id, "gen");
assert_eq!(m.tools.len(), 1);
assert_eq!(m.tools[0].name, "image");
assert_eq!(m.tools[0].risk, Risk::Act);
}
#[test]
fn image_args_need_a_prompt_and_sane_sizes() {
let a = parse_image_args(r#"{"prompt":"a red bicycle on the moon"}"#).unwrap();
assert_eq!((a.width, a.height), (1024, 1024));
let b = parse_image_args(r#"{"prompt":"x","width":512,"height":768}"#).unwrap();
assert_eq!((b.width, b.height), (512, 768));
assert!(parse_image_args(r#"{"width":512}"#).unwrap_err().contains("prompt"));
assert!(parse_image_args(r#"{"prompt":"x","width":16}"#).unwrap_err().contains("256"));
assert!(parse_image_args("nope").is_err());
}
#[test]
fn file_names_are_slugs_with_the_right_extension() {
assert_eq!(slug("A red bicycle, on the Moon!"), "a-red-bicycle-on-the-moon");
assert_eq!(slug(" "), "picture");
assert!(slug(&"word ".repeat(30)).len() <= 41);
assert_eq!(extension_for("image/png"), "png");
assert_eq!(extension_for("image/jpeg; charset=binary"), "jpg");
assert_eq!(extension_for("application/octet-stream"), "png");
}
}

View file

@ -1,89 +0,0 @@
//! aichat as a library: the panel widget that owns the engine, the WM-bus
//! client half, the settings, and the module root. A host links this,
//! calls [`script_mod`] after the widgets' own, and puts `AiChatPanel{}`
//! where the chat goes — the standalone binary and the WM pane child do
//! that themselves; a plain `Window` does it for free through its F10
//! slot, which instantiates `mod.widgets.AiChatOverlay{}` by name; the
//! superbuild seats the same overlay in its pane.
//!
//! Linking this crate also gives the bridge its `/ai` routes: `script_mod`
//! installs `Cx::ai_callback`, the way the widgets crate installs the
//! tweaker's, so `/ai?on=1`, `/ai?say=…` and `/ai/transcript` drive and
//! read the chat in a hidden instance.
pub use makepad_widgets;
use makepad_widgets::ai_slot::AiSlotRequests;
use makepad_widgets::makepad_platform::ScriptVmCx;
use makepad_widgets::*;
pub mod bus;
pub mod gen;
pub mod overlay;
pub mod panel;
pub mod settings;
pub use bus::ServiceBus;
pub use overlay::{AiChatOverlay, AiTranscript};
pub use panel::{AiChatPanel, AiChatPanelAction};
pub use settings::AiSettings;
/// Register the panel and the overlay, and give the bridge its `/ai`
/// routes. Call once after `makepad_widgets::script_mod`.
pub fn script_mod(vm: &mut ScriptVm) {
crate::panel::script_mod(vm);
crate::overlay::script_mod(vm);
vm.cx_mut().ai_callback = Some(ai_callback);
}
fn arg<'a>(args: &'a [(String, String)], keys: &[&str]) -> Option<&'a str> {
for key in keys {
if let Some((_, value)) = args.iter().find(|(k, _)| k == key) {
return Some(value.as_str());
}
}
None
}
/// The bridge's `/ai` dispatcher. `toggle` (`on=1|0`, or flip) and `say`
/// (`say=TEXT`, opening the overlay if it is closed) are requests the
/// slot and the overlay take on their next event; `transcript` is what
/// the overlay last published. Never borrows a widget.
fn ai_callback(cx: &mut Cx, op: &str, args: &[(String, String)]) -> Result<String, String> {
match op {
"toggle" => {
let is_open = cx.global::<AiSlotRequests>().is_open;
let on = match arg(args, &["on"]) {
Some(value) => !matches!(value, "0" | "false" | "off" | "no"),
None => !is_open,
};
cx.global::<AiSlotRequests>().open = Some(on);
// The slot takes the request on its next event: make one.
cx.new_next_frame();
cx.redraw_all();
Ok(format!("{{\"on\":{}}}", on as u8))
}
"say" => {
let text = arg(args, &["say", "t"]).unwrap_or("").trim().to_string();
if text.is_empty() {
return Err("need say=TEXT".into());
}
let req = cx.global::<AiSlotRequests>();
if !req.is_open {
req.open = Some(true);
}
req.say.push(text);
cx.new_next_frame();
cx.redraw_all();
Ok("{\"ok\":1}".into())
}
"transcript" => {
let json = cx.global::<AiTranscript>().json.clone();
if json.is_empty() {
Ok("{\"status\":\"closed\",\"provider\":\"\",\"apps\":[],\"entries\":[],\"generation\":0}".into())
} else {
Ok(json)
}
}
other => Err(format!("no ai op `{other}`; there are toggle, say, transcript")),
}
}

View file

@ -1,79 +0,0 @@
//! aichat: the assistant as its own window, and as the window manager's
//! special child. Both are this binary: standalone it is a window with
//! the panel in it and whatever services are linked in-process (none,
//! until an app embeds it); under the WM (`--stdin-loop`) the WM seats it
//! in the pane slot and every other app's service reaches it over the bus
//! as studio `Custom` frames, which the panel turns into registry links.
pub use makepad_widgets;
use makepad_aichat::AiChatPanelAction;
use makepad_widgets::*;
app_main!(App);
script_mod! {
use mod.prelude.widgets.*
use mod.widgets.*
startup() do #(App::script_component(vm)){
ui: Root{
main_window := Window{
window.inner_size: vec2(460, 760)
window.title: "AI"
pass +: { clear_color: theme.color_bg_app }
body +: {
panel := AiChatPanel{
width: Fill
height: Fill
}
}
}
}
}
}
#[derive(Script, ScriptHook)]
pub struct App {
#[live]
ui: WidgetRef,
}
impl MatchEvent for App {
fn handle_startup(&mut self, cx: &mut Cx) {
makepad_wm_api::set_title(cx, "AI");
}
fn handle_actions(&mut self, cx: &mut Cx, actions: &Actions) {
for action in actions {
if let Some(widget_action) = action.as_widget_action() {
if let AiChatPanelAction::Close = widget_action.cast() {
// Under the WM the pane hides; standalone the window stays.
let _ = makepad_wm_api::send(cx, &makepad_wm_api::WmRequest::Close);
}
}
}
}
}
impl AppMain for App {
fn script_mod(vm: &mut ScriptVm) -> ScriptValue {
crate::makepad_widgets::script_mod(vm);
makepad_wm_theme::apply(vm);
makepad_aichat::script_mod(vm);
self::script_mod(vm)
}
fn handle_event(&mut self, cx: &mut Cx, event: &Event) {
// Bus frames reach the panel through its own handle_event; of the
// WM's own frames only the polite close is ours (the pane hides us
// by leaving us running; this is the desktop going down).
if let Event::Custom(json) = event {
if let Some(makepad_wm_api::WmEvent::CloseRequested) = makepad_wm_api::WmEvent::parse(json) {
cx.quit();
return;
}
}
self.match_event(cx, event);
self.ui.handle_event(cx, event, &mut Scope::empty());
}
}

View file

@ -1,261 +0,0 @@
//! The chat's module root: what a host seats in-process.
//!
//! `mod.widgets.AiChatOverlay{}` is what the Window's AI slot
//! (`widgets/src/ai_slot.rs`) instantiates by name on F10, and what the
//! superbuild seats in its pane. It is a `View` with the panel in it and
//! three duties around it: adopt the in-process service links the apps
//! parked on `Cx` ([`PendingServiceLinks`]) into the panel's registry,
//! send the lines the bridge asked to say ([`AiSlotRequests::say`]),
//! and publish the transcript as JSON ([`AiTranscript`]) after each draw
//! so `/ai/transcript` answers without touching a widget. The panel's
//! Escape (an idle, empty composer) asks the slot to close.
use crate::panel::{AiChatPanel, AiChatPanelAction};
use makepad_ai_services::port::PendingServiceLinks;
use makepad_ai_services::state::{Entry, EngineState, Status, ToolStatus};
use makepad_widgets::ai_slot::AiSlotRequests;
use makepad_widgets::makepad_micro_serde::*;
use makepad_widgets::*;
script_mod! {
use mod.prelude.widgets_internal.*
use mod.widgets.*
mod.widgets.AiChatOverlayBase = #(AiChatOverlay::register_widget(vm))
mod.widgets.AiChatOverlay = set_type_default() do mod.widgets.AiChatOverlayBase{
width: Fill
height: Fill
panel := AiChatPanel{
width: Fill
height: Fill
}
}
}
/// The transcript as the bridge reads it: a `Cx` global the overlay
/// rewrites after every draw of the panel.
#[derive(Default)]
pub struct AiTranscript {
pub json: String,
}
#[derive(SerJson)]
struct TranscriptRow {
kind: String,
text: String,
title: String,
status: String,
note: String,
}
#[derive(SerJson)]
struct Transcript {
status: String,
provider: String,
apps: Vec<String>,
entries: Vec<TranscriptRow>,
generation: u64,
}
/// The transcript JSON for a state: `status`, `provider`, the connected
/// apps, and one row per entry with what its card says.
pub fn transcript_json(state: &EngineState) -> String {
let status = match &state.status {
Status::Idle => "idle".to_string(),
Status::Loading { phase, fraction } => format!("loading {phase} {:.0}%", fraction * 100.0),
Status::Thinking => "thinking".to_string(),
Status::Streaming => "streaming".to_string(),
Status::WaitingForTool => "waiting_for_tool".to_string(),
Status::Error(e) => format!("error: {e}"),
};
let entries = state
.entries
.iter()
.map(|entry| match entry {
Entry::User { text } => TranscriptRow {
kind: "user".into(),
text: text.clone(),
title: String::new(),
status: String::new(),
note: String::new(),
},
Entry::Event(event) => TranscriptRow {
kind: "event".into(),
text: event.text.clone(),
title: format!("{} · {}", event.service_label, event.topic),
status: if event.final_ { "final".into() } else { "message".into() },
note: if event.dropped == 0 {
format!("sub_id: {}", event.sub_id)
} else {
format!("sub_id: {} · dropped: {}", event.sub_id, event.dropped)
},
},
Entry::Assistant { text, streaming } => TranscriptRow {
kind: "assistant".into(),
text: text.clone(),
title: String::new(),
status: if *streaming { "streaming".into() } else { "done".into() },
note: String::new(),
},
Entry::Tool(t) => {
let (status, note, text) = match &t.status {
ToolStatus::Confirm => ("confirm".to_string(), String::new(), String::new()),
ToolStatus::Running { note, permille } => ("running".to_string(), format!("{note} {permille}"), String::new()),
ToolStatus::Done { outcome, note, text } => (outcome.slug().to_string(), note.clone(), text.clone()),
};
TranscriptRow { kind: "tool".into(), text, title: t.title.clone(), status, note }
}
Entry::System { text } => TranscriptRow {
kind: "system".into(),
text: text.clone(),
title: String::new(),
status: String::new(),
note: String::new(),
},
})
.collect();
Transcript {
status,
provider: state.provider_label.clone(),
apps: state.services.iter().filter(|s| s.connected).map(|s| s.label.clone()).collect(),
entries,
generation: state.generation,
}
.serialize_json()
}
#[derive(Script, ScriptHook, Widget)]
pub struct AiChatOverlay {
#[source]
source: ScriptObjectRef,
#[deref]
view: View,
/// The state generation last published as JSON.
#[rust]
published: Option<u64>,
}
impl AiChatOverlay {
/// The transcript JSON, rewritten when the engine's state moved — on
/// every event as well as every draw, so a closed slot (no draws) still
/// answers `/ai/transcript` with the turn that finished behind it.
fn publish_transcript(&mut self, cx: &mut Cx) {
let json = self
.view
.widget(cx, ids!(panel))
.borrow::<AiChatPanel>()
.and_then(|panel| {
let state = panel.state()?;
if self.published == Some(state.generation) {
return None;
}
Some((state.generation, transcript_json(state)))
});
if let Some((generation, json)) = json {
self.published = Some(generation);
cx.global::<AiTranscript>().json = json;
}
}
/// The links the apps parked and the lines the bridge asked for, into
/// the panel.
fn adopt_requests(&mut self, cx: &mut Cx) {
let links = cx.global::<PendingServiceLinks>().take();
let says = std::mem::take(&mut cx.global::<AiSlotRequests>().say);
if links.is_empty() && says.is_empty() {
return;
}
let panel = self.view.widget(cx, ids!(panel));
let Some(mut panel) = panel.borrow_mut::<AiChatPanel>() else {
return;
};
for link in links {
let label = link.manifest.label.clone();
match panel.registry().register(link, "in this window", None) {
Ok(endpoint) => log!("aichat: {label} joined in-process as {}", endpoint.as_str()),
Err(e) => log!("aichat: {label} refused: {e}"),
}
}
for text in says {
panel.say(cx, text);
}
}
}
impl Widget for AiChatOverlay {
fn handle_event(&mut self, cx: &mut Cx, event: &Event, scope: &mut Scope) {
self.adopt_requests(cx);
self.view.handle_event(cx, event, scope);
self.publish_transcript(cx);
if let Event::Actions(actions) = event {
for action in actions {
if let Some(widget_action) = action.as_widget_action() {
if let AiChatPanelAction::Close = widget_action.cast() {
cx.global::<AiSlotRequests>().open = Some(false);
cx.redraw_all();
}
}
}
}
}
fn draw_walk(&mut self, cx: &mut Cx2d, scope: &mut Scope, walk: Walk) -> DrawStep {
let step = self.view.draw_walk(cx, scope, walk);
if step.is_done() {
self.publish_transcript(cx);
}
step
}
}
#[cfg(test)]
mod tests {
use super::*;
use makepad_ai_services::state::{EventEntry, ServiceInfo, ToolEntry};
use makepad_ai_services::wire::ToolOutcome;
#[test]
fn the_transcript_reads_as_json_rows() {
let mut state = EngineState::default();
state.provider_label = "No model".into();
state.services.push(ServiceInfo {
id: "sheets".into(),
endpoint: "e1".into(),
label: "Sheets".into(),
parent: None,
location: "in this window".into(),
connected: true,
launchable: false,
tool_count: 1,
});
state.entries.push(Entry::User { text: "/sheets.summary {}".into() });
state.entries.push(Entry::Event(EventEntry {
sub_id: "s1".into(),
service_label: "Sheets".into(),
topic: "changes".into(),
text: "A1 changed".into(),
data: None,
dropped: 2,
final_: false,
}));
state.entries.push(Entry::Tool(ToolEntry {
call_id: "c1".into(),
service: "sheets".into(),
service_label: "Sheets".into(),
tool: "summary".into(),
title: "Sheets · summary".into(),
args: "{}".into(),
status: ToolStatus::Done { outcome: ToolOutcome::Ok, note: "3 × 4".into(), text: "Sheet1, 3 rows".into() },
preview: false,
expanded: false,
}));
state.generation = 7;
let json = transcript_json(&state);
assert!(json.contains(r#""status":"idle""#), "{json}");
assert!(json.contains(r#""apps":["Sheets"]"#), "{json}");
assert!(json.contains(r#""kind":"user""#) && json.contains(r#""kind":"tool""#), "{json}");
assert!(json.contains(r#""kind":"event""#) && json.contains("sub_id: s1") && json.contains("dropped: 2"), "{json}");
assert!(json.contains(r#""title":"Sheets · summary""#) && json.contains(r#""status":"ok""#), "{json}");
assert!(json.contains(r#""generation":7"#), "{json}");
}
}

View file

@ -1,589 +0,0 @@
//! The chat panel: the one widget every host shows. It owns the engine —
//! the service registry, the model, the transcript — and draws
//! `EngineState` as a transcript of user lines, assistant text, tool cards
//! (running, done, waiting for a confirm), and system lines, over a
//! composer.
//!
//! Hosts talk to it in two ways: they hand it service links
//! (`registry()`) or bus frames (`on_custom`), and they listen for
//! [`AiChatPanelAction`]s. The engine runs on the panel's own events: every
//! event pumps it, and while a turn is in flight the panel asks for the
//! next frame so streaming, deadlines and the cloud provider's polling all
//! advance without a host timer.
use crate::bus::ServiceBus;
use crate::gen::GenService;
use crate::settings::AiSettings;
#[cfg(feature = "engine")]
use makepad_ai_services::engine::models::{build_model, provider_rows};
use makepad_ai_services::engine::NoModelWithReason;
use makepad_ai_services::engine::{EngineCore, EngineEvent, ServiceRegistry};
use makepad_ai_services::state::*;
use makepad_ai_services::wire::ToolOutcome;
use makepad_widgets::*;
script_mod! {
use mod.prelude.widgets_internal.*
use mod.widgets.*
mod.widgets.AiChatPanelBase = #(AiChatPanel::register_widget(vm))
let Line = Label{
width: Fill
height: Fit
draw_text +: {
color: theme.color_text
text_style: theme.font_regular{font_size: 9.5}
}
}
let Row = View{
width: Fill
height: Fit
flow: Down
padding: Inset{left: 14 right: 14 top: 4 bottom: 4}
}
mod.widgets.AiChatPanel = set_type_default() do mod.widgets.AiChatPanelBase{
width: Fill
height: Fill
flow: Down
draw_bg +: { color: theme.color_bg_app }
header := SolidView{
width: Fill
height: 36
flow: Right
spacing: 10
padding: Inset{left: 14 right: 8}
align: Align{y: 0.5}
draw_bg +: { color: theme.color_bg_container }
title := Label{
text: "AI"
draw_text +: {
color: theme.color_text
text_style: theme.font_bold{font_size: 10.5}
}
}
provider := Label{
width: Fill
text: ""
draw_text +: {
color: theme.color_text_meta
text_style: theme.font_regular{font_size: 8.5}
}
}
clear_button := ButtonFlatter{ text: "Clear" }
}
apps_row := Label{
width: Fill
height: Fit
padding: Inset{left: 14 right: 14 top: 6 bottom: 2}
max_lines: 2
text: ""
draw_text +: {
color: theme.color_text_meta
text_style: theme.font_regular{font_size: 8.5}
}
}
transcript := PortalList{
width: Fill
height: Fill
auto_tail: true
UserRow := Row{
padding: Inset{left: 14 right: 14 top: 10 bottom: 4}
user_text := Line{
draw_text +: {
color: theme.color_text
text_style: theme.font_bold{font_size: 9.5}
}
}
}
EventRow := Row{
margin: Inset{left: 10 right: 10 top: 5 bottom: 5}
padding: Inset{left: 10 right: 10 top: 7 bottom: 7}
draw_bg +: { color: theme.color_bg_container }
event_title := Line{
draw_text +: {
color: theme.color_text_hl
text_style: theme.font_bold{font_size: 8.5}
}
}
event_text := Line{}
event_meta := Line{
draw_text +: {
color: theme.color_text_meta
text_style: theme.font_regular{font_size: 8.0}
}
}
}
AssistantRow := Row{
assistant_md := Markdown{
width: Fill
height: Fit
body: ""
}
}
StreamRow := Row{
stream_text := Line{}
}
ToolRow := Row{
padding: Inset{left: 22 right: 14 top: 3 bottom: 3}
tool_head := View{
width: Fill
height: Fit
cursor: MouseCursor.Hand
tool_title := Line{
draw_text +: {
color: theme.color_text_meta
text_style: theme.font_regular{font_size: 8.5}
}
}
}
tool_note := Line{
draw_text +: {
color: theme.color_text_meta
text_style: theme.font_regular{font_size: 8.5}
}
}
tool_bar := SolidView{
width: Fill
height: 2
margin: Inset{top: 3}
draw_bg +: { color: theme.color_text_hl }
}
tool_detail := Line{
visible: false
draw_text +: {
color: theme.color_text_meta
text_style: theme.font_regular{font_size: 8.0}
}
}
}
ConfirmRow := Row{
padding: Inset{left: 22 right: 14 top: 6 bottom: 6}
confirm_title := Line{
draw_text +: {
color: theme.color_text
text_style: theme.font_regular{font_size: 9.0}
}
}
confirm_buttons := View{
width: Fill
height: Fit
flow: Right
spacing: 8
margin: Inset{top: 4}
run_button := Button{ text: "Run" }
deny_button := ButtonFlat{ text: "Cancel" }
}
}
SystemRow := Row{
system_text := Line{
draw_text +: {
color: theme.color_text_hl
text_style: theme.font_regular{font_size: 8.5}
}
}
}
}
status := Label{
width: Fill
height: Fit
padding: Inset{left: 14 right: 14 top: 4 bottom: 2}
max_lines: 2
text: ""
draw_text +: {
color: theme.color_text_meta
text_style: theme.font_regular{font_size: 8.5}
}
}
composer := View{
width: Fill
height: Fit
flow: Right
spacing: 8
padding: Inset{left: 12 right: 12 top: 6 bottom: 10}
align: Align{y: 0.5}
input := TextInput{
width: Fill
height: Fit
empty_text: "Ask AI"
// The prompt is a hint, not text: a dark grey in every state,
// never the typed colour (the composer is always focused).
draw_text +: {
color_empty: #666666
color_empty_hover: #777777
color_empty_focus: #666666
}
}
send_button := Button{ text: "Send" }
}
}
}
/// What the panel tells its host.
#[derive(Clone, Debug, PartialEq, Default)]
pub enum AiChatPanelAction {
/// Esc with an empty composer and no turn in flight: the host may hide
/// the pane.
Close,
#[default]
None,
}
#[derive(Script, ScriptHook, Widget)]
pub struct AiChatPanel {
#[source]
source: ScriptObjectRef,
#[deref]
view: View,
#[rust]
engine: Option<EngineCore>,
#[rust]
registry: ServiceRegistry,
#[rust]
bus: ServiceBus,
/// The assistant's own `gen` service (pictures from the fleet), joined
/// to the registry with the engine.
#[rust]
gen: Option<GenService>,
#[rust]
settings: Option<AiSettings>,
#[rust]
drawn_generation: u64,
#[rust]
next_frame: NextFrame,
/// The composer took the keyboard once it existed on screen — a
/// focus set before the first draw lands on no area at all.
#[rust]
composer_focused: bool,
}
impl AiChatPanel {
/// The registry a host plugs in-process links into.
pub fn registry(&self) -> &ServiceRegistry {
&self.registry
}
/// A studio `Custom` frame that may be a bus frame from the WM.
pub fn on_custom(&mut self, json: &str) -> bool {
self.bus.on_custom(&self.registry, json)
}
pub fn state(&self) -> Option<&EngineState> {
self.engine.as_ref().map(|e| e.state())
}
fn settings(&mut self) -> &AiSettings {
if self.settings.is_none() {
self.settings = Some(AiSettings::load());
}
self.settings.as_ref().unwrap()
}
/// The engine comes up on first use — a local model is a long load
/// and no host pays it before the person opens the pane.
fn ensure_engine(&mut self) -> &mut EngineCore {
if self.engine.is_none() {
let lease_id = self.widget_uid().0;
let settings = self.settings().clone();
// The real models ride the `engine` feature; a build without a
// runtime (the web page) says so and keeps the tool console.
#[cfg(feature = "engine")]
let (model, rows): (Box<dyn makepad_ai_services::Model>, Vec<ProviderRow>) = (
match build_model(&settings.provider, settings.local_only) {
Ok(m) => m,
Err(reason) => Box::new(NoModelWithReason::new(reason)),
},
provider_rows(settings.local_only),
);
#[cfg(not(feature = "engine"))]
let (model, rows): (Box<dyn makepad_ai_services::Model>, Vec<ProviderRow>) =
(Box::new(NoModelWithReason::new("this build has no model runtime")), Vec::new());
let mut core = EngineCore::new(self.registry.clone(), model, None, lease_id);
// The state is the panel's window into the core; the core owns
// it, so provider facts go in through the core.
core.set_provider_facts(settings.provider.clone(), rows, settings.local_only);
self.engine = Some(core);
self.gen = GenService::open(&self.registry);
}
self.engine.as_mut().unwrap()
}
fn now(cx: &Cx) -> f64 {
cx.seconds_since_app_start()
}
/// A line as if typed and sent — the bridge's `/ai?say=`.
pub fn say(&mut self, cx: &mut Cx, text: String) {
self.send(cx, text);
}
fn send(&mut self, cx: &mut Cx, text: String) {
let now = Self::now(cx);
self.ensure_engine().send(&text, now);
let input = self.view.text_input(cx, ids!(input));
input.set_text(cx, "");
// The widget drops the keyboard on submit; a chat composer keeps
// it, so the next line can be typed straight away.
input.take_key_focus(cx);
self.view.redraw(cx);
}
fn apps_line(&self, state: &EngineState) -> String {
if state.services.is_empty() {
return "No apps connected.".into();
}
state
.services
.iter()
.map(|s| if s.connected { s.label.clone() } else { format!("{} (not running)", s.label) })
.collect::<Vec<_>>()
.join(" · ")
}
fn status_line(state: &EngineState) -> String {
let rate = state.rate.map(|r| format!(" · {r:.0} tok/s")).unwrap_or_default();
match &state.status {
Status::Idle => rate.trim_start_matches(" · ").to_string(),
Status::Loading { phase, fraction } => format!("loading {phase} {:.0}%", fraction * 100.0),
Status::Thinking => {
if state.thinking.is_empty() {
"thinking…".to_string()
} else {
let tail: String = state.thinking.chars().rev().take(120).collect::<Vec<_>>().into_iter().rev().collect();
format!("thinking… {tail}")
}
}
Status::Streaming => format!("writing{rate}"),
Status::WaitingForTool => "waiting for the app…".to_string(),
Status::Error(e) => e.clone(),
}
}
}
impl Drop for AiChatPanel {
fn drop(&mut self) {
// The engine only enqueues Unsubscribe frames. Flush them while the
// hosted bus and registry still exist; field destruction is too late.
if let Some(engine) = self.engine.as_mut() {
engine.shutdown();
}
self.bus.relay_down(&self.registry);
}
}
impl Widget for AiChatPanel {
fn handle_event(&mut self, cx: &mut Cx, event: &Event, scope: &mut Scope) {
if let Event::Custom(json) = event {
self.on_custom(json);
}
// Esc: stop a running turn; with nothing running and an empty
// composer, ask the host to hide the pane.
if let Event::KeyDown(ke) = event {
if ke.key_code == KeyCode::Escape {
let busy = self.engine.as_ref().map(|e| e.state().status.is_busy()).unwrap_or(false);
let empty = self.view.text_input(cx, ids!(input)).text().trim().is_empty();
if busy {
let now = Self::now(cx);
self.ensure_engine().cancel(now);
self.view.redraw(cx);
} else if empty {
cx.widget_action(self.widget_uid(), AiChatPanelAction::Close);
}
}
}
// The built-in services answer before the engine pumps, so a
// finished picture lands in this same event.
if let Some(gen) = self.gen.as_mut() {
gen.handle_event(cx, event);
}
// Drive the engine on every event; the bus relays what it sent.
let now = Self::now(cx);
let mut changed = false;
let mut busy = false;
if let Some(engine) = self.engine.as_mut() {
for ev in engine.pump(now) {
match ev {
EngineEvent::Changed => changed = true,
EngineEvent::Confirm { .. } => changed = true,
}
}
busy = engine.needs_pump()
|| engine.state().status.is_busy()
|| matches!(engine.state().status, Status::Loading { .. });
}
self.bus.relay_down(&self.registry);
if changed {
self.view.redraw(cx);
}
if busy {
self.next_frame = cx.new_next_frame();
}
self.view.handle_event(cx, event, scope);
self.widget_match_event(cx, event, scope);
}
fn draw_walk(&mut self, cx: &mut Cx2d, scope: &mut Scope, walk: Walk) -> DrawStep {
// The engine exists once the panel is on screen.
let _ = self.ensure_engine();
let state = self.engine.as_ref().unwrap().state().clone();
self.drawn_generation = state.generation;
self.view.label(cx, ids!(provider)).set_text(cx, &format!("{}{}", state.provider_label, if state.local_only { " · local only" } else { "" }));
self.view.label(cx, ids!(apps_row)).set_text(cx, &self.apps_line(&state));
self.view.label(cx, ids!(status)).set_text(cx, &Self::status_line(&state));
while let Some(item) = self.view.draw_walk(cx, scope, walk).step() {
let Some(mut list) = item.borrow_mut::<PortalList>() else { continue };
let total = state.entries.len();
list.set_item_range(cx, 0, total);
while let Some(index) = list.next_visible_item(cx) {
if index >= total {
continue;
}
let entry = &state.entries[index];
let row = match entry {
Entry::User { text } => {
let row = list.item(cx, index, id!(UserRow));
row.label(cx, ids!(user_text)).set_text(cx, text);
row
}
Entry::Event(event) => {
let row = list.item(cx, index, id!(EventRow));
row.label(cx, ids!(event_title))
.set_text(cx, &format!("{} · {}", event.service_label, event.topic));
row.label(cx, ids!(event_text)).set_text(cx, &event.text);
let mut meta = format!("sub_id: {}", event.sub_id);
if event.dropped != 0 {
meta.push_str(&format!(" · dropped: {}", event.dropped));
}
if event.final_ {
meta.push_str(" · final");
}
if let Some(data) = &event.data {
meta.push_str(&format!("\n{data}"));
}
row.label(cx, ids!(event_meta)).set_text(cx, &meta);
row
}
Entry::Assistant { text, streaming: false } => {
let row = list.item(cx, index, id!(AssistantRow));
// No text, no row: a blank block above a card is a gap.
row.set_visible(cx, !text.trim().is_empty());
if let Some(mut md) = row.widget(cx, ids!(assistant_md)).borrow_mut::<Markdown>() {
md.set_text(cx, text);
}
row
}
Entry::Assistant { text, streaming: true } => {
let row = list.item(cx, index, id!(StreamRow));
row.set_visible(cx, !text.trim().is_empty());
row.label(cx, ids!(stream_text)).set_text(cx, text);
row
}
Entry::Tool(t) if matches!(t.status, ToolStatus::Confirm) => {
let row = list.item(cx, index, id!(ConfirmRow));
row.label(cx, ids!(confirm_title)).set_text(cx, &format!("{} — this changes things outside the app. Run it?", t.title));
row
}
Entry::Tool(t) => {
let row = list.item(cx, index, id!(ToolRow));
let (glyph, note, permille, detail) = match &t.status {
ToolStatus::Running { note, permille } => ("", note.clone(), *permille, String::new()),
ToolStatus::Done { outcome, note, text } => {
let glyph = if outcome.is_ok() { "" } else { "" };
let note = if note.is_empty() { outcome.slug().to_string() } else { note.clone() };
(glyph, note, 1000, text.clone())
}
ToolStatus::Confirm => ("?", String::new(), 0, String::new()),
};
row.label(cx, ids!(tool_title)).set_text(cx, &format!("{glyph} {}", t.title));
row.label(cx, ids!(tool_note)).set_text(cx, &note);
let bar_visible = matches!(t.status, ToolStatus::Running { .. });
row.view(cx, ids!(tool_bar)).set_visible(cx, bar_visible);
if bar_visible {
// The bar is a fraction of the row's inner width
// (the row's rect is last frame's; the first
// frame of a card draws it at a token width).
let inner = (row.area().rect(cx).size.x - 36.0).max(24.0);
let px = inner * (permille as f64 / 1000.0).max(0.05);
let mut bar = row.view(cx, ids!(tool_bar));
script_apply_eval!(cx, bar, { width: #(px) });
}
let detail_label = row.label(cx, ids!(tool_detail));
detail_label.set_visible(cx, t.expanded && !detail.is_empty());
if t.expanded {
detail_label.set_text(cx, &format!("{}\n{}", t.args, detail));
}
let _ = matches!(t.status, ToolStatus::Done { outcome: ToolOutcome::Ok, .. });
row
}
Entry::System { text } => {
let row = list.item(cx, index, id!(SystemRow));
row.label(cx, ids!(system_text)).set_text(cx, text);
row
}
};
row.draw_all(cx, &mut Scope::empty());
}
}
if !self.composer_focused {
self.composer_focused = true;
self.view.text_input(cx, ids!(input)).take_key_focus(cx);
}
DrawStep::done()
}
}
impl WidgetMatchEvent for AiChatPanel {
fn handle_actions(&mut self, cx: &mut Cx, actions: &Actions, scope: &mut Scope) {
let input = self.view.text_input(cx, ids!(input));
let returned = input.returned(actions).map(|(text, _)| text);
if let Some(text) = returned {
self.send(cx, text);
} else if self.view.button(cx, ids!(send_button)).clicked(actions) {
let text = input.text();
self.send(cx, text);
}
if self.view.button(cx, ids!(clear_button)).clicked(actions) {
let now = Self::now(cx);
self.ensure_engine().clear(now);
self.view.redraw(cx);
}
// Cards: confirm buttons and click-to-expand.
let list = self.view.portal_list(cx, ids!(transcript));
let items = list.items_with_actions(actions);
if !items.is_empty() {
let now = Self::now(cx);
let call_ids: Vec<Option<String>> = {
let state = self.ensure_engine().state();
items
.iter()
.map(|(index, _)| match state.entries.get(*index) {
Some(Entry::Tool(t)) => Some(t.call_id.clone()),
_ => None,
})
.collect()
};
for ((_, item), call_id) in items.iter().zip(call_ids) {
let Some(call_id) = call_id else { continue };
if item.button(cx, ids!(run_button)).clicked(actions) {
self.ensure_engine().confirm(&call_id, true, now);
self.view.redraw(cx);
} else if item.button(cx, ids!(deny_button)).clicked(actions) {
self.ensure_engine().confirm(&call_id, false, now);
self.view.redraw(cx);
} else if item.view(cx, ids!(tool_head)).finger_up(actions).is_some() {
self.ensure_engine().toggle_tool(&call_id);
self.view.redraw(cx);
}
}
}
let _ = scope;
}
}

View file

@ -1,101 +0,0 @@
//! What the person chose: which model answers, and whether cloud models
//! are locked out. Two lines in `~/.makepad/aichat/settings`, the lock a
//! promise kept in code (a cloud choice under the lock normalises to
//! local at load and is refused at set), never a menu filter.
use makepad_ai_services::state::ProviderChoice;
use std::path::PathBuf;
#[derive(Clone, Debug, PartialEq)]
pub struct AiSettings {
pub provider: ProviderChoice,
/// Default ON: the person must switch it off before any cloud model
/// can be picked.
pub local_only: bool,
}
impl Default for AiSettings {
fn default() -> Self {
AiSettings { provider: ProviderChoice::Local, local_only: true }
}
}
impl AiSettings {
/// The lock, applied: a cloud provider under the lock becomes local.
/// "none" (no model) is always allowed — it reaches nothing.
pub fn normalized(mut self) -> Self {
if self.local_only {
if let ProviderChoice::Cloud(slug) = &self.provider {
if slug != "none" {
self.provider = ProviderChoice::Local;
}
}
}
self
}
/// Can this choice be made under the current lock?
pub fn allows(&self, choice: &ProviderChoice) -> Result<(), String> {
match choice {
ProviderChoice::Cloud(slug) if self.local_only && slug != "none" => Err("Local AI only is on".into()),
_ => Ok(()),
}
}
pub fn parse(text: &str) -> AiSettings {
let mut s = AiSettings::default();
for line in text.lines() {
let Some((k, v)) = line.split_once('=') else { continue };
match k.trim() {
"provider" => s.provider = ProviderChoice::from_slug(v.trim()),
"local_only" => s.local_only = v.trim() != "false",
_ => {}
}
}
s.normalized()
}
pub fn render(&self) -> String {
format!("provider={}\nlocal_only={}\n", self.provider.slug(), self.local_only)
}
pub fn path() -> Option<PathBuf> {
if cfg!(target_arch = "wasm32") {
return None;
}
let home = std::env::var_os("HOME").or_else(|| std::env::var_os("USERPROFILE"))?;
Some(PathBuf::from(home).join(".makepad").join("aichat").join("settings"))
}
pub fn load() -> AiSettings {
match Self::path().and_then(|p| std::fs::read_to_string(p).ok()) {
Some(text) => Self::parse(&text),
None => AiSettings::default(),
}
}
pub fn save(&self) -> Result<(), String> {
let Some(path) = Self::path() else { return Ok(()) };
if let Some(dir) = path.parent() {
std::fs::create_dir_all(dir).map_err(|e| e.to_string())?;
}
std::fs::write(&path, self.render()).map_err(|e| e.to_string())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_lock_is_kept_at_load_and_at_set() {
let s = AiSettings::parse("provider=claude-api\nlocal_only=true\n");
assert_eq!(s.provider, ProviderChoice::Local, "a cloud pref under the lock cannot resurrect a cloud model");
assert!(s.allows(&ProviderChoice::Cloud("claude-api".into())).is_err());
assert!(s.allows(&ProviderChoice::Cloud("none".into())).is_ok());
let open = AiSettings::parse("provider=claude-api\nlocal_only=false\n");
assert_eq!(open.provider, ProviderChoice::Cloud("claude-api".into()));
assert_eq!(AiSettings::parse(&open.render()), open);
assert_eq!(AiSettings::parse("garbage"), AiSettings::default());
}
}

View file

@ -19,7 +19,6 @@ makepad-render = { path = "../../libs/render" }
makepad-gltf = { path = "../../libs/gltf" }
# Splat viewer: ViewSplat + XrSceneView desktop host.
makepad-xr = { path = "../../xr" }
makepad-media-view = { path = "../../libs/media_view" }
# REAL Asset Server frontend: shared session lifecycle (discovery/auth/
# retry), catalog runtimes, committed-event subscriber, verified cache. The
# VJ worker owns these crates; this app consumes the public API only.

View file

@ -44,7 +44,7 @@ use makepad_audio_lyrics::bake::LyricsBaker;
use makepad_audio_lyrics::TrackLyrics;
use makepad_audio_sidechannels::{encode_stem_oggs, publish_side_channels};
use makepad_widgets::log;
use makepad_widgets::makepad_platform::thread::{SignalToUI, ThreadOptions, ThreadSpawner};
use makepad_widgets::makepad_platform::thread::SignalToUI;
use std::collections::HashSet;
use std::path::PathBuf;
use std::str::FromStr;
@ -545,8 +545,14 @@ pub struct AnalysisQueue {
fetch_generation: u64,
}
impl Default for AnalysisQueue {
fn default() -> Self {
AnalysisQueue::start()
}
}
impl AnalysisQueue {
pub fn start(spawner: ThreadSpawner) -> AnalysisQueue {
pub fn start() -> AnalysisQueue {
let (bake_tx, bake_requests) = channel::<BakeRequest>();
let (bake_done, bake_rx) = channel::<BakeMsg>();
let (fetch_tx, fetch_requests) = channel::<FetchRequest>();
@ -555,26 +561,12 @@ impl AnalysisQueue {
let worker_batch = Arc::clone(&batch);
// A failed spawn is not fatal: the queue simply never runs, and
// every enqueue reports it instead of pretending to work.
match spawner.spawn_worker(
ThreadOptions {
name: Some("asset-ui-stem-bake".into()),
..Default::default()
},
move || bake_loop(bake_requests, bake_done, worker_batch),
) {
Ok(handle) => handle.detach(),
Err(error) => log!("analysis bake worker unavailable: {error}"),
}
match spawner.spawn_worker(
ThreadOptions {
name: Some("asset-ui-stem-fetch".into()),
..Default::default()
},
move || fetch_loop(fetch_requests, fetch_done),
) {
Ok(handle) => handle.detach(),
Err(error) => log!("analysis fetch worker unavailable: {error}"),
}
let _ = std::thread::Builder::new()
.name("asset-ui-stem-bake".into())
.spawn(move || bake_loop(bake_requests, bake_done, worker_batch));
let _ = std::thread::Builder::new()
.name("asset-ui-stem-fetch".into())
.spawn(move || fetch_loop(fetch_requests, fetch_done));
AnalysisQueue {
bake_tx,
bake_rx,
@ -1481,8 +1473,7 @@ mod tests {
#[test]
fn the_queue_counts_a_batch_and_keeps_its_verdict() {
let cx = makepad_widgets::Cx::new(Box::new(|_, _| {}));
let mut queue = AnalysisQueue::start(cx.thread_spawner());
let mut queue = AnalysisQueue::start();
assert!(!queue.busy());
assert_eq!(queue.status_line(), "");
assert_eq!(queue.progress_fraction(), 0.0);

View file

@ -13,11 +13,12 @@
//! - gallery preview decodes run on a small worker pool (2..=8 threads)
//! pulling a last-in-first-out stack, capped so old off-screen work drops.
use makepad_widgets::makepad_platform::thread::{SignalToUI, ThreadOptions, ThreadSpawner};
use makepad_widgets::makepad_platform::thread::SignalToUI;
use makepad_widgets::{decode_image_from_data, ImageBuffer};
use std::collections::HashSet;
use std::path::PathBuf;
use std::sync::mpsc::{channel, Receiver, Sender};
use std::sync::{Arc, Condvar, Mutex};
/// What one background read is for.
pub enum IoPurpose {
@ -163,39 +164,35 @@ pub enum IoDone {
pub struct ArtifactIo {
tx: Sender<IoRequest>,
rx: Receiver<IoDone>,
gallery: Arc<GalleryStack>,
}
impl ArtifactIo {
pub fn start(spawner: ThreadSpawner) -> Self {
pub fn start() -> Self {
let (request_tx, request_rx) = channel::<IoRequest>();
let (done_tx, done_rx) = channel::<IoDone>();
let (gallery_tx, gallery_rx) = channel::<IoRequest>();
spawner
.spawn_worker(
ThreadOptions {
name: Some("asset-ui-artifact-io".into()),
..Default::default()
},
{
let done_tx = done_tx.clone();
move || dispatch_loop(request_rx, done_tx, gallery_tx)
},
)
.expect("artifact io dispatcher")
.detach();
spawner
.spawn_worker(
ThreadOptions {
name: Some("asset-ui-preview".into()),
..Default::default()
},
move || gallery_loop(gallery_rx, done_tx),
)
.expect("gallery decode worker")
.detach();
let gallery = Arc::new(GalleryStack::new());
std::thread::Builder::new()
.name("asset-ui-artifact-io".into())
.spawn({
let done_tx = done_tx.clone();
let gallery = Arc::clone(&gallery);
move || dispatch_loop(request_rx, done_tx, gallery)
})
.expect("artifact io dispatcher");
let n = gallery_worker_count();
for i in 0..n {
let done_tx = done_tx.clone();
let gallery = Arc::clone(&gallery);
std::thread::Builder::new()
.name(format!("asset-ui-preview-{i}"))
.spawn(move || gallery_loop(gallery, done_tx))
.expect("gallery decode worker");
}
Self {
tx: request_tx,
rx: done_rx,
gallery,
}
}
@ -209,6 +206,12 @@ impl ArtifactIo {
}
}
impl Drop for ArtifactIo {
fn drop(&mut self) {
self.gallery.shutdown();
}
}
fn is_gallery(purpose: &IoPurpose) -> bool {
matches!(
purpose,
@ -218,13 +221,13 @@ fn is_gallery(purpose: &IoPurpose) -> bool {
)
}
fn dispatch_loop(rx: Receiver<IoRequest>, tx: Sender<IoDone>, gallery: Sender<IoRequest>) {
fn dispatch_loop(rx: Receiver<IoRequest>, tx: Sender<IoDone>, gallery: Arc<GalleryStack>) {
// One connected client per session, reused across opens: against the
// local server a fresh connect costs more than the payload.
let mut store: Option<(crate::import::ServerSession, makepad_asset_client::AssetClient)> = None;
while let Ok(request) = rx.recv() {
if is_gallery(&request.purpose) {
let _ = gallery.send(request);
gallery.push_latest(request);
continue;
}
let done = process_with_store(request, &mut store);
@ -233,27 +236,18 @@ fn dispatch_loop(rx: Receiver<IoRequest>, tx: Sender<IoDone>, gallery: Sender<Io
}
SignalToUI::set_ui_signal();
}
gallery.shutdown();
}
fn gallery_loop(rx: Receiver<IoRequest>, tx: Sender<IoDone>) {
let mut gallery = GalleryStack::new();
while let Ok(request) = rx.recv() {
gallery.push_latest(request);
for request in rx.try_iter() {
gallery.push_latest(request);
}
while let Some(request) = gallery.pop_latest() {
let file = request.file.clone();
let done = process(request);
gallery.finish(&file);
if tx.send(done).is_err() {
return;
}
SignalToUI::set_ui_signal();
for request in rx.try_iter() {
gallery.push_latest(request);
}
fn gallery_loop(gallery: Arc<GalleryStack>, tx: Sender<IoDone>) {
while let Some(request) = gallery.pop_latest() {
let file = request.file.clone();
let done = process(request);
gallery.finish(&file);
if tx.send(done).is_err() {
return;
}
SignalToUI::set_ui_signal();
}
}
@ -270,22 +264,24 @@ struct GalleryInner {
}
struct GalleryStack {
inner: GalleryInner,
inner: Mutex<GalleryInner>,
cv: Condvar,
}
impl GalleryStack {
fn new() -> Self {
Self {
inner: GalleryInner {
inner: Mutex::new(GalleryInner {
stack: Vec::new(),
decoding: HashSet::new(),
shutdown: false,
},
}),
cv: Condvar::new(),
}
}
fn push_latest(&mut self, request: IoRequest) {
let g = &mut self.inner;
fn push_latest(&self, request: IoRequest) {
let mut g = self.inner.lock().expect("gallery stack");
if g.shutdown {
return;
}
@ -298,30 +294,43 @@ impl GalleryStack {
let drop_n = g.stack.len() - GALLERY_STACK_CAP;
g.stack.drain(0..drop_n);
}
self.cv.notify_one();
}
fn pop_latest(&mut self) -> Option<IoRequest> {
let g = &mut self.inner;
if g.shutdown {
return None;
}
while let Some(request) = g.stack.pop() {
if g.decoding.insert(request.file.clone()) {
return Some(request);
fn pop_latest(&self) -> Option<IoRequest> {
let mut g = self.inner.lock().expect("gallery stack");
loop {
if g.shutdown {
return None;
}
while let Some(request) = g.stack.pop() {
if g.decoding.insert(request.file.clone()) {
return Some(request);
}
}
g = self.cv.wait(g).expect("gallery stack");
}
None
}
fn finish(&mut self, file: &str) {
self.inner.decoding.remove(file);
fn finish(&self, file: &str) {
let mut g = self.inner.lock().expect("gallery stack");
g.decoding.remove(file);
}
fn shutdown(&mut self) {
self.inner.shutdown = true;
fn shutdown(&self) {
let mut g = self.inner.lock().expect("gallery stack");
g.shutdown = true;
self.cv.notify_all();
}
}
fn gallery_worker_count() -> usize {
std::thread::available_parallelism()
.map(|n| n.get())
.unwrap_or(4)
.clamp(2, 8)
}
/// A read whose bytes may come from the store. Everything past the fetch is
/// the same decode the local-file path uses — one viewer, two sources.
fn process_with_store(
@ -856,8 +865,7 @@ mod tests {
std::fs::write(&payload, b"mp4-bytes").unwrap();
let copy = dir.join("viewer-open.mp4");
let cx = makepad_widgets::Cx::new(Box::new(|_, _| {}));
let io = ArtifactIo::start(cx.thread_spawner());
let io = ArtifactIo::start();
io.request(IoRequest {
file: "clip.mp4".into(),
path: payload.clone(),
@ -1135,7 +1143,7 @@ mod tests {
#[test]
fn gallery_stack_is_last_requested_first_and_rebumps() {
let mut stack = GalleryStack::new();
let stack = GalleryStack::new();
let mk = |file: &str| IoRequest {
file: file.into(),
path: PathBuf::from(file),
@ -1160,7 +1168,7 @@ mod tests {
#[test]
fn gallery_stack_drops_oldest_when_capped() {
let mut stack = GalleryStack::new();
let stack = GalleryStack::new();
for i in 0..(GALLERY_STACK_CAP + 10) {
stack.push_latest(IoRequest {
file: format!("f{i}"),

View file

@ -56,9 +56,6 @@ use makepad_asset_client::{
use makepad_asset_data::{AssetId, AssetRevisionId};
pub use makepad_asset_data::AssetKind;
use makepad_widgets::log;
use makepad_widgets::makepad_platform::thread::{
Lane, TaskHandle, TaskPool, ThreadOptions, ThreadSpawner,
};
use std::collections::VecDeque;
use std::net::{IpAddr, SocketAddr};
use std::path::{Path, PathBuf};
@ -363,8 +360,6 @@ pub struct SearchResults {
/// happens through `start`/`poll`/`submit_search`/`select` only.
#[derive(Default)]
pub struct AssetStore {
pool: Option<TaskPool>,
spawner: Option<ThreadSpawner>,
/// Continuous ai-content-library → catalog publisher. Declared BEFORE
/// `embedded` so it is joined while the server it publishes into is
/// still alive.
@ -402,7 +397,7 @@ pub struct AssetStore {
/// boxes directly (the store advertises nothing any more — generation
/// is client-driven, aicore §9).
pub profiles: Remote<Vec<JobProfileDto>>,
profiles_task: Option<TaskHandle<Vec<JobProfileDto>>>,
profiles_rx: Option<std::sync::mpsc::Receiver<Vec<JobProfileDto>>>,
/// Committed catalog events, newest first, capped.
pub events: VecDeque<CatalogEventDto>,
/// The event feed delivered its initial cursor and is following commits.
@ -485,7 +480,7 @@ pub struct AssetStore {
succession_note: Option<String>,
/// The previous session is being torn down off-thread. Flips when its
/// cache roots are free for the next session to open.
releasing: Option<TaskHandle<()>>,
releasing: Option<Arc<AtomicBool>>,
/// What to connect to once `releasing` flips.
pending_session: Option<PendingSession>,
}
@ -503,13 +498,11 @@ impl AssetStore {
/// the client finds it through the same discovery/health path any LAN
/// peer would. Set the env var to skip embed and talk to a standalone
/// server instead.
pub fn start(&mut self, library_dir: PathBuf, pool: TaskPool, spawner: ThreadSpawner) {
pub fn start(&mut self, library_dir: PathBuf) {
if self.connector.is_some() || self.server.is_some() {
return;
}
self.library_dir = library_dir;
self.pool = Some(pool);
self.spawner = Some(spawner);
self.server_root = default_asset_server_root();
self.beacon = beacon_from_env();
self.embed = embed_policy_from_env();
@ -649,7 +642,7 @@ impl AssetStore {
label: handles.server_label.clone(),
server_id: handles.server_id,
});
self.endpoints = handles.endpoints;
self.endpoints = Some(handles.endpoints);
self.token = handles.token.clone();
self.handles = Some(*handles);
self.connector = None;
@ -662,15 +655,17 @@ impl AssetStore {
}
}
// Fleet-built generation profiles landing from their worker thread.
if let Some(result) = self.profiles_task.as_mut().and_then(TaskHandle::try_take) {
self.profiles_task = None;
match result {
if let Some(rx) = &self.profiles_rx {
match rx.try_recv() {
Ok(profiles) => {
self.profiles_rx = None;
self.profiles = Remote::Ready(profiles);
changed = true;
}
Err(error) => {
self.profiles = Remote::Failed(format!("fleet profile job failed: {error}"));
Err(std::sync::mpsc::TryRecvError::Empty) => {}
Err(std::sync::mpsc::TryRecvError::Disconnected) => {
self.profiles_rx = None;
self.profiles = Remote::Failed("fleet probe thread died".to_string());
changed = true;
}
}
@ -811,23 +806,13 @@ impl AssetStore {
/// Start the embedded server's background loops and become the host.
fn begin_hosting(&mut self, server: makepad_asset_store::AssetServer, token: &str) {
if self.host_loops == HostLoops::Run {
let Some(spawner) = self.spawner.as_ref() else {
log!("asset store: host workers unavailable (thread runtime not configured)");
self.role = ServerRole::Host;
self.embedded = Some(server);
return;
};
self.publish = start_publish_loop(
spawner,
&server,
token,
self.library_dir.clone(),
);
self.publish =
start_publish_loop(&server, token, self.library_dir.clone());
// LIVECODING: observed origin directories, catalogued in place.
// Only the HOST runs this — reference admission is loopback
// privilege, so an attached client never observes for somebody
// else's store.
self.observe = start_observe_loop(spawner, &server, token);
self.observe = start_observe_loop(&server, token);
}
self.role = ServerRole::Host;
self.embedded = Some(server);
@ -871,39 +856,39 @@ impl AssetStore {
self.search_continuation = false;
self.next_cursor = None;
self.detail_req = None;
self.profiles_task = None;
self.profiles_rx = None;
self.probe_req = None;
self.gc_req = None;
self.gc_cancel_req = None;
self.retire_reqs.clear();
let Some(pool) = &self.pool else {
log!("asset store: session release refused (runtime pool unavailable)");
return;
};
let slot = match pool.reserve(Lane::Heavy) {
Ok(slot) => slot,
Err(error) => {
log!("asset store: session release delayed ({error})");
return;
}
};
let released = Arc::new(AtomicBool::new(false));
self.releasing = Some(released.clone());
let Some(handles) = self.handles.take() else {
self.releasing = None;
released.store(true, Ordering::Release);
return;
};
self.releasing = Some(slot.submit(move || handles.shutdown()));
let done = released.clone();
let spawned = std::thread::Builder::new()
.name("asset-ui-session-release".to_string())
.spawn(move || {
handles.shutdown();
done.store(true, Ordering::Release);
});
if let Err(error) = spawned {
// The closure (and the session inside it) was dropped, which
// already joined the runtimes right here. Nothing is left to
// wait for, so never leave the swap parked on a thread that
// does not exist.
log!("asset store: session release thread refused ({error}); released inline");
released.store(true, Ordering::Release);
}
}
/// Open the decided session once the previous one has let go.
fn finish_swap(&mut self) -> bool {
if self.releasing.is_none() && self.handles.is_some() {
self.begin_release();
return false;
}
if let Some(releasing) = &mut self.releasing {
let Some(result) = releasing.try_take() else { return false };
if let Err(error) = result {
log!("asset store: session release failed: {error}");
if let Some(released) = &self.releasing {
if !released.load(Ordering::Acquire) {
return false;
}
}
self.releasing = None;
@ -1165,20 +1150,18 @@ impl AssetStore {
/// they can execute right now. Runs on its own thread — LAN probes must
/// never stall a frame — and lands through `profiles_rx` in [`Self::poll`].
fn submit_profiles(&mut self) {
let (tx, rx) = std::sync::mpsc::channel();
self.profiles_rx = Some(rx);
self.profiles = Remote::Loading;
let Some(pool) = &self.pool else {
self.profiles = Remote::Failed("runtime task pool unavailable".into());
return;
};
match pool.submit(Lane::Light, move || {
let _ = std::thread::Builder::new()
.name("asset-ui-profiles".to_string())
.spawn(move || {
let snapshots = makepad_asset_creator::runner::fleet_snapshots();
makepad_asset_importer::gen_profiles::build_profiles(&snapshots, "gen")
}) {
Ok(handle) => self.profiles_task = Some(handle),
Err(error) => {
self.profiles = Remote::Failed(format!("fleet profile job refused: {error}"));
}
}
let profiles = makepad_asset_importer::gen_profiles::build_profiles(
&snapshots, "gen",
);
let _ = tx.send(profiles);
});
}
fn on_catalog_event(&mut self, event: ClientEvent) -> bool {
@ -1690,17 +1673,21 @@ fn start_embedded_asset_server_at(
Ok((server, token))
}
/// Stop flag for the single in-process publish loop. A `static` (not an
/// `Arc`) because `watch::run` borrows it for the thread's whole life and
/// there is at most one loop per process.
static PUBLISH_STOP: AtomicBool = AtomicBool::new(false);
/// Owns the publisher thread; dropping the store stops and joins it.
struct PublishLoop {
stop: Arc<AtomicBool>,
task: Option<TaskHandle<()>>,
join: Option<std::thread::JoinHandle<()>>,
}
impl Drop for PublishLoop {
fn drop(&mut self) {
self.stop.store(true, Ordering::Release);
if let Some(task) = self.task.take() {
task.detach();
PUBLISH_STOP.store(true, Ordering::Release);
if let Some(join) = self.join.take() {
let _ = join.join();
}
}
}
@ -1710,7 +1697,6 @@ impl Drop for PublishLoop {
/// slow or refused connection can never stall the UI, and every failure is
/// a log line — never a panic.
fn start_publish_loop(
spawner: &ThreadSpawner,
server: &makepad_asset_store::AssetServer,
token: &str,
library_dir: PathBuf,
@ -1719,14 +1705,10 @@ fn start_publish_loop(
let server_id = server.server_id();
let token = token.to_string();
let cache = asset_ui_home().join("publish-cache");
let stop = Arc::new(AtomicBool::new(false));
let worker_stop = stop.clone();
let task = spawner.spawn_worker(
ThreadOptions {
name: Some("asset-ui-publish".into()),
..Default::default()
},
move || {
PUBLISH_STOP.store(false, Ordering::Release);
let join = std::thread::Builder::new()
.name("asset-ui-publish".to_string())
.spawn(move || {
let mut config = makepad_asset_client::ClientConfig::new(cache);
config.token = Some(token);
let mut client = match makepad_asset_client::AssetClient::connect(
@ -1754,13 +1736,12 @@ fn start_publish_loop(
// Log publications, failures and retries; out-of-scope rows
// (the pack-import bulk) stay silent by design.
true,
&worker_stop,
&PUBLISH_STOP,
);
log!("publish loop: stopped");
},
);
match task {
Ok(task) => Some(PublishLoop { stop, task: Some(task) }),
});
match join {
Ok(join) => Some(PublishLoop { join: Some(join) }),
Err(error) => {
log!("publish loop: could not spawn: {error}");
None
@ -1768,17 +1749,19 @@ fn start_publish_loop(
}
}
/// Stop flag for the single in-process observe loop.
static OBSERVE_STOP: AtomicBool = AtomicBool::new(false);
/// Owns the observer thread; dropping the store stops and joins it.
struct ObserveLoop {
stop: Arc<AtomicBool>,
task: Option<TaskHandle<()>>,
join: Option<std::thread::JoinHandle<()>>,
}
impl Drop for ObserveLoop {
fn drop(&mut self) {
self.stop.store(true, Ordering::Release);
if let Some(task) = self.task.take() {
task.detach();
OBSERVE_STOP.store(true, Ordering::Release);
if let Some(join) = self.join.take() {
let _ = join.join();
}
}
}
@ -1806,7 +1789,6 @@ fn observe_origins() -> Vec<PathBuf> {
/// Observe effect-document origins into the server this process hosts.
/// Connect + watch happen on the thread; every failure is a log line.
fn start_observe_loop(
spawner: &ThreadSpawner,
server: &makepad_asset_store::AssetServer,
token: &str,
) -> Option<ObserveLoop> {
@ -1815,14 +1797,10 @@ fn start_observe_loop(
let token = token.to_string();
let cache = asset_ui_home().join("observe-cache");
let origins = observe_origins();
let stop = Arc::new(AtomicBool::new(false));
let worker_stop = stop.clone();
let task = spawner.spawn_worker(
ThreadOptions {
name: Some("asset-ui-observe".into()),
..Default::default()
},
move || {
OBSERVE_STOP.store(false, Ordering::Release);
let join = std::thread::Builder::new()
.name("asset-ui-observe".to_string())
.spawn(move || {
let mut config = makepad_asset_client::ClientConfig::new(cache);
config.token = Some(token);
let mut client = match makepad_asset_client::AssetClient::connect(
@ -1839,13 +1817,12 @@ fn start_observe_loop(
makepad_asset_store::observe::run(
&mut client,
&makepad_asset_store::observe::ObserveConfig::vjfx(origins),
&worker_stop,
&OBSERVE_STOP,
);
log!("observe loop: stopped");
},
);
match task {
Ok(task) => Some(ObserveLoop { stop, task: Some(task) }),
});
match join {
Ok(join) => Some(ObserveLoop { join: Some(join) }),
Err(error) => {
log!("observe loop: could not spawn: {error}");
None
@ -2101,13 +2078,6 @@ mod tests {
namespace: "game".into(),
kind: None,
title: title.into(),
creator: String::new(),
artist: String::new(),
artist_url: String::new(),
album: String::new(),
source_url: String::new(),
license: String::new(),
license_url: String::new(),
snippet: String::new(),
score: 0,
live: true,
@ -2398,14 +2368,6 @@ mod tests {
(server, token)
}
fn ensure_test_runtime(store: &mut AssetStore) {
if store.pool.is_none() {
let cx = makepad_widgets::Cx::new(Box::new(|_, _| {}));
store.pool = Some(cx.task_pool());
store.spawner = Some(cx.thread_spawner());
}
}
/// Publish one minimal real asset directly (the synchronous
/// `AssetClient`, not through `AssetStore` — publishing is the import
/// pipeline's job, already covered elsewhere; this test starts from
@ -2450,7 +2412,6 @@ mod tests {
/// `AssetStore::poll()` loop (no discovery — explicit endpoints), and
/// wait for the initial auto-search `poll()` fires on connect to land.
fn connect_store_to(store: &mut AssetStore, server: &makepad_asset_store::AssetServer, token: &str) {
ensure_test_runtime(store);
let config = SessionConfig {
endpoints: Some(ApiEndpoints { control: server.control_addr(), data: server.data_addr() }),
server_id: Some(server.server_id()),
@ -2487,7 +2448,6 @@ mod tests {
secs: u64,
ready: F,
) {
ensure_test_runtime(store);
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(secs);
loop {
store.poll();

View file

@ -1,16 +1,15 @@
//! Artifact playback + waveform strip.
//!
//! The UI owns a transport handle and the device callback owns the playback
//! engine. Commands and atomic snapshots cross between them; neither side
//! waits on a mutex. The service emits PCM16 WAV (kokoro: mono 24kHz,
//! sa3-sfx: stereo 44.1kHz), decoded here with a minimal RIFF parser
//! (libs/asset/ai wav.rs is encode-only).
//! Same shape as the sandbox's `VideoAudio` mixer (apps/sandbox/src/
//! video_player.rs): a process-global resampling stereo queue mixed
//! additively from the `cx.audio_output` callback, so playback needs no
//! plumbing through the widget tree. The service emits PCM16 WAV (kokoro:
//! mono 24kHz, sa3-sfx: stereo 44.1kHz), decoded here with a minimal RIFF
//! parser (libs/asset/ai wav.rs is encode-only).
use makepad_widgets::makepad_platform::audio::AudioBuffer;
use makepad_widgets::makepad_platform::thread::{Lane, TaskHandle, TaskPool};
use std::cell::RefCell;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::{mpsc, Arc};
use std::sync::{Arc, LazyLock, Mutex};
#[derive(Clone)]
pub struct WavPcm {
@ -104,22 +103,24 @@ pub fn parse_wav(bytes: &[u8]) -> Result<WavPcm, String> {
/// advances it; UI code only loads, pauses and seeks.
const FP_ONE: u64 = 1 << 32;
struct AudioSnapshot {
struct WavMixer {
clip: Mutex<Option<Arc<WavPcm>>>,
cursor_fp: AtomicU64,
playing: AtomicBool,
ack: AtomicU64,
}
impl Default for AudioSnapshot {
impl Default for WavMixer {
fn default() -> Self {
Self {
clip: Mutex::new(None),
cursor_fp: AtomicU64::new(0),
playing: AtomicBool::new(false),
ack: AtomicU64::new(0),
}
}
}
static WAV_MIXER: LazyLock<WavMixer> = LazyLock::new(WavMixer::default);
// ---------------------------------------------------------------------------
// Separated layers ("split audio layers")
// ---------------------------------------------------------------------------
@ -149,301 +150,33 @@ pub const STEM_LANES: usize = 4;
/// files are published and fetched in, so no index ever has to be remapped.
pub const STEM_LANE_NAMES: [&str; STEM_LANES] = ["drums", "bass", "vocals", "other"];
enum AudioCommand {
InstallClip {
serial: u64,
clip: Arc<WavPcm>,
},
ClearClip {
serial: u64,
},
InstallStems {
serial: u64,
lanes: Arc<[StemPcm; STEM_LANES]>,
},
ClearStems {
serial: u64,
},
Play {
serial: u64,
},
Pause {
serial: u64,
},
Stop {
serial: u64,
},
Seek {
serial: u64,
cursor_fp: u64,
},
MuteLane {
serial: u64,
lane: usize,
muted: bool,
},
struct StemMixer {
lanes: Mutex<Option<Arc<[StemPcm; STEM_LANES]>>>,
/// Per-lane mute. Read by the audio callback, written by the UI.
mute: [AtomicBool; STEM_LANES],
/// Whether the layers are what the transport plays. Kept separate from
/// the lock so the callback can tell "no stems" from "stems, but the UI
/// holds the lock this quantum" — the second must be silence, not a
/// blip of the mixed track.
active: AtomicBool,
/// The clip generation these layers belong to. The mixed audio and the
/// four stems arrive on two different workers in either order; this is
/// what lets the second one to land know it is the same track.
generation: AtomicU64,
}
enum RetiredAudio {
Clip(Arc<WavPcm>),
Stems(Arc<[StemPcm; STEM_LANES]>),
}
struct PendingDecode {
generation: u64,
task: TaskHandle<Result<WavPcm, String>>,
}
/// UI-thread transport handle. It owns the requested state and communicates
/// with the realtime callback exclusively through commands and atomics.
struct AudioMixer {
commands: mpsc::Sender<AudioCommand>,
retired: mpsc::Receiver<RetiredAudio>,
snapshot: Arc<AudioSnapshot>,
engine: Option<AudioEngine>,
clip: Option<Arc<WavPcm>>,
stems: Option<Arc<[StemPcm; STEM_LANES]>>,
stem_generation: u64,
muted: [bool; STEM_LANES],
cursor_fp: u64,
playing: bool,
serial: u64,
load_generation: u64,
pending_decodes: Vec<PendingDecode>,
}
/// Realtime-owned state. Once installed in `cx.audio_output`, only the audio
/// callback touches these payloads and cursors.
pub struct AudioEngine {
commands: mpsc::Receiver<AudioCommand>,
retired: mpsc::Sender<RetiredAudio>,
snapshot: Arc<AudioSnapshot>,
clip: Option<Arc<WavPcm>>,
stems: Option<Arc<[StemPcm; STEM_LANES]>>,
muted: [bool; STEM_LANES],
cursor_fp: u64,
playing: bool,
ack: u64,
}
impl AudioMixer {
fn new() -> Self {
let (command_tx, command_rx) = mpsc::channel();
let (retired_tx, retired_rx) = mpsc::channel();
let snapshot = Arc::new(AudioSnapshot::default());
let engine = AudioEngine {
commands: command_rx,
retired: retired_tx,
snapshot: snapshot.clone(),
clip: None,
stems: None,
muted: [false; STEM_LANES],
cursor_fp: 0,
playing: false,
ack: 0,
};
impl Default for StemMixer {
fn default() -> Self {
Self {
commands: command_tx,
retired: retired_rx,
snapshot,
engine: Some(engine),
clip: None,
stems: None,
stem_generation: u64::MAX,
muted: [false; STEM_LANES],
cursor_fp: 0,
playing: false,
serial: 0,
load_generation: 0,
pending_decodes: Vec::new(),
lanes: Mutex::new(None),
mute: Default::default(),
active: AtomicBool::new(false),
generation: AtomicU64::new(u64::MAX),
}
}
fn take_engine(&mut self) -> Option<AudioEngine> {
self.engine.take()
}
fn next_serial(&mut self) -> u64 {
self.serial = self.serial.wrapping_add(1).max(1);
self.serial
}
fn send(&self, command: AudioCommand) {
let _ = self.commands.send(command);
}
fn refresh_snapshot(&mut self) {
let ack = self.snapshot.ack.load(Ordering::Acquire);
if ack >= self.serial {
self.cursor_fp = self.snapshot.cursor_fp.load(Ordering::Acquire);
self.playing = self.snapshot.playing.load(Ordering::Acquire);
}
}
fn pump(&mut self) {
self.refresh_snapshot();
for retired in self.retired.try_iter() {
match retired {
RetiredAudio::Clip(clip) => drop(clip),
RetiredAudio::Stems(stems) => drop(stems),
}
}
let pending_decodes = std::mem::take(&mut self.pending_decodes);
let mut waiting = Vec::with_capacity(pending_decodes.len());
for mut pending in pending_decodes {
let Some(result) = pending.task.try_take() else {
waiting.push(pending);
continue;
};
match result {
Ok(Ok(pcm)) if pending.generation == self.load_generation => {
self.install(pcm, pending.generation);
}
Ok(Err(error)) if pending.generation == self.load_generation => {
makepad_widgets::log!("audio decode failed: {error}");
}
Err(error) if pending.generation == self.load_generation => {
makepad_widgets::log!("audio decode task failed: {error}");
}
_ => {}
}
}
self.pending_decodes = waiting;
}
fn clear_stems(&mut self) {
self.stems = None;
self.stem_generation = u64::MAX;
self.muted = [false; STEM_LANES];
let serial = self.next_serial();
self.send(AudioCommand::ClearStems { serial });
}
fn clear(&mut self) {
self.clear_stems();
self.clip = None;
self.cursor_fp = 0;
self.playing = false;
let serial = self.next_serial();
self.send(AudioCommand::ClearClip { serial });
}
fn install(&mut self, pcm: WavPcm, generation: u64) -> bool {
if self.load_generation != generation {
return false;
}
if pcm.frames.is_empty() || pcm.sample_rate == 0 {
self.clear();
return false;
}
if self.stem_generation != generation {
self.clear_stems();
}
let clip = Arc::new(pcm);
self.clip = Some(clip.clone());
self.cursor_fp = 0;
self.playing = false;
let serial = self.next_serial();
self.send(AudioCommand::InstallClip { serial, clip });
true
}
fn load(&mut self, pcm: WavPcm) -> bool {
self.load_generation = self.load_generation.wrapping_add(1);
self.install(pcm, self.load_generation)
}
fn set_stems(&mut self, lanes: [StemPcm; STEM_LANES], generation: u64) -> bool {
if self.load_generation != generation {
return false;
}
if lanes
.iter()
.any(|lane| lane.frames.is_empty() || lane.sample_rate == 0)
{
self.clear_stems();
return false;
}
let lanes = Arc::new(lanes);
self.stems = Some(lanes.clone());
self.stem_generation = generation;
self.muted = [false; STEM_LANES];
let serial = self.next_serial();
self.send(AudioCommand::InstallStems { serial, lanes });
true
}
fn play(&mut self) {
let Some(clip) = self.clip.as_ref() else {
return;
};
let end = (clip.frames.len() as u64) << 32;
if self.cursor_fp >= end {
self.cursor_fp = 0;
}
self.playing = true;
let serial = self.next_serial();
self.send(AudioCommand::Play { serial });
}
fn pause(&mut self) {
self.playing = false;
let serial = self.next_serial();
self.send(AudioCommand::Pause { serial });
}
fn stop(&mut self) {
self.playing = false;
self.cursor_fp = 0;
let serial = self.next_serial();
self.send(AudioCommand::Stop { serial });
}
fn seek_fraction(&mut self, fraction: f64) {
let Some(clip) = self.clip.as_ref() else {
return;
};
let frame = (fraction.clamp(0.0, 1.0) * clip.frames.len() as f64) as u64;
self.cursor_fp = frame.min(clip.frames.len() as u64) << 32;
let serial = self.next_serial();
self.send(AudioCommand::Seek {
serial,
cursor_fp: self.cursor_fp,
});
}
fn set_lane_muted(&mut self, lane: usize, muted: bool) {
let Some(slot) = self.muted.get_mut(lane) else {
return;
};
*slot = muted;
let serial = self.next_serial();
self.send(AudioCommand::MuteLane {
serial,
lane,
muted,
});
}
}
thread_local! {
static AUDIO_MIXER: RefCell<AudioMixer> = RefCell::new(AudioMixer::new());
}
fn with_mixer<T>(f: impl FnOnce(&mut AudioMixer) -> T) -> T {
AUDIO_MIXER.with(|mixer| f(&mut mixer.borrow_mut()))
}
/// Move the engine into the app's one audio callback. It is intentionally
/// one-shot: there must never be a second owner of realtime state.
pub fn take_engine() -> AudioEngine {
with_mixer(|mixer| mixer.take_engine()).expect("audio engine is installed once")
}
/// Poll background decodes and reclaim callback-retired payloads on the UI.
pub fn pump() {
with_mixer(AudioMixer::pump);
}
static STEM_MIXER: LazyLock<StemMixer> = LazyLock::new(StemMixer::default);
/// Install four separated layers over the loaded clip. From here the
/// transport plays their SUM instead of the mixed track — which is also the
@ -457,32 +190,63 @@ pub fn pump() {
/// Refused (and the layers cleared) when a lane is empty: half a stem set is
/// a lie about what the asset carries.
pub fn set_stems(lanes: [StemPcm; STEM_LANES], generation: u64) -> bool {
with_mixer(|mixer| mixer.set_stems(lanes, generation))
if LOAD_GENERATION.load(Ordering::Acquire) != generation {
return false;
}
if lanes
.iter()
.any(|lane| lane.frames.is_empty() || lane.sample_rate == 0)
{
clear_stems();
return false;
}
for mute in &STEM_MIXER.mute {
mute.store(false, Ordering::Release);
}
*STEM_MIXER.lanes.lock().unwrap() = Some(Arc::new(lanes));
STEM_MIXER.generation.store(generation, Ordering::Release);
STEM_MIXER.active.store(true, Ordering::Release);
true
}
/// Back to the mixed track. Called whenever the clip changes, so a new
/// selection can never play the previous track's layers.
pub fn clear_stems() {
with_mixer(AudioMixer::clear_stems);
STEM_MIXER.active.store(false, Ordering::Release);
STEM_MIXER.generation.store(u64::MAX, Ordering::Release);
*STEM_MIXER.lanes.lock().unwrap() = None;
for mute in &STEM_MIXER.mute {
mute.store(false, Ordering::Release);
}
}
/// True when the transport is playing separated layers.
pub fn stems_ready() -> bool {
with_mixer(|mixer| mixer.stems.is_some())
STEM_MIXER.active.load(Ordering::Acquire)
}
pub fn lane_muted(lane: usize) -> bool {
with_mixer(|mixer| mixer.muted.get(lane).copied().unwrap_or(false))
STEM_MIXER
.mute
.get(lane)
.is_some_and(|mute| mute.load(Ordering::Acquire))
}
pub fn set_lane_muted(lane: usize, muted: bool) {
with_mixer(|mixer| mixer.set_lane_muted(lane, muted));
if let Some(mute) = STEM_MIXER.mute.get(lane) {
mute.store(muted, Ordering::Release);
}
}
/// Length of the installed layers, for the honest "these are this track's
/// stems" check a host wants before it draws the toggles.
pub fn stems_seconds() -> f64 {
with_mixer(|mixer| mixer.stems.as_ref().map_or(0.0, |lanes| lanes[0].seconds()))
STEM_MIXER
.lanes
.lock()
.unwrap()
.as_ref()
.map_or(0.0, |lanes| lanes[0].seconds())
}
/// One lane at a fixed-point cursor, linearly interpolated — the same
@ -512,44 +276,88 @@ fn sample_lane(lane: &StemPcm, cursor: u64) -> (f32, f32) {
/// a new clip generation, which invalidates any separated layers installed
/// for the previous one.
pub fn load(pcm: WavPcm) -> bool {
with_mixer(|mixer| mixer.load(pcm))
let generation = LOAD_GENERATION.fetch_add(1, Ordering::AcqRel) + 1;
install(pcm, generation)
}
/// Install a decoded clip under the generation it was decoded FOR. A newer
/// pick having happened in the meantime drops it.
///
/// The layers are cleared unless they were installed for THIS clip: the
/// fetch of a track's stems and the decode of its mixed audio run on two
/// workers, and whichever finishes second must not wipe the first.
fn install(pcm: WavPcm, generation: u64) -> bool {
if LOAD_GENERATION.load(Ordering::Acquire) != generation {
return false;
}
if pcm.frames.is_empty() || pcm.sample_rate == 0 {
clear();
return false;
}
if STEM_MIXER.generation.load(Ordering::Acquire) != generation {
clear_stems();
}
WAV_MIXER.playing.store(false, Ordering::Release);
*WAV_MIXER.clip.lock().unwrap() = Some(Arc::new(pcm));
WAV_MIXER.cursor_fp.store(0, Ordering::Release);
true
}
/// Discard the loaded clip and make the transport unavailable.
pub fn clear() {
with_mixer(AudioMixer::clear);
clear_stems();
WAV_MIXER.playing.store(false, Ordering::Release);
*WAV_MIXER.clip.lock().unwrap() = None;
WAV_MIXER.cursor_fp.store(0, Ordering::Release);
}
/// Which load request is current. A user clicking down a list starts several;
/// only the newest may install itself, or a slow decode of the track before
/// last lands on top of the one they are looking at.
static LOAD_GENERATION: AtomicU64 = AtomicU64::new(0);
/// Take a track's bytes — WAV, MP3 or Ogg — and make the transport play them.
///
/// Decoding happens in the runtime pool: the music library is MP3s, and
/// turning six minutes of one into PCM on the frame thread is a visible
/// stall. The UI polls the task and sends the accepted result to the engine.
/// Decoding happens on a worker: the music library is MP3s, and turning six
/// minutes of one into PCM on the frame thread is a visible stall. The mixer
/// is process-global, so the worker installs the result itself; there is
/// nothing to plumb back through the widget tree.
///
/// The transport goes unavailable immediately, because the previous track is
/// no longer what the well is showing — a stale clip left loaded is a play
/// button that plays the wrong song.
/// Returns the clip generation this request claimed, which is what a
/// side-channel fetch for the same track carries back into [`set_stems`].
pub fn load_clip_async(pool: &TaskPool, bytes: Vec<u8>) -> u64 {
with_mixer(|mixer| {
mixer.load_generation = mixer.load_generation.wrapping_add(1);
let generation = mixer.load_generation;
mixer.clear();
match pool.submit(Lane::Heavy, move || decode_clip(&bytes)) {
Ok(task) => mixer
.pending_decodes
.push(PendingDecode { generation, task }),
Err(error) => makepad_widgets::log!("audio decode job refused: {error}"),
pub fn load_clip_async(bytes: Vec<u8>) -> u64 {
let generation = LOAD_GENERATION.fetch_add(1, Ordering::AcqRel) + 1;
clear();
let bytes = Arc::new(bytes);
let spawned = std::thread::Builder::new()
.name("asset-ui-audio-decode".into())
.spawn({
let bytes = Arc::clone(&bytes);
move || {
let Ok(pcm) = decode_clip(&bytes) else {
return;
};
// A newer pick happened while this was decoding: drop it.
install(pcm, generation);
}
});
if spawned.is_err() {
// No worker to be had: decode here rather than leave a dead
// transport under a drawn waveform.
if let Ok(pcm) = decode_clip(&bytes) {
install(pcm, generation);
}
generation
})
}
generation
}
/// The clip generation currently claimed. A side-channel fetch records it
/// with the request and hands it back to [`set_stems`].
pub fn clip_generation() -> u64 {
with_mixer(|mixer| mixer.load_generation)
LOAD_GENERATION.load(Ordering::Acquire)
}
/// Any container the catalog carries, in the mixer's shape. RIFF is parsed
@ -580,45 +388,48 @@ pub fn decode_clip(bytes: &[u8]) -> Result<WavPcm, String> {
/// Start or resume. Starting from the end restarts at zero.
pub fn play() {
with_mixer(AudioMixer::play);
let clip = WAV_MIXER.clip.lock().unwrap();
let Some(clip) = clip.as_ref() else { return };
let end = (clip.frames.len() as u64) << 32;
if WAV_MIXER.cursor_fp.load(Ordering::Acquire) >= end {
WAV_MIXER.cursor_fp.store(0, Ordering::Release);
}
WAV_MIXER.playing.store(true, Ordering::Release);
}
pub fn pause() {
with_mixer(AudioMixer::pause);
WAV_MIXER.playing.store(false, Ordering::Release);
}
/// Stop returns to the start but retains the decoded clip for replay.
pub fn stop() {
with_mixer(AudioMixer::stop);
pause();
WAV_MIXER.cursor_fp.store(0, Ordering::Release);
}
pub fn is_ready() -> bool {
with_mixer(|mixer| mixer.clip.is_some())
WAV_MIXER.clip.lock().unwrap().is_some()
}
pub fn is_playing() -> bool {
with_mixer(|mixer| {
mixer.refresh_snapshot();
let end = mixer
.clip
.as_ref()
.map_or(0, |clip| (clip.frames.len() as u64) << 32);
mixer.playing && mixer.cursor_fp < end
})
WAV_MIXER.playing.load(Ordering::Acquire) && !at_end()
}
pub fn duration_secs() -> f64 {
with_mixer(|mixer| mixer.clip.as_ref().map_or(0.0, |clip| clip.seconds()))
WAV_MIXER
.clip
.lock()
.unwrap()
.as_ref()
.map_or(0.0, |clip| clip.seconds())
}
/// Truthful device-clocked playhead, never derived from a UI timer.
pub fn playhead_secs() -> f64 {
with_mixer(|mixer| {
mixer.refresh_snapshot();
mixer.clip.as_ref().map_or(0.0, |clip| {
(mixer.cursor_fp as f64 / FP_ONE as f64) / clip.sample_rate as f64
})
})
let clip = WAV_MIXER.clip.lock().unwrap();
let Some(clip) = clip.as_ref() else { return 0.0 };
(WAV_MIXER.cursor_fp.load(Ordering::Acquire) as f64 / FP_ONE as f64)
/ clip.sample_rate as f64
}
/// Normalized playhead across the loaded clip for the waveform overlay:
@ -632,18 +443,19 @@ pub fn playhead_fraction() -> f64 {
}
pub fn at_end() -> bool {
with_mixer(|mixer| {
mixer.refresh_snapshot();
mixer
.clip
.as_ref()
.is_some_and(|clip| mixer.cursor_fp >= (clip.frames.len() as u64) << 32)
})
let clip = WAV_MIXER.clip.lock().unwrap();
let Some(clip) = clip.as_ref() else { return false };
WAV_MIXER.cursor_fp.load(Ordering::Acquire) >= (clip.frames.len() as u64) << 32
}
/// Sample-accurate fractional seek, clamped to the decoded clip.
pub fn seek_fraction(frac: f64) {
with_mixer(|mixer| mixer.seek_fraction(frac));
let clip = WAV_MIXER.clip.lock().unwrap();
let Some(clip) = clip.as_ref() else { return };
let frame = (frac.clamp(0.0, 1.0) * clip.frames.len() as f64) as u64;
WAV_MIXER
.cursor_fp
.store((frame.min(clip.frames.len() as u64)) << 32, Ordering::Release);
}
/// Long-form threshold for the audition policy below: at/under this a voice
@ -672,170 +484,95 @@ pub fn format_time(secs: f64) -> String {
format!("{minutes}:{:04.1}", secs - minutes as f64 * 60.0)
}
impl AudioEngine {
fn retire(&self, retired: RetiredAudio) {
let _ = self.retired.send(retired);
/// One additive source in the app's single `cx.audio_output` callback.
/// The callback never blocks on a UI load/seek: a contended quantum is silent.
pub fn mix_into(output: &mut AudioBuffer, device_rate: f64) {
if !WAV_MIXER.playing.load(Ordering::Acquire) || device_rate <= 0.0 {
return;
}
fn drain_commands(&mut self) {
while let Ok(command) = self.commands.try_recv() {
let serial = match command {
AudioCommand::InstallClip { serial, clip } => {
if let Some(old) = self.clip.replace(clip) {
self.retire(RetiredAudio::Clip(old));
}
self.cursor_fp = 0;
self.playing = false;
serial
}
AudioCommand::ClearClip { serial } => {
if let Some(old) = self.clip.take() {
self.retire(RetiredAudio::Clip(old));
}
self.cursor_fp = 0;
self.playing = false;
serial
}
AudioCommand::InstallStems { serial, lanes } => {
if let Some(old) = self.stems.replace(lanes) {
self.retire(RetiredAudio::Stems(old));
}
self.muted = [false; STEM_LANES];
serial
}
AudioCommand::ClearStems { serial } => {
if let Some(old) = self.stems.take() {
self.retire(RetiredAudio::Stems(old));
}
self.muted = [false; STEM_LANES];
serial
}
AudioCommand::Play { serial } => {
if let Some(clip) = self.clip.as_ref() {
let end = (clip.frames.len() as u64) << 32;
if self.cursor_fp >= end {
self.cursor_fp = 0;
}
self.playing = true;
}
serial
}
AudioCommand::Pause { serial } => {
self.playing = false;
serial
}
AudioCommand::Stop { serial } => {
self.playing = false;
self.cursor_fp = 0;
serial
}
AudioCommand::Seek { serial, cursor_fp } => {
self.cursor_fp = self.clip.as_ref().map_or(0, |clip| {
cursor_fp.min((clip.frames.len() as u64) << 32)
});
serial
}
AudioCommand::MuteLane {
serial,
lane,
muted,
} => {
if let Some(slot) = self.muted.get_mut(lane) {
*slot = muted;
}
serial
}
};
self.ack = serial;
}
let Ok(clip) = WAV_MIXER.clip.try_lock() else { return };
let Some(clip) = clip.as_ref() else {
WAV_MIXER.playing.store(false, Ordering::Release);
return;
};
let end = (clip.frames.len() as u64) << 32;
let mut cursor = WAV_MIXER.cursor_fp.load(Ordering::Acquire);
if cursor >= end {
WAV_MIXER.playing.store(false, Ordering::Release);
return;
}
fn publish(&self) {
self.snapshot.cursor_fp.store(self.cursor_fp, Ordering::Relaxed);
self.snapshot.playing.store(self.playing, Ordering::Relaxed);
self.snapshot.ack.store(self.ack, Ordering::Release);
let step = ((clip.sample_rate as f64 / device_rate) * FP_ONE as f64) as u64;
if step == 0 {
return;
}
const GAIN: f32 = 0.9;
/// Add this transport to the app's output. Commands are drained first;
/// no application lock or wait occurs on the realtime callback.
pub fn mix_into(&mut self, output: &mut AudioBuffer, device_rate: f64) {
self.drain_commands();
if !self.playing || device_rate <= 0.0 {
self.publish();
return;
}
let Some(clip) = self.clip.as_ref() else {
self.playing = false;
self.publish();
return;
};
let end = (clip.frames.len() as u64) << 32;
if self.cursor_fp >= end {
self.playing = false;
self.publish();
return;
}
let step = ((clip.sample_rate as f64 / device_rate) * FP_ONE as f64) as u64;
if step == 0 {
self.publish();
return;
}
const GAIN: f32 = 0.9;
if let Some(lanes) = self.stems.as_ref() {
let stem_rate = lanes[0].sample_rate.max(1) as f64;
let stem_step = ((stem_rate / device_rate) * FP_ONE as f64) as u64;
let secs = (self.cursor_fp as f64 / FP_ONE as f64) / clip.sample_rate as f64;
let mut stem_cursor = (secs * stem_rate * FP_ONE as f64) as u64;
for frame in 0..output.frame_count() {
if self.cursor_fp >= end {
self.playing = false;
self.cursor_fp = end;
break;
}
let (mut l, mut r) = (0.0f32, 0.0f32);
for (index, lane) in lanes.iter().enumerate() {
if self.muted[index] {
continue;
}
let (ll, rr) = sample_lane(lane, stem_cursor);
l += ll;
r += rr;
}
l *= GAIN;
r *= GAIN;
for channel in 0..output.channel_count() {
output.channel_mut(channel)[frame] += if channel == 0 { l } else { r };
}
self.cursor_fp = self.cursor_fp.saturating_add(step);
stem_cursor = stem_cursor.saturating_add(stem_step);
}
self.cursor_fp = self.cursor_fp.min(end);
self.publish();
return;
}
// Separated layers REPLACE the mixed track while they are installed.
// A contended lane lock is silence for this quantum, never a blip of
// the original — the same rule the clip lock above follows.
if STEM_MIXER.active.load(Ordering::Acquire) {
let Ok(guard) = STEM_MIXER.lanes.try_lock() else { return };
let Some(lanes) = guard.as_ref() else { return };
// ONE cursor for all four lanes — they cannot drift from each other
// — re-derived from the track cursor at every quantum, so they
// cannot drift from the timeline the waveform and the transport
// draw either. The layers are at the model's rate; the clip may not
// be, hence the second step.
let stem_rate = lanes[0].sample_rate.max(1) as f64;
let stem_step = ((stem_rate / device_rate) * FP_ONE as f64) as u64;
let secs = (cursor as f64 / FP_ONE as f64) / clip.sample_rate as f64;
let mut stem_cursor = (secs * stem_rate * FP_ONE as f64) as u64;
// The mute flags are read ONCE per quantum, not per sample: a
// toggle lands on the next buffer, which is inaudible, and the
// callback stays free of per-sample atomics.
let audible: [bool; STEM_LANES] =
std::array::from_fn(|lane| !STEM_MIXER.mute[lane].load(Ordering::Relaxed));
for frame in 0..output.frame_count() {
if self.cursor_fp >= end {
self.playing = false;
self.cursor_fp = end;
if cursor >= end {
WAV_MIXER.playing.store(false, Ordering::Release);
cursor = end;
break;
}
let index = (self.cursor_fp >> 32) as usize;
let fraction = (self.cursor_fp & (FP_ONE - 1)) as f32 / FP_ONE as f32;
let next = (index + 1).min(clip.frames.len() - 1);
let (al, ar) = clip.frames[index];
let (bl, br) = clip.frames[next];
let l = (al + (bl - al) * fraction) * GAIN;
let r = (ar + (br - ar) * fraction) * GAIN;
let (mut l, mut r) = (0.0f32, 0.0f32);
for (lane, on) in lanes.iter().zip(audible.iter()) {
if !on {
continue;
}
let (ll, rr) = sample_lane(lane, stem_cursor);
l += ll;
r += rr;
}
l *= GAIN;
r *= GAIN;
for channel in 0..output.channel_count() {
output.channel_mut(channel)[frame] += if channel == 0 { l } else { r };
}
self.cursor_fp = self.cursor_fp.saturating_add(step);
cursor = cursor.saturating_add(step);
stem_cursor = stem_cursor.saturating_add(stem_step);
}
self.cursor_fp = self.cursor_fp.min(end);
self.publish();
WAV_MIXER.cursor_fp.store(cursor.min(end), Ordering::Release);
return;
}
for frame in 0..output.frame_count() {
if cursor >= end {
WAV_MIXER.playing.store(false, Ordering::Release);
cursor = end;
break;
}
let index = (cursor >> 32) as usize;
let fraction = (cursor & (FP_ONE - 1)) as f32 / FP_ONE as f32;
let next = (index + 1).min(clip.frames.len() - 1);
let (al, ar) = clip.frames[index];
let (bl, br) = clip.frames[next];
let l = (al + (bl - al) * fraction) * GAIN;
let r = (ar + (br - ar) * fraction) * GAIN;
for channel in 0..output.channel_count() {
output.channel_mut(channel)[frame] += if channel == 0 { l } else { r };
}
cursor = cursor.saturating_add(step);
}
WAV_MIXER.cursor_fp.store(cursor.min(end), Ordering::Release);
}
// ---------------------------------------------------------------------------
@ -923,10 +660,7 @@ pub fn waveform_bgra(pcm: &WavPcm, width: usize, height: usize) -> Vec<u32> {
mod tests {
use super::*;
fn reset_transport() -> AudioEngine {
AUDIO_MIXER.with(|slot| *slot.borrow_mut() = AudioMixer::new());
take_engine()
}
static TRANSPORT_TEST_LOCK: Mutex<()> = Mutex::new(());
fn transport_pcm() -> WavPcm {
WavPcm {
@ -981,7 +715,7 @@ mod tests {
#[test]
fn transport_is_device_clocked_pauseable_seekable_and_restarts_at_end() {
let mut engine = reset_transport();
let _serial = TRANSPORT_TEST_LOCK.lock().unwrap();
clear();
assert!(load(transport_pcm()));
assert!(is_ready());
@ -994,7 +728,7 @@ mod tests {
assert_eq!(playhead_secs(), 0.0);
assert_eq!(playhead_fraction(), 0.0);
let mut output = AudioBuffer::new_with_size(2, 2);
engine.mix_into(&mut output, 10.0);
mix_into(&mut output, 10.0);
assert!((playhead_secs() - 0.2).abs() < 1e-9);
// The drawn playhead tracks the same device-clocked cursor.
assert!((playhead_fraction() - 0.5).abs() < 1e-9);
@ -1003,7 +737,7 @@ mod tests {
pause();
let paused_at = playhead_secs();
let mut silent = AudioBuffer::new_with_size(2, 2);
engine.mix_into(&mut silent, 10.0);
mix_into(&mut silent, 10.0);
assert_eq!(playhead_secs(), paused_at);
assert!(silent.channel(0).iter().all(|sample| *sample == 0.0));
@ -1018,7 +752,7 @@ mod tests {
assert_eq!(playhead_secs(), 0.0);
let mut to_end = AudioBuffer::new_with_size(8, 2);
engine.mix_into(&mut to_end, 10.0);
mix_into(&mut to_end, 10.0);
assert!(at_end());
assert!(!is_playing());
clear();
@ -1034,7 +768,7 @@ mod tests {
#[test]
fn layers_replace_the_mixed_track_and_mute_one_at_a_time() {
let mut engine = reset_transport();
let _serial = TRANSPORT_TEST_LOCK.lock().unwrap();
clear();
assert!(!stems_ready(), "no clip, no layers");
assert!(load(transport_pcm()));
@ -1054,7 +788,7 @@ mod tests {
play();
let mut all = AudioBuffer::new_with_size(1, 2);
engine.mix_into(&mut all, 10.0);
mix_into(&mut all, 10.0);
assert!(
(all.channel(0)[0] - expect(1_000 + 2_000 + 4_000 + 8_000)).abs() < 1e-4,
"all four layers sum: {}",
@ -1069,7 +803,7 @@ mod tests {
set_lane_muted(2, true);
assert!(lane_muted(2) && !lane_muted(0));
let mut without_vocals = AudioBuffer::new_with_size(1, 2);
engine.mix_into(&mut without_vocals, 10.0);
mix_into(&mut without_vocals, 10.0);
assert!(
(without_vocals.channel(0)[0] - expect(1_000 + 2_000 + 8_000)).abs() < 1e-4,
"vocals muted: {}",
@ -1082,7 +816,7 @@ mod tests {
set_lane_muted(index, true);
}
let mut silent = AudioBuffer::new_with_size(1, 2);
engine.mix_into(&mut silent, 10.0);
mix_into(&mut silent, 10.0);
assert!(silent.channel(0)[0].abs() < 1e-6, "{}", silent.channel(0)[0]);
// Clearing the layers hands playback back to the mixed track.
@ -1090,7 +824,7 @@ mod tests {
assert!(!stems_ready());
seek_fraction(0.5);
let mut mixed = AudioBuffer::new_with_size(1, 2);
engine.mix_into(&mut mixed, 10.0);
mix_into(&mut mixed, 10.0);
assert!(
(mixed.channel(0)[0] - 0.8 * GAIN).abs() < 1e-4,
"the clip's own third frame: {}",
@ -1120,7 +854,7 @@ mod tests {
#[test]
fn an_empty_layer_is_refused_rather_than_played_as_a_hole() {
let _engine = reset_transport();
let _serial = TRANSPORT_TEST_LOCK.lock().unwrap();
clear();
assert!(load(transport_pcm()));
assert!(!set_stems(
@ -1175,7 +909,7 @@ mod tests {
#[test]
fn empty_clip_is_unavailable() {
let _engine = reset_transport();
let _serial = TRANSPORT_TEST_LOCK.lock().unwrap();
clear();
assert!(!load(WavPcm {
frames: Vec::new(),

View file

@ -30,7 +30,6 @@ use makepad_chat_ui::{ChatFeed, ClientTools, FeedConfig};
use makepad_asset_client::dto::ChatToolOutcomeDto;
use makepad_asset_client::json::{self, Value};
use makepad_asset_client::{ApiEndpoints, ChatAttachment};
use makepad_widgets::Cx;
use std::path::PathBuf;
use std::sync::mpsc::{self, Receiver, Sender};
use std::sync::{Arc, Mutex};
@ -494,13 +493,7 @@ impl ChatBridge {
/// The store session is up: open the chat on its broker. The session
/// itself is created lazily, on the first turn.
pub fn connect(
&mut self,
cx: &Cx,
endpoints: ApiEndpoints,
token: Option<String>,
cache: PathBuf,
) {
pub fn connect(&mut self, endpoints: ApiEndpoints, token: Option<String>, cache: PathBuf) {
let tools = AppTools {
defaults: self.defaults.clone(),
fleet: self.fleet.clone(),
@ -510,7 +503,6 @@ impl ChatBridge {
self.feed = Some(ChatFeed::start(
FeedConfig::new(endpoints, token, cache, "gen", "gen"),
Box::new(tools),
cx.thread_spawner(),
));
}
@ -947,13 +939,12 @@ mod tests {
fn snap(url: &str, domain: &str, id: &str, state: &str) -> BoxSnapshot {
BoxSnapshot {
base_url: url.into(),
health: Some(HealthJson { realtime: None, activity: None,
health: Some(HealthJson { realtime: None,
service: "makepad-asset-ai".into(),
version: "t".into(),
gpu: Some("RTX".into()),
vram_free_mb: Some(20000),
vram_total_mb: Some(24576),
vram_usable_mb: None,
models_loaded: vec![id.into()],
jobs_pending: Some(0),
node_id: Some(1),
@ -962,7 +953,6 @@ mod tests {
capabilities: Some(vec![domain.into()]),
vram_reserve_mb: Some(1024),
queue_limit: Some(4),
max_job_body_bytes: None,
fleet: None,
lanes: None,
}),

View file

@ -18,13 +18,13 @@ use makepad_asset_client::{
AnnotationUpload, ApiEndpoints, AssetClient, ClientConfig,
};
use makepad_asset_data::{sha256, AssetKind, BlobId};
use makepad_widgets::makepad_platform::thread::{Lane, TaskPool};
use makepad_widgets::{log, vec3f};
use std::collections::BTreeSet;
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{self, Receiver, TryRecvError};
use std::sync::Arc;
use std::thread;
/// Import-card preview strip size.
/// How many imported items the LOAD grid remembers a picture for. A grid,
@ -835,7 +835,6 @@ pub struct ImportPage {
/// status prefix — without it the bar restarted per pack, which over a
/// 38-pack run read as noise.
all_run: Option<(usize, usize)>,
pool: Option<TaskPool>,
}
impl Default for ImportPage {
@ -855,16 +854,11 @@ impl Default for ImportPage {
rx: None,
icon_resume: IconResumeGate::default(),
all_run: None,
pool: None,
}
}
}
impl ImportPage {
pub fn set_task_pool(&mut self, pool: TaskPool) {
self.pool = Some(pool);
}
/// Selected full Kenney kit (name, official page).
pub fn selected_pack_id(&self) -> (String, String) {
let packs = on_disk_kenney_packs();
@ -1538,10 +1532,9 @@ impl ImportPage {
self.icon_resume = icon_resume;
self.kenney_phase = ImportPhase::compiling(pack_name.clone());
let cancel = self.cancel.clone();
self.pool
.as_ref()
.ok_or("runtime task pool is not configured")?
.submit(Lane::Heavy, move || {
thread::Builder::new()
.name("asset-ui-kenney-import".into())
.spawn(move || {
let phase = run_kenney_import(
&dir,
&out,
@ -1555,8 +1548,7 @@ impl ImportPage {
);
let _ = tx.send(phase);
})
.map(|handle| handle.detach())
.map_err(|e| format!("failed to submit compile job: {e}"))?;
.map_err(|e| format!("failed to start compile thread: {e}"))?;
Ok(())
}
@ -1588,15 +1580,13 @@ impl ImportPage {
self.icon_resume = icon_resume;
self.kenney_phase = ImportPhase::compiling("kaykit");
let cancel = self.cancel.clone();
self.pool
.as_ref()
.ok_or("runtime task pool is not configured")?
.submit(Lane::Heavy, move || {
thread::Builder::new()
.name("asset-ui-kaykit-import".into())
.spawn(move || {
let phase = run_kaykit_import(&dir, &out, spec, server, &tx, &cancel, &icon_resume_rx);
let _ = tx.send(phase);
})
.map(|handle| handle.detach())
.map_err(|e| format!("failed to submit KayKit import job: {e}"))?;
.map_err(|e| format!("failed to start KayKit import thread: {e}"))?;
Ok(())
}
@ -1629,10 +1619,9 @@ impl ImportPage {
self.kenney_phase = ImportPhase::compiling("all");
self.all_run = Some((0, present.len()));
let cancel = self.cancel.clone();
self.pool
.as_ref()
.ok_or("runtime task pool is not configured")?
.submit(Lane::Heavy, move || {
thread::Builder::new()
.name("asset-ui-kenney-import-all".into())
.spawn(move || {
let total = present.len();
let mut ok = Vec::new();
let mut failed = Vec::new();
@ -1760,8 +1749,7 @@ impl ImportPage {
skipped,
});
})
.map(|handle| handle.detach())
.map_err(|e| format!("failed to submit import-all job: {e}"))?;
.map_err(|e| format!("failed to start import-all thread: {e}"))?;
Ok(())
}
@ -4015,12 +4003,6 @@ fn publish_compiled_pack(
categories: vec!["kenney".into(), pack_name.to_string()],
tags,
creator: KENNEY_CREDITS.to_string(),
artist: String::new(),
artist_url: String::new(),
album: String::new(),
source_url: String::new(),
license: String::new(),
license_url: String::new(),
generator: "pack_import".into(),
backend: "asset-ui".into(),
model: pack_name.to_string(),

View file

@ -30,6 +30,7 @@ use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{self, Receiver, TryRecvError};
use std::sync::Arc;
use std::thread;
use crate::import::{KENNEY_MODULE, KAYKIT_MODULE, NASA_SKY_MODULE, PackModule};
use makepad_asset_importer::tdm_zipsync::{
@ -305,7 +306,6 @@ pub struct ClassicImportCard {
/// a classic pack publishes real thumbnails and its staging can be
/// reclaimed the moment publish succeeds. See [`IconResumeGate`].
icon_resume: IconResumeGate,
pool: Option<TaskPool>,
}
struct IsoSync {
@ -358,7 +358,6 @@ impl ClassicImportCard {
tdm: None,
iso: None,
icon_resume: IconResumeGate::default(),
pool: None,
}
}
@ -659,7 +658,6 @@ impl ClassicImportCard {
path_override: String,
server: Option<ServerSession>,
) -> Result<(), String> {
self.pool = Some(cx.task_pool());
if self.compiling() {
return Err(format!(
"a {} import is already running",
@ -1542,10 +1540,9 @@ impl ClassicImportCard {
// until the UI has taken every landing for icon rendering.
let (gate, icon_resume_rx) = IconResumeGate::armed();
self.icon_resume = gate;
self.pool
.as_ref()
.ok_or("runtime task pool is not configured")?
.submit(Lane::Heavy, move || {
thread::Builder::new()
.name(format!("asset-ui-{}-import", source.id()))
.spawn(move || {
let phase = run_classic_import(
&dir,
&out,
@ -1558,8 +1555,7 @@ impl ClassicImportCard {
);
let _ = tx.send(phase);
})
.map(|handle| handle.detach())
.map_err(|e| format!("failed to submit classic import job: {e}"))?;
.map_err(|e| format!("failed to start classic import thread: {e}"))?;
Ok(())
}
@ -2271,12 +2267,6 @@ fn publish_classic_pack(
categories: vec![source.id().into(), pack_name.to_string()],
tags,
creator: source.credits().to_string(),
artist: String::new(),
artist_url: String::new(),
album: String::new(),
source_url: String::new(),
license: String::new(),
license_url: String::new(),
generator: "classic_import".into(),
backend: "asset-ui".into(),
model: pack_name.to_string(),

View file

@ -55,9 +55,7 @@ mod import_classic;
mod store_content;
mod library;
mod mask_paint;
mod mesh_view {
pub use makepad_media_view::mesh_view::*;
}
mod mesh_view;
mod music_page;
use crate::mask_paint::{MaskPaint, MaskPaintAction};
mod pipeline;
@ -65,9 +63,7 @@ mod runs_chip;
mod scheduler;
mod store_views;
mod thumbnail_renderer;
mod video_player {
pub use makepad_media_view::{FileVideoPlayer as VideoPlayer, VideoDecoder};
}
mod video_player;
mod webcam;
use crate::artifact_io::{
@ -248,7 +244,7 @@ use crate::store_views::{
StoreListPanel, StoreRow,
TileDelete,
};
use crate::video_player::{VideoDecoder, VideoPlayer};
use crate::video_player::VideoPlayer;
use makepad_micro_serde::SerJson;
use makepad_widgets::*;
@ -4014,7 +4010,7 @@ pub struct App {
fleet_timer: Timer,
/// LAN beacon listener; polled on the fleet timer.
#[rust]
discovered: Option<makepad_ai_hub::discovery::Discovery>,
discovered: Option<makepad_ai_hub::discovery::Discovered>,
#[rust]
job_timer: Timer,
#[rust]
@ -4040,8 +4036,6 @@ pub struct App {
library: Option<Library>,
#[rust]
video: Option<VideoPlayer>,
#[rust]
video_decoder: Option<VideoDecoder>,
/// The file the viewer's current video came from — Restart and the loop
/// toggle re-open it.
video_path: Option<PathBuf>,
@ -4329,16 +4323,8 @@ impl App {
// -- setup ---------------------------------------------------------------
fn setup(&mut self, cx: &mut Cx) {
let pool = cx.task_pool();
let spawner = cx.thread_spawner();
let (video_decoder, mut video_audio) =
VideoDecoder::start(spawner.clone()).expect("asset-ui video decoder worker");
self.video_decoder = Some(video_decoder);
let _ = std::fs::create_dir_all(artifacts_dir());
self.artifact_io = Some(ArtifactIo::start(spawner.clone()));
self.analysis = Some(AnalysisQueue::start(spawner.clone()));
self.import_page.set_task_pool(pool.clone());
self.music_import_page.set_task_pool(pool.clone());
self.artifact_io = Some(ArtifactIo::start());
self.load_fleet_prefs();
self.library = Some(Library::open(repo_path("local/ai_content_library")));
self.saved_presets = fast_presets::load(&fast_presets::store_path());
@ -4363,11 +4349,7 @@ impl App {
// The store hosts the embedded Asset Server; hand it the library it
// must publish. Library::open ran above, so the product backfill is
// already on disk when the watcher's first poll reads index.json.
self.store.start(
PathBuf::from(repo_path("local/ai_content_library")),
pool,
spawner,
);
self.store.start(PathBuf::from(repo_path("local/ai_content_library")));
self.asset_store_timer = cx.start_interval(0.2);
// Opening stays metadata-only; every missing preview is queued here
// and regenerated a bounded slice at a time once frames are flowing.
@ -4566,12 +4548,11 @@ impl App {
self.refresh_voice_ui(cx);
self.sync_preset_name_box(cx);
// Speakers: both engines move into the callback and own their state.
let mut audio_engine = crate::audio::take_engine();
// Speakers: wav artifacts + video soundtrack.
cx.audio_output(0, move |info, output| {
output.zero();
audio_engine.mix_into(output, info.sample_rate);
video_audio.mix_into(output, info.sample_rate);
crate::audio::mix_into(output, info.sample_rate);
crate::video_player::mix_into(output, info.sample_rate);
});
// Headless drive.
@ -7560,7 +7541,7 @@ impl App {
// WAV must not call play() or a 200ms DS_* / Quake
// shot becomes a loop (play-at-end restarts).
if audition && audio::autoplay_one_shot(domain, pcm.seconds()) {
self.stop_video_audio();
crate::video_player::stop_audio();
audio::play();
self.arm_audio_pump(cx);
}
@ -7608,8 +7589,7 @@ impl App {
// behind a new open or an error state.
self.stop_video_playback();
self.clear_video_frame(cx);
let decoder = self.video_decoder.as_ref().expect("video decoder started");
match VideoPlayer::new(&path.to_string_lossy(), decoder) {
match VideoPlayer::new(&path.to_string_lossy()) {
Ok(player) => {
self.ui.label(cx, ids!(video_info)).set_text(
cx,
@ -9121,7 +9101,7 @@ impl App {
&& audio::is_ready()
&& !audio::is_playing()
{
self.stop_video_audio();
crate::video_player::stop_audio();
audio::play();
self.arm_audio_pump(cx);
self.sync_audio_ui(cx);
@ -9983,7 +9963,7 @@ impl App {
if !self.chat.is_linked() {
if let Some(endpoints) = self.store.endpoints {
let cache = session_config_from_env().cache_parent.join("cache-chat");
self.chat.connect(cx, endpoints, self.store.token.clone(), cache);
self.chat.connect(endpoints, self.store.token.clone(), cache);
// The pane says "waiting for the asset server" until
// something redraws it, and the feed only marks itself
// dirty once a turn runs — so the line would sit there
@ -10734,8 +10714,7 @@ impl App {
if let Some(path) = item.as_ref().and_then(|item| item.payload.clone()) {
self.stop_video_playback();
self.clear_video_frame(cx);
let decoder = self.video_decoder.as_ref().expect("video decoder started");
match VideoPlayer::new(&path.to_string_lossy(), decoder) {
match VideoPlayer::new(&path.to_string_lossy()) {
Ok(player) => {
self.library_video_file = Some(file.clone());
self.video = Some(player);
@ -10810,8 +10789,7 @@ impl App {
self.library_audio_file = Some(file.clone());
// The transport: decoded off the frame thread and
// installed when it lands.
let pool = cx.task_pool();
let clip_gen = crate::audio::load_clip_async(&pool, bytes.clone());
let clip_gen = crate::audio::load_clip_async(bytes.clone());
// And, exactly once per track, what the store
// already holds BESIDE the mixed audio: the four
// separated layers and the transcript. The clip
@ -10952,9 +10930,11 @@ impl App {
// -- "Split audio layers": the bake queue and its consumers -----------
/// The bake + fetch lanes, started once with the app and fed by channels.
/// The bake + fetch lanes, started on first use. Two threads parked on
/// a channel is the whole cost of having them.
fn analysis(&mut self) -> &mut analysis::AnalysisQueue {
self.analysis.as_mut().expect("analysis workers started")
self.analysis
.get_or_insert_with(analysis::AnalysisQueue::start)
}
/// The selected catalog hit when it is an AUDIO asset: id and title.
@ -12420,13 +12400,7 @@ impl App {
/// [`Self::clear_video_frame`] is also called.
fn stop_video_playback(&mut self) {
self.video = None;
self.stop_video_audio();
}
fn stop_video_audio(&self) {
if let Some(decoder) = &self.video_decoder {
decoder.stop_audio();
}
crate::video_player::stop_audio();
}
/// Blank the actual video WIDGET texture (not only the app-side handle),
@ -12818,8 +12792,7 @@ impl App {
fn restart_viewer_video(&mut self, cx: &mut Cx) -> bool {
let Some(path) = self.video_path.clone() else { return false };
self.stop_video_playback();
let decoder = self.video_decoder.as_ref().expect("video decoder started");
match VideoPlayer::new(&path.to_string_lossy(), decoder) {
match VideoPlayer::new(&path.to_string_lossy()) {
Ok(player) => {
self.video = Some(player);
self.sync_video_transport(cx);
@ -14160,7 +14133,7 @@ impl MatchEvent for App {
} else {
// A user-resumed WAV preview wins over a stale video
// soundtrack in the shared device callback.
self.stop_video_audio();
crate::video_player::stop_audio();
audio::play();
self.arm_audio_pump(cx);
}
@ -14461,11 +14434,13 @@ impl AppMain for App {
fn script_mod(vm: &mut ScriptVm) -> ScriptValue {
crate::makepad_widgets::script_mod(vm);
// Draw shaders must register before the widgets that declare them.
makepad_media_view::script_mod(vm);
makepad_render::script_mod(vm);
makepad_xr::script_mod(vm);
// Shared preview widgets (ContentPreview / AudioView): the pool this
// app draws catalog content with, and the same one the VJ and DJ
// surfaces adopt.
makepad_asset_widgets::script_mod(vm);
crate::mesh_view::script_mod(vm);
crate::mask_paint::script_mod(vm);
crate::billboard_view::script_mod(vm);
crate::thumbnail_renderer::script_mod(vm);
@ -14716,17 +14691,14 @@ impl AppMain for App {
}
}
self.scrub_audio(cx, event);
if self.audio_timer.is_event(event).is_some() {
audio::pump();
if audio::is_ready() {
self.sync_audio_ui(cx);
// The Library rail has its own transport over the same mixer.
if self.surface == Surface::Library && self.library_audio_file.is_some() {
self.refresh_library_audio(cx);
// Playback that started from the transport re-arms the
// transcript's own per-frame follow.
self.arm_lyrics_pump(cx);
}
if self.audio_timer.is_event(event).is_some() && audio::is_ready() {
self.sync_audio_ui(cx);
// The Library rail has its own transport over the same mixer.
if self.surface == Surface::Library && self.library_audio_file.is_some() {
self.refresh_library_audio(cx);
// Playback that started from the transport re-arms the
// transcript's own per-frame follow.
self.arm_lyrics_pump(cx);
}
}
if self.audio_timer.is_event(event).is_some() && self.webcam.capturing {
@ -15946,13 +15918,6 @@ mod world_style_tests {
namespace: namespace.to_string(),
kind: Some(makepad_asset_data::AssetKind::World),
title: title.to_string(),
creator: String::new(),
artist: String::new(),
artist_url: String::new(),
album: String::new(),
source_url: String::new(),
license: String::new(),
license_url: String::new(),
snippet: String::new(),
score: 0,
live: true,

View file

@ -30,7 +30,6 @@
//! PNG captures at fixed ticks, exit once all are on disk (the rig
//! example's RIG_CAPTURE_DIR pattern).
use crate::{is_glb, MediaFit, MediaKind, MediaViewAction};
use makepad_render::play::{LocoState, Locomotion, PlayInput};
use makepad_render::skin::{PoseBuffer, SkinnedModel, SKIN_VERTEX_FLOATS};
use makepad_render::{
@ -44,7 +43,7 @@ use makepad_widgets::*;
// The static-PBR branch lives in its own file; declared from here (not
// main.rs, which another lane owns) — `#[path]` resolves the sibling in src/.
#[path = "pbr_preview.rs"]
pub mod pbr_preview;
pub(crate) mod pbr_preview;
use pbr_preview::{PbrDisplayControls, PbrPreview, PbrStatus};
script_mod! {
@ -615,9 +614,6 @@ pub struct MeshView {
draw_hud: DrawText,
#[live(vec4(0.03, 0.045, 0.075, 1.0))]
clear_color: Vec4f,
/// Hosts stack this pane beside other media panes and show one at a time.
#[live(true)]
visible: bool,
#[new]
pass: DrawPass,
#[new]
@ -647,8 +643,6 @@ pub struct MeshView {
/// Dark backdrop: near-black ground + sky; the model stays fully lit.
#[rust(false)]
dark_enabled: bool,
#[rust(true)]
show_hud: bool,
/// Studio light for the PBR lane: softbox environment (bright boxes
/// for metals and gloss to reflect) + a strong warm key. Off = the
/// procedural sky environment and the neutral rig.
@ -793,14 +787,14 @@ fn is_playable_skin_shape(joints: usize, clips: usize) -> bool {
joints > 0 && joints <= 256 && clips > 0
}
pub fn is_playable_skin(model: &SkinnedModel) -> bool {
pub(crate) fn is_playable_skin(model: &SkinnedModel) -> bool {
is_playable_skin_shape(model.joint_count(), model.clips.len())
}
/// Base-color image bytes out of a GLB, if it embeds one: material 0's
/// baseColorTexture source, else image 0 (skin.rs ignores materials by
/// design — the host binds the texture itself).
pub fn extract_base_color(glb: &[u8]) -> Option<Vec<u8>> {
pub(crate) fn extract_base_color(glb: &[u8]) -> Option<Vec<u8>> {
let loaded = makepad_gltf::load_gltf_from_bytes(glb, None).ok()?;
let doc = &loaded.document;
let image_index = doc
@ -823,7 +817,7 @@ fn load_sprite_frames(cx: &mut Cx, path: &std::path::Path) -> Vec<Texture> {
let ext = path.extension().and_then(|e| e.to_str()).unwrap_or("");
if ext.eq_ignore_ascii_case("billboard") {
let text = std::fs::read_to_string(path).unwrap_or_default();
if let Ok(bb) = makepad_asset_data::stateful_billboard::StatefulBillboard::parse(&text)
if let Ok(bb) = makepad_asset_importer::stateful_billboard::StatefulBillboard::parse(&text)
{
let mut out = Vec::new();
for frame in bb.preview_frames() {
@ -848,7 +842,7 @@ fn load_sprite_frames(cx: &mut Cx, path: &std::path::Path) -> Vec<Texture> {
/// Decode PNG or JPEG bytes into a texture (Blender/SkinTokens exports carry
/// either), falling back to a 1x1 white so an untextured rig still draws.
pub fn image_texture(cx: &mut Cx, bytes: Option<Vec<u8>>) -> Texture {
pub(crate) fn image_texture(cx: &mut Cx, bytes: Option<Vec<u8>>) -> Texture {
if let Some(bytes) = bytes {
let decoded = if bytes.starts_with(&[0xff, 0xd8]) {
ImageBuffer::from_jpg(&bytes).ok()
@ -914,62 +908,6 @@ const CAPTURES: [(u64, &str); 14] = [
];
impl MeshView {
/// Load a GLB from host-owned bytes. Parsing/upload remains deferred to
/// draw, exactly as in asset-ui's original viewer.
pub fn load_bytes(
&mut self,
cx: &mut Cx,
bytes: &[u8],
content_type: &str,
) -> Result<(), String> {
if !is_glb(bytes) {
let error = format!("{content_type} is not a GLB payload");
cx.widget_action(self.widget_uid(), MediaViewAction::Failed(error.clone()));
return Err(error);
}
self.set_model_bytes(cx, bytes.to_vec(), None);
cx.widget_action(self.widget_uid(), MediaViewAction::Loaded(MediaKind::Mesh));
Ok(())
}
/// Forget every branch of the currently displayed model.
pub fn clear(&mut self, cx: &mut Cx) {
self.pending = None;
self.instance = None;
self.character = None;
self.walk_cam = None;
self.extra_instances.clear();
self.placed_sprites.clear();
self.pending_placed_models.clear();
self.pbr.clear(&mut self.draw_pbr);
self.status = "no mesh yet".into();
self.area.redraw(cx);
}
/// Select the camera framing used for ordinary embedded media surfaces.
pub fn set_fit(&mut self, cx: &mut Cx, fit: MediaFit) {
self.reset_studio_camera();
self.look.distance = match fit {
MediaFit::Contain => 4.2,
MediaFit::Cover => 3.4,
MediaFit::Stretch => 3.8,
};
self.area.redraw(cx);
}
pub fn set_size(&mut self, cx: &mut Cx, width: Size, height: Size) {
self.walk.width = width;
self.walk.height = height;
self.area.redraw(cx);
}
pub fn is_loaded(&self) -> bool {
self.pending.is_some()
|| self.instance.is_some()
|| self.character.is_some()
|| self.pbr.bounds().is_some()
}
/// Queue a GLB (and optional base-color PNG) for display; parsed and
/// uploaded during the next draw. Routing is automatic: playable rig →
/// play mode, static material-bearing GLB → PBR, anything else → statue.
@ -1086,7 +1024,6 @@ impl MeshView {
color_adjust: vec4(0.0, 1.0, 1.0, 0.0),
dynamic: true,
depth_order: 0.0,
custom_material: None,
part_poses: Vec::new(),
});
}
@ -1202,11 +1139,6 @@ impl MeshView {
self.dark_enabled
}
pub fn set_show_hud(&mut self, cx: &mut Cx, show: bool) {
self.show_hud = show;
self.area.redraw(cx);
}
pub fn set_dark_enabled(&mut self, cx: &mut Cx, on: bool) {
if self.dark_enabled == on {
return;
@ -1567,7 +1499,6 @@ impl MeshView {
color_adjust: vec4(0.0, 1.0, 1.0, 0.0),
dynamic: true,
depth_order: 0.0,
custom_material: None,
part_poses: Vec::new(),
});
self.status = format!("{triangles} tris{ao_note} · CSM · WASD walk, drag look");
@ -1589,7 +1520,6 @@ impl MeshView {
// Realtime CSM only collects `dynamic` movers.
dynamic: true,
depth_order: 0.0,
custom_material: None,
part_poses: Vec::new(),
});
self.status =
@ -1709,17 +1639,6 @@ impl WidgetNode for MeshView {
fn redraw(&mut self, cx: &mut Cx) {
self.area.redraw(cx);
}
fn set_visible(&mut self, cx: &mut Cx, visible: bool) {
if self.visible != visible {
self.visible = visible;
self.area.redraw(cx);
}
}
fn visible(&self) -> bool {
self.visible
}
}
impl Widget for MeshView {
@ -1877,9 +1796,6 @@ impl Widget for MeshView {
}
fn draw_walk(&mut self, cx: &mut Cx2d, _scope: &mut Scope, walk: Walk) -> DrawStep {
if !self.visible {
return DrawStep::done();
}
let rect = cx.walk_turtle_with_area(&mut self.area, walk);
if rect.size.x <= 1.0 || rect.size.y <= 1.0 {
return DrawStep::done();
@ -2100,13 +2016,11 @@ impl Widget for MeshView {
None => format!("{} drag orbit, wheel zoom", self.status),
},
};
if self.show_hud {
self.draw_hud.draw_abs(
cx,
dvec2(rect.pos.x + 10.0, rect.pos.y + rect.size.y - 22.0),
&help,
);
}
self.draw_hud.draw_abs(
cx,
dvec2(rect.pos.x + 10.0, rect.pos.y + rect.size.y - 22.0),
&help,
);
DrawStep::done()
}
}

View file

@ -12,11 +12,11 @@
use crate::import::{ImportPhase, ServerSession};
use makepad_asset_client::{AssetClient, ClientConfig};
use makepad_asset_importer::music_import::{self, MusicReport};
use makepad_widgets::makepad_platform::thread::{Lane, TaskPool};
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{self, Receiver, TryRecvError};
use std::sync::Arc;
use std::thread;
/// The card's whole state: which folder, what the worker is doing, and the
/// one-line result of the last run.
@ -44,7 +44,6 @@ pub struct MusicImportPage {
/// Aliases the finished run landed, waiting to be handed to the
/// analysis queue. Non-empty only when `split_layers` was on.
pending_analysis: Vec<String>,
pool: Option<TaskPool>,
}
/// What the worker sends back: live progress, then exactly one verdict.
@ -66,7 +65,6 @@ impl Default for MusicImportPage {
split_layers: false,
bake_lyrics: false,
pending_analysis: Vec::new(),
pool: None,
}
}
}
@ -76,10 +74,6 @@ impl Default for MusicImportPage {
pub const MUSIC_NAMESPACE: &str = "music";
impl MusicImportPage {
pub fn set_task_pool(&mut self, pool: TaskPool) {
self.pool = Some(pool);
}
/// The job this card would enqueue right now, or `None` while no folder
/// has been picked.
pub fn job(&self) -> Option<crate::import::ImportJob> {
@ -215,18 +209,13 @@ impl MusicImportPage {
total: 0,
current: String::new(),
};
let Some(pool) = self.pool.clone() else {
return Err(self.refuse("runtime task pool is not configured".into()));
};
match pool.submit(Lane::Heavy, move || {
thread::Builder::new()
.name("asset-ui-music-import".into())
.spawn(move || {
let msg = run_music_import(&dir, server, &tx, &cancel);
let _ = tx.send(msg);
}) {
Ok(handle) => handle.detach(),
Err(error) => {
return Err(self.refuse(format!("failed to submit music import job: {error}")));
}
}
})
.map_err(|e| self.refuse(format!("failed to start music import thread: {e}")))?;
Ok(())
}

View file

@ -20,13 +20,10 @@
//! behavior (geometric normal, factor-only metallic/roughness, occlusion 1)
//! instead of guessing.
//!
//! This file is a child module of `mesh_view.rs` in `makepad-media-view`.
//! This file is a child module of mesh_view.rs (declared there via
//! `#[path]`) because main.rs is owned by another lane and must not change.
use makepad_gltf::{decode_mesh_primitive, load_gltf_from_bytes, LoadedGltf};
use makepad_zune_core::bit_depth::BitDepth;
use makepad_zune_core::colorspace::ColorSpace;
use makepad_zune_core::options::EncoderOptions;
use makepad_zune_png::PngEncoder;
use makepad_widgets::*;
use makepad_xr::render::{GltfDrawObject, GltfMaterialState, GltfRenderer};
use makepad_widgets::shader::draw_pbr::{DrawPbrMaterialState, DrawPbrTextureSet, PbrMeshHandle};
@ -775,15 +772,7 @@ pub fn studio_equirect_png() -> Vec<u8> {
rgba[i + 3] = 255;
}
}
let options = EncoderOptions::default()
.set_width(W)
.set_height(H)
.set_depth(BitDepth::Eight)
.set_colorspace(ColorSpace::RGBA);
let mut encoder = PngEncoder::new(&rgba, options);
let mut png = Vec::new();
encoder.encode(&mut png).expect("studio equirect encodes");
png
makepad_ai_hub::testpattern::encode_png_rgba(&rgba, W, H).expect("studio equirect encodes")
}
#[cfg(test)]

View file

@ -218,16 +218,16 @@ impl Preset {
/// Python rigger, or another arbitrary model merely because it is resident.
// This is the warm, local Makepad-Llama model advertised by the Mac fleet
// node. Keeping it as fallback avoids an accidental large-model cold pull.
use makepad_asset_creator::character::CHARACTER_LLM_MODEL;
const CHARACTER_LLM_MODEL: &str = "qwen3.5-9b";
/// Fleet-wide default for text expansion once a node has the audited weights
/// ready. This is a preference, not a hard pin: first-run provisioning stays
/// explicit and the warm 9B lane remains immediately usable.
const PREFERRED_EXPAND_MODEL: &str = "qwen3.8-27b";
use makepad_asset_creator::character::CHARACTER_IMAGE_MODEL;
use makepad_asset_creator::character::CHARACTER_MATTE_MODEL;
use makepad_asset_creator::character::CHARACTER_MESH_MODEL;
use makepad_asset_creator::character::CHARACTER_RIG_MODEL;
use makepad_asset_creator::character::CHARACTER_MOTION_MODEL;
const CHARACTER_IMAGE_MODEL: &str = "flux1-dev";
const CHARACTER_MATTE_MODEL: &str = "birefnet-hr";
const CHARACTER_MESH_MODEL: &str = "trellis-2";
const CHARACTER_RIG_MODEL: &str = "skintokens";
const CHARACTER_MOTION_MODEL: &str = "hy-motion";
/// Instruction image editing (reference image + "change …" prompt).
const EDIT_MODEL: &str = "flux2-klein-4b";
/// Sprite enhancement runs on the 32B dev DiT, NOT the 4-step distilled
@ -257,7 +257,7 @@ const SPLAT_MODEL: &str = "triposplat";
/// A character expansion substantially shorter than the 40-90 words asked
/// for by `expand_rig.txt` is not a usable rig-safe brief. Refuse to quietly
/// continue with it; the user can see and retry the failed LLM stage.
// The shared character contract owns brief validation.
const CHARACTER_EXPANSION_MIN_WORDS: usize = 24;
/// A character reconstruction gets two deterministic second chances when the
/// rig or animated-skin quality gate rejects it. The matte/image are
@ -544,7 +544,7 @@ pub const PRESETS: &[Preset] = &[
// (idle/walk/jump locomotion, see mesh_view play mode).
Preset::linear(
"character (playable)",
makepad_asset_creator::character::CHARACTER_DOMAINS,
&["text", "image", "matte", "mesh", "rig", "motion"],
// Character geometry is downstream of this one image: Schnell's
// four-step distillation is useful for previews, but it is the wrong
// silent affinity fallback for the rig master. Pin the validated
@ -1329,7 +1329,21 @@ impl Pipeline {
return unusable("was empty");
}
if self.is_character_pipeline() {
makepad_asset_creator::character::validate_brief(&self.prompt, text)?;
let words = text.split_whitespace().count();
if words < CHARACTER_EXPANSION_MIN_WORDS {
return Err(format!(
"LLM character brief is too short ({words} words, need at least {CHARACTER_EXPANSION_MIN_WORDS}); refusing to start image generation"
));
}
if !text
.to_lowercase()
.contains(&self.prompt.trim().to_lowercase())
{
return Err(format!(
"LLM character brief dropped identity anchor {:?}; refusing to start image generation",
self.prompt.trim()
));
}
}
return Ok(text.to_string());
}
@ -1487,7 +1501,16 @@ impl Pipeline {
let is_music_target = target == "music";
request.target_domain = Some(target);
if self.is_character_pipeline() {
makepad_asset_creator::character::configure_expansion(&mut request, &self.prompt);
request.identity_anchor = Some(self.prompt.trim().to_string());
// Named-character identity and rig-safe presentation are
// constraints, not a variant hunt. Keep this expansion
// low-temperature and deterministic enough to avoid
// inventing conflicting signature traits.
request.temperature = Some(0.0);
request.style = Some(
"When the intent names an established character, preserve the exact named identity and canonical official design unchanged. Do not redesign, genericize, or guess traits. If a visual trait is uncertain, omit it instead of inventing it; it is better to say 'canonical official design unchanged' and spend the remaining prompt on full-body framing, a relaxed wide A-pose with straight diagonal arms and hands clear above the hips, visible gaps between every limb and the torso, even studio light, a uniform plain background, and a clean separated silhouette. Rigging constraints may change pose and spacing but never delete canonical anatomy or worn pieces."
.to_string(),
);
}
// Music expansion carries a compact structured production
// brief AND original section-tagged lyrics. Scale its budget
@ -3358,7 +3381,7 @@ impl Pipeline {
} => {
let bytes = response
.filter(|response| !failed && response.status_code == 200)
.and_then(|response| response.body.as_deref().map(<[u8]>::to_vec));
.and_then(|response| response.body.clone());
let Some(bytes) = bytes else {
return self.candidate_failed(
cx,
@ -3566,7 +3589,7 @@ impl Pipeline {
Req::Artifact(stage, artifact) => {
let bytes = response
.filter(|r| !failed && r.status_code == 200)
.and_then(|r| r.body.as_deref().map(<[u8]>::to_vec));
.and_then(|r| r.body.clone());
let Some(bytes) = bytes else {
return self.fail_stage_or_skip_expander(
cx,
@ -3927,13 +3950,12 @@ mod tests {
use makepad_ai_hub::protocol::{HealthJson, ModelInfoJson, MODEL_STATE_LOADED};
BoxSnapshot {
base_url: url.to_string(),
health: Some(HealthJson { realtime: None, activity: None,
health: Some(HealthJson { realtime: None,
service: "test".to_string(),
version: "1".to_string(),
gpu: Some("GPU".to_string()),
vram_free_mb: Some(24_000),
vram_total_mb: Some(24_000),
vram_usable_mb: None,
models_loaded: vec!["flux1-schnell".to_string()],
jobs_pending: Some(0),
node_id: Some(1),
@ -3942,7 +3964,6 @@ mod tests {
capabilities: Some(vec!["image".to_string()]),
vram_reserve_mb: Some(0),
queue_limit: Some(8),
max_job_body_bytes: None,
fleet: None,
lanes: None,
}),
@ -3974,13 +3995,12 @@ mod tests {
use makepad_ai_hub::protocol::{HealthJson, ModelInfoJson, MODEL_STATE_LOADED};
BoxSnapshot {
base_url: url.to_string(),
health: Some(HealthJson { realtime: None, activity: None,
health: Some(HealthJson { realtime: None,
service: "test".to_string(),
version: "1".to_string(),
gpu: Some("24 GB GPU".to_string()),
vram_free_mb: Some(24 * 1024),
vram_total_mb: Some(24 * 1024),
vram_usable_mb: None,
models_loaded: models
.iter()
.filter(|(_, state)| *state == MODEL_STATE_LOADED)
@ -3993,7 +4013,6 @@ mod tests {
capabilities: Some(vec!["text".to_string()]),
vram_reserve_mb: Some(0),
queue_limit: Some(8),
max_job_body_bytes: None,
fleet: None,
lanes: None,
}),
@ -4938,10 +4957,7 @@ Arrangement: Pulsing bass, gated drums and widening analog pads."
let mut want: Vec<(String, String)> = [
("audio", "moss-sfx"),
("audio", "sa3-sfx"),
("audio", "salamander-drumkit"),
("audio", "woosh-sfx"),
("beats", "beat-this"),
("body", "sam3dbody"),
("control", "flux1-canny-dev"),
("control", "flux1-depth-dev"),
("depth", "da3-metric-large"),
@ -4959,8 +4975,6 @@ Arrangement: Pulsing bass, gated drums and widening analog pads."
("music", "minimax-music3"),
("music", "minimax-music3-q4"),
("music", "ace-step-1.5-xl"),
("notes", "basic-pitch"),
("ocr", "chandra-ocr-2"),
("paint", "hunyuan3d-paint-2.1"),
("rig", "skintokens"),
("rig", "skintokens-oracle"),
@ -4968,8 +4982,6 @@ Arrangement: Pulsing bass, gated drums and widening analog pads."
("splat", "triposplat"),
("speech", "indextts-2.5"),
("speech", "kokoro"),
("stems", "bs-roformer-4stem"),
("stt", "whisper-large-v3-turbo"),
("text", "qwen3.8-27b"),
("upscale", "realesrgan-x4plus"),
("vision", "qwen3.8-27b-vision"),
@ -5030,7 +5042,7 @@ Arrangement: Pulsing bass, gated drums and widening analog pads."
/// fails visibly as a service gap instead of rerouting.
const DOCUMENTED_OVERRIDE_PINS: &[(&str, &str)] = &[("text", CHARACTER_LLM_MODEL)];
/// Every pin must reference a model the registry can serve for the
/// Every pin must reference a model the registry actually has in the
/// pinned domain — or be a documented cache-registry override above.
/// Catches typos and silent registry drift.
#[test]
@ -5045,13 +5057,7 @@ Arrangement: Pulsing bass, gated drums and widening analog pads."
registry
.models
.iter()
.any(|entry| {
entry.id == *model
&& (entry.domain.as_str() == *domain
|| (*domain == "edit"
&& entry.domain.as_str() == "image"
&& model.starts_with("flux2-dev")))
}),
.any(|entry| entry.id == *model && entry.domain.as_str() == *domain),
"preset {:?} pins unknown model {domain}/{model}",
preset.name
);

View file

@ -2542,13 +2542,6 @@ mod tests {
namespace: "gen".into(),
kind: Some(AssetKind::Prop),
title: title.into(),
creator: String::new(),
artist: String::new(),
artist_url: String::new(),
album: String::new(),
source_url: String::new(),
license: String::new(),
license_url: String::new(),
snippet: "a thing".into(),
score: 10,
live,

View file

@ -823,7 +823,6 @@ impl ThumbnailRenderer {
color_adjust: vec4(0.0, 1.0, 1.0, 0.0),
dynamic: true,
depth_order: 0.0,
custom_material: None,
part_poses: Vec::new(),
}),
frame,

View file

@ -0,0 +1,637 @@
//! Video artifact playback — the sandbox's proven decode pattern
//! (apps/sandbox/src/video_player.rs): a decode thread pulls frames + audio
//! from the platform video-file seam (`makepad_platform::video_file`,
//! hardware codecs), a small ring buffer hands BGRA frames to the render
//! thread paced by pts against a wall clock, and the audio track mixes into
//! this app's `cx.audio_output` closure.
//!
//! Playback is PLAY-ONCE: at end-of-stream the remaining audio drains and
//! the clip stops (the sandbox pattern's loop-forever reopen was what users
//! heard as "the soundtrack never ends"). Loading a new artifact drops the
//! previous player (its `Drop` silences the queue); `stop_audio()` silences
//! immediately and stays muted until the next clip starts.
//!
//! Copied rather than imported: the sandbox is an app crate under active
//! concurrent development, not a library — and this pattern is ~250 lines.
use makepad_widgets::log;
use makepad_widgets::makepad_platform::audio::AudioBuffer;
use makepad_widgets::makepad_platform::video_file::{nv12, VideoFileDecoder};
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
const RING_FRAMES: usize = 3;
const AUDIO_AHEAD_SECS: f64 = 1.0;
struct Frame {
pts_100ns: i64,
bgra: Vec<u32>,
}
struct Shared {
frames: Mutex<VecDeque<Frame>>,
stop: AtomicBool,
/// The decode thread exited for good — a honored stop or a fatal
/// decode error. End-of-stream no longer ends the thread: it PARKS
/// (see `eos`) so seeks — loop restarts, scrubs — stay instant.
done: AtomicBool,
/// The stream is fully decoded and the thread is parked waiting for a
/// seek or a stop. With an empty ring this is the player's EOS state.
eos: AtomicBool,
/// Requested playback position in 100ns units; -1 = none. The decode
/// thread consumes it (an in-place decoder seek — no reopen).
seek_100ns: AtomicI64,
}
/// Soundtrack-queue ownership ticket. Every player claims a fresh epoch; a
/// DETACHED decode thread of a dropped player still holds its old epoch and
/// its pushes bounce off the queue (see [`VideoPlayer::drop`]).
static NEXT_CLIP_EPOCH: AtomicU64 = AtomicU64::new(1);
/// Half a frame at 60 fps: a seek target inside a frame's span lands on
/// that frame instead of the next.
const FRAME_EPS_100NS: i64 = 83_000;
pub struct VideoPlayer {
pub width: u32,
pub height: u32,
/// Container-reported duration; 0 when the container does not say.
pub duration_100ns: i64,
shared: Arc<Shared>,
started: Option<Instant>,
last_pts: i64,
/// While paused: when the pause began. The clock rebases by the paused
/// span on resume, so playback continues where it stopped instead of
/// skipping the frames "missed" on the wall clock.
paused_at: Option<Instant>,
epoch: u64,
}
impl VideoPlayer {
pub fn new(path: &str) -> Result<Self, String> {
let info = VideoFileDecoder::open(path)
.map_err(|e| e.to_string())?
.info()
.clone();
if info.width == 0 || info.height == 0 {
return Err(format!(
"video reports zero size: {}x{}",
info.width, info.height
));
}
log!(
"video: {} {}x{} {}/{} fps codec {:?} audio {} ({} Hz, {} ch)",
path,
info.width,
info.height,
info.fps_num,
info.fps_den,
info.video_codec,
info.has_audio,
info.audio_sample_rate,
info.audio_channels
);
// Fresh clip: take ownership of the soundtrack queue. Claiming a new
// epoch drops any previous tail, lifts the sticky stop_audio() mute,
// and locks every straggler push from an older detached decode
// thread out of the queue.
let epoch = NEXT_CLIP_EPOCH.fetch_add(1, Ordering::Relaxed);
{
let mut audio = video_audio().lock().unwrap();
audio.clear();
audio.muted = false;
audio.owner = epoch;
}
let shared = Arc::new(Shared {
frames: Mutex::new(VecDeque::new()),
stop: AtomicBool::new(false),
done: AtomicBool::new(false),
eos: AtomicBool::new(false),
seek_100ns: AtomicI64::new(-1),
});
let thread_shared = shared.clone();
let thread_path = path.to_string();
// The JoinHandle is deliberately dropped: teardown must never join a
// possibly wedged hardware decoder on the UI thread. The thread is
// detached; `stop` + the audio epoch make that safe.
std::thread::Builder::new()
.name("asset-ui-video-decode".into())
.spawn(move || {
match VideoFileDecoder::open(&thread_path) {
Ok(decoder) => decode_loop(thread_path, decoder, &thread_shared, epoch),
Err(e) => log!("video: decode thread open failed: {}", e),
}
thread_shared.done.store(true, Ordering::Release);
})
.map_err(|e| e.to_string())?;
Ok(Self {
width: info.width,
height: info.height,
duration_100ns: info.duration_100ns,
shared,
started: None,
last_pts: 0,
paused_at: None,
epoch,
})
}
/// The newest frame whose pts has been reached; `None` keeps whatever is
/// on the texture. Call once per render frame.
pub fn is_paused(&self) -> bool {
self.paused_at.is_some()
}
/// Freeze the picture; the soundtrack mutes with it. Idempotent.
pub fn pause(&mut self) {
if self.paused_at.is_none() {
self.paused_at = Some(Instant::now());
video_audio().lock().unwrap().muted = true;
}
}
/// Continue from the paused position by pushing the clock base forward
/// by the paused span. Idempotent.
pub fn resume(&mut self) {
if let Some(paused_at) = self.paused_at.take() {
if let Some(started) = &mut self.started {
*started += paused_at.elapsed();
}
video_audio().lock().unwrap().muted = false;
}
}
pub fn take_due_frame(&mut self) -> Option<Vec<u32>> {
if self.paused_at.is_some() {
// Paused normally: hold the picture. Freshly seeked while
// paused (clock unset): show the seek target frame once.
if self.started.is_some() {
return None;
}
let mut frames = self.shared.frames.lock().unwrap();
let frame = frames.pop_front()?;
self.last_pts = frame.pts_100ns;
// Keep the clock unset: resume rebases at the next frame.
return Some(frame.bgra);
}
let mut frames = self.shared.frames.lock().unwrap();
let first_pts = frames.front()?.pts_100ns;
let started = *self.started.get_or_insert_with(|| {
Instant::now() - Duration::from_nanos(first_pts.max(0) as u64 * 100)
});
let media_100ns = (started.elapsed().as_nanos() / 100) as i64;
let mut due = None;
while frames.front().is_some_and(|f| f.pts_100ns <= media_100ns) {
due = frames.pop_front();
}
if let Some(frame) = &due {
self.last_pts = frame.pts_100ns;
}
due.map(|f| f.bgra)
}
pub fn position_secs(&self) -> f64 {
self.last_pts as f64 / 10_000_000.0
}
pub fn duration_secs(&self) -> f64 {
self.duration_100ns as f64 / 10_000_000.0
}
/// Jump playback to `secs`. The decode thread reopens the file and
/// discards up to the target; the picture clock rebases on the first
/// frame that arrives, so play continues from there (paused stays
/// paused, showing the seeked frame).
pub fn seek(&mut self, secs: f64) {
let target = (secs.max(0.0) * 10_000_000.0) as i64;
self.shared.seek_100ns.store(target, Ordering::Release);
self.shared.frames.lock().unwrap().clear();
self.started = None;
self.last_pts = target;
let mut audio = video_audio().lock().unwrap();
if audio.owner == self.epoch {
audio.clear();
}
}
/// True end-of-playback for the frame pump: the decode thread has exited
/// (end of stream, decode error, or stop) and every buffered frame has
/// been taken. The soundtrack tail may still be draining in the audio
/// callback — that needs no frame pump.
pub fn at_end(&self) -> bool {
(self.shared.eos.load(Ordering::Acquire) || self.shared.done.load(Ordering::Acquire))
&& self.shared.frames.lock().unwrap().is_empty()
}
/// True while a seek request is still unconsumed by the decode thread —
/// the host coalesces scrub drags on this instead of flooding.
pub fn seek_pending(&self) -> bool {
self.shared.seek_100ns.load(Ordering::Acquire) >= 0
}
}
impl Drop for VideoPlayer {
fn drop(&mut self) {
self.shared.stop.store(true, Ordering::Relaxed);
// NEVER join the decode thread here: a wedged hardware decoder call
// would hang the UI thread. The detached thread observes `stop`
// between packets and exits on its own; until then the epoch guard
// keeps its audio pushes out of the queue, and its frame ring dies
// with the last Arc.
let mut audio = video_audio().lock().unwrap();
if audio.owner == self.epoch {
audio.clear();
audio.owner = 0;
}
}
}
fn decode_loop(_path: String, mut decoder: VideoFileDecoder, shared: &Shared, epoch: u64) {
let info = decoder.info().clone();
let mut audio_eos = false;
let mut rgb_scratch = Vec::new();
loop {
if shared.stop.load(Ordering::Relaxed) {
return;
}
let seek = shared.seek_100ns.swap(-1, Ordering::AcqRel);
if seek >= 0 {
shared.eos.store(false, Ordering::Release);
// In-place decoder seek (SetCurrentPosition / reader rebuild in
// the platform layer, ~10 ms) — never a full reopen, which is
// what makes SCRUBBING realtime and a loop restart seamless.
match decoder.seek(seek) {
Ok(()) => {
audio_eos = !info.has_audio;
shared.frames.lock().unwrap().clear();
loop {
if shared.stop.load(Ordering::Relaxed) {
return;
}
if shared.seek_100ns.load(Ordering::Acquire) >= 0 {
break; // newer scrub target supersedes this one
}
match decoder.next_frame() {
Ok(Some(frame)) if frame.pts_100ns + FRAME_EPS_100NS < seek => {}
Ok(Some(frame)) => {
nv12::nv12_to_rgb8(
&frame.nv12,
frame.width,
frame.height,
&mut rgb_scratch,
);
let mut bgra =
Vec::with_capacity((frame.width * frame.height) as usize);
for px in rgb_scratch.chunks_exact(3) {
bgra.push(
0xff00_0000
| ((px[0] as u32) << 16)
| ((px[1] as u32) << 8)
| px[2] as u32,
);
}
shared
.frames
.lock()
.unwrap()
.push_back(Frame { pts_100ns: frame.pts_100ns, bgra });
break;
}
Ok(None) => break,
Err(e) => {
log!("video: seek decode error: {}", e);
break;
}
}
}
// Audio follows the picture: drop queued samples and
// skip the soundtrack forward to the target.
if info.has_audio {
video_audio().lock().unwrap().clear_for(epoch);
loop {
if shared.stop.load(Ordering::Relaxed) {
return;
}
match decoder.next_audio() {
Ok(Some(chunk)) if chunk.pts_100ns < seek => {}
Ok(Some(chunk)) => {
video_audio().lock().unwrap().push_i16(
epoch,
&chunk.samples,
chunk.channels,
chunk.sample_rate,
);
break;
}
Ok(None) => {
audio_eos = true;
break;
}
Err(_) => {
audio_eos = true;
break;
}
}
}
}
}
Err(e) => log!("video: decoder seek failed: {}", e),
}
continue;
}
if info.has_audio && !audio_eos {
while video_audio().lock().unwrap().buffered_secs() < AUDIO_AHEAD_SECS {
match decoder.next_audio() {
Ok(Some(chunk)) => video_audio().lock().unwrap().push_i16(
epoch,
&chunk.samples,
chunk.channels,
chunk.sample_rate,
),
Ok(None) => {
audio_eos = true;
break;
}
Err(e) => {
log!("video: audio decode error: {}", e);
audio_eos = true;
break;
}
}
}
}
if shared.frames.lock().unwrap().len() >= RING_FRAMES {
std::thread::sleep(Duration::from_millis(4));
continue;
}
match decoder.next_frame() {
Ok(Some(frame)) => {
nv12::nv12_to_rgb8(&frame.nv12, frame.width, frame.height, &mut rgb_scratch);
let mut bgra = Vec::with_capacity((frame.width * frame.height) as usize);
for px in rgb_scratch.chunks_exact(3) {
bgra.push(
0xff00_0000
| ((px[0] as u32) << 16)
| ((px[1] as u32) << 8)
| px[2] as u32,
);
}
shared.frames.lock().unwrap().push_back(Frame {
pts_100ns: frame.pts_100ns,
bgra,
});
}
Ok(None) => {
// End of stream: drain the soundtrack tail, then PARK. The
// thread stays alive serving seeks — a loop restart or a
// scrub back into the clip is a ~10 ms decoder seek, not a
// teardown and reopen.
if info.has_audio && !audio_eos {
loop {
if shared.stop.load(Ordering::Relaxed) {
return;
}
match decoder.next_audio() {
Ok(Some(chunk)) => video_audio().lock().unwrap().push_i16(
epoch,
&chunk.samples,
chunk.channels,
chunk.sample_rate,
),
Ok(None) => break,
Err(e) => {
log!("video: audio tail decode error: {}", e);
break;
}
}
}
audio_eos = true;
}
shared.eos.store(true, Ordering::Release);
while shared.eos.load(Ordering::Acquire) {
if shared.stop.load(Ordering::Relaxed) {
return;
}
if shared.seek_100ns.load(Ordering::Acquire) >= 0 {
break; // the top of the loop consumes it
}
std::thread::sleep(Duration::from_millis(10));
}
continue;
}
Err(e) => {
log!("video: decode error: {}", e);
return;
}
}
}
}
// ---------------------------------------------------------------------------
// Soundtrack queue (identical mixer shape to the wav player in audio.rs)
// ---------------------------------------------------------------------------
impl VideoAudio {
/// Epoch-guarded queue drop for a seek: only the owning player's decode
/// thread may flush what it queued.
fn clear_for(&mut self, epoch: u64) {
if self.owner == epoch {
self.frames.clear();
self.cursor = 0.0;
}
}
}
pub struct VideoAudio {
frames: VecDeque<(f32, f32)>,
cursor: f64,
source_rate: f64,
/// Sticky mute raised by [`stop_audio`]: the decode thread may still be
/// refilling the queue, so a plain clear would go audible again ~a
/// second later. Cleared when the next clip starts.
muted: bool,
/// Epoch of the ONE player allowed to push (0 = none). Detached decode
/// threads of dropped players carry stale epochs and are locked out.
owner: u64,
}
impl VideoAudio {
const fn new() -> Self {
Self {
frames: VecDeque::new(),
cursor: 0.0,
source_rate: 0.0,
muted: false,
owner: 0,
}
}
fn clear(&mut self) {
self.frames.clear();
self.cursor = 0.0;
}
fn buffered_secs(&self) -> f64 {
if self.source_rate <= 0.0 {
return 0.0;
}
(self.frames.len() as f64 - self.cursor).max(0.0) / self.source_rate
}
fn push_i16(&mut self, epoch: u64, samples: &[i16], channels: u16, rate: u32) {
if self.muted || self.owner != epoch {
return;
}
self.source_rate = rate as f64;
let ch = channels.max(1) as usize;
const GAIN: f32 = 0.9;
for frame in samples.chunks_exact(ch) {
let l = frame[0] as f32 / 32768.0 * GAIN;
let r = frame[ch - 1] as f32 / 32768.0 * GAIN;
self.frames.push_back((l, r));
}
}
fn mix(&mut self, output: &mut AudioBuffer, device_rate: f64) {
if self.frames.is_empty() || self.source_rate <= 0.0 || device_rate <= 0.0 {
return;
}
let step = self.source_rate / device_rate;
let channels = output.channel_count();
for frame in 0..output.frame_count() {
let index = self.cursor as usize;
if index + 1 >= self.frames.len() {
break;
}
let fraction = (self.cursor - index as f64) as f32;
let (al, ar) = self.frames[index];
let (bl, br) = self.frames[index + 1];
let l = al + (bl - al) * fraction;
let r = ar + (br - ar) * fraction;
for channel in 0..channels {
let s = if channel == 0 { l } else { r };
output.channel_mut(channel)[frame] += s;
}
self.cursor += step;
}
let consumed = self.cursor as usize;
if consumed > 0 {
self.frames.drain(..consumed.min(self.frames.len()));
self.cursor -= consumed as f64;
}
// A lone trailing frame can never be interpolated: once the queue is
// down to it the clip is over (or hard-underrun) — release it so the
// soundtrack ends instead of pinning the final sample forever.
if self.frames.len() <= 1 {
self.clear();
}
}
}
static VIDEO_AUDIO: Mutex<VideoAudio> = Mutex::new(VideoAudio::new());
fn video_audio() -> &'static Mutex<VideoAudio> {
&VIDEO_AUDIO
}
/// Mix queued video audio into the device buffer (one line in the app's
/// `cx.audio_output` closure).
pub fn mix_into(output: &mut AudioBuffer, device_rate: f64) {
if let Ok(mut audio) = video_audio().lock() {
audio.mix(output, device_rate);
}
}
/// Silences the video soundtrack immediately and keeps it silent (the
/// decode thread may still be refilling) until the next clip starts —
/// stop-button / tab-switch hook. Revoking ownership locks every live
/// decode thread out of the queue, not just muting it.
pub fn stop_audio() {
if let Ok(mut audio) = video_audio().lock() {
audio.clear();
audio.muted = true;
audio.owner = 0;
}
}
// ---------------------------------------------------------------------------
#[cfg(test)]
mod tests {
use super::*;
/// Both tests touch the process-global soundtrack queue.
static VIDEO_TEST_LOCK: Mutex<()> = Mutex::new(());
#[test]
fn at_end_needs_decode_exit_and_a_drained_ring() {
let _serial = VIDEO_TEST_LOCK.lock().unwrap();
let shared = Arc::new(Shared {
frames: Mutex::new(VecDeque::new()),
stop: AtomicBool::new(false),
done: AtomicBool::new(false),
eos: AtomicBool::new(false),
seek_100ns: AtomicI64::new(-1),
});
shared.frames.lock().unwrap().push_back(Frame {
pts_100ns: 0,
bgra: vec![0xff00_0000; 4],
});
let mut player = VideoPlayer {
width: 2,
height: 2,
duration_100ns: 0,
shared: shared.clone(),
started: None,
last_pts: 0,
paused_at: None,
epoch: u64::MAX,
};
// Still decoding: never EOS, with or without buffered frames.
assert!(!player.at_end());
// Decode exited, but a due frame is still buffered: pump keeps going.
shared.done.store(true, Ordering::Release);
assert!(!player.at_end());
// The pts-0 frame is immediately due; taking it drains the ring.
assert!(player.take_due_frame().is_some());
assert!(player.at_end(), "decode done + drained ring is EOS");
// A stopped-but-undrained ring is also not EOS until taken.
assert!(player.take_due_frame().is_none());
}
#[test]
fn soundtrack_queue_ignores_stale_epochs_and_revoked_ownership() {
let _serial = VIDEO_TEST_LOCK.lock().unwrap();
{
let mut audio = video_audio().lock().unwrap();
audio.clear();
audio.muted = false;
audio.owner = 7;
audio.source_rate = 0.0;
}
// A dropped player's detached thread (stale epoch) cannot push.
video_audio()
.lock()
.unwrap()
.push_i16(6, &[1000, 1000], 2, 48_000);
assert_eq!(video_audio().lock().unwrap().frames.len(), 0);
// The owning epoch pushes fine.
video_audio()
.lock()
.unwrap()
.push_i16(7, &[1000, 1000, 2000, 2000], 2, 48_000);
assert_eq!(video_audio().lock().unwrap().frames.len(), 2);
// stop_audio revokes ownership: even the former owner is locked out.
stop_audio();
video_audio()
.lock()
.unwrap()
.push_i16(7, &[1000, 1000], 2, 48_000);
let audio = video_audio().lock().unwrap();
assert_eq!(audio.frames.len(), 0);
assert_eq!(audio.owner, 0);
assert!(audio.muted);
}
}

View file

@ -1,29 +0,0 @@
# browser — a Chrome-like browser as a plain full-window Makepad app.
#
# Chromium (CEF, libs/cef) renders ONLY the page, GPU-accelerated straight
# into an IOSurface-backed Makepad texture. All browser chrome — the tab
# strip, the toolbar with back/forward/reload, the omnibox and the menu —
# is Makepad splash UI, styled from the wm theme when hosted by
# makepad-wm (`MAKEPAD_WM_THEME_SPLASH`) and from a Chrome-dark palette otherwise.
#
# Runs standalone, and unmodified inside makepad-wm / Studio tiles via the
# shared --stdin-loop client runtime every Makepad app has.
[package]
name = "makepad-browser"
version = "0.1.0"
edition = "2021"
default-run = "browser"
[[bin]]
name = "browser"
path = "src/main.rs"
[dependencies]
makepad-widgets = { path = "../../widgets", features = ["cef"] }
makepad-cef = { path = "../../libs/cef" }
makepad-ai-services = { path = "../../libs/ai/services" }
makepad-strict-json = { path = "../../libs/strict_json" }
makepad-wm-theme = { path = "../../libs/wm_theme" }
# The window manager's vocabulary: the bar's title, the polite close.
makepad-wm-api = { path = "../../libs/wm_api" }

View file

@ -1,220 +0,0 @@
//! The browser on the desktop's AI bus.
//!
//! The current CEF wrapper mirrors navigation metadata but does not bind its
//! frame text/source callbacks, so `page` reports the active title and URL.
use makepad_ai_services::wire::{Risk, ServiceCall, ServiceManifest, ToolDef, ToolResult};
use makepad_strict_json::{self as json, Value};
pub struct PageState {
pub title: String,
pub url: String,
}
pub struct TabState {
pub title: String,
pub url: String,
pub active: bool,
}
/// The bus-facing subset of the webview. Keeping it as a trait makes the
/// closed dispatcher testable without starting CEF or a window.
pub trait BrowserTarget {
fn page(&self) -> Option<PageState>;
fn tabs(&self) -> Vec<TabState>;
fn navigate(&mut self, url: &str) -> bool;
fn new_tab(&mut self, url: &str);
}
pub fn manifest() -> ServiceManifest {
ServiceManifest::new(
"browser",
"Browser",
"The live web browser. Its read tools report the active page and all tabs; its action tools steer the active tab or open a new one.",
)
.with_tool(ToolDef::new(
"page",
"Read the active tab's displayed title and URL. The current CEF binding does not expose visible page text or source, so this tool cannot return page text.",
r#"{"type":"object","properties":{},"additionalProperties":false}"#,
Risk::Read,
))
.with_tool(ToolDef::new(
"tabs",
"Read every open tab's displayed title and URL, with the active tab marked.",
r#"{"type":"object","properties":{},"additionalProperties":false}"#,
Risk::Read,
))
.with_tool(ToolDef::new(
"navigate",
"Navigate the active tab to an http:// or https:// URL, or to about:blank.",
r#"{"type":"object","properties":{"url":{"type":"string"}},"required":["url"],"additionalProperties":false}"#,
Risk::Act,
))
.with_tool(ToolDef::new(
"new_tab",
"Open and activate a new tab at an http:// or https:// URL, or at about:blank.",
r#"{"type":"object","properties":{"url":{"type":"string"}},"required":["url"],"additionalProperties":false}"#,
Risk::Act,
))
}
/// Answer one browser call through a closed match over the four advertised
/// names. URL actions use the webview's existing navigation methods.
pub fn answer(call: &ServiceCall, target: &mut impl BrowserTarget) -> ToolResult {
match call.tool.as_str() {
"page" => {
if let Err(error) = empty_args(&call.args) {
return ToolResult::refused(&call.call_id, error);
}
match target.page() {
Some(page) => ToolResult::ok(
&call.call_id,
format!("title: {}\nurl: {}", page.title, page.url),
format!("{}{}", page.title, page.url),
),
None => ToolResult::unavailable(&call.call_id, "there is no active browser tab"),
}
}
"tabs" => {
if let Err(error) = empty_args(&call.args) {
return ToolResult::refused(&call.call_id, error);
}
let tabs = target.tabs();
let mut text = String::new();
for (index, tab) in tabs.iter().enumerate() {
if index > 0 {
text.push('\n');
}
text.push_str(if tab.active { "[active] " } else { " " });
text.push_str(&tab.title);
text.push_str("");
text.push_str(&tab.url);
}
if text.is_empty() {
text.push_str("no tabs open");
}
ToolResult::ok(&call.call_id, text, format!("{} tabs", tabs.len()))
}
"navigate" => {
let url = match url_arg(&call.args, "navigate") {
Ok(url) => url,
Err(error) => return ToolResult::refused(&call.call_id, error),
};
if target.navigate(&url) {
ToolResult::ok(&call.call_id, format!("navigating to {url}"), "navigating")
} else {
ToolResult::unavailable(&call.call_id, "there is no active browser tab")
}
}
"new_tab" => {
let url = match url_arg(&call.args, "new_tab") {
Ok(url) => url,
Err(error) => return ToolResult::refused(&call.call_id, error),
};
target.new_tab(&url);
ToolResult::ok(&call.call_id, format!("opened a new tab at {url}"), "navigating")
}
other => ToolResult::refused(
&call.call_id,
format!("browser has no tool `{other}`; it has page, tabs, navigate, new_tab"),
),
}
}
fn empty_args(args: &str) -> Result<(), String> {
let fields = object_args(args)?;
if let Some((key, _)) = fields.first() {
return Err(format!("unknown argument `{key}`"));
}
Ok(())
}
fn url_arg(args: &str, tool: &str) -> Result<String, String> {
let fields = object_args(args)?;
if let Some((key, _)) = fields.iter().find(|(key, _)| key != "url") {
return Err(format!("unknown argument `{key}`"));
}
let Some(url) = fields
.iter()
.find(|(key, _)| key == "url")
.and_then(|(_, value)| value.as_str())
else {
return Err(format!("{tool}.url must be a string"));
};
if !allowed_url(url) {
return Err(format!(
"{tool}.url must start with http:// or https://, or be exactly about:blank"
));
}
Ok(url.to_string())
}
fn object_args(args: &str) -> Result<Vec<(String, Value)>, String> {
match json::parse(args.as_bytes()) {
Ok(Value::Obj(fields)) => Ok(fields),
Ok(_) => Err("tool arguments must be a JSON object".to_string()),
Err(error) => Err(format!("invalid tool arguments: {error}")),
}
}
fn allowed_url(url: &str) -> bool {
if url == "about:blank" {
return true;
}
if url.chars().any(char::is_whitespace) || url.chars().any(char::is_control) {
return false;
}
url.strip_prefix("http://")
.or_else(|| url.strip_prefix("https://"))
.is_some_and(|rest| !rest.is_empty())
}
#[cfg(test)]
mod tests {
use super::*;
use makepad_ai_services::wire::ToolOutcome;
#[derive(Default)]
struct FakeTarget {
navigated: Vec<String>,
}
impl BrowserTarget for FakeTarget {
fn page(&self) -> Option<PageState> {
None
}
fn tabs(&self) -> Vec<TabState> {
Vec::new()
}
fn navigate(&mut self, url: &str) -> bool {
self.navigated.push(url.to_string());
true
}
fn new_tab(&mut self, url: &str) {
self.navigated.push(url.to_string());
}
}
#[test]
fn browser_manifest_validates() {
manifest().validate().expect("a valid browser manifest");
}
#[test]
fn navigate_refuses_non_http_urls() {
let mut target = FakeTarget::default();
for url in ["file:///etc/passwd", "javascript:alert(1)", "data:text/plain,no"] {
let call = ServiceCall {
call_id: "c1".into(),
tool: "navigate".into(),
args: format!(r#"{{"url":"{url}"}}"#),
};
let result = answer(&call, &mut target);
assert_eq!(result.outcome, ToolOutcome::Refused);
}
assert!(target.navigated.is_empty());
}
}

View file

@ -18,7 +18,7 @@ gltf = ["dep:makepad-fab-loader-gltf"]
makepad-widgets = { path = "../../widgets", version = "2.0.0" }
makepad-fab-shell = { path = "../../libs/fab", version = "0.1.0" }
makepad-fab-loader-gltf = { path = "loaders/gltf", version = "0.1.0", optional = true }
makepad-video = { package = "makepad-platform-video", path = "../../platform/video", version = "1.0.0" }
makepad-video = { path = "../../platform/video", version = "1.0.0" }
makepad-zune-png = { path = "../../libs/zune/zune-png", version = "0.5.2" }
[dev-dependencies]

View file

@ -1,5 +1,5 @@
//! Encode a directory of `frame_%06d.png` files into an H.264 mp4 via the
//! platform hardware encoder (`makepad-platform-video` / VideoToolbox on macOS).
//! platform hardware encoder (`makepad-video` / VideoToolbox on macOS).
//!
//! ```text
//! frames_to_mp4 <dir> <out.mp4> [--fps 24] [--start N] [--end M] [--bitrate BPS] [--crf N]

View file

@ -1,18 +0,0 @@
[package]
name = "makepad-fabric"
version = "0.1.0"
edition = "2021"
license = "MIT OR Apache-2.0"
description = "Fabric: a photo in, a fitted sewing pattern out — body model, measurements, draft, cut sheet"
[dependencies]
# The window manager's vocabulary: the bar's title and the polite close.
makepad-wm-api = { path = "../../libs/wm_api" }
makepad-widgets = { path = "../../widgets" }
makepad-fabric-measure = { path = "../../libs/fabric/measure" }
makepad-fabric-draft = { path = "../../libs/fabric/draft" }
# The body model runs in-process (Metal on the Mac, CUDA on a box); the
# hub does install, licence acknowledgement and weight location.
makepad-ai-body = { path = "../../libs/ai/models/body" }
makepad-ai-hub = { path = "../../libs/ai/hub", default-features = false, features = ["local", "body-native"] }
makepad-ai-hub-ui = { path = "../../libs/ai/hub_ui" }

View file

@ -1,539 +0,0 @@
use makepad_fabric_measure::{BodyMesh, Line, Measured, Ring};
use makepad_widgets::*;
use std::sync::Arc;
script_mod! {
use mod.prelude.widgets_internal.*
use mod.widgets.*
set_type_default() do #(DrawBodyPoint::script_shader(vm)) {
..mod.draw.DrawQuad
pixel: fn() {
let d = length(self.pos - vec2(0.5, 0.5))
let a = clamp((0.5 - d) * 7.0, 0.0, 1.0)
let near = #x80e7ff
let far = #x27435d
let c = far.mix(near, 1.0 - self.depth)
return vec4(c.xyz * a, a)
}
}
set_type_default() do #(DrawFabricLine::script_shader(vm)) {
..mod.draw.DrawQuad
pixel: fn() {
// A line as a distance field inside its bounding quad. The
// endpoints are LOCAL to the quad (the turtle may still shift
// rect_pos after the instance is written), like the chart's
// segment shader.
let p = self.pos * self.rect_size
let ab = self.p1 - self.p0
let t = clamp(dot(p - self.p0, ab) / max(dot(ab, ab), 0.0001), 0.0, 1.0)
let d = length(p - (self.p0 + ab * t))
let aa = 1.0 - smoothstep(self.half_width - 0.6, self.half_width + 0.6, d)
let alpha = aa * self.color.w
return vec4(self.color.xyz * alpha, alpha)
}
}
mod.widgets.FabricBodyViewBase = #(FabricBodyView::register_widget(vm))
mod.widgets.FabricBodyView = set_type_default() do mod.widgets.FabricBodyViewBase {
width: Fill
height: Fill
draw_bg +: {color: #x11161d}
draw_point +: {}
draw_line +: {}
draw_text +: {
color: #x9aa8b7
text_style: theme.font_regular{font_size: 9.0}
}
}
}
#[derive(Script, ScriptHook)]
#[repr(C)]
pub struct DrawBodyPoint {
#[deref]
draw_super: DrawQuad,
#[live]
depth: f32,
}
#[derive(Script, ScriptHook)]
#[repr(C)]
pub struct DrawFabricLine {
#[deref]
draw_super: DrawQuad,
#[live]
pub color: Vec4f,
#[live]
p0: Vec2f,
#[live]
p1: Vec2f,
#[live]
half_width: f32,
}
impl DrawFabricLine {
pub fn segment(&mut self, cx: &mut Cx2d, from: DVec2, to: DVec2, width: f64) {
if (to - from).length() < 0.01 {
return;
}
let half = width * 0.5;
let pad = half + 1.0;
let min = dvec2(from.x.min(to.x) - pad, from.y.min(to.y) - pad);
let max = dvec2(from.x.max(to.x) + pad, from.y.max(to.y) + pad);
self.p0 = v2f(from - min);
self.p1 = v2f(to - min);
self.half_width = half as f32;
self.draw_abs(cx, Rect { pos: min, size: max - min });
}
}
fn v2f(value: DVec2) -> Vec2f {
Vec2f {
x: value.x as f32,
y: value.y as f32,
}
}
#[derive(Clone, Copy)]
struct BodyDrag {
from: DVec2,
yaw: f64,
pitch: f64,
pan: DVec2,
panning: bool,
}
#[derive(Clone, Debug, Default)]
pub(crate) struct BodyPoseMapping {
ring_vertices: Vec<Vec<usize>>,
line_vertices: Vec<[usize; 2]>,
}
pub(crate) fn map_measurements_to_vertices(
mesh: &BodyMesh,
measured: &Measured,
) -> BodyPoseMapping {
let nearest = |point| nearest_vertex_index(&mesh.vertices, measured.scale, point);
BodyPoseMapping {
ring_vertices: measured
.rings
.iter()
.map(|ring| ring.points.iter().copied().map(nearest).collect())
.collect(),
line_vertices: measured
.lines
.iter()
.map(|line| [nearest(line.from), nearest(line.to)])
.collect(),
}
}
fn nearest_vertex_index(vertices: &[[f32; 3]], scale: f32, point: [f32; 3]) -> usize {
vertices
.iter()
.enumerate()
.map(|(index, vertex)| {
let dx = vertex[0] * scale - point[0];
let dy = vertex[1] * scale - point[1];
let dz = vertex[2] * scale - point[2];
(index, dx * dx + dy * dy + dz * dz)
})
.min_by(|left, right| left.1.total_cmp(&right.1))
.map(|(index, _)| index)
.unwrap_or(0)
}
fn mirror_x(value: f64, mirrored: bool) -> f64 {
if mirrored { -value } else { value }
}
fn bounds(points: &[[f32; 3]], scale: f32) -> Option<([f32; 3], f32)> {
let mut min = [f32::INFINITY; 3];
let mut max = [f32::NEG_INFINITY; 3];
for point in points {
for axis in 0..3 {
let value = point[axis] * scale;
min[axis] = min[axis].min(value);
max[axis] = max[axis].max(value);
}
}
if !min[0].is_finite() {
return None;
}
let centre = [
(min[0] + max[0]) * 0.5,
(min[1] + max[1]) * 0.5,
(min[2] + max[2]) * 0.5,
];
let dx = max[0] - min[0];
let dy = max[1] - min[1];
let dz = max[2] - min[2];
let radius = (dx * dx + dy * dy + dz * dz).sqrt().max(1.0) * 0.5;
Some((centre, radius))
}
#[derive(Script, ScriptHook, Widget)]
pub struct FabricBodyView {
#[uid]
uid: WidgetUid,
#[source]
source: ScriptObjectRef,
#[walk]
walk: Walk,
#[layout]
layout: Layout,
#[redraw]
#[area]
area: Area,
#[live]
draw_bg: DrawColor,
#[live]
draw_point: DrawBodyPoint,
#[live]
draw_line: DrawFabricLine,
#[live]
draw_text: DrawText,
#[rust]
mesh: Option<Arc<BodyMesh>>,
#[rust]
posed: Option<Arc<Vec<[f32; 3]>>>,
#[rust]
rings: Vec<Ring>,
#[rust]
lines: Vec<Line>,
#[rust]
pose_mapping: BodyPoseMapping,
#[rust]
mesh_scale: f32,
#[rust]
centre: [f32; 3],
#[rust]
radius: f32,
#[rust(0.35)]
yaw: f64,
#[rust(-0.06)]
pitch: f64,
#[rust(1.0)]
zoom: f64,
#[rust]
pan: DVec2,
#[rust]
drag: Option<BodyDrag>,
#[rust(true)]
mirrored: bool,
}
impl FabricBodyView {
pub fn set_body(
&mut self,
cx: &mut Cx,
mesh: Arc<BodyMesh>,
measured: &Measured,
pose_mapping: BodyPoseMapping,
) {
self.mesh_scale = measured.scale;
self.rings = measured.rings.clone();
self.lines = measured.lines.clone();
self.pose_mapping = pose_mapping;
if self.posed.is_none() {
self.fit_bounds(&mesh.vertices);
self.reset_camera();
}
self.mesh = Some(mesh);
self.redraw(cx);
}
pub fn set_pose(&mut self, cx: &mut Cx, posed: Option<Arc<Vec<[f32; 3]>>>) {
match posed {
Some(posed)
if self
.mesh
.as_ref()
.is_some_and(|mesh| mesh.vertices.len() == posed.len()) =>
{
if self.posed.is_none() {
self.fit_bounds(posed.as_slice());
self.reset_camera();
}
self.posed = Some(posed);
}
_ => {
self.posed = None;
if let Some(fit) = self
.mesh
.as_ref()
.and_then(|mesh| bounds(&mesh.vertices, self.mesh_scale))
{
(self.centre, self.radius) = fit;
self.reset_camera();
}
}
}
self.redraw(cx);
}
pub fn set_mirrored(&mut self, cx: &mut Cx, mirrored: bool) {
self.mirrored = mirrored;
self.redraw(cx);
}
fn fit_bounds(&mut self, points: &[[f32; 3]]) {
if let Some((centre, radius)) = bounds(points, self.mesh_scale) {
self.centre = centre;
self.radius = radius;
}
}
fn reset_camera(&mut self) {
self.yaw = 0.35;
self.pitch = -0.06;
self.zoom = 1.0;
self.pan = dvec2(0.0, 0.0);
}
fn project(&self, point: [f32; 3], mesh_point: bool, rect: Rect) -> Option<(DVec2, f32)> {
let scale = if mesh_point { self.mesh_scale } else { 1.0 } as f64;
let x = point[0] as f64 * scale - self.centre[0] as f64;
let y = point[1] as f64 * scale - self.centre[1] as f64;
let z = point[2] as f64 * scale - self.centre[2] as f64;
let (sy, cy) = self.yaw.sin_cos();
let (sp, cp) = self.pitch.sin_cos();
let rx = mirror_x(cy * x + sy * z, self.mirrored);
let rz = -sy * x + cy * z;
let ry = cp * y - sp * rz;
let rz = sp * y + cp * rz;
let fov = 35.0_f64.to_radians();
let fit_distance = self.radius as f64 / (fov * 0.5).tan() * 1.2;
let camera_z = fit_distance / self.zoom.max(0.08) - rz;
if camera_z <= 0.01 {
return None;
}
let focal = rect.size.y.max(1.0) * 0.5 / (fov * 0.5).tan();
let centre = rect.pos + rect.size * 0.5 + self.pan;
let screen = dvec2(centre.x + focal * rx / camera_z, centre.y - focal * ry / camera_z);
Some((screen, camera_z as f32))
}
fn projected_polyline(&self, points: &[[f32; 3]], rect: Rect) -> Vec<(DVec2, f32)> {
points
.iter()
.filter_map(|point| self.project(*point, false, rect))
.collect()
}
}
impl Widget for FabricBodyView {
fn draw_walk(&mut self, cx: &mut Cx2d, _scope: &mut Scope, walk: Walk) -> DrawStep {
let rect = cx.walk_turtle(walk);
self.draw_bg.draw_abs(cx, rect);
cx.push_clip_rect(rect);
let Some(mesh) = self.mesh.as_ref() else {
self.draw_text.color = Vec4f {
x: 0.48,
y: 0.55,
z: 0.62,
w: 1.0,
};
self.draw_text.draw_abs(
cx,
rect.pos + rect.size * 0.5 - dvec2(70.0, 5.0),
"drop a photo to start",
);
cx.pop_clip_rect();
cx.add_aligned_rect_area(&mut self.area, rect);
return DrawStep::done();
};
let posed = self
.posed
.as_deref()
.filter(|posed| posed.len() == mesh.vertices.len())
.map(Vec::as_slice);
let display_vertices = posed.unwrap_or(mesh.vertices.as_slice());
let mut points: Vec<(DVec2, f32)> = display_vertices
.iter()
.filter_map(|point| self.project(*point, true, rect))
.collect();
points.sort_by(|a, b| b.1.total_cmp(&a.1));
let min_depth = points.iter().map(|point| point.1).fold(f32::INFINITY, f32::min);
let max_depth = points
.iter()
.map(|point| point.1)
.fold(f32::NEG_INFINITY, f32::max);
let depth_span = (max_depth - min_depth).max(0.001);
self.draw_point.begin_many_instances(cx);
for (point, depth) in points {
self.draw_point.depth = (depth - min_depth) / depth_span;
self.draw_point.draw_abs(
cx,
Rect {
pos: point - dvec2(1.25, 1.25),
size: dvec2(2.5, 2.5),
},
);
}
self.draw_point.end_many_instances(cx);
let rings: Vec<(String, Vec<(DVec2, f32)>)> = self
.rings
.iter()
.enumerate()
.map(|(ring_index, ring)| {
let points = match (
posed,
self.pose_mapping.ring_vertices.get(ring_index),
) {
(Some(posed), Some(indices)) if indices.len() == ring.points.len() => indices
.iter()
.filter_map(|&index| self.project(*posed.get(index)?, true, rect))
.collect(),
_ => self.projected_polyline(&ring.points, rect),
};
(ring.key.replace('_', " "), points)
})
.collect();
let lines: Vec<(DVec2, DVec2)> = self
.lines
.iter()
.enumerate()
.filter_map(|(line_index, line)| {
let (from, to, mesh_points) = match (
posed,
self.pose_mapping.line_vertices.get(line_index),
) {
(Some(posed), Some([from, to])) =>
(*posed.get(*from)?, *posed.get(*to)?, true),
_ => (line.from, line.to, false),
};
Some((
self.project(from, mesh_points, rect)?.0,
self.project(to, mesh_points, rect)?.0,
))
})
.collect();
self.draw_line.begin_many_instances(cx);
self.draw_line.color = Vec4f {
x: 1.0,
y: 0.38,
z: 0.18,
w: 0.92,
};
for (_, points) in &rings {
for pair in points.windows(2) {
self.draw_line.segment(cx, pair[0].0, pair[1].0, 1.5);
}
if let (Some(first), Some(last)) = (points.first(), points.last()) {
self.draw_line.segment(cx, last.0, first.0, 1.5);
}
}
self.draw_line.color = Vec4f {
x: 0.42,
y: 0.78,
z: 1.0,
w: 0.9,
};
for (from, to) in lines {
self.draw_line.segment(cx, from, to, 1.25);
}
self.draw_line.end_many_instances(cx);
self.draw_text.color = Vec4f {
x: 1.0,
y: 0.66,
z: 0.48,
w: 1.0,
};
for (key, points) in rings {
if let Some(front) = points.iter().min_by(|a, b| a.1.total_cmp(&b.1)) {
self.draw_text
.draw_abs(cx, front.0 + dvec2(4.0, -5.0), &key);
}
}
cx.pop_clip_rect();
cx.add_aligned_rect_area(&mut self.area, rect);
DrawStep::done()
}
fn handle_event(&mut self, cx: &mut Cx, event: &Event, _scope: &mut Scope) {
match event.hits(cx, self.area) {
Hit::FingerDown(event) if event.device.is_primary_hit() => {
if event.tap_count >= 2 {
self.reset_camera();
self.redraw(cx);
return;
}
self.drag = Some(BodyDrag {
from: event.abs,
yaw: self.yaw,
pitch: self.pitch,
pan: self.pan,
panning: event.modifiers.shift,
});
}
Hit::FingerMove(event) => {
if let Some(drag) = self.drag {
let delta = event.abs - drag.from;
if drag.panning {
self.pan = drag.pan + delta;
} else {
self.yaw = drag.yaw - delta.x * 0.008;
self.pitch = (drag.pitch + delta.y * 0.007).clamp(-1.35, 1.35);
}
self.redraw(cx);
}
}
Hit::FingerUp(_) => self.drag = None,
Hit::FingerScroll(event) => {
self.zoom = (self.zoom * (-event.scroll.y * 0.004).exp()).clamp(0.15, 12.0);
self.redraw(cx);
}
Hit::FingerHoverIn(_) | Hit::FingerHoverOver(_) => {
cx.set_cursor(MouseCursor::Grab);
}
_ => {}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn nearest_vertex_mapping_uses_scaled_rest_mesh_positions() {
let mesh = BodyMesh {
vertices: vec![[0.0, 0.0, 0.0], [4.0, 0.0, 0.0], [9.0, 0.0, 0.0]],
faces: Vec::new(),
landmarks: None,
};
let measured = Measured {
values: makepad_fabric_measure::Measurements::sample(),
scale: 2.0,
rings: vec![Ring {
key: "test_ring",
y_cm: 0.0,
points: vec![[0.2, 0.0, 0.0], [7.5, 0.0, 0.0]],
skin_perimeter_cm: 0.0,
tape_perimeter_cm: 0.0,
}],
lines: vec![Line {
key: "test_line",
from: [7.5, 0.0, 0.0],
to: [17.0, 0.0, 0.0],
}],
};
let mapping = map_measurements_to_vertices(&mesh, &measured);
assert_eq!(mapping.ring_vertices, vec![vec![0, 1]]);
assert_eq!(mapping.line_vertices, vec![[1, 2]]);
}
#[test]
fn mirror_transform_only_flips_horizontal_view_axis() {
assert_eq!(mirror_x(12.5, false), 12.5);
assert_eq!(mirror_x(12.5, true), -12.5);
assert_eq!(mirror_x(-3.0, true), 3.0);
}
}

View file

@ -1,303 +0,0 @@
use makepad_widgets::{
makepad_platform::video::{
CameraFrameLayout, CameraFrameRef, VideoFormatId, VideoInputId, VideoInputsEvent,
VideoPixelFormat,
},
Cx, CxMediaApi,
};
use std::{
cmp::Reverse,
sync::{
atomic::{AtomicBool, AtomicU64, Ordering},
Arc, Mutex,
},
time::{Duration, Instant},
};
pub const SEND_MAX_WIDTH: usize = 640;
const PREVIEW_MAX_WIDTH: usize = 320;
const PREVIEW_INTERVAL: Duration = Duration::from_millis(100);
#[derive(Clone, Debug, PartialEq)]
pub struct CameraRgbFrame {
pub width: u32,
pub height: u32,
pub rgb: Vec<u8>,
pub serial: u64,
}
#[derive(Default)]
struct PreviewSlot {
frame: Option<CameraRgbFrame>,
updated_at: Option<Instant>,
}
#[derive(Default)]
struct CameraMailboxInner {
want: AtomicBool,
serial: AtomicU64,
frame: Mutex<Option<CameraRgbFrame>>,
model_size: Mutex<Option<(u32, u32)>>,
preview: Mutex<PreviewSlot>,
}
/// A one-frame handoff from the camera callback to the model worker. The
/// callback only converts a model-sized frame after the worker asks for one;
/// the smaller preview is independent and limited to ten updates per second.
#[derive(Clone, Default)]
pub struct CameraMailbox {
inner: Arc<CameraMailboxInner>,
}
impl CameraMailbox {
pub fn request(&self) {
if let Ok(mut frame) = self.inner.frame.lock() {
*frame = None;
}
self.inner.want.store(true, Ordering::Release);
}
pub fn take(&self) -> Option<CameraRgbFrame> {
self.inner.frame.lock().ok()?.take()
}
pub fn peek_preview(&self) -> Option<CameraRgbFrame> {
self.inner.preview.lock().ok()?.frame.clone()
}
pub fn model_size(&self) -> Option<(u32, u32)> {
*self.inner.model_size.lock().ok()?
}
fn capture(&self, frame: &CameraFrameRef<'_>) {
let wanted = self.inner.want.swap(false, Ordering::AcqRel);
let preview_due = self
.inner
.preview
.lock()
.ok()
.map(|slot| {
slot.updated_at
.map(|updated| updated.elapsed() >= PREVIEW_INTERVAL)
.unwrap_or(true)
})
.unwrap_or(false);
if !wanted && !preview_due {
return;
}
if wanted {
if let Some((rgb, width, height)) = frame_to_rgb(frame) {
let serial = self.inner.serial.fetch_add(1, Ordering::Relaxed) + 1;
if let Ok(mut size) = self.inner.model_size.lock() {
*size = Some((width, height));
}
if let Ok(mut slot) = self.inner.frame.lock() {
*slot = Some(CameraRgbFrame {
width,
height,
rgb,
serial,
});
}
}
}
if preview_due {
if let Some((rgb, width, height)) = frame_to_rgb_max(frame, PREVIEW_MAX_WIDTH) {
let serial = self.inner.serial.fetch_add(1, Ordering::Relaxed) + 1;
if let Ok(mut slot) = self.inner.preview.lock() {
slot.frame = Some(CameraRgbFrame {
width,
height,
rgb,
serial,
});
slot.updated_at = Some(Instant::now());
}
}
}
}
}
/// Register the capture callback. Calling this also asks the platform to
/// enumerate cameras, which produces `Event::VideoInputs` on the UI thread.
pub fn install_camera(cx: &mut Cx, mailbox: CameraMailbox) {
cx.camera_frame_input(0, move |frame| mailbox.capture(&frame));
}
/// Choose the first device's smallest raw-YUV format at least 640x360.
pub fn pick_camera(event: &VideoInputsEvent) -> Option<(VideoInputId, VideoFormatId)> {
let device = event.descs.first()?;
let format = device
.formats
.iter()
.filter(|format| {
format.width >= 640
&& format.height >= 360
&& matches!(
format.pixel_format,
VideoPixelFormat::NV12 | VideoPixelFormat::YUY2
)
})
.min_by_key(|format| {
(
format.width.saturating_mul(format.height),
if format.pixel_format == VideoPixelFormat::NV12 {
0
} else {
1
},
Reverse((format.frame_rate.unwrap_or(0.0) * 1000.0) as u64),
)
})?;
Some((device.input_id, format.format_id))
}
/// Convert NV12 or YUY2 to packed RGB8, using an integer sampling step so
/// the entire frame fits within [`SEND_MAX_WIDTH`].
pub fn frame_to_rgb(frame: &CameraFrameRef<'_>) -> Option<(Vec<u8>, u32, u32)> {
frame_to_rgb_max(frame, SEND_MAX_WIDTH)
}
fn frame_to_rgb_max(
frame: &CameraFrameRef<'_>,
max_width: usize,
) -> Option<(Vec<u8>, u32, u32)> {
let (width, height) = (frame.width, frame.height);
if width == 0 || height == 0 || max_width == 0 {
return None;
}
let step = width.div_ceil(max_width).max(1);
let out_width = width.div_ceil(step);
let out_height = height.div_ceil(step);
let mut rgb = Vec::with_capacity(out_width.checked_mul(out_height)?.checked_mul(3)?);
for out_y in 0..out_height {
let source_y = (out_y * step).min(height - 1);
for out_x in 0..out_width {
let source_x = (out_x * step).min(width - 1);
let (y, u, v) = match frame.layout {
CameraFrameLayout::NV12 => nv12_pixel(frame, source_x, source_y)?,
CameraFrameLayout::YUY2 => yuy2_pixel(frame, source_x, source_y)?,
_ => return None,
};
rgb.extend_from_slice(&yuv_to_rgb(y, u, v));
}
}
Some((
rgb,
u32::try_from(out_width).ok()?,
u32::try_from(out_height).ok()?,
))
}
fn nv12_pixel(frame: &CameraFrameRef<'_>, x: usize, y: usize) -> Option<(u8, u8, u8)> {
if frame.plane_count < 2 {
return None;
}
let y_plane = frame.planes[0];
let uv_plane = frame.planes[1];
let y_index = y
.checked_mul(y_plane.row_stride)?
.checked_add(x.checked_mul(y_plane.pixel_stride)?)?;
let uv_index = (y / 2)
.checked_mul(uv_plane.row_stride)?
.checked_add((x / 2).checked_mul(uv_plane.pixel_stride)?)?;
Some((
*y_plane.bytes.get(y_index)?,
*uv_plane.bytes.get(uv_index)?,
*uv_plane.bytes.get(uv_index + 1)?,
))
}
fn yuy2_pixel(frame: &CameraFrameRef<'_>, x: usize, y: usize) -> Option<(u8, u8, u8)> {
if frame.plane_count < 1 {
return None;
}
let plane = frame.planes[0];
let pixel_stride = plane.pixel_stride.max(2);
let pair = y
.checked_mul(plane.row_stride)?
.checked_add((x / 2).checked_mul(pixel_stride.checked_mul(2)?)?)?;
Some((
*plane.bytes.get(pair + (x & 1) * pixel_stride)?,
*plane.bytes.get(pair + 1)?,
*plane.bytes.get(pair + pixel_stride + 1)?,
))
}
/// One BT.709, video-range YUV pixel to RGB8.
pub(crate) fn yuv_to_rgb(y: u8, u: u8, v: u8) -> [u8; 3] {
let c = i32::from(y) - 16;
let d = i32::from(u) - 128;
let e = i32::from(v) - 128;
let clip = |value: i32| ((value + 128) >> 8).clamp(0, 255) as u8;
[
clip(298 * c + 459 * e),
clip(298 * c - 55 * d - 136 * e),
clip(298 * c + 541 * d),
]
}
#[cfg(test)]
mod tests {
use super::*;
use makepad_widgets::makepad_platform::video::{
CameraColorMatrix, CameraFramePlaneRef,
};
#[test]
fn yuv_grey_red_and_blue() {
let grey = yuv_to_rgb(126, 128, 128);
assert!(grey.iter().all(|channel| (126i16 - i16::from(*channel)).abs() <= 2));
let red = yuv_to_rgb(81, 90, 240);
assert!(red[0] > 220 && red[1] < 60 && red[2] < 60, "{red:?}");
let blue = yuv_to_rgb(41, 240, 110);
assert!(blue[2] > 220 && blue[0] < 60 && blue[1] < 60, "{blue:?}");
}
#[test]
fn nv12_integer_downsample_keeps_the_whole_frame() {
const WIDTH: usize = 64;
const HEIGHT: usize = 32;
let mut y_plane = vec![0u8; WIDTH * HEIGHT];
for y in 0..HEIGHT {
for x in 0..WIDTH {
y_plane[y * WIDTH + x] = 16 + ((x + y) % 200) as u8;
}
}
let uv_plane = vec![128u8; WIDTH * HEIGHT / 2];
let frame = CameraFrameRef {
timestamp_ns: 0,
width: WIDTH,
height: HEIGHT,
layout: CameraFrameLayout::NV12,
matrix: CameraColorMatrix::BT709,
plane_count: 2,
planes: [
CameraFramePlaneRef {
bytes: &y_plane,
row_stride: WIDTH,
pixel_stride: 1,
},
CameraFramePlaneRef {
bytes: &uv_plane,
row_stride: WIDTH,
pixel_stride: 2,
},
CameraFramePlaneRef::empty(),
],
};
let (rgb, width, height) = frame_to_rgb_max(&frame, 16).unwrap();
assert_eq!((width, height), (16, 8));
assert_eq!(rgb.len(), 16 * 8 * 3);
assert_eq!(&rgb[..3], &yuv_to_rgb(y_plane[0], 128, 128));
let last_source = y_plane[28 * WIDTH + 60];
assert_eq!(&rgb[rgb.len() - 3..], &yuv_to_rgb(last_source, 128, 128));
}
}

View file

@ -1,73 +0,0 @@
use makepad_ai_hub::{
local::{InstallState, LocalModels},
registry::LicenseRestriction,
};
use makepad_ai_hub_ui::{ModelRowInstallState, ModelRowState};
pub const BODY_MODEL_ID: &str = "sam3dbody";
pub const BODY_MODEL_ROLE: &str = "native-body";
pub fn body_model_row(models: &LocalModels) -> ModelRowState {
let spec = models.spec(BODY_MODEL_ID);
let bytes_from_spec = spec
.map(|spec| spec.files.iter().filter_map(|file| file.size).sum())
.unwrap_or(0);
let (bytes_done, bytes_total, state) = match models.install_state(BODY_MODEL_ID) {
InstallState::NotInstalled { bytes_total } => {
(0, bytes_total.max(bytes_from_spec), ModelRowInstallState::NotInstalled)
}
InstallState::Partial {
bytes_done,
bytes_total,
} => (bytes_done, bytes_total, ModelRowInstallState::NotInstalled),
InstallState::Installed => (
bytes_from_spec,
bytes_from_spec,
ModelRowInstallState::Installed,
),
};
let license = spec.and_then(|spec| spec.license.as_ref());
ModelRowState {
model_id: BODY_MODEL_ID.to_string(),
name: "SAM 3D Body".to_string(),
bytes_total,
bytes_done,
state,
license_name: license
.map(|license| license.name.clone())
.unwrap_or_else(|| "Licence unavailable".to_string()),
restriction: license
.map(|license| restriction_name(license.restriction).to_string())
.unwrap_or_else(|| "restricted".to_string()),
}
}
pub fn body_model_status(models: &LocalModels, downloading: bool) -> String {
if !models.license_acknowledged(BODY_MODEL_ID) {
return "licence not accepted".to_string();
}
match models.install_state(BODY_MODEL_ID) {
InstallState::Installed => "installed · 2.8 GB · Metal".to_string(),
InstallState::NotInstalled { .. } => "not installed · 2.8 GB".to_string(),
InstallState::Partial {
bytes_done,
bytes_total,
} => {
let percent = bytes_done.saturating_mul(100) / bytes_total.max(1);
if downloading {
format!("downloading {percent} %")
} else {
format!("not installed · {percent} % downloaded")
}
}
}
}
fn restriction_name(restriction: LicenseRestriction) -> &'static str {
match restriction {
LicenseRestriction::None => "none",
LicenseRestriction::NonCommercial => "non-commercial",
LicenseRestriction::Community => "community",
LicenseRestriction::Restricted => "restricted",
}
}

File diff suppressed because it is too large Load diff

View file

@ -1,398 +0,0 @@
use crate::body_view::DrawFabricLine;
use makepad_fabric_draft::{flatten, nest, offset, Layout as PatternLayout, Part, Pattern, Point};
use makepad_widgets::*;
script_mod! {
use mod.prelude.widgets_internal.*
use mod.widgets.*
mod.widgets.FabricPatternViewBase = #(FabricPatternView::register_widget(vm))
mod.widgets.FabricPatternView = set_type_default() do mod.widgets.FabricPatternViewBase {
width: Fill
height: Fill
draw_bg +: {color: #x0d1218}
draw_line +: {}
draw_text +: {
color: #xc5d0dc
text_style: theme.font_regular{font_size: 8.5}
}
}
}
#[derive(Clone, Copy)]
struct PatternDrag {
from: DVec2,
pan: DVec2,
}
#[derive(Clone, Copy, Default)]
struct Bounds {
min: DVec2,
max: DVec2,
valid: bool,
}
impl Bounds {
fn include(&mut self, point: DVec2) {
if !self.valid {
self.min = point;
self.max = point;
self.valid = true;
} else {
self.min.x = self.min.x.min(point.x);
self.min.y = self.min.y.min(point.y);
self.max.x = self.max.x.max(point.x);
self.max.y = self.max.y.max(point.y);
}
}
fn size(self) -> DVec2 {
let size = self.max - self.min;
dvec2(size.x.max(1.0), size.y.max(1.0))
}
fn centre(self) -> DVec2 {
(self.min + self.max) * 0.5
}
}
#[derive(Script, ScriptHook, Widget)]
pub struct FabricPatternView {
#[uid]
uid: WidgetUid,
#[source]
source: ScriptObjectRef,
#[walk]
walk: Walk,
#[layout]
layout: Layout,
#[redraw]
#[area]
area: Area,
#[live]
draw_bg: DrawColor,
#[live]
draw_line: DrawFabricLine,
#[live]
draw_text: DrawText,
#[rust]
pattern: Option<Pattern>,
#[rust]
nested: Option<PatternLayout>,
#[rust]
error: String,
#[rust]
bounds: Bounds,
#[rust(1.0)]
zoom: f64,
/// The view aspect the current nest was chosen for.
#[rust(1.0)]
nest_aspect: f64,
#[rust]
pan: DVec2,
#[rust]
drag: Option<PatternDrag>,
}
impl FabricPatternView {
pub fn set_pattern(&mut self, cx: &mut Cx, pattern: Pattern) {
self.pattern = Some(pattern);
// Nested on the next draw, for the pane's shape.
self.nested = None;
self.error.clear();
self.zoom = 1.0;
self.pan = dvec2(0.0, 0.0);
self.redraw(cx);
}
pub fn set_error(&mut self, cx: &mut Cx, error: impl Into<String>) {
self.pattern = None;
self.nested = None;
self.error = error.into();
self.redraw(cx);
}
/// Nest onto the fabric width whose finished layout has the pane's
/// aspect ratio, so the pieces fill the view instead of a tall strip.
fn ensure_nest(&mut self, rect: Rect) {
let target = (rect.size.x - 24.0).max(1.0) / (rect.size.y - 24.0).max(1.0);
if self.nested.is_some() && (self.nest_aspect / target).ln().abs() < 0.12 {
return;
}
const WIDTHS: [f64; 9] = [
900.0, 1200.0, 1500.0, 2000.0, 2500.0, 3000.0, 4000.0, 5000.0, 6500.0,
];
let best = {
let Some(pattern) = &self.pattern else { return };
let mut best: Option<(f64, PatternLayout, Bounds)> = None;
for width in WIDTHS {
let layout = nest(pattern, width);
let bounds = pattern_bounds(pattern, &layout);
let size = bounds.size();
let aspect = size.x.max(1.0) / size.y.max(1.0);
let score = (aspect / target).ln().abs();
if best.as_ref().map_or(true, |(other, _, _)| score < *other) {
best = Some((score, layout, bounds));
}
}
best
};
if let Some((_, layout, bounds)) = best {
self.nested = Some(layout);
self.bounds = bounds;
self.nest_aspect = target;
}
}
fn to_screen(&self, point: DVec2, rect: Rect) -> DVec2 {
let size = self.bounds.size();
let fit = ((rect.size.x - 24.0) / size.x)
.min((rect.size.y - 24.0) / size.y)
.max(0.0001);
rect.pos + rect.size * 0.5
+ self.pan
+ (point - self.bounds.centre()) * fit * self.zoom
}
fn part_offset<'a>(
&'a self,
layout: &'a PatternLayout,
part_index: usize,
) -> (Point, f64) {
layout
.placements
.iter()
.find(|placement| placement.part == part_index)
.map(|placement| (placement.offset, placement.rotation_deg))
.unwrap_or((Point::default(), 0.0))
}
fn path_segments(
&self,
path: &makepad_fabric_draft::Path,
offset_by: Point,
rotation: f64,
rect: Rect,
) -> Vec<(DVec2, DVec2)> {
let points = flatten(path, 1.0);
let transformed: Vec<DVec2> = points
.iter()
.map(|point| self.to_screen(place(*point, offset_by, rotation), rect))
.collect();
let mut segments: Vec<_> = transformed
.windows(2)
.map(|pair| (pair[0], pair[1]))
.collect();
if path.closed {
if let (Some(first), Some(last)) = (transformed.first(), transformed.last()) {
segments.push((*last, *first));
}
}
segments
}
}
impl Widget for FabricPatternView {
fn draw_walk(&mut self, cx: &mut Cx2d, _scope: &mut Scope, walk: Walk) -> DrawStep {
let rect = cx.walk_turtle(walk);
self.draw_bg.draw_abs(cx, rect);
cx.push_clip_rect(rect);
self.ensure_nest(rect);
let (Some(pattern), Some(layout)) = (&self.pattern, &self.nested) else {
let message = if self.error.is_empty() {
"pattern preview"
} else {
&self.error
};
self.draw_text.color = Vec4f {
x: 0.52,
y: 0.58,
z: 0.65,
w: 1.0,
};
self.draw_text
.draw_abs(cx, rect.pos + dvec2(14.0, 16.0), message);
cx.pop_clip_rect();
cx.add_aligned_rect_area(&mut self.area, rect);
return DrawStep::done();
};
struct Stroke {
from: DVec2,
to: DVec2,
color: Vec4f,
width: f64,
}
let mut strokes = Vec::new();
let mut labels = Vec::new();
let cut_color = Vec4f {
x: 0.93,
y: 0.96,
z: 0.99,
w: 1.0,
};
let seam_color = Vec4f {
x: 0.40,
y: 0.49,
z: 0.58,
w: 0.72,
};
let mark_color = Vec4f {
x: 1.0,
y: 0.40,
z: 0.18,
w: 0.95,
};
for (part_index, part) in pattern.parts.iter().enumerate() {
let (part_offset, rotation) = self.part_offset(layout, part_index);
let cut_path = offset(&part.outline, part.seam_allowance_mm);
for (from, to) in self.path_segments(&cut_path, part_offset, rotation, rect) {
strokes.push(Stroke {
from,
to,
color: cut_color,
width: 1.25,
});
}
for (from, to) in self.path_segments(&part.outline, part_offset, rotation, rect) {
strokes.push(Stroke {
from,
to,
color: seam_color,
width: 0.75,
});
}
for path in &part.internal {
for (from, to) in self.path_segments(path, part_offset, rotation, rect) {
strokes.push(Stroke {
from,
to,
color: seam_color,
width: 0.75,
});
}
}
for notch in &part.notches {
let at = self.to_screen(place(*notch, part_offset, rotation), rect);
strokes.push(Stroke {
from: at + dvec2(-3.0, -3.0),
to: at + dvec2(3.0, 3.0),
color: mark_color,
width: 1.0,
});
strokes.push(Stroke {
from: at + dvec2(-3.0, 3.0),
to: at + dvec2(3.0, -3.0),
color: mark_color,
width: 1.0,
});
}
let grain_from = self.to_screen(place(part.grainline.0, part_offset, rotation), rect);
let grain_to = self.to_screen(place(part.grainline.1, part_offset, rotation), rect);
strokes.push(Stroke {
from: grain_from,
to: grain_to,
color: mark_color,
width: 1.0,
});
let label_at = part
.labels
.first()
.map(|label| label.at)
.unwrap_or(part.outline.start);
labels.push((
self.to_screen(place(label_at, part_offset, rotation), rect),
format!(
"{} · {}",
part.name,
if part.on_fold {
"cut on fold".to_string()
} else {
format!("cut {}", part.cut_count)
}
),
));
}
self.draw_line.begin_many_instances(cx);
for stroke in strokes {
self.draw_line.color = stroke.color;
self.draw_line
.segment(cx, stroke.from, stroke.to, stroke.width);
}
self.draw_line.end_many_instances(cx);
self.draw_text.color = Vec4f {
x: 0.78,
y: 0.84,
z: 0.90,
w: 1.0,
};
for (at, text) in labels {
self.draw_text.draw_abs(cx, at + dvec2(4.0, -5.0), &text);
}
cx.pop_clip_rect();
cx.add_aligned_rect_area(&mut self.area, rect);
DrawStep::done()
}
fn handle_event(&mut self, cx: &mut Cx, event: &Event, _scope: &mut Scope) {
match event.hits(cx, self.area) {
Hit::FingerDown(event) if event.device.is_primary_hit() => {
self.drag = Some(PatternDrag {
from: event.abs,
pan: self.pan,
});
}
Hit::FingerMove(event) => {
if let Some(drag) = self.drag {
self.pan = drag.pan + event.abs - drag.from;
self.redraw(cx);
}
}
Hit::FingerUp(_) => self.drag = None,
Hit::FingerScroll(event) => {
self.zoom = (self.zoom * (-event.scroll.y * 0.004).exp()).clamp(0.1, 30.0);
self.redraw(cx);
}
Hit::FingerHoverIn(_) | Hit::FingerHoverOver(_) => {
cx.set_cursor(MouseCursor::Grab)
}
_ => {}
}
}
}
fn place(point: Point, offset: Point, rotation_deg: f64) -> DVec2 {
let angle = rotation_deg.to_radians();
let (sin, cos) = angle.sin_cos();
dvec2(
point.x * cos - point.y * sin + offset.x,
point.x * sin + point.y * cos + offset.y,
)
}
fn pattern_bounds(pattern: &Pattern, layout: &PatternLayout) -> Bounds {
let mut bounds = Bounds::default();
for (part_index, part) in pattern.parts.iter().enumerate() {
let (part_offset, rotation) = layout
.placements
.iter()
.find(|placement| placement.part == part_index)
.map(|placement| (placement.offset, placement.rotation_deg))
.unwrap_or((Point::default(), 0.0));
include_part(&mut bounds, part, part_offset, rotation);
}
if !bounds.valid && layout.width_mm > 0.0 && layout.height_mm > 0.0 {
bounds.include(dvec2(0.0, 0.0));
bounds.include(dvec2(layout.width_mm, layout.height_mm));
}
bounds
}
fn include_part(bounds: &mut Bounds, part: &Part, part_offset: Point, rotation: f64) {
let cut = offset(&part.outline, part.seam_allowance_mm);
for point in flatten(&cut, 1.0) {
bounds.include(place(point, part_offset, rotation));
}
}

View file

@ -1,676 +0,0 @@
use crate::{
body_view::{map_measurements_to_vertices, BodyPoseMapping},
camera::CameraMailbox,
};
use makepad_ai_body::model::BodyModel;
use makepad_fabric_measure::{measure, BodyMesh, MeasureOptions, Measured};
use makepad_widgets::image_cache::ImageBuffer;
use makepad_widgets::makepad_platform::thread::SignalToUI;
use std::{
collections::VecDeque,
path::{Path, PathBuf},
sync::{
atomic::{AtomicBool, Ordering},
mpsc::{self, Receiver, Sender},
Arc,
},
thread,
time::{Duration, Instant},
};
const PHOTO_CROP_SIZE: usize = 512;
const LIVE_CROP_SIZE: usize = 384;
const SHAPE_ALPHA: f32 = 0.35;
const POSE_ALPHA: f32 = 0.6;
const SHAPE_RESET_GAP: Duration = Duration::from_secs(1);
const FRAME_TIMEOUT: Duration = Duration::from_secs(2);
const FRAME_POLL: Duration = Duration::from_millis(5);
pub enum PipelineMessage {
Stage(String),
LiveFrame {
fps: f32,
model_ms: f32,
pose_ms: f32,
person: bool,
bbox: Option<[f32; 4]>,
},
Done {
measured: Box<Measured>,
mesh: Arc<BodyMesh>,
posed: Option<Arc<Vec<[f32; 3]>>>,
pose_mapping: BodyPoseMapping,
reset_pose: bool,
},
Failed(String),
}
struct RunRequest {
photo: PathBuf,
weights: PathBuf,
height_cm: Option<f32>,
}
struct LiveRequest {
weights: PathBuf,
height_cm: Option<f32>,
mailbox: CameraMailbox,
}
enum WorkerRequest {
Photo(RunRequest),
Live(LiveRequest),
}
pub struct Pipeline {
request_tx: Sender<WorkerRequest>,
message_rx: Receiver<PipelineMessage>,
live_stop: Arc<AtomicBool>,
}
impl Pipeline {
pub fn new() -> Self {
let (request_tx, request_rx) = mpsc::channel();
let (message_tx, message_rx) = mpsc::channel();
let live_stop = Arc::new(AtomicBool::new(true));
let worker_stop = live_stop.clone();
thread::Builder::new()
.name("fabric-body-pipeline".to_string())
.spawn(move || worker(request_rx, message_tx, worker_stop))
.expect("spawn fabric body worker");
Self {
request_tx,
message_rx,
live_stop,
}
}
pub fn run(
&self,
photo: PathBuf,
weights: PathBuf,
height_cm: Option<f32>,
) -> Result<(), String> {
self.request_tx
.send(WorkerRequest::Photo(RunRequest {
photo,
weights,
height_cm,
}))
.map_err(|_| "the body model worker stopped".to_string())
}
pub fn start_live(
&self,
weights: PathBuf,
height_cm: Option<f32>,
mailbox: CameraMailbox,
) -> Result<(), String> {
self.live_stop.store(false, Ordering::Release);
if self
.request_tx
.send(WorkerRequest::Live(LiveRequest {
weights,
height_cm,
mailbox,
}))
.is_err()
{
self.live_stop.store(true, Ordering::Release);
return Err("the body model worker stopped".to_string());
}
Ok(())
}
pub fn stop_live(&self) {
self.live_stop.store(true, Ordering::Release);
}
pub fn poll(&self) -> Vec<PipelineMessage> {
self.message_rx.try_iter().collect()
}
}
fn emit(sender: &Sender<PipelineMessage>, message: PipelineMessage) -> bool {
if sender.send(message).is_err() {
return false;
}
SignalToUI::set_ui_signal();
true
}
fn worker(
requests: Receiver<WorkerRequest>,
messages: Sender<PipelineMessage>,
live_stop: Arc<AtomicBool>,
) {
let mut loaded: Option<(PathBuf, BodyModel)> = None;
while let Ok(request) = requests.recv() {
let result = match request {
WorkerRequest::Photo(request) => run_one(&request, &messages, &mut loaded),
WorkerRequest::Live(request) => {
run_live(&request, &messages, &mut loaded, &live_stop)
}
};
if let Err(error) = result {
if !emit(&messages, PipelineMessage::Failed(error)) {
return;
}
}
}
}
fn run_one(
request: &RunRequest,
messages: &Sender<PipelineMessage>,
loaded: &mut Option<(PathBuf, BodyModel)>,
) -> Result<(), String> {
emit(messages, PipelineMessage::Stage("decoding photo…".to_string()));
let (rgb, width, height) = decode_rgb(&request.photo)?;
ensure_model(&request.weights, messages, loaded)?;
emit(messages, PipelineMessage::Stage("inferring…".to_string()));
let model = &mut loaded.as_mut().expect("model was loaded above").1;
model
.set_crop_size(PHOTO_CROP_SIZE)
.map_err(|error| format!("could not configure the body model: {error}"))?;
let packet = model
.infer(&rgb, width, height, None)
.map_err(|error| format!("body inference failed: {error}"))?;
let person = packet
.people
.first()
.ok_or_else(|| "no person found".to_string())?;
let posed = Arc::new(posed_vertices(
model,
&person.shape,
&person.expr,
person.mhr,
person.global_rot,
)?);
let vertices = model.rig().rest_vertices(&person.shape, &person.expr);
let face_indices = model
.weights
.i64_shaped("head_pose.faces", &[36_874, 3])
.map_err(|error| format!("could not read the body mesh faces: {error}"))?;
let mesh = Arc::new(body_mesh_from_flat(&vertices, &face_indices)?);
emit(messages, PipelineMessage::Stage("measuring…".to_string()));
let measured = measure(
&mesh,
&MeasureOptions {
height_cm: request.height_cm,
},
)
.map_err(|error| error.to_string())?;
let pose_mapping = map_measurements_to_vertices(&mesh, &measured);
emit(
messages,
PipelineMessage::Done {
measured: Box::new(measured),
mesh,
posed: Some(posed),
pose_mapping,
reset_pose: true,
},
);
Ok(())
}
fn ensure_model(
weights: &Path,
messages: &Sender<PipelineMessage>,
loaded: &mut Option<(PathBuf, BodyModel)>,
) -> Result<(), String> {
if loaded
.as_ref()
.map(|(path, _)| path != weights)
.unwrap_or(true)
{
emit(
messages,
PipelineMessage::Stage("loading model 2.8 GB…".to_string()),
);
let model = BodyModel::load(weights)
.map_err(|error| format!("could not load the body model: {error}"))?;
*loaded = Some((weights.to_path_buf(), model));
}
Ok(())
}
fn run_live(
request: &LiveRequest,
messages: &Sender<PipelineMessage>,
loaded: &mut Option<(PathBuf, BodyModel)>,
stop: &AtomicBool,
) -> Result<(), String> {
if stop.load(Ordering::Acquire) {
return Ok(());
}
ensure_model(&request.weights, messages, loaded)?;
let model = &mut loaded.as_mut().expect("model was loaded above").1;
model
.set_crop_size(LIVE_CROP_SIZE)
.map_err(|error| format!("could not configure live body inference: {error}"))?;
let face_indices = model
.weights
.i64_shaped("head_pose.faces", &[36_874, 3])
.map_err(|error| format!("could not read the body mesh faces: {error}"))?;
let started = Instant::now();
let mut previous_bbox = None;
let mut smoother = ShapeSmoother::default();
let mut pose_smoother = PoseSmoother::default();
let mut fps = FpsCounter::default();
let mut reset_pose = true;
while !stop.load(Ordering::Acquire) {
request.mailbox.request();
let wait_started = Instant::now();
let frame = loop {
if stop.load(Ordering::Acquire) {
return Ok(());
}
if let Some(frame) = request.mailbox.take() {
break Some(frame);
}
if wait_started.elapsed() >= FRAME_TIMEOUT {
break None;
}
thread::sleep(FRAME_POLL);
};
let Some(frame) = frame else {
if !emit(
messages,
PipelineMessage::Stage("no camera frames".to_string()),
) {
return Ok(());
}
continue;
};
let model_started = Instant::now();
let packet = model
.infer(&frame.rgb, frame.width, frame.height, previous_bbox)
.map_err(|error| format!("live body inference failed: {error}"))?;
let model_ms = model_started.elapsed().as_secs_f32() * 1000.0;
let now = started.elapsed();
let person = packet.people.first();
let bbox = person.map(|person| person.bbox);
let mut pose_ms = 0.0;
let mut posed = None;
let smoothed_shape = if let Some(person) = person {
previous_bbox = Some(expand_bbox(
person.bbox,
frame.width,
frame.height,
0.15,
));
let (mhr, global_rot) = pose_smoother.observe(person.mhr, person.global_rot);
let pose_started = Instant::now();
posed = Some(Arc::new(posed_vertices(
model,
&person.shape,
&person.expr,
mhr,
global_rot,
)?));
pose_ms = pose_started.elapsed().as_secs_f32() * 1000.0;
Some(smoother.observe_person(person.shape, now))
} else {
previous_bbox = None;
smoother.observe_miss(now);
pose_smoother.reset();
None
};
let current_fps = fps.tick(now);
if !emit(
messages,
PipelineMessage::LiveFrame {
fps: current_fps,
model_ms,
pose_ms,
person: person.is_some(),
bbox,
},
) {
return Ok(());
}
if stop.load(Ordering::Acquire) {
return Ok(());
}
let Some(shape) = smoothed_shape else {
continue;
};
let expression = [0.0f32; 72];
let vertices = model.rig().rest_vertices(&shape, &expression);
let mesh = Arc::new(body_mesh_from_flat(&vertices, &face_indices)?);
let measured = measure(
&mesh,
&MeasureOptions {
height_cm: request.height_cm,
},
)
.map_err(|error| error.to_string())?;
let pose_mapping = map_measurements_to_vertices(&mesh, &measured);
if !emit(
messages,
PipelineMessage::Done {
measured: Box::new(measured),
mesh,
posed,
pose_mapping,
reset_pose,
},
) {
return Ok(());
}
reset_pose = false;
}
Ok(())
}
#[derive(Default)]
struct PoseSmoother {
mhr: Option<[f32; 204]>,
global_rot: Option<[f32; 3]>,
}
impl PoseSmoother {
fn observe(
&mut self,
mhr: [f32; 204],
global_rot: [f32; 3],
) -> ([f32; 204], [f32; 3]) {
let global_rot = match self.global_rot {
Some(previous) => std::array::from_fn(|index| {
previous[index] + POSE_ALPHA * (global_rot[index] - previous[index])
}),
None => global_rot,
};
let mut mhr = match self.mhr {
Some(previous) => std::array::from_fn(|index| {
previous[index] + POSE_ALPHA * (mhr[index] - previous[index])
}),
None => mhr,
};
// The packet's 204 values are exactly the rig's [pose 136 | scales 68]
// input. Global rotation is pose slots 3..6; keep the separately
// smoothed copy authoritative before MhrRig::forward pads to 249.
mhr[3..6].copy_from_slice(&global_rot);
self.mhr = Some(mhr);
self.global_rot = Some(global_rot);
(mhr, global_rot)
}
fn reset(&mut self) {
self.mhr = None;
self.global_rot = None;
}
}
#[derive(Default)]
struct ShapeSmoother {
shape: Option<[f32; 45]>,
last_person: Option<Duration>,
}
impl ShapeSmoother {
fn observe_person(&mut self, shape: [f32; 45], now: Duration) -> [f32; 45] {
if self
.last_person
.is_some_and(|last| now.saturating_sub(last) > SHAPE_RESET_GAP)
{
self.shape = None;
}
self.last_person = Some(now);
let smoothed = match self.shape {
Some(previous) => std::array::from_fn(|index| {
previous[index] + SHAPE_ALPHA * (shape[index] - previous[index])
}),
None => shape,
};
self.shape = Some(smoothed);
smoothed
}
fn observe_miss(&mut self, now: Duration) {
if self
.last_person
.is_some_and(|last| now.saturating_sub(last) > SHAPE_RESET_GAP)
{
self.shape = None;
self.last_person = None;
}
}
}
#[derive(Default)]
struct FpsCounter {
samples: VecDeque<Duration>,
}
impl FpsCounter {
fn tick(&mut self, now: Duration) -> f32 {
self.samples.push_back(now);
while self.samples.len() > 10 {
self.samples.pop_front();
}
let Some(first) = self.samples.front().copied() else {
return 0.0;
};
let seconds = now.saturating_sub(first).as_secs_f32();
if seconds <= f32::EPSILON {
0.0
} else {
(self.samples.len().saturating_sub(1)) as f32 / seconds
}
}
}
pub(crate) fn expand_bbox(
bbox: [f32; 4],
width: u32,
height: u32,
amount: f32,
) -> [f32; 4] {
let box_width = (bbox[2] - bbox[0]).max(0.0);
let box_height = (bbox[3] - bbox[1]).max(0.0);
[
(bbox[0] - box_width * amount).clamp(0.0, width as f32),
(bbox[1] - box_height * amount).clamp(0.0, height as f32),
(bbox[2] + box_width * amount).clamp(0.0, width as f32),
(bbox[3] + box_height * amount).clamp(0.0, height as f32),
]
}
fn posed_vertices(
model: &BodyModel,
shape: &[f32; 45],
expression: &[f32; 72],
mut mhr: [f32; 204],
global_rot: [f32; 3],
) -> Result<Vec<[f32; 3]>, String> {
// BodyPerson::mhr is already model_params(): pose 0..136 followed by
// scales 136..204. The root translation remains zero and the model's
// global rotation occupies 3..6. forward() pads the 45 identity slots
// to the rig's 249-wide internal vector, then returns rig-space cm.
mhr[3..6].copy_from_slice(&global_rot);
let output = model.rig().forward(shape, &mhr, expression, true);
vertices_from_flat(&output.verts)
}
fn decode_rgb(path: &Path) -> Result<(Vec<u8>, u32, u32), String> {
let bytes = std::fs::read(path)
.map_err(|error| format!("could not read {}: {error}", path.display()))?;
let extension = path
.extension()
.and_then(|extension| extension.to_str())
.unwrap_or_default()
.to_ascii_lowercase();
let image = match extension.as_str() {
"jpg" | "jpeg" => ImageBuffer::from_jpg(&bytes),
"png" => ImageBuffer::from_png(&bytes),
_ => return Err("choose a JPG or PNG photo".to_string()),
}
.map_err(|error| format!("could not decode {}: {error}", path.display()))?;
let pixel_count = image
.width
.checked_mul(image.height)
.ok_or_else(|| "photo dimensions are too large".to_string())?;
if image.data.len() < pixel_count {
return Err("decoded photo has too few pixels".to_string());
}
let width = u32::try_from(image.width).map_err(|_| "photo is too wide".to_string())?;
let height = u32::try_from(image.height).map_err(|_| "photo is too tall".to_string())?;
Ok((argb_to_rgb(&image.data[..pixel_count]), width, height))
}
pub(crate) fn argb_to_rgb(pixels: &[u32]) -> Vec<u8> {
let mut rgb = Vec::with_capacity(pixels.len() * 3);
for pixel in pixels {
rgb.push((pixel >> 16) as u8);
rgb.push((pixel >> 8) as u8);
rgb.push(*pixel as u8);
}
rgb
}
pub(crate) fn faces_i64_to_u32(
values: &[i64],
vertex_count: usize,
) -> Result<Vec<[u32; 3]>, String> {
if values.len() % 3 != 0 {
return Err("body face index buffer is not made of triangles".to_string());
}
values
.chunks_exact(3)
.enumerate()
.map(|(face_index, triangle)| {
let mut face = [0; 3];
for corner in 0..3 {
let index = usize::try_from(triangle[corner]).map_err(|_| {
format!("body face {face_index} contains a negative vertex index")
})?;
if index >= vertex_count {
return Err(format!(
"body face {face_index} references vertex {index}, but there are {vertex_count} vertices"
));
}
face[corner] = u32::try_from(index)
.map_err(|_| format!("body vertex index {index} exceeds u32"))?;
}
Ok(face)
})
.collect()
}
pub(crate) fn body_mesh_from_flat(
vertices: &[f32],
face_indices: &[i64],
) -> Result<BodyMesh, String> {
let vertices = vertices_from_flat(vertices)?;
let faces = faces_i64_to_u32(face_indices, vertices.len())?;
Ok(BodyMesh {
vertices,
faces,
landmarks: None,
})
}
fn vertices_from_flat(vertices: &[f32]) -> Result<Vec<[f32; 3]>, String> {
if vertices.is_empty() || vertices.len() % 3 != 0 {
return Err("body vertex buffer is empty or malformed".to_string());
}
Ok(vertices
.chunks_exact(3)
.map(|point| [point[0], point[1], point[2]])
.collect())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn converts_aarrggbb_to_rgb() {
assert_eq!(
argb_to_rgb(&[0xff_12_34_56, 0x00_ab_cd_ef]),
vec![0x12, 0x34, 0x56, 0xab, 0xcd, 0xef]
);
}
#[test]
fn builds_a_tiny_checked_mesh() {
let mesh = body_mesh_from_flat(
&[0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0],
&[0, 1, 2],
)
.unwrap();
assert_eq!(mesh.vertices.len(), 3);
assert_eq!(mesh.faces, vec![[0, 1, 2]]);
assert!(faces_i64_to_u32(&[0, 1, 3], 3).is_err());
assert!(faces_i64_to_u32(&[-1, 1, 2], 3).is_err());
}
#[test]
fn live_shape_ema_converges_and_resets_after_a_gap() {
let mut smoother = ShapeSmoother::default();
assert_eq!(
smoother.observe_person([0.0; 45], Duration::ZERO),
[0.0; 45]
);
let first = smoother.observe_person([10.0; 45], Duration::from_millis(250));
assert!((first[0] - 3.5).abs() < 0.0001);
let second = smoother.observe_person([10.0; 45], Duration::from_millis(500));
assert!((second[0] - 5.775).abs() < 0.0001);
smoother.observe_miss(Duration::from_millis(1_501));
let reset = smoother.observe_person([8.0; 45], Duration::from_millis(1_750));
assert_eq!(reset, [8.0; 45]);
}
#[test]
fn live_pose_ema_smooths_parameters_and_keeps_global_slots_in_sync() {
let mut smoother = PoseSmoother::default();
let mut initial_mhr = [0.0; 204];
initial_mhr[20] = 2.0;
let (initial, initial_rot) = smoother.observe(initial_mhr, [1.0, 2.0, 3.0]);
assert_eq!(initial[3..6], initial_rot);
let mut next_mhr = [10.0; 204];
next_mhr[20] = 12.0;
let (smoothed, smoothed_rot) = smoother.observe(next_mhr, [3.0, 4.0, 5.0]);
assert!((smoothed[20] - 8.0).abs() < 0.0001);
assert_eq!(smoothed_rot, [2.2, 3.2, 4.2]);
assert_eq!(smoothed[3..6], smoothed_rot);
smoother.reset();
let (reset, _) = smoother.observe([7.0; 204], [0.5, 1.0, 1.5]);
assert_eq!(reset[20], 7.0);
}
#[test]
fn bbox_expansion_is_fifteen_percent_and_clamped() {
let expanded = expand_bbox([100.0, 50.0, 300.0, 250.0], 640, 360, 0.15);
for (actual, expected) in expanded.into_iter().zip([70.0, 20.0, 330.0, 280.0]) {
assert!((actual - expected).abs() < 0.0001, "{expanded:?}");
}
assert_eq!(
expand_bbox([5.0, 10.0, 635.0, 350.0], 640, 360, 0.15),
[0.0, 0.0, 640.0, 360.0]
);
}
#[test]
fn fps_counter_tracks_recent_frame_cadence() {
let mut counter = FpsCounter::default();
assert_eq!(counter.tick(Duration::ZERO), 0.0);
for quarter in 1..=4 {
let fps = counter.tick(Duration::from_millis(quarter * 250));
assert!((fps - 4.0).abs() < 0.0001, "{fps}");
}
}
}

View file

@ -1,25 +0,0 @@
[package]
name = "makepad-files"
version = "0.1.0"
edition = "2021"
[[bin]]
name = "files"
path = "src/main.rs"
[features]
default = ["chat"]
chat = ["dep:makepad-ai-hub"]
demo = []
[dependencies]
makepad-widgets = { path = "../../widgets" }
makepad-wm-theme = { path = "../../libs/wm_theme" }
makepad-wm-api = { path = "../../libs/wm_api" }
# The AI services wire: bounded read and confirmed mutation tools on the bus
# (hosted port, id-correlated runner — see ai_service.rs).
makepad-ai-services = { path = "../../libs/ai/services" }
makepad-strict-json = { path = "../../libs/strict_json" }
# The ask panel's local model: an in-process Qwen GGUF on makepad-ggml, loaded
# only when the panel is first opened.
makepad-ai-hub = { path = "../../libs/ai/hub", default-features = false, features = ["llm"], optional = true }

View file

@ -1,64 +0,0 @@
#[path = "../../libs/ai/models/paint/src/png.rs"]
#[allow(dead_code)]
mod png;
use std::{env, fs, path::Path};
const WIDTH: u32 = 256;
const HEIGHT: u32 = 128;
const PALETTES: [(&str, [u8; 3], [u8; 3], [u8; 3]); 6] = [
("aurora-vignette.png", [28, 31, 78], [69, 175, 170], [227, 151, 232]),
("canyon-vignette.png", [86, 35, 38], [221, 119, 70], [255, 211, 128]),
("lagoon-vignette.png", [12, 55, 77], [28, 154, 166], [151, 232, 207]),
("meadow-vignette.png", [36, 68, 45], [124, 167, 79], [234, 213, 126]),
("twilight-vignette.png", [36, 25, 65], [103, 71, 141], [238, 144, 116]),
("cinema-still.png", [19, 24, 39], [56, 75, 105], [235, 176, 91]),
];
fn mix(a: u8, b: u8, amount: u32) -> u32 {
(u32::from(a) * (255 - amount) + u32::from(b) * amount) / 255
}
fn picture(top: [u8; 3], bottom: [u8; 3], accent: [u8; 3], seed: u32) -> Vec<u8> {
let mut pixels = Vec::with_capacity((WIDTH * HEIGHT * 3) as usize);
let glow_x = 36 + seed * 31;
let glow_y = 22 + (seed * 17) % 54;
for y in 0..HEIGHT {
for x in 0..WIDTH {
let blend = (y * 190 / (HEIGHT - 1) + x * 65 / (WIDTH - 1)).min(255);
let edge_x = (x as i32 * 2 - (WIDTH - 1) as i32).unsigned_abs();
let edge_y = (y as i32 * 2 - (HEIGHT - 1) as i32).unsigned_abs();
let vignette = edge_x * edge_x * 18 / ((WIDTH - 1) * (WIDTH - 1))
+ edge_y * edge_y * 24 / ((HEIGHT - 1) * (HEIGHT - 1));
let glow_dx = (x as i32 - glow_x as i32).unsigned_abs();
let glow_dy = (y as i32 - glow_y as i32).unsigned_abs();
let glow = 72u32.saturating_sub(glow_dx * glow_dx / 360 + glow_dy * glow_dy / 90);
let texture = ((x * 17 + y * 31 + seed * 43) & 7) as i32 - 3;
for channel in 0..3 {
let base = mix(top[channel], bottom[channel], blend);
let lit = (base * (72 - glow) + u32::from(accent[channel]) * glow) / 72;
pixels.push((lit as i32 - vignette as i32 + texture).clamp(0, 255) as u8);
}
}
}
pixels
}
fn write_if_changed(path: &Path, bytes: &[u8]) {
if fs::read(path).ok().as_deref() != Some(bytes) {
fs::write(path, bytes).expect("write generated files demo picture");
}
}
fn main() {
println!("cargo:rerun-if-changed=../../libs/ai/models/paint/src/png.rs");
println!("cargo:rerun-if-changed=build.rs");
let out_dir = env::var_os("OUT_DIR").expect("Cargo did not set OUT_DIR");
for (seed, (name, top, bottom, accent)) in PALETTES.iter().copied().enumerate() {
let pixels = picture(top, bottom, accent, seed as u32);
let encoded = png::encode_png(WIDTH, HEIGHT, png::PngColor::Rgb, &pixels);
write_if_changed(&Path::new(&out_dir).join(name), &encoded);
}
}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 307 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 374 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 143 KiB

View file

@ -1,250 +0,0 @@
//! The file browser on the desktop's AI bus.
//!
//! Hosted by the window manager, the app opens one [`AiServicePort`] with
//! the manifest from `chat_tools::service_manifest` and answers the calls
//! that come back through it. The tools are the same seven the app's own
//! panel has; what is new is how they run: every call carries the
//! engine's `call_id`, the answer carries it back, and the person (or the
//! router) can give up on a call mid-walk. The old panel's runner is
//! order-only and cannot do either, so this one sits beside it rather
//! than inside it, and the two never share a job.
//!
//! One worker thread, one job at a time. A job in flight is cancelled by
//! a flag the walk checks on every entry; a job still queued behind it is
//! cancelled before it starts. Progress from the walk and the finished
//! results come back on channels the UI drains on its signal.
use std::{
collections::HashMap,
path::PathBuf,
sync::{
atomic::{AtomicBool, Ordering},
mpsc::{channel, Receiver, Sender},
Arc,
},
};
use makepad_ai_services::wire::{ServiceCall, ToolResult};
use makepad_strict_json as json;
use makepad_widgets::makepad_platform::thread::{SignalToUI, ThreadOptions, ThreadSpawner};
use crate::chat_tools::{run_with, ToolJob, ToolOutcome};
#[cfg(test)]
use std::thread;
/// One call on its way to the worker.
struct ServiceJob {
call_id: String,
job: ToolJob,
cancel: Arc<AtomicBool>,
}
/// What the worker sends back, in the order it happened.
pub enum ServiceReply {
Result { result: ToolResult, mutated: bool },
Progress { call_id: String, note: String, permille: u16 },
}
/// The bus's tool worker: correlated by call id, cancellable.
pub struct ServiceRunner {
jobs: Sender<ServiceJob>,
replies: Receiver<ServiceReply>,
/// The cancel flag of every call not yet answered.
live: HashMap<String, Arc<AtomicBool>>,
}
impl ServiceRunner {
pub fn new(spawner: &ThreadSpawner) -> Self {
let (jobs, job_rx) = channel::<ServiceJob>();
let (reply_tx, replies) = channel::<ServiceReply>();
if let Ok(handle) = spawner.spawn_worker(
ThreadOptions { name: Some("files-ai-service".into()), ..Default::default() },
move || {
while let Ok(ServiceJob { call_id, job, cancel }) = job_rx.recv() {
let (result, mutated) = if cancel.load(Ordering::Relaxed) {
// Given up on while it waited its turn: never started.
(ToolResult::cancelled(&call_id), false)
} else {
let progress_tx = reply_tx.clone();
let progress_id = call_id.clone();
let progress = move |permille: u16| {
let _ = progress_tx.send(ServiceReply::Progress {
call_id: progress_id.clone(),
note: "measuring…".to_string(),
permille,
});
SignalToUI::set_ui_signal();
};
let outcome = run_with(&job, &cancel, &progress);
if cancel.load(Ordering::Relaxed) {
// The walk stopped early on the flag: what it has
// is a floor nobody asked for any more.
(ToolResult::cancelled(&call_id), false)
} else {
let mutated = outcome.mutated;
(result_for(&call_id, outcome), mutated)
}
};
if reply_tx.send(ServiceReply::Result { result, mutated }).is_err() {
return;
}
SignalToUI::set_ui_signal();
}
},
) {
handle.detach();
}
Self { jobs, replies, live: HashMap::new() }
}
/// Queue one call. `cwd` is the folder the person is looking at — what
/// a relative path is read from — and `home` the jail.
pub fn submit(&mut self, call: &ServiceCall, cwd: PathBuf, home: PathBuf) {
let cancel = Arc::new(AtomicBool::new(false));
self.live.insert(call.call_id.clone(), cancel.clone());
let job = ToolJob {
name: call.tool.clone(),
args: flat_args(&call.args),
cwd,
home,
};
let _ = self.jobs.send(ServiceJob { call_id: call.call_id.clone(), job, cancel });
}
/// Give up on one call, running or queued. Unknown ids are nothing.
pub fn cancel(&mut self, call_id: &str) {
if let Some(flag) = self.live.get(call_id) {
flag.store(true, Ordering::Relaxed);
}
}
/// Everything the worker sent since the last drain. A result retires
/// its call's flag.
pub fn drain(&mut self) -> Vec<ServiceReply> {
let replies: Vec<ServiceReply> = self.replies.try_iter().collect();
for reply in &replies {
if let ServiceReply::Result { result, .. } = reply {
self.live.remove(&result.call_id);
}
}
replies
}
}
/// A tool's outcome as the wire says it. The tools already sort their own
/// failures into "refused" (the jail, an unknown name) and "could not"
/// (the disk said no); the wire keeps that distinction.
fn result_for(call_id: &str, outcome: ToolOutcome) -> ToolResult {
if !outcome.is_error {
return ToolResult::ok(call_id, outcome.text, outcome.note);
}
if outcome.note.starts_with("refused") || outcome.note.starts_with("unknown tool") {
ToolResult::refused(call_id, outcome.text)
} else {
ToolResult::failed(call_id, outcome.text)
}
}
/// The call's JSON argument object as the tools read it: one string per
/// key. Numbers and booleans become their text; nested values are dropped,
/// since no tool here takes one.
pub fn flat_args(args: &str) -> Vec<(String, String)> {
let Ok(json::Value::Obj(fields)) = json::parse(args.as_bytes()) else {
return Vec::new();
};
fields
.into_iter()
.filter_map(|(key, value)| {
let text = match value {
json::Value::Str(s) => s,
json::Value::Int(i) => i.to_string(),
json::Value::F64(f) => f.to_string(),
json::Value::Bool(b) => b.to_string(),
_ => return None,
};
Some((key, text))
})
.collect()
}
#[cfg(test)]
// Native worker tests use std deadlines while exercising blocking threads.
#[allow(clippy::disallowed_types, clippy::disallowed_methods)]
mod tests {
use super::*;
use makepad_ai_services::wire::ToolOutcome as Outcome;
use std::time::{Duration, Instant};
fn call(id: &str, tool: &str, args: &str) -> ServiceCall {
ServiceCall { call_id: id.into(), tool: tool.into(), args: args.into() }
}
fn wait_for(runner: &mut ServiceRunner, n: usize) -> Vec<ToolResult> {
let deadline = Instant::now() + Duration::from_secs(20);
let mut out = Vec::new();
while out.len() < n && Instant::now() < deadline {
for reply in runner.drain() {
if let ServiceReply::Result { result: r, .. } = reply {
out.push(r);
}
}
thread::sleep(Duration::from_millis(5));
}
out
}
#[test]
fn json_args_become_the_tools_flat_pairs() {
let args = flat_args(r#"{"path":"~/Downloads","top":3,"deep":{"x":1},"quick":true}"#);
assert_eq!(
args,
vec![
("path".to_string(), "~/Downloads".to_string()),
("top".to_string(), "3".to_string()),
("quick".to_string(), "true".to_string()),
]
);
assert!(flat_args("not json").is_empty());
assert!(flat_args("[1,2]").is_empty());
}
#[test]
fn results_carry_their_own_call_ids_and_a_queued_cancel_never_runs() {
let home = crate::model::home_dir();
let spawner = makepad_widgets::Cx::new(Box::new(|_, _| {})).thread_spawner();
let mut runner = ServiceRunner::new(&spawner);
// An unknown tool is refused; a jail escape is refused; both keep
// their ids whatever order the worker answers in.
runner.submit(&call("a", "stat", r#"{"path":"/etc/passwd"}"#), home.clone(), home.clone());
runner.submit(&call("b", "rm_rf", r#"{"path":"~"}"#), home.clone(), home.clone());
runner.submit(&call("c", "stat", r#"{"path":"~"}"#), home.clone(), home.clone());
// Cancelled while it is still queued: it must come back cancelled
// without the tool ever running.
runner.cancel("c");
assert_eq!(runner.live.len(), 3);
let results = wait_for(&mut runner, 3);
assert_eq!(results.len(), 3);
let by_id = |id: &str| results.iter().find(|r| r.call_id == id).unwrap();
assert_eq!(by_id("a").outcome, Outcome::Refused);
assert!(by_id("a").text.contains("refused"));
assert_eq!(by_id("b").outcome, Outcome::Refused);
assert!(by_id("b").text.contains("no tool called"));
assert_eq!(by_id("c").outcome, Outcome::Cancelled);
assert_eq!(runner.live.len(), 0);
// An unknown id is nothing.
runner.cancel("zzz");
}
#[test]
fn a_folder_walk_stops_on_the_flag() {
let cancel = AtomicBool::new(true);
let started = Instant::now();
let (bytes, files, complete) = crate::chat_tools::measure_for_test(
std::path::Path::new(env!("CARGO_MANIFEST_DIR")),
&cancel,
);
assert_eq!((bytes, files, complete), (0, 0, false));
assert!(started.elapsed() < Duration::from_secs(1));
}
}

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -1,509 +0,0 @@
//! The filesystem seam.
//!
//! Every place the browser touches "the filesystem" goes through one [`Vfs`]:
//! listing a folder, measuring it, mapping it for the treemap, opening a file
//! in a viewer, and every operation that changes something. There are two
//! implementations — [`RealVfs`], which is `std::fs` and the app's normal life,
//! and the demo one, which is a plausible home held in memory so a screen
//! recording can show the whole app without showing anybody's real disk.
//!
//! A process has exactly one filesystem for its whole life, so the choice is
//! installed once at startup and read from anywhere, worker threads included.
//! Threading a handle through every signature would buy nothing: no part of
//! this app ever wants a *different* filesystem than the rest of it.
//!
//! Virtual files are also statted and read through this seam. `native_path`
//! is reserved for native integrations backed by [`RealVfs`]; the closed
//! demo returns typed [`VfsError::Unavailable`] instead of mapping a virtual
//! name onto the host disk.
use std::{
path::{Path, PathBuf},
sync::{atomic::AtomicBool, Arc, OnceLock},
};
use makepad_widgets::makepad_platform::thread::TaskPool;
use crate::{
model::{self, FileEntry},
ops::{OpKind, OpRequest, Undo},
sizecache::Cached,
treemap::{self, Node, ScanProgress, ScanRules, ScanStep},
};
/// A capability the active filesystem deliberately does not provide.
///
/// Virtual paths have no honest host path or native cache file. Keeping that
/// answer typed makes an accidentally reached native integration fail closed
/// instead of turning the virtual path into a host path and reaching the
/// platform's unsupported-filesystem trap.
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum VfsError {
Unavailable(&'static str),
Io(String),
}
impl std::fmt::Display for VfsError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
VfsError::Unavailable(capability) => write!(f, "{capability} is unavailable"),
VfsError::Io(message) => f.write_str(message),
}
}
}
impl std::error::Error for VfsError {}
/// What an operation did, once it is done: the sentence for the status bar,
/// how to reverse it, and the paths worth selecting afterwards.
pub struct OpOutcome {
pub message: String,
pub undo: Option<Undo>,
pub touched: Vec<PathBuf>,
}
/// The filesystem the browser is looking at.
pub trait Vfs: Send + Sync {
/// The folder a fresh window opens in.
fn home(&self) -> PathBuf;
/// The filesystem's wall clock, in seconds since the epoch. Demo
/// filesystems pin this to the same instant as their generated dates.
fn now_secs(&self) -> u64;
/// One directory listing, sorted the way [`model::read_directory`] sorts.
/// Real disks are dispatched to a worker; instant backends run inline.
fn read_dir(&self, path: &Path, show_hidden: bool) -> Result<Vec<FileEntry>, String>;
/// Metadata for one path, including paths below the current listing.
fn stat(&self, path: &Path) -> Result<FileEntry, String>;
/// At most `max` bytes of a file. Consumers must not assume the host has
/// a corresponding path.
fn read_bytes(&self, path: &Path, max: usize) -> Result<Vec<u8>, String>;
fn is_dir(&self, path: &Path) -> bool;
/// Whether the filesystem has anything at this path at all. The default
/// asks the parent folder for its listing, which is the only question a
/// virtual filesystem can always answer; a real one knows directly.
fn exists(&self, path: &Path) -> bool {
if self.is_dir(path) {
return true;
}
let Some(parent) = path.parent() else {
return false;
};
self.read_dir(parent, true)
.map(|entries| entries.iter().any(|e| e.path == path))
.unwrap_or(false)
}
/// Create a directory path. Callers validate collisions before this
/// reaches the backend; recursive creation also lets Trash bootstrap its
/// platform-specific parent folders.
fn mkdir(&self, path: &Path) -> Result<(), String>;
/// Move or rename one path without replacing an existing target.
fn rename(&self, source: &Path, target: &Path) -> Result<(), String>;
/// The real file on disk behind a path, for native integrations that
/// cannot consume bytes. A virtual filesystem returns typed Unavailable.
fn native_path(&self, _path: &Path) -> Result<PathBuf, VfsError> {
Err(VfsError::Unavailable("native filesystem path"))
}
/// Unix permission bits for the properties panel. Other backends have no
/// inode mode to report.
fn unix_mode(&self, _path: &Path) -> Result<u32, VfsError> {
Err(VfsError::Unavailable("Unix file mode"))
}
/// Resolve links and `..` components for callers that enforce a path
/// boundary. Virtual filesystems may return the already-normalized path.
fn canonicalize(&self, _path: &Path) -> Result<PathBuf, VfsError> {
Err(VfsError::Unavailable("path canonicalization"))
}
/// Whether a path is a link without following it.
fn is_symlink(&self, _path: &Path) -> Result<bool, VfsError> {
Err(VfsError::Unavailable("symbolic-link metadata"))
}
/// The native size-map cache. Virtual filesystems have no cache file;
/// their scans are already instant.
fn load_scan_cache(&self, _root: &Path) -> Result<Option<Cached>, VfsError> {
Err(VfsError::Unavailable("native size-map cache"))
}
fn store_scan_cache(&self, _root: &Path, _bytes: &[u8]) -> Result<(), VfsError> {
Err(VfsError::Unavailable("native size-map cache"))
}
fn forget_scan_cache(&self, _root: &Path) -> Result<(), VfsError> {
Err(VfsError::Unavailable("native size-map cache"))
}
/// Recursive byte total, for the properties panel. Stops when cancelled.
fn total_bytes(&self, path: &Path, cancel: &AtomicBool) -> u64;
/// The tree the treemap draws.
fn scan(
&self,
root: &Path,
cancel: &AtomicBool,
progress: &dyn Fn(ScanProgress),
) -> Option<Node>;
/// The same tree, streamed back through `sink` as it is discovered, so a
/// map of a full disk is drawable after one `read_dir` instead of after
/// the whole walk. Returns false when the walk was cancelled.
///
/// The default hands the finished tree over in one step, which is exactly
/// right for a filesystem that answers instantly — there is nothing to
/// stream when there is nothing to wait for. A real disk overrides it.
fn scan_stream(
&self,
root: &Path,
cancel: &AtomicBool,
sink: &(dyn Fn(ScanStep) + Sync),
_pool: &TaskPool,
) -> bool {
match self.scan(root, cancel, &|_| {}) {
Some(node) => {
sink(ScanStep::Opened {
at: Vec::new(),
children: node.children,
denied: false,
});
true
}
None => false,
}
}
/// Perform an operation *synchronously*. Only a filesystem that can do so
/// in no time at all implements this — see [`Vfs::is_instant`]; the real
/// one hands its work to the operations engine's worker instead.
fn perform(&self, request: &OpRequest) -> Result<OpOutcome, String>;
/// Reverse a finished operation, synchronously. Same rule as `perform`.
fn perform_undo(&self, undo: &Undo) -> Result<OpOutcome, String>;
/// True when operations finish instantly and need no worker thread and no
/// progress row — which is exactly what an in-memory tree is.
fn is_instant(&self) -> bool;
/// True when this is not the user's real disk, so the window can say so.
fn is_demo(&self) -> bool {
self.is_instant()
}
}
/// `std::fs` — the app's normal life. Everything here delegates to the
/// modules that already own the behaviour, so there is exactly one
/// implementation of each rule.
pub struct RealVfs;
impl Vfs for RealVfs {
fn home(&self) -> PathBuf {
model::home_dir()
}
fn now_secs(&self) -> u64 {
model::real_now_secs()
}
fn read_dir(&self, path: &Path, show_hidden: bool) -> Result<Vec<FileEntry>, String> {
model::read_directory(path, show_hidden)
}
fn stat(&self, path: &Path) -> Result<FileEntry, String> {
model::real_entry_at(path).ok_or_else(|| format!("No such file: {}", path.display()))
}
fn read_bytes(&self, path: &Path, max: usize) -> Result<Vec<u8>, String> {
use std::io::Read;
let file = std::fs::File::open(path)
.map_err(|error| format!("Could not read {}: {error}", path.display()))?;
let mut data = Vec::with_capacity(max.min(64 * 1024));
file.take(max as u64)
.read_to_end(&mut data)
.map_err(|error| format!("Could not read {}: {error}", path.display()))?;
Ok(data)
}
fn is_dir(&self, path: &Path) -> bool {
path.is_dir()
}
fn exists(&self, path: &Path) -> bool {
path.exists()
}
fn mkdir(&self, path: &Path) -> Result<(), String> {
std::fs::create_dir_all(path)
.map_err(|error| format!("Could not create {}: {error}", path.display()))
}
fn rename(&self, source: &Path, target: &Path) -> Result<(), String> {
if target.exists() {
return Err(format!("{} already exists", target.display()));
}
crate::ops::move_path(source, target, &AtomicBool::new(false), &|_| {})
.map_err(|error| format!("Could not move {}: {error}", source.display()))
}
fn native_path(&self, path: &Path) -> Result<PathBuf, VfsError> {
#[cfg(not(target_arch = "wasm32"))]
{
Ok(path.to_path_buf())
}
#[cfg(target_arch = "wasm32")]
{
let _ = path;
Err(VfsError::Unavailable("native filesystem path"))
}
}
fn unix_mode(&self, path: &Path) -> Result<u32, VfsError> {
#[cfg(all(unix, not(target_arch = "wasm32")))]
{
use std::os::unix::fs::PermissionsExt;
std::fs::metadata(path)
.map(|meta| meta.permissions().mode() & 0o7777)
.map_err(|error| VfsError::Io(error.to_string()))
}
#[cfg(any(not(unix), target_arch = "wasm32"))]
{
let _ = path;
Err(VfsError::Unavailable("Unix file mode"))
}
}
fn canonicalize(&self, path: &Path) -> Result<PathBuf, VfsError> {
#[cfg(not(target_arch = "wasm32"))]
{
std::fs::canonicalize(path).map_err(|error| VfsError::Io(error.to_string()))
}
#[cfg(target_arch = "wasm32")]
{
let _ = path;
Err(VfsError::Unavailable("path canonicalization"))
}
}
fn is_symlink(&self, path: &Path) -> Result<bool, VfsError> {
#[cfg(not(target_arch = "wasm32"))]
{
std::fs::symlink_metadata(path)
.map(|metadata| metadata.file_type().is_symlink())
.map_err(|error| VfsError::Io(error.to_string()))
}
#[cfg(target_arch = "wasm32")]
{
let _ = path;
Err(VfsError::Unavailable("symbolic-link metadata"))
}
}
fn load_scan_cache(&self, root: &Path) -> Result<Option<Cached>, VfsError> {
#[cfg(not(target_arch = "wasm32"))]
{
Ok(crate::sizecache::load(root))
}
#[cfg(target_arch = "wasm32")]
{
let _ = root;
Err(VfsError::Unavailable("native size-map cache"))
}
}
fn store_scan_cache(&self, root: &Path, bytes: &[u8]) -> Result<(), VfsError> {
#[cfg(not(target_arch = "wasm32"))]
{
crate::sizecache::store(root, bytes);
Ok(())
}
#[cfg(target_arch = "wasm32")]
{
let _ = (root, bytes);
Err(VfsError::Unavailable("native size-map cache"))
}
}
fn forget_scan_cache(&self, root: &Path) -> Result<(), VfsError> {
#[cfg(not(target_arch = "wasm32"))]
{
crate::sizecache::forget(root);
Ok(())
}
#[cfg(target_arch = "wasm32")]
{
let _ = root;
Err(VfsError::Unavailable("native size-map cache"))
}
}
fn total_bytes(&self, path: &Path, cancel: &AtomicBool) -> u64 {
crate::ops::total_bytes(path, cancel)
}
fn scan(
&self,
root: &Path,
cancel: &AtomicBool,
progress: &dyn Fn(ScanProgress),
) -> Option<Node> {
let classify = |p: &Path, is_dir: bool| model::kind_for(p, is_dir) as u8;
let home = self.home();
let skip = |path: &Path| model::skip_for_scan(path, &home);
treemap::scan(root, &treemap::ScanRules { classify: &classify, skip: &skip }, cancel, progress)
}
fn scan_stream(
&self,
root: &Path,
cancel: &AtomicBool,
sink: &(dyn Fn(ScanStep) + Sync),
pool: &TaskPool,
) -> bool {
let classify = |p: &Path, is_dir: bool| model::kind_for(p, is_dir) as u8;
let home = self.home();
let skip = |path: &Path| model::skip_for_scan(path, &home);
let rules = ScanRules {
classify: &classify,
skip: &skip,
};
treemap::scan_stream(root, &rules, cancel, sink, pool)
}
fn perform(&self, _request: &OpRequest) -> Result<OpOutcome, String> {
Err("the real filesystem runs its operations on the worker".to_string())
}
fn perform_undo(&self, _undo: &Undo) -> Result<OpOutcome, String> {
Err("the real filesystem runs its operations on the worker".to_string())
}
fn is_instant(&self) -> bool {
false
}
}
static VFS: OnceLock<Arc<dyn Vfs>> = OnceLock::new();
/// Choose the filesystem for this process. Called once, before the UI reads
/// anything; a second call is ignored, because a browser that changed
/// filesystems underneath itself would be showing two different worlds.
pub fn install(vfs: Arc<dyn Vfs>) {
let _ = VFS.set(vfs);
}
/// The filesystem this process is browsing.
pub fn vfs() -> &'static Arc<dyn Vfs> {
VFS.get_or_init(|| {
#[cfg(all(target_arch = "wasm32", feature = "demo"))]
{
Arc::new(crate::demo::DemoVfs::new())
}
#[cfg(not(all(target_arch = "wasm32", feature = "demo")))]
{
Arc::new(RealVfs)
}
})
}
/// True when the browser is showing the demo home rather than a real disk.
pub fn is_demo() -> bool {
vfs().is_demo()
}
/// The active filesystem's wall clock.
pub fn now_secs() -> u64 {
vfs().now_secs()
}
/// The demo is asked for by `--demo` on the command line or `MAKEPAD_FILES_DEMO=1`
/// in the environment, so it can be started from a launcher that has no
/// argument list of its own.
pub fn demo_requested() -> bool {
cfg!(feature = "demo")
|| std::env::args().any(|a| a == "--demo")
|| std::env::var("MAKEPAD_FILES_DEMO").is_ok_and(|v| v != "0" && !v.is_empty())
}
/// The description of an operation, for the message an instant filesystem
/// hands back. Shared so the demo's sentences read like the real ones.
pub fn outcome_message(kind: OpKind, count: usize, where_to: &Path) -> String {
let items = format!("{} item{}", count, if count == 1 { "" } else { "s" });
match kind {
OpKind::Copy => format!("Copied {items} to {}", model::display_name(where_to)),
OpKind::Move => format!("Moved {items} to {}", model::display_name(where_to)),
OpKind::Trash => format!("Moved {items} to the Trash"),
OpKind::Rename => format!("Renamed {items}"),
OpKind::NewFolder => format!("Created {}", model::display_name(where_to)),
OpKind::Delete => format!("Deleted {items} permanently"),
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_real_filesystem_is_the_identity_on_paths() {
let real = RealVfs;
let path = Path::new("/a/b/c.png");
assert_eq!(real.native_path(path).unwrap(), path);
assert!(!real.is_instant());
assert!(!real.is_demo());
}
#[test]
fn virtual_native_entry_points_are_typed_unavailable() {
let virtual_fs = crate::demo::DemoVfs::new();
assert_eq!(
virtual_fs.native_path(Path::new("/Demo/file")),
Err(VfsError::Unavailable("native filesystem path"))
);
assert!(matches!(
virtual_fs.forget_scan_cache(Path::new("/Demo")),
Err(VfsError::Unavailable("native size-map cache"))
));
}
#[test]
fn real_stat_and_bounded_reads_use_the_same_entry_shape() {
let real = RealVfs;
let path = std::env::temp_dir().join(format!("files-vfs-stat-{}", std::process::id()));
std::fs::write(&path, b"abcdef").unwrap();
let entry = real.stat(&path).unwrap();
assert_eq!(entry.path, path);
assert_eq!(entry.size, 6);
assert!(!entry.is_dir);
assert_eq!(real.read_bytes(&path, 3).unwrap(), b"abc");
std::fs::remove_file(path).ok();
}
#[test]
fn the_default_filesystem_is_the_real_one() {
// Nothing installed anything in this test binary, so asking for the
// filesystem must still answer — with the disk.
assert!(!vfs().is_demo());
}
#[test]
fn operation_sentences_read_the_same_either_way() {
let dir = Path::new("/x/Documents");
assert_eq!(outcome_message(OpKind::Copy, 1, dir), "Copied 1 item to Documents");
assert_eq!(outcome_message(OpKind::Move, 3, dir), "Moved 3 items to Documents");
assert_eq!(outcome_message(OpKind::Trash, 2, dir), "Moved 2 items to the Trash");
assert_eq!(
outcome_message(OpKind::Delete, 1, dir),
"Deleted 1 item permanently"
);
}
}

View file

@ -14,13 +14,7 @@ default-run = "finance"
name = "finance"
path = "src/main.rs"
[features]
default = []
demo = []
[dependencies]
makepad-widgets = { path = "../../widgets" }
makepad-wm-theme = { path = "../../libs/wm_theme" }
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
mp-theme = { path = "../../libs/mp_theme" }
makepad-sqlite = { path = "../../libs/sqlite_query" }

View file

@ -388,7 +388,10 @@ impl fmt::Display for DateRange {
/// Today, from the system clock. The one place time enters the app.
pub fn today() -> Day {
let secs = makepad_widgets::Cx::time_now().max(0.0) as i64;
let secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs() as i64)
.unwrap_or(0);
(secs / 86_400) as Day
}

View file

@ -455,37 +455,12 @@ impl Db {
}
pub fn insert_transaction(&mut self, txn: &Transaction) -> Result<Id, String> {
Ok(self.insert_transaction_with_ids(txn)?.0)
}
/// Insert a transaction and report the database ids assigned to both
/// the row and its splits. First-run persistence uses every returned id
/// to build an explicit map from generated ids to stored ids.
pub(crate) fn insert_transaction_with_ids(
&mut self,
txn: &Transaction,
) -> Result<(Id, Vec<Id>), String> {
insert_transaction_on(&mut self.conn, txn)?;
let id = self.last_id("transactions")?;
let mut split_ids = Vec::with_capacity(txn.splits.len());
for split in &txn.splits {
insert_split_on(&mut self.conn, id, split)?;
split_ids.push(self.last_id("splits")?);
}
Ok((id, split_ids))
}
pub(crate) fn insert_payee(&mut self, payee: &Payee) -> Result<Id, String> {
self.conn
.execute(
"INSERT INTO payees(name, default_category) VALUES(?, ?)",
&[
Value::text(payee.name.as_str()),
payee.default_category.map(Value::Integer).unwrap_or(Value::Null),
],
)
.map_err(|e| format!("insert payee: {e:?}"))?;
self.last_id("payees")
Ok(id)
}
pub fn insert_budget(&mut self, entry: &BudgetEntry) -> Result<(), String> {

View file

@ -1,36 +0,0 @@
use crate::model::{Id, Ledger};
use crate::runtime::{ImportState, Runtime, Start};
use makepad_widgets::{Actions, Cx};
#[derive(Default)]
pub(crate) struct Backend;
impl Runtime for Backend {
fn start(&mut self) -> Start {
let today = crate::runtime::demo_today();
Start {
today,
ledger: crate::seed::generate(crate::seed::DEFAULT_YEARS, today),
status: "Demo household loaded".to_string(),
}
}
fn has_import(&self) -> bool {
false
}
fn pick_statement(&mut self, _cx: &mut Cx) {}
fn prepare_from_actions(
&mut self,
_actions: &Actions,
_ledger: &Ledger,
_account_filter: Option<Id>,
) -> Option<Result<ImportState, String>> {
None
}
fn commit_import(&mut self, _state: ImportState) -> Result<(Ledger, String), String> {
Err("statement import is unavailable in the demo".to_string())
}
}

View file

@ -14,13 +14,11 @@ use makepad_widgets::*;
mod chart;
mod csv;
mod date;
#[cfg(all(not(target_arch = "wasm32"), not(feature = "demo")))]
mod db;
mod import;
mod model;
mod money;
mod report;
mod runtime;
mod seed;
mod theme;
mod view;
@ -55,7 +53,7 @@ impl MatchEvent for App {}
impl AppMain for App {
fn script_mod(vm: &mut ScriptVm) -> ScriptValue {
crate::makepad_widgets::script_mod(vm);
makepad_wm_theme::apply(vm);
mp_theme::apply(vm);
crate::theme::install(vm);
crate::chart::script_mod(vm);
crate::view::script_mod(vm);

View file

@ -1,455 +0,0 @@
//! Native SQLite persistence and statement import.
use crate::date;
use crate::db::Db;
use crate::model::*;
use crate::runtime::{ImportState, Runtime, Start};
use makepad_widgets::makepad_platform::file_dialogs::{FileDialog, FileDialogAction};
use makepad_widgets::*;
use std::collections::{BTreeSet, HashMap};
const PICK_STATEMENT: LiveId = live_id!(finance_pick_statement);
#[derive(Default)]
pub(crate) struct Backend {
db: Option<Db>,
}
impl Runtime for Backend {
fn start(&mut self) -> Start {
let today = date::today();
let path = std::path::PathBuf::from("local/finance/finance.db");
let mut db = match Db::open(&path) {
Ok(db) => db,
Err(error) => {
return Start {
today,
ledger: Ledger::default(),
status: format!("cannot open {}: {error}", path.display()),
};
}
};
let mut status = String::new();
match db.is_empty() {
Ok(true) => {
let ledger = crate::seed::generate(crate::seed::DEFAULT_YEARS, today);
match persist(&mut db, &ledger) {
Ok(_) => {
let start = date::month_start(date::add_months(
today,
-(crate::seed::DEFAULT_YEARS * 12 - 1),
));
status = format!(
"Demo file created: {} transactions across {} accounts, {} to {}",
ledger.transactions.len(),
ledger.accounts.len(),
date::format_short(start),
date::format_short(today)
);
}
Err(error) => status = format!("demo data failed: {error}"),
}
}
Ok(false) => {}
Err(error) => status = format!("cannot read {}: {error}", path.display()),
}
let ledger = match db.load() {
Ok(ledger) => ledger,
Err(error) => {
status = format!("load failed: {error}");
Ledger::default()
}
};
self.db = Some(db);
Start { today, ledger, status }
}
fn has_import(&self) -> bool {
true
}
fn pick_statement(&mut self, cx: &mut Cx) {
let dialog = FileDialog::new()
.set_id(PICK_STATEMENT)
.set_title("Choose a statement".to_string())
.add_filter("Comma-separated values".to_string(), vec!["csv".to_string()])
.add_filter("Text".to_string(), vec!["txt".to_string()])
.add_filter("All Files".to_string(), vec!["*".to_string()]);
cx.open_select_file_dialog(dialog);
}
fn prepare_from_actions(
&mut self,
actions: &Actions,
ledger: &Ledger,
account_filter: Option<Id>,
) -> Option<Result<ImportState, String>> {
for action in actions {
let Some(picked) = action.downcast_ref::<FileDialogAction>() else { continue };
if picked.id() == PICK_STATEMENT {
if let Some(path) = picked.path() {
return Some(self.prepare_import(path, ledger, account_filter));
}
}
}
None
}
fn commit_import(&mut self, state: ImportState) -> Result<(Ledger, String), String> {
let db = self.db.as_mut().ok_or_else(|| "database is not open".to_string())?;
let rows: Vec<Transaction> = state.plan.to_import().cloned().collect();
let count = rows.len();
db.transact(|conn| {
for txn in &rows {
crate::db::insert_transaction_on(conn, txn)?;
}
Ok(())
})?;
let ledger = db.load()?;
let status = format!("Imported {count} transactions from {}", state.path);
Ok((ledger, status))
}
}
impl Backend {
fn prepare_import(
&mut self,
path: &std::path::Path,
ledger: &Ledger,
account_filter: Option<Id>,
) -> Result<ImportState, String> {
let bytes = std::fs::read(path)
.map_err(|error| format!("cannot read {}: {error}", path.display()))?;
let csv = crate::csv::parse(&String::from_utf8_lossy(&bytes));
let guess = crate::import::Mapping::guess(&csv);
let account = account_filter
.or_else(|| ledger.accounts.first().map(|account| account.id))
.ok_or_else(|| "no account to import into".to_string())?;
let account = ledger
.account(account)
.ok_or_else(|| "no account to import into".to_string())?;
let known = self
.db
.as_mut()
.ok_or_else(|| "database is not open".to_string())?
.known_fingerprints()?;
let plan = crate::import::plan(&csv, &guess.mapping, account, &ledger.rules, &known);
Ok(ImportState {
path: path.display().to_string(),
plan,
ask_date_order: guess.ask_date_order,
})
}
}
#[derive(Default)]
struct PersistedIds {
accounts: HashMap<Id, Id>,
categories: HashMap<Id, Id>,
payees: HashMap<Id, Id>,
transactions: HashMap<Id, Id>,
splits: HashMap<Id, Id>,
transfer_groups: HashMap<Id, Id>,
rules: HashMap<Id, Id>,
scheduled: HashMap<Id, Id>,
}
fn mapped(ids: &HashMap<Id, Id>, old: Id, kind: &str) -> Result<Id, String> {
ids.get(&old)
.copied()
.ok_or_else(|| format!("missing {kind} id {old} while persisting generated ledger"))
}
fn mapped_opt(
ids: &HashMap<Id, Id>,
old: Option<Id>,
kind: &str,
) -> Result<Option<Id>, String> {
old.map(|id| mapped(ids, id, kind)).transpose()
}
/// Persist a generated ledger while translating every local id to the id
/// SQLite assigned. Keeping the maps explicit prevents insertion order from
/// leaking into parent links or any downstream reference.
fn persist(db: &mut Db, ledger: &Ledger) -> Result<PersistedIds, String> {
let mut ids = PersistedIds::default();
for account in &ledger.accounts {
ids.accounts.insert(account.id, db.insert_account(account)?);
}
// Parents must be assigned before their children, regardless of the
// display order in the generated vector.
let mut categories: Vec<&Category> = ledger.categories.categories.iter().collect();
while !categories.is_empty() {
let Some(index) = categories.iter().position(|category| {
category.parent.is_none_or(|parent| ids.categories.contains_key(&parent))
}) else {
return Err("category tree contains a missing or cyclic parent".to_string());
};
let category = categories.remove(index);
let mut stored = category.clone();
stored.parent = mapped_opt(&ids.categories, category.parent, "category parent")?;
ids.categories.insert(category.id, db.insert_category(&stored)?);
}
for payee in &ledger.payees {
let mut stored = payee.clone();
stored.default_category =
mapped_opt(&ids.categories, payee.default_category, "payee category")?;
ids.payees.insert(payee.id, db.insert_payee(&stored)?);
}
let groups: BTreeSet<Id> =
ledger.transactions.iter().filter_map(|txn| txn.transfer_group).collect();
for (index, group) in groups.into_iter().enumerate() {
ids.transfer_groups.insert(group, index as Id + 1);
}
for txn in &ledger.transactions {
let mut stored = txn.clone();
stored.account = mapped(&ids.accounts, txn.account, "transaction account")?;
stored.category = mapped_opt(&ids.categories, txn.category, "transaction category")?;
stored.transfer_group =
mapped_opt(&ids.transfer_groups, txn.transfer_group, "transfer group")?;
for split in &mut stored.splits {
split.category = mapped_opt(&ids.categories, split.category, "split category")?;
}
let (transaction_id, split_ids) = db.insert_transaction_with_ids(&stored)?;
ids.transactions.insert(txn.id, transaction_id);
for (split, stored_id) in txn.splits.iter().zip(split_ids) {
ids.splits.insert(split.id, stored_id);
}
}
for budget in &ledger.budgets {
let mut stored = *budget;
stored.category = mapped(&ids.categories, budget.category, "budget category")?;
db.insert_budget(&stored)?;
}
for rule in &ledger.rules {
let mut stored = rule.clone();
stored.set_category = mapped_opt(&ids.categories, rule.set_category, "rule category")?;
ids.rules.insert(rule.id, db.insert_rule(&stored)?);
}
for scheduled in &ledger.scheduled {
let mut stored = scheduled.clone();
stored.account = mapped(&ids.accounts, scheduled.account, "scheduled account")?;
stored.category =
mapped_opt(&ids.categories, scheduled.category, "scheduled category")?;
ids.scheduled.insert(scheduled.id, db.insert_scheduled(&stored)?);
}
db.set_setting("base_currency", ledger.base_currency.code)?;
Ok(ids)
}
#[cfg(test)]
mod tests {
use super::*;
fn temp_path(name: &str) -> std::path::PathBuf {
let mut path = std::env::temp_dir();
path.push(format!("finance-native-{name}-{}.db", std::process::id()));
let _ = std::fs::remove_file(&path);
path
}
fn assert_references_resolve(ledger: &Ledger) {
for category in &ledger.categories.categories {
if let Some(parent) = category.parent {
assert!(ledger.categories.get(parent).is_some(), "missing parent {parent}");
}
}
for txn in &ledger.transactions {
assert!(ledger.account(txn.account).is_some(), "missing account {}", txn.account);
if let Some(category) = txn.category {
assert!(ledger.categories.get(category).is_some(), "missing category {category}");
}
for split in &txn.splits {
if let Some(category) = split.category {
assert!(ledger.categories.get(category).is_some(), "missing split category");
}
}
}
for budget in &ledger.budgets {
assert!(ledger.categories.get(budget.category).is_some(), "missing budget category");
}
for payee in &ledger.payees {
if let Some(category) = payee.default_category {
assert!(ledger.categories.get(category).is_some(), "missing payee category");
}
}
for rule in &ledger.rules {
if let Some(category) = rule.set_category {
assert!(ledger.categories.get(category).is_some(), "missing rule category");
}
}
for item in &ledger.scheduled {
assert!(ledger.account(item.account).is_some(), "missing scheduled account");
if let Some(category) = item.category {
assert!(ledger.categories.get(category).is_some(), "missing scheduled category");
}
}
}
#[test]
fn generated_ledger_round_trips_through_sqlite_with_all_links_remapped() {
let path = temp_path("generated-roundtrip");
let generated = crate::seed::generate(2, date::from_ymd(2026, 8, 28));
let mut db = Db::open(&path).expect("open temp database");
let ids = persist(&mut db, &generated).expect("persist generated ledger");
let loaded = db.load().expect("load generated ledger");
assert_eq!(loaded.accounts.len(), generated.accounts.len());
assert_eq!(loaded.categories.categories.len(), generated.categories.categories.len());
assert_eq!(loaded.transactions.len(), generated.transactions.len());
assert_eq!(
loaded.transactions.iter().map(|txn| txn.splits.len()).sum::<usize>(),
generated.transactions.iter().map(|txn| txn.splits.len()).sum::<usize>()
);
assert_eq!(loaded.payees.len(), generated.payees.len());
assert_eq!(loaded.budgets.len(), generated.budgets.len());
assert_eq!(loaded.rules.len(), generated.rules.len());
assert_eq!(loaded.scheduled.len(), generated.scheduled.len());
assert_references_resolve(&loaded);
for account in &generated.accounts {
let loaded_id = ids.accounts[&account.id];
let mut expected = account.clone();
expected.id = loaded_id;
let actual = loaded.account(loaded_id).expect("mapped account");
assert_eq!(format!("{actual:?}"), format!("{expected:?}"));
assert_eq!(loaded.balance(loaded_id), generated.balance(account.id), "{}", account.name);
}
for category in &generated.categories.categories {
let mut expected = category.clone();
expected.id = ids.categories[&category.id];
expected.parent = category.parent.map(|parent| ids.categories[&parent]);
let actual = loaded.categories.get(expected.id).expect("mapped category");
assert_eq!(format!("{actual:?}"), format!("{expected:?}"));
}
for txn in &generated.transactions {
let mut expected = txn.clone();
expected.id = ids.transactions[&txn.id];
expected.account = ids.accounts[&txn.account];
expected.category = txn.category.map(|category| ids.categories[&category]);
expected.transfer_group =
txn.transfer_group.map(|group| ids.transfer_groups[&group]);
for split in &mut expected.splits {
split.id = ids.splits[&split.id];
split.category = split.category.map(|category| ids.categories[&category]);
}
let actual = loaded.transaction(expected.id).expect("mapped transaction");
assert_eq!(format!("{actual:?}"), format!("{expected:?}"));
}
for budget in &generated.budgets {
let mut expected = *budget;
expected.category = ids.categories[&budget.category];
let actual = loaded
.budgets
.iter()
.find(|item| item.category == expected.category && item.month == expected.month)
.expect("mapped budget");
assert_eq!(format!("{actual:?}"), format!("{expected:?}"));
}
for rule in &generated.rules {
let mut expected = rule.clone();
expected.id = ids.rules[&rule.id];
expected.set_category = rule.set_category.map(|category| ids.categories[&category]);
let actual = loaded.rules.iter().find(|item| item.id == expected.id).expect("mapped rule");
assert_eq!(format!("{actual:?}"), format!("{expected:?}"));
}
for item in &generated.scheduled {
let mut expected = item.clone();
expected.id = ids.scheduled[&item.id];
expected.account = ids.accounts[&item.account];
expected.category = item.category.map(|category| ids.categories[&category]);
let actual = loaded
.scheduled
.iter()
.find(|candidate| candidate.id == expected.id)
.expect("mapped scheduled entry");
assert_eq!(format!("{actual:?}"), format!("{expected:?}"));
}
let mut generated_tree: Vec<_> = generated
.categories
.categories
.iter()
.map(|category| {
(
category.name.clone(),
category.parent.map(|parent| generated.categories.name(parent).to_string()),
)
})
.collect();
let mut loaded_tree: Vec<_> = loaded
.categories
.categories
.iter()
.map(|category| {
(
category.name.clone(),
category.parent.map(|parent| loaded.categories.name(parent).to_string()),
)
})
.collect();
generated_tree.sort();
loaded_tree.sort();
assert_eq!(loaded_tree, generated_tree);
assert_eq!(ids.accounts.len(), generated.accounts.len());
assert_eq!(ids.categories.len(), generated.categories.categories.len());
assert_eq!(ids.payees.len(), generated.payees.len());
assert_eq!(ids.transactions.len(), generated.transactions.len());
assert_eq!(
ids.splits.len(),
generated.transactions.iter().map(|txn| txn.splits.len()).sum::<usize>()
);
assert_eq!(ids.rules.len(), generated.rules.len());
assert_eq!(ids.scheduled.len(), generated.scheduled.len());
assert_eq!(
ids.transfer_groups.len(),
generated
.transactions
.iter()
.filter_map(|txn| txn.transfer_group)
.collect::<BTreeSet<_>>()
.len()
);
drop(db);
let _ = std::fs::remove_file(path);
}
#[test]
fn native_backend_advertises_import() {
assert!(Backend::default().has_import());
}
#[test]
fn payee_category_reference_is_remapped() {
let path = temp_path("payee-remap");
let mut ledger = Ledger::default();
let mut category = Category::group("Food", CategoryKind::Expense);
category.id = 40;
ledger.categories.categories.push(category);
ledger.payees.push(Payee {
id: 90,
name: "Market".to_string(),
default_category: Some(40),
transactions: 0,
});
let mut db = Db::open(&path).expect("open temp database");
let ids = persist(&mut db, &ledger).expect("persist payee");
let loaded = db.load().expect("load payee");
assert_eq!(loaded.payees.len(), 1);
assert_eq!(loaded.payees[0].id, ids.payees[&90]);
assert_eq!(loaded.payees[0].default_category, Some(ids.categories[&40]));
drop(db);
let _ = std::fs::remove_file(path);
}
}

View file

@ -1,54 +0,0 @@
//! Build-specific storage and import capabilities behind one UI-facing API.
use crate::date::{self, Day};
use crate::model::{Id, Ledger};
use makepad_widgets::{Actions, Cx};
pub(crate) struct Start {
pub today: Day,
pub ledger: Ledger,
pub status: String,
}
pub(crate) struct ImportState {
pub path: String,
pub plan: crate::import::Plan,
pub ask_date_order: bool,
}
pub(crate) fn demo_today() -> Day {
date::from_ymd(2026, 8, 28)
}
#[cfg(all(not(target_arch = "wasm32"), not(feature = "demo")))]
#[path = "native.rs"]
mod imp;
#[cfg(any(target_arch = "wasm32", feature = "demo"))]
#[path = "demo.rs"]
mod imp;
pub(crate) use imp::Backend;
pub(crate) trait Runtime {
fn start(&mut self) -> Start;
fn has_import(&self) -> bool;
fn pick_statement(&mut self, cx: &mut Cx);
fn prepare_from_actions(
&mut self,
actions: &Actions,
ledger: &Ledger,
account_filter: Option<Id>,
) -> Option<Result<ImportState, String>>;
fn commit_import(&mut self, state: ImportState) -> Result<(Ledger, String), String>;
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn demo_clock_is_pinned_late_in_august() {
assert_eq!(date::to_ymd(demo_today()), (2026, 8, 28));
assert_eq!(date::month_key(demo_today()), date::month_key(date::from_ymd(2026, 8, 1)));
}
}

View file

@ -12,11 +12,12 @@
//! empty current month. The generator is seeded and deterministic, so the
//! same day always produces the same file and a screenshot is reproducible.
//!
//! Generation is pure. Native first-run persistence writes the resulting
//! ledger through the ordinary database paths, while demo builds can use it
//! directly without a filesystem.
//! Everything here is ordinary ledger data written through the ordinary
//! [`crate::db`] paths. There is no "demo mode" in the app: the rows are
//! real rows, editable and deletable like any other.
use crate::date::{self, Day};
use crate::db::Db;
use crate::model::*;
use crate::money::{Currency, EUR};
@ -91,14 +92,15 @@ struct Cats {
insurance: Id,
}
/// Generate a complete household ledger deterministically for `today`.
pub fn generate(years: i32, today: Day) -> Ledger {
/// Fill an empty file with a generated household. Returns a one-line
/// summary for the status bar.
pub fn populate(db: &mut Db, years: i32) -> Result<String, String> {
let today = date::today();
let start = date::month_start(date::add_months(today, -(years * 12 - 1)));
let currency = EUR;
let mut ledger = Ledger { base_currency: currency, ..Ledger::default() };
let accounts = insert_accounts(&mut ledger.accounts, currency, start);
let cats = insert_categories(&mut ledger.categories.categories);
let accounts = insert_accounts(db, currency, start)?;
let cats = insert_categories(db)?;
let mut rng = Rng::new(0x5EED_F1_A2_C3);
let mut txns: Vec<Transaction> = Vec::new();
@ -129,18 +131,29 @@ pub fn generate(years: i32, today: Day) -> Ledger {
};
}
txns.sort_by_key(|t| t.date);
for (index, txn) in txns.iter_mut().enumerate() {
txn.id = index as Id + 1;
}
add_splits(&mut txns, &cats);
ledger.transactions = txns;
ledger.budgets = generate_budgets(&cats, today, years);
ledger.rules = generate_rules(&cats);
ledger.scheduled = generate_scheduled(&accounts, &cats, today);
ledger.categories.categories.sort_by_key(|category| (category.sort_order, category.id));
ledger.scheduled.sort_by_key(|item| (item.next_due, item.id));
ledger
let count = txns.len();
db.transact(|conn| {
for txn in &txns {
crate::db::insert_transaction_on(conn, txn)?;
}
Ok(())
})?;
// Splits need the ids the insert assigned, so they go in a second pass
// over the file rather than being carried along.
add_splits(db, &cats)?;
insert_budgets(db, &cats, today, years)?;
insert_rules(db, &cats)?;
insert_scheduled(db, &accounts, &cats, today)?;
db.set_setting("base_currency", currency.code)?;
Ok(format!(
"{count} transactions across {} accounts, {} to {}",
accounts.all().len(),
date::format_short(start),
date::format_short(today)
))
}
struct Accounts {
@ -167,99 +180,91 @@ impl Accounts {
}
}
fn insert_accounts(accounts: &mut Vec<Account>, currency: Currency, start: Day) -> Accounts {
fn insert_accounts(db: &mut Db, currency: Currency, start: Day) -> Result<Accounts, String> {
let mut make = |name: &str,
kind: AccountKind,
institution: &str,
opening: i64,
order: i32| {
order: i32|
-> Result<Id, String> {
let mut account = Account::new(name, kind, currency);
account.id = accounts.len() as Id + 1;
account.institution = institution.to_string();
account.opening_balance = opening;
account.opening_date = start - 1;
account.sort_order = order;
let id = account.id;
accounts.push(account);
id
db.insert_account(&account)
};
Accounts {
checking: make("Everyday", AccountKind::Checking, "ING", 342_150, 0),
savings: make("Savings", AccountKind::Savings, "ING", 1_480_000, 1),
card: make("Rewards Card", AccountKind::CreditCard, "Amex", -84_320, 2),
cash: make("Cash", AccountKind::Cash, "", 12_000, 3),
brokerage: make("Brokerage", AccountKind::Investment, "DEGIRO", 2_650_000, 4),
Ok(Accounts {
checking: make("Everyday", AccountKind::Checking, "ING", 342_150, 0)?,
savings: make("Savings", AccountKind::Savings, "ING", 1_480_000, 1)?,
card: make("Rewards Card", AccountKind::CreditCard, "Amex", -84_320, 2)?,
cash: make("Cash", AccountKind::Cash, "", 12_000, 3)?,
brokerage: make("Brokerage", AccountKind::Investment, "DEGIRO", 2_650_000, 4)?,
// A mortgage is a debt: negative, and paid down over the run.
mortgage: make("Mortgage", AccountKind::Loan, "Rabobank", -24_800_000, 5),
house: make("Apartment", AccountKind::Asset, "", 41_500_000, 6),
}
mortgage: make("Mortgage", AccountKind::Loan, "Rabobank", -24_800_000, 5)?,
house: make("Apartment", AccountKind::Asset, "", 41_500_000, 6)?,
})
}
fn insert_categories(categories: &mut Vec<Category>) -> Cats {
let mut group = |name: &str, kind: CategoryKind, order: i32| {
fn insert_categories(db: &mut Db) -> Result<Cats, String> {
let mut group = |name: &str, kind: CategoryKind, order: i32| -> Result<Id, String> {
let mut category = Category::group(name, kind);
category.id = categories.len() as Id + 1;
category.sort_order = order;
let id = category.id;
categories.push(category);
id
db.insert_category(&category)
};
let income = group("Income", CategoryKind::Income, 0);
let housing = group("Housing", CategoryKind::Expense, 1);
let food = group("Food", CategoryKind::Expense, 2);
let transport = group("Transport", CategoryKind::Expense, 3);
let shopping = group("Shopping", CategoryKind::Expense, 4);
let health = group("Health", CategoryKind::Expense, 5);
let fun = group("Fun", CategoryKind::Expense, 6);
let travel = group("Travel", CategoryKind::Expense, 7);
let money = group("Money", CategoryKind::Expense, 8);
let family = group("Family", CategoryKind::Expense, 9);
drop(group);
let income = group("Income", CategoryKind::Income, 0)?;
let housing = group("Housing", CategoryKind::Expense, 1)?;
let food = group("Food", CategoryKind::Expense, 2)?;
let transport = group("Transport", CategoryKind::Expense, 3)?;
let shopping = group("Shopping", CategoryKind::Expense, 4)?;
let health = group("Health", CategoryKind::Expense, 5)?;
let fun = group("Fun", CategoryKind::Expense, 6)?;
let travel = group("Travel", CategoryKind::Expense, 7)?;
let money = group("Money", CategoryKind::Expense, 8)?;
let family = group("Family", CategoryKind::Expense, 9)?;
let mut child = |name: &str,
parent: Id,
kind: CategoryKind,
rollover: bool,
order: i32| {
order: i32|
-> Result<Id, String> {
let mut category = Category::child(name, parent, kind);
category.id = categories.len() as Id + 1;
category.budgeted = kind == CategoryKind::Expense;
category.rollover = rollover;
category.sort_order = order;
let id = category.id;
categories.push(category);
id
db.insert_category(&category)
};
Cats {
salary: child("Salary", income, CategoryKind::Income, false, 0),
interest: child("Interest", income, CategoryKind::Income, false, 1),
housing: child("Mortgage", housing, CategoryKind::Expense, false, 0),
utilities: child("Energy", housing, CategoryKind::Expense, false, 1),
internet: child("Internet", housing, CategoryKind::Expense, false, 2),
phone: child("Phone", housing, CategoryKind::Expense, false, 3),
groceries: child("Groceries", food, CategoryKind::Expense, false, 0),
restaurants: child("Restaurants", food, CategoryKind::Expense, false, 1),
coffee: child("Coffee", food, CategoryKind::Expense, false, 2),
household: child("Household", shopping, CategoryKind::Expense, false, 0),
fuel: child("Fuel", transport, CategoryKind::Expense, false, 0),
transit: child("Transit", transport, CategoryKind::Expense, false, 1),
Ok(Cats {
salary: child("Salary", income, CategoryKind::Income, false, 0)?,
interest: child("Interest", income, CategoryKind::Income, false, 1)?,
housing: child("Mortgage", housing, CategoryKind::Expense, false, 0)?,
utilities: child("Energy", housing, CategoryKind::Expense, false, 1)?,
internet: child("Internet", housing, CategoryKind::Expense, false, 2)?,
phone: child("Phone", housing, CategoryKind::Expense, false, 3)?,
groceries: child("Groceries", food, CategoryKind::Expense, false, 0)?,
restaurants: child("Restaurants", food, CategoryKind::Expense, false, 1)?,
coffee: child("Coffee", food, CategoryKind::Expense, false, 2)?,
household: child("Household", shopping, CategoryKind::Expense, false, 0)?,
fuel: child("Fuel", transport, CategoryKind::Expense, false, 0)?,
transit: child("Transit", transport, CategoryKind::Expense, false, 1)?,
// Car maintenance is lumpy, so it rolls over: three quiet months
// pay for the fourth.
car: child("Car upkeep", transport, CategoryKind::Expense, true, 2),
clothing: child("Clothing", shopping, CategoryKind::Expense, false, 1),
electronics: child("Electronics", shopping, CategoryKind::Expense, true, 2),
pharmacy: child("Pharmacy", health, CategoryKind::Expense, false, 0),
gym: child("Gym", health, CategoryKind::Expense, false, 1),
streaming: child("Streaming", fun, CategoryKind::Expense, false, 0),
events: child("Going out", fun, CategoryKind::Expense, false, 1),
flights: child("Flights", travel, CategoryKind::Expense, true, 0),
hotels: child("Hotels", travel, CategoryKind::Expense, true, 1),
fees: child("Bank fees", money, CategoryKind::Expense, false, 0),
insurance: child("Insurance", money, CategoryKind::Expense, false, 1),
gifts: child("Gifts", family, CategoryKind::Expense, true, 0),
childcare: child("Childcare", family, CategoryKind::Expense, false, 1),
}
car: child("Car upkeep", transport, CategoryKind::Expense, true, 2)?,
clothing: child("Clothing", shopping, CategoryKind::Expense, false, 1)?,
electronics: child("Electronics", shopping, CategoryKind::Expense, true, 2)?,
pharmacy: child("Pharmacy", health, CategoryKind::Expense, false, 0)?,
gym: child("Gym", health, CategoryKind::Expense, false, 1)?,
streaming: child("Streaming", fun, CategoryKind::Expense, false, 0)?,
events: child("Going out", fun, CategoryKind::Expense, false, 1)?,
flights: child("Flights", travel, CategoryKind::Expense, true, 0)?,
hotels: child("Hotels", travel, CategoryKind::Expense, true, 1)?,
fees: child("Bank fees", money, CategoryKind::Expense, false, 0)?,
insurance: child("Insurance", money, CategoryKind::Expense, false, 1)?,
gifts: child("Gifts", family, CategoryKind::Expense, true, 0)?,
childcare: child("Childcare", family, CategoryKind::Expense, false, 1)?,
})
}
/// A payday that lands on a working day: paid on the 25th, moved back to
@ -655,40 +660,39 @@ fn mortgage(
/// Turn a handful of supermarket trips into split transactions, so the
/// split UI has real examples the moment the app opens.
fn add_splits(transactions: &mut [Transaction], cats: &Cats) {
let mut next_split_id = 1;
for txn in transactions
.iter_mut()
fn add_splits(db: &mut Db, cats: &Cats) -> Result<(), String> {
let ledger = db.load()?;
let candidates: Vec<Transaction> = ledger
.transactions
.iter()
.filter(|t| {
t.category == Some(cats.groceries) && t.amount < -7_000 && t.splits.is_empty()
})
.take(6)
{
.cloned()
.collect();
for mut txn in candidates {
// A third of a big shop was household goods, not food.
let household = txn.amount / 3;
let food = txn.amount - household;
txn.splits = vec![
Split { id: 0, category: Some(cats.groceries), amount: food, memo: "Food".into() },
Split {
id: next_split_id,
category: Some(cats.groceries),
amount: food,
memo: "Food".into(),
},
Split {
id: next_split_id + 1,
id: 0,
category: Some(cats.household),
amount: household,
memo: "Cleaning, paper".into(),
},
];
next_split_id += 2;
debug_assert_eq!(txn.split_imbalance(), 0);
db.update_transaction(&txn)?;
}
Ok(())
}
/// Budgets for every month of history, so the budget screen opens on real
/// numbers and the "assigned vs spent" bars mean something.
fn generate_budgets(cats: &Cats, today: Day, years: i32) -> Vec<BudgetEntry> {
fn insert_budgets(db: &mut Db, cats: &Cats, today: Day, years: i32) -> Result<(), String> {
let plan: [(Id, i64); 17] = [
(cats.housing, 92_400),
(cats.utilities, 14_000),
@ -710,23 +714,23 @@ fn generate_budgets(cats: &Cats, today: Day, years: i32) -> Vec<BudgetEntry> {
];
let months = years * 12;
let first = date::month_key(date::add_months(today, -(months - 1)));
let mut budgets = Vec::new();
db.transact(|_conn| Ok(()))?;
for offset in 0..months {
let month = first + offset;
for (category, assigned) in plan {
budgets.push(BudgetEntry {
db.insert_budget(&BudgetEntry {
category,
month,
assigned,
rollover: matches!(category, c if c == cats.car),
});
})?;
}
}
budgets
Ok(())
}
/// The rules a person would have written after a month of imports.
fn generate_rules(cats: &Cats) -> Vec<Rule> {
fn insert_rules(db: &mut Db, cats: &Cats) -> Result<(), String> {
let rules = [
("Albert Heijn", "AH TO GO", Some(cats.groceries), Some("Albert Heijn")),
("Shell", "SHELL NEDERLAND", Some(cats.fuel), Some("Shell")),
@ -734,10 +738,9 @@ fn generate_rules(cats: &Cats) -> Vec<Rule> {
("Netflix", "NETFLIX.COM", Some(cats.streaming), Some("Netflix")),
("Amazon", "AMZN MKTP", Some(cats.household), Some("Amazon")),
];
let mut generated = Vec::new();
for (index, (name, pattern, category, rename)) in rules.into_iter().enumerate() {
generated.push(Rule {
id: index as Id + 1,
db.insert_rule(&Rule {
id: 0,
name: name.to_string(),
match_on: MatchOn::Raw,
how: MatchHow::Contains,
@ -751,13 +754,13 @@ fn generate_rules(cats: &Cats) -> Vec<Rule> {
priority: index as i32,
enabled: true,
hits: 0,
});
})?;
}
generated
Ok(())
}
/// The recurring bills, as the app's detector would have found them.
fn generate_scheduled(accounts: &Accounts, cats: &Cats, today: Day) -> Vec<Scheduled> {
fn insert_scheduled(db: &mut Db, accounts: &Accounts, cats: &Cats, today: Day) -> Result<(), String> {
let next = |day: u32| -> Day {
let (y, m, _) = date::to_ymd(today);
let candidate = date::from_ymd(y, m, day.min(date::days_in_month(y, m)));
@ -778,11 +781,9 @@ fn generate_scheduled(accounts: &Accounts, cats: &Cats, today: Day) -> Vec<Sched
(accounts.card, "SportCity", -2_995, cats.gym, next(18)),
(accounts.checking, "Bergman Design BV", 492_400, cats.salary, next(25)),
];
items
.into_iter()
.enumerate()
.map(|(index, (account, payee, amount, category, due))| Scheduled {
id: index as Id + 1,
for (account, payee, amount, category, due) in items {
db.insert_scheduled(&Scheduled {
id: 0,
account,
payee: payee.to_string(),
amount,
@ -793,143 +794,28 @@ fn generate_scheduled(accounts: &Accounts, cats: &Cats, today: Day) -> Vec<Sched
auto_post: false,
enabled: true,
detected: true,
})
.collect()
})?;
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::{HashMap, HashSet};
fn fingerprint(ledger: &Ledger) -> u64 {
// Every model here derives Debug over all of its named fields. Sort
// each table by its durable key, then length-frame and hash those
// complete structural records so neither ordering nor concatenation
// ambiguity can hide a difference.
let mut rows = vec![format!("currency:{:?}", ledger.base_currency)];
let mut accounts: Vec<_> = ledger.accounts.iter().collect();
accounts.sort_by_key(|account| account.id);
rows.extend(accounts.into_iter().map(|account| format!("account:{account:?}")));
let mut categories: Vec<_> = ledger.categories.categories.iter().collect();
categories.sort_by_key(|category| category.id);
rows.extend(categories.into_iter().map(|category| format!("category:{category:?}")));
let mut transactions = ledger.transactions.clone();
transactions.sort_by_key(|txn| txn.id);
for txn in &mut transactions {
txn.splits.sort_by_key(|split| split.id);
}
rows.extend(transactions.into_iter().map(|txn| format!("transaction:{txn:?}")));
let mut payees: Vec<_> = ledger.payees.iter().collect();
payees.sort_by_key(|payee| payee.id);
rows.extend(payees.into_iter().map(|payee| format!("payee:{payee:?}")));
let mut budgets = ledger.budgets.clone();
budgets.sort_by_key(|budget| (budget.category, budget.month));
rows.extend(budgets.into_iter().map(|budget| format!("budget:{budget:?}")));
let mut rules: Vec<_> = ledger.rules.iter().collect();
rules.sort_by_key(|rule| rule.id);
rows.extend(rules.into_iter().map(|rule| format!("rule:{rule:?}")));
let mut scheduled: Vec<_> = ledger.scheduled.iter().collect();
scheduled.sort_by_key(|item| item.id);
rows.extend(scheduled.into_iter().map(|item| format!("scheduled:{item:?}")));
let mut hash = 0xcbf2_9ce4_8422_2325u64;
for row in rows {
for byte in (row.len() as u64).to_le_bytes().into_iter().chain(row.bytes()) {
hash ^= u64::from(byte);
hash = hash.wrapping_mul(0x0000_0100_0000_01b3);
}
}
hash
}
fn entity_ids(ledger: &Ledger) -> Vec<(&'static str, Vec<Id>)> {
vec![
("accounts", ledger.accounts.iter().map(|item| item.id).collect()),
(
"categories",
ledger.categories.categories.iter().map(|item| item.id).collect(),
),
("transactions", ledger.transactions.iter().map(|item| item.id).collect()),
(
"splits",
ledger
.transactions
.iter()
.flat_map(|txn| txn.splits.iter().map(|split| split.id))
.collect(),
),
("payees", ledger.payees.iter().map(|item| item.id).collect()),
("rules", ledger.rules.iter().map(|item| item.id).collect()),
("scheduled", ledger.scheduled.iter().map(|item| item.id).collect()),
]
}
fn assert_unique_ids(ledger: &Ledger) {
for (kind, ids) in entity_ids(ledger) {
let mut unique = HashSet::new();
for id in ids {
assert_ne!(id, NO_ID, "{kind} contains an unassigned id");
assert!(unique.insert(id), "duplicate {kind} id {id}");
}
}
let budget_keys: HashSet<_> =
ledger.budgets.iter().map(|entry| (entry.category, entry.month)).collect();
assert_eq!(budget_keys.len(), ledger.budgets.len(), "duplicate budget key");
}
fn assert_references_resolve(ledger: &Ledger) {
let accounts: HashSet<_> = ledger.accounts.iter().map(|account| account.id).collect();
let categories: HashSet<_> =
ledger.categories.categories.iter().map(|category| category.id).collect();
for category in &ledger.categories.categories {
if let Some(parent) = category.parent {
assert!(categories.contains(&parent), "missing parent category {parent}");
}
}
for txn in &ledger.transactions {
assert!(accounts.contains(&txn.account), "missing transaction account {}", txn.account);
if let Some(category) = txn.category {
assert!(categories.contains(&category), "missing transaction category {category}");
}
for split in &txn.splits {
if let Some(category) = split.category {
assert!(categories.contains(&category), "missing split category {category}");
}
}
}
for payee in &ledger.payees {
if let Some(category) = payee.default_category {
assert!(categories.contains(&category), "missing payee category {category}");
}
}
for budget in &ledger.budgets {
assert!(categories.contains(&budget.category), "missing budget category");
}
for rule in &ledger.rules {
if let Some(category) = rule.set_category {
assert!(categories.contains(&category), "missing rule category {category}");
}
}
for item in &ledger.scheduled {
assert!(accounts.contains(&item.account), "missing scheduled account");
if let Some(category) = item.category {
assert!(categories.contains(&category), "missing scheduled category");
}
}
fn temp_db(name: &str) -> (Db, std::path::PathBuf) {
let mut path = std::env::temp_dir();
path.push(format!("finance-seed-{name}-{}.db", std::process::id()));
let _ = std::fs::remove_file(&path);
(Db::open(&path).expect("open"), path)
}
#[test]
fn the_demo_file_is_a_coherent_household() {
let today = date::from_ymd(2026, 8, 28);
let ledger = generate(DEFAULT_YEARS, today);
let (mut db, path) = temp_db("household");
let summary = populate(&mut db, DEFAULT_YEARS).expect("populate");
assert!(summary.contains("transactions"));
let ledger = db.load().expect("load");
// Enough to fill every screen.
assert!(
@ -945,19 +831,13 @@ mod tests {
// Every transfer pair balances — the invariant the whole
// net-worth number rests on.
let mut groups: HashMap<Id, Vec<&Transaction>> = HashMap::new();
for txn in ledger.transactions.iter().filter(|txn| txn.transfer_group.is_some()) {
groups.entry(txn.transfer_group.unwrap()).or_default().push(txn);
}
let groups: std::collections::HashSet<Id> =
ledger.transactions.iter().filter_map(|t| t.transfer_group).collect();
assert!(groups.len() > 20, "expected many transfers, got {}", groups.len());
for (group, rows) in groups {
assert_eq!(rows.len(), 2, "transfer {group} must have exactly two rows");
assert_eq!(rows[0].amount, -rows[1].amount, "transfer {group} must cancel");
for group in groups {
assert!(ledger.transfer_is_balanced(group), "transfer {group} does not cancel");
}
assert_unique_ids(&ledger);
assert_references_resolve(&ledger);
// Splits sum to their transaction.
let split_count = ledger.transactions.iter().filter(|t| t.is_split()).count();
assert!(split_count >= 5, "expected split examples, got {split_count}");
@ -974,9 +854,10 @@ mod tests {
ledger.balance(mortgage.id) > mortgage.opening_balance,
"the mortgage should have been paid down"
);
assert!(ledger.net_worth_on(today) > 0);
assert!(ledger.net_worth_on(date::today()) > 0);
// Nothing in the future, and history reaches back two years.
let today = date::today();
assert!(ledger.transactions.iter().all(|t| t.date <= today));
let oldest = ledger.transactions.iter().map(|t| t.date).min().unwrap();
assert!(today - oldest > 660, "expected ~2 years of history");
@ -991,16 +872,23 @@ mod tests {
.iter()
.any(|t| t.cleared == Cleared::Reconciled));
assert!(ledger.cleared_balance(checking.id) != ledger.balance(checking.id));
let _ = std::fs::remove_file(&path);
}
#[test]
fn the_same_seed_produces_the_same_file() {
let today = date::from_ymd(2026, 8, 28);
let one = generate(2, today);
let two = generate(2, today);
assert_eq!(fingerprint(&one), fingerprint(&two));
assert_eq!(entity_ids(&one), entity_ids(&two), "generated ids must be stable");
assert_unique_ids(&one);
assert_unique_ids(&two);
let (mut a, path_a) = temp_db("determinism-a");
let (mut b, path_b) = temp_db("determinism-b");
populate(&mut a, 1).expect("a");
populate(&mut b, 1).expect("b");
let one = a.load().expect("load a");
let two = b.load().expect("load b");
assert_eq!(one.transactions.len(), two.transactions.len());
let sum_a: i64 = one.transactions.iter().map(|t| t.amount).sum();
let sum_b: i64 = two.transactions.iter().map(|t| t.amount).sum();
assert_eq!(sum_a, sum_b);
let _ = std::fs::remove_file(&path_a);
let _ = std::fs::remove_file(&path_b);
}
}

View file

@ -15,13 +15,18 @@
use crate::chart::{FinanceChartWidgetExt, MeterWidgetRefExt};
use crate::date::{self, DateRange, Day, MonthKey};
use crate::db::Db;
use crate::model::*;
use crate::money::{format_compact, format_minor, format_money, Currency};
use crate::report;
use crate::runtime::{Backend, ImportState, Runtime};
use crate::theme;
use makepad_widgets::makepad_platform::file_dialogs::{FileDialog, FileDialogAction};
use makepad_widgets::*;
/// The dialog that picks a statement, so its answer is not confused with
/// any other file dialog the app might grow.
const PICK_STATEMENT: LiveId = live_id!(finance_pick_statement);
script_mod! {
use mod.prelude.widgets.*
use mod.widgets.*
@ -760,11 +765,9 @@ pub struct Finance {
view: View,
#[rust]
backend: Backend,
db: Option<Db>,
#[rust]
ledger: Ledger,
#[rust]
today: Day,
#[rust(Screen::Overview)]
screen: Screen,
#[rust(Layout::Wide)]
@ -793,28 +796,50 @@ pub struct Finance {
import: Option<ImportState>,
}
struct ImportState {
path: String,
csv: crate::csv::Csv,
mapping: crate::import::Mapping,
plan: crate::import::Plan,
account: Id,
ask_date_order: bool,
}
impl Finance {
fn currency(&self) -> Currency {
self.ledger.base_currency
}
/// Load either the generated demo household or the native database.
/// Open the file, generating a demo household if it is empty, and load
/// it into memory.
fn start(&mut self, cx: &mut Cx) {
if self.started {
return;
}
self.started = true;
let started = self.backend.start();
self.today = started.today;
self.ledger = started.ledger;
self.status = started.status;
let has_import = self.backend.has_import();
self.widget(cx, ids!(nav_import)).set_visible(cx, has_import);
self.widget(cx, ids!(tab_import)).set_visible(cx, has_import);
self.view(cx, ids!(import)).set_visible(cx, false);
self.budget_month = date::month_key(self.today);
let path = std::path::PathBuf::from("local/finance/finance.db");
let mut db = match Db::open(&path) {
Ok(db) => db,
Err(error) => {
self.status = format!("cannot open {}: {error}", path.display());
error!("finance: {}", self.status);
return;
}
};
match db.is_empty() {
Ok(true) => match crate::seed::populate(&mut db, crate::seed::DEFAULT_YEARS) {
Ok(summary) => self.status = format!("Demo file created: {summary}"),
Err(error) => self.status = format!("demo data failed: {error}"),
},
Ok(false) => {}
Err(error) => self.status = format!("cannot read {}: {error}", path.display()),
}
match db.load() {
Ok(ledger) => self.ledger = ledger,
Err(error) => self.status = format!("load failed: {error}"),
}
self.db = Some(db);
self.budget_month = date::month_key(date::today());
self.rebuild_rows();
self.show_only_current_screen(cx);
self.chrome_synced = false;
@ -875,11 +900,11 @@ impl Finance {
.iter()
.map(|t| t.date)
.min()
.unwrap_or(self.today)
.unwrap_or_else(date::today)
}
fn range(&self) -> DateRange {
self.range.resolve(self.today, self.earliest())
self.range.resolve(date::today(), self.earliest())
}
/// Show the screen, and make the chrome agree with it.
@ -895,11 +920,7 @@ impl Finance {
fn show_only_current_screen(&mut self, cx: &mut Cx) {
for screen in Screen::ALL {
self.view(cx, screen.view_id())
.set_visible(
cx,
screen == self.screen
&& (screen != Screen::Import || self.backend.has_import()),
);
.set_visible(cx, screen == self.screen);
}
}
@ -927,7 +948,7 @@ impl Finance {
/// something changed, rather than tracking what.
fn sync_chrome(&mut self, cx: &mut Cx) {
let currency = self.currency();
let today = self.today;
let today = date::today();
let range = self.range();
self.label(cx, ids!(screen_title)).set_text(cx, self.screen.title());
@ -1187,13 +1208,72 @@ impl Finance {
self.chrome_synced = true;
}
fn open_statement(&mut self, cx: &mut Cx) {
let dialog = FileDialog::new()
.set_id(PICK_STATEMENT)
.set_title("Choose a statement".to_string())
.add_filter("Comma-separated values".to_string(), vec!["csv".to_string()])
.add_filter("Text".to_string(), vec!["txt".to_string()])
.add_filter("All Files".to_string(), vec!["*".to_string()]);
cx.open_select_file_dialog(dialog);
}
/// Read a chosen file and build the plan, without writing anything.
fn prepare_import(&mut self, cx: &mut Cx, path: &std::path::Path) {
let text = match std::fs::read(path) {
Ok(bytes) => String::from_utf8_lossy(&bytes).into_owned(),
Err(error) => {
self.status = format!("cannot read {}: {error}", path.display());
self.chrome_synced = false;
self.redraw(cx);
return;
}
};
let csv = crate::csv::parse(&text);
let guess = crate::import::Mapping::guess(&csv);
let account = self
.account_filter
.or_else(|| self.ledger.accounts.first().map(|a| a.id))
.unwrap_or(0);
let Some(account_ref) = self.ledger.account(account).cloned() else {
self.status = "no account to import into".to_string();
return;
};
let known = self
.db
.as_mut()
.and_then(|db| db.known_fingerprints().ok())
.unwrap_or_default();
let plan = crate::import::plan(&csv, &guess.mapping, &account_ref, &self.ledger.rules, &known);
self.import = Some(ImportState {
path: path.display().to_string(),
csv,
mapping: guess.mapping,
plan,
account,
ask_date_order: guess.ask_date_order,
});
self.set_screen(cx, Screen::Import);
}
/// Write the plan. Everything or nothing.
fn commit_import(&mut self, cx: &mut Cx) {
let Some(state) = self.import.take() else { return };
match self.backend.commit_import(state) {
Ok((ledger, status)) => {
self.status = status;
self.ledger = ledger;
let Some(db) = self.db.as_mut() else { return };
let rows: Vec<Transaction> = state.plan.to_import().cloned().collect();
let count = rows.len();
let result = db.transact(|conn| {
for txn in &rows {
crate::db::insert_transaction_on(conn, txn)?;
}
Ok(())
});
match result {
Ok(()) => {
self.status = format!("Imported {count} transactions from {}", state.path);
if let Ok(ledger) = db.load() {
self.ledger = ledger;
}
self.rebuild_rows();
self.set_screen(cx, Screen::Ledger);
}
@ -1496,17 +1576,12 @@ impl WidgetMatchEvent for Finance {
(Screen::Ledger, ids!(nav_ledger), ids!(tab_ledger)),
(Screen::Budget, ids!(nav_budget), ids!(tab_budget)),
(Screen::Reports, ids!(nav_reports), ids!(tab_reports)),
(Screen::Import, ids!(nav_import), ids!(tab_import)),
] {
if self.button(cx, nav).clicked(actions) || self.button(cx, tab).clicked(actions) {
self.set_screen(cx, screen);
}
}
if self.backend.has_import()
&& (self.button(cx, ids!(nav_import)).clicked(actions)
|| self.button(cx, ids!(tab_import)).clicked(actions))
{
self.set_screen(cx, Screen::Import);
}
for (range, id) in [
(Range::Month, ids!(range_month)),
@ -1532,10 +1607,10 @@ impl WidgetMatchEvent for Finance {
self.redraw(cx);
}
if self.backend.has_import() && self.button(cx, ids!(import_pick)).clicked(actions) {
self.backend.pick_statement(cx);
if self.button(cx, ids!(import_pick)).clicked(actions) {
self.open_statement(cx);
}
if self.backend.has_import() && self.button(cx, ids!(import_apply)).clicked(actions) {
if self.button(cx, ids!(import_apply)).clicked(actions) {
self.commit_import(cx);
}
if self.button(cx, ids!(import_cancel)).clicked(actions) {
@ -1570,18 +1645,12 @@ impl WidgetMatchEvent for Finance {
}
}
if let Some(prepared) =
self.backend.prepare_from_actions(actions, &self.ledger, self.account_filter)
{
match prepared {
Ok(state) => {
self.import = Some(state);
self.set_screen(cx, Screen::Import);
}
Err(error) => {
self.status = error;
self.chrome_synced = false;
self.redraw(cx);
for action in actions {
if let Some(picked) = action.downcast_ref::<FileDialogAction>() {
if picked.id() == PICK_STATEMENT {
if let Some(path) = picked.path().cloned() {
self.prepare_import(cx, &path);
}
}
}
}

View file

@ -1,8 +0,0 @@
[package]
name = "makepad-flow-server"
version = "0.1.0"
edition = "2021"
publish = false
[dependencies]
makepad-flow = { path = "../../libs/flow", features = ["host"] }

View file

@ -1,116 +0,0 @@
use makepad_flow::embed::default_root;
use makepad_flow::host::{FlowServer, FlowServerConfig};
use std::net::{IpAddr, Ipv4Addr, SocketAddr};
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::Duration;
const USAGE: &str = "\
makepad-flow-server [options]
Options:
--root <dir> Server root (default: ~/.makepad/flow)
--bind <ip> Bind IP (default: 127.0.0.1)
--control-port <port> Control port (default: 0, ephemeral)
--data-port <port> Data port (default: 0, ephemeral)
--help Show this help
";
static STOP: AtomicBool = AtomicBool::new(false);
extern "C" fn on_signal(_signal: i32) {
STOP.store(true, Ordering::SeqCst);
}
fn install_signal_handlers() {
#[cfg(unix)]
{
unsafe extern "C" {
fn signal(signum: i32, handler: usize) -> usize;
}
const SIGINT: i32 = 2;
const SIGTERM: i32 = 15;
unsafe {
signal(SIGINT, on_signal as *const () as usize);
signal(SIGTERM, on_signal as *const () as usize);
}
}
}
fn fail(message: &str) -> ! {
eprintln!("makepad-flow-server: {message}");
eprintln!("{USAGE}");
std::process::exit(2);
}
fn value(name: &str, args: &mut impl Iterator<Item = String>) -> String {
args.next().unwrap_or_else(|| fail(&format!("{name} needs a value")))
}
fn parse_config() -> FlowServerConfig {
let mut args = std::env::args().skip(1);
let mut root: Option<PathBuf> = None;
let mut bind = IpAddr::V4(Ipv4Addr::LOCALHOST);
let mut control_port = 0u16;
let mut data_port = 0u16;
while let Some(argument) = args.next() {
match argument.as_str() {
"--root" => root = Some(PathBuf::from(value("--root", &mut args))),
"--bind" => {
bind = value("--bind", &mut args)
.parse()
.unwrap_or_else(|_| fail("--bind must be an IP address"));
}
"--control-port" => {
control_port = value("--control-port", &mut args)
.parse()
.unwrap_or_else(|_| fail("--control-port must be 0..65535"));
}
"--data-port" => {
data_port = value("--data-port", &mut args)
.parse()
.unwrap_or_else(|_| fail("--data-port must be 0..65535"));
}
"--help" | "-h" => {
println!("{USAGE}");
std::process::exit(0);
}
other => fail(&format!("unknown option {other}")),
}
}
let mut config = FlowServerConfig::new(root.unwrap_or_else(default_root));
config.asset.token = std::fs::read_to_string(
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("../../local/asset-ui/asset-server/admin-token"),
)
.ok()
.map(|token| token.trim().to_string())
.filter(|token| !token.is_empty());
config.control_addr = SocketAddr::new(bind, control_port).to_string();
config.data_addr = SocketAddr::new(bind, data_port).to_string();
config
}
fn main() {
let config = parse_config();
let root = config.root.clone();
install_signal_handlers();
let server = match FlowServer::start(config) {
Ok(server) => server,
Err(error) => {
eprintln!("makepad-flow-server: failed to start: {error}");
std::process::exit(1);
}
};
let endpoints = server.endpoints();
println!(
"[flow-server] listening control={} data={} root={}",
endpoints.control,
endpoints.data,
root.display()
);
while !STOP.load(Ordering::SeqCst) {
std::thread::sleep(Duration::from_millis(200));
}
server.shutdown();
}

View file

@ -1,18 +0,0 @@
[package]
name = "makepad-app-flow-ui"
version = "0.1.0"
edition = "2021"
publish = false
[dependencies]
makepad-widgets = { path = "../../widgets" }
makepad-code-editor = { path = "../../code_editor" }
makepad-strict-json = { path = "../../libs/strict_json" }
makepad-flow = { path = "../../libs/flow", features = ["host"] }
makepad-flowgraph = { path = "../../libs/flowgraph" }
makepad-media-view = { path = "../../libs/media_view" }
makepad-aichat = { path = "../aichat" }
makepad-ai-services = { path = "../../libs/ai/services" }
makepad-ai-hub = { path = "../../libs/ai/hub", default-features = false, features = ["llm"] }
makepad-app-asset-server = { path = "../asset-server" }
makepad-asset-client = { path = "../../libs/asset/client" }

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><circle cx="12" cy="12" r="10" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M12 8v4" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M12 16h.01" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 434 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M7.9 20A9 9 0 1 0 4 16.1L2 22Z" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M9.09 9a3 3 0 0 1 5.83 1c0 2-3 3-3 3" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M12 17h.01" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 473 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M2 10v3" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M6 6v11" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M10 3v18" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M14 8v7" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M18 5v13" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M22 10v3" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 772 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><rect x="14" y="14" width="4" height="6" rx="2" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><rect x="6" y="4" width="4" height="6" rx="2" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M6 20h4" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M14 10h4" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M6 14h2v6" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M14 4h2v6" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 832 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.937 15.5A2 2 0 0 0 8.5 14.063l-6.135-1.582a.5.5 0 0 1 0-.962L8.5 9.936A2 2 0 0 0 9.937 8.5l1.582-6.135a.5.5 0 0 1 .963 0L14.063 8.5A2 2 0 0 0 15.5 9.937l6.135 1.581a.5.5 0 0 1 0 .964L15.5 14.063a2 2 0 0 0-1.437 1.437l-1.582 6.135a.5.5 0 0 1-.963 0z" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M20 3v4" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M22 5h-4" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 663 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 6 9 17l-5-5" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 192 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="m6 9 6 6 6-6" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 189 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><circle cx="12" cy="12" r="10" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M12 6v6l4 2" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 318 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M6 6l12 12M18 6L6 18" stroke="currentColor" stroke-width="2" stroke-linecap="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 173 B

View file

@ -1,8 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16">
<rect x="1.5" y="1.5" width="13" height="13" rx="2.5" fill="none" stroke="#000" stroke-width="1.5"/>
<circle cx="5" cy="5" r="1.1" fill="#000"/>
<circle cx="11" cy="5" r="1.1" fill="#000"/>
<circle cx="8" cy="8" r="1.1" fill="#000"/>
<circle cx="5" cy="11" r="1.1" fill="#000"/>
<circle cx="11" cy="11" r="1.1" fill="#000"/>
</svg>

Before

Width:  |  Height:  |  Size: 405 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M8 3H5a2 2 0 0 0-2 2v3" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M21 8V5a2 2 0 0 0-2-2h-3" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M3 16v3a2 2 0 0 0 2 2h3" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M16 21h3a2 2 0 0 0 2-2v-3" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 601 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><rect width="8" height="8" x="3" y="3" rx="2" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M7 11v4a2 2 0 0 0 2 2h4" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><rect width="8" height="8" x="13" y="13" rx="2" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 492 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M8 3H7a2 2 0 0 0-2 2v5a2 2 0 0 1-2 2 2 2 0 0 1 2 2v5c0 1.1.9 2 2 2h1" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M16 21h1a2 2 0 0 0 2-2v-5c0-1.1.9-2 2-2a2 2 0 0 1-2-2V5a2 2 0 0 0-2-2h-1" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 427 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="m21.64 3.64-1.28-1.28a1.21 1.21 0 0 0-1.72 0L2.36 18.64a1.21 1.21 0 0 0 0 1.72l1.28 1.28a1.2 1.2 0 0 0 1.72 0L21.64 5.36a1.2 1.2 0 0 0 0-1.72" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="m14 7 3 3" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M5 6v4" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M19 14v4" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M10 2v2" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M7 8H3" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M21 16h-4" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M11 3H9" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 1.1 KiB

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><circle cx="9" cy="12" r="1.4" fill="currentColor"/><circle cx="9" cy="5" r="1.4" fill="currentColor"/><circle cx="9" cy="19" r="1.4" fill="currentColor"/><circle cx="15" cy="12" r="1.4" fill="currentColor"/><circle cx="15" cy="5" r="1.4" fill="currentColor"/><circle cx="15" cy="19" r="1.4" fill="currentColor"/></svg>

Before

Width:  |  Height:  |  Size: 380 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><circle cx="12" cy="12" r="10" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M12 2a14.5 14.5 0 0 0 0 20 14.5 14.5 0 0 0 0-20" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M2 12h20" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 472 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><rect width="18" height="18" x="3" y="3" rx="2" ry="2" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><circle cx="9" cy="9" r="2" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="m21 15-3.086-3.086a2 2 0 0 0-2.828 0L6 21" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 499 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 7V4h16v3" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M9 20h6" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M12 4v16" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 423 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M8 3H7a2 2 0 0 0-2 2v5a2 2 0 0 1-2 2 2 2 0 0 1 2 2v5c0 1.1.9 2 2 2h1" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M16 21h1a2 2 0 0 0 2-2v-5c0-1.1.9-2 2-2a2 2 0 0 1-2-2V5a2 2 0 0 0-2-2h-1" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 427 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 12h.01" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M3 18h.01" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M3 6h.01" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M8 12h13" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M8 18h13" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M8 6h13" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 776 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M21 8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73l7 4a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16Z" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="m3.3 7 8.7 5 8.7-5" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M12 22V12" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 542 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M21 13v6a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V5a2 2 0 0 1 2-2h6" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="m21 3-9 9" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M15 3h6v6" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 471 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M6 3 20 12 6 21Z" fill="currentColor" stroke="currentColor" stroke-width="2" stroke-linejoin="round"/></svg>

Before

Width:  |  Height:  |  Size: 178 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M5 12h14" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M12 5v14" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 303 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><rect width="14" height="14" x="5" y="5" rx="2" fill="currentColor"/></svg>

Before

Width:  |  Height:  |  Size: 136 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 7V4h16v3" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M9 20h6" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M12 4v16" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 423 B

View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><rect width="18" height="18" x="3" y="3" rx="2" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M7 3v18" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M3 7.5h4" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M3 12h18" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M3 16.5h4" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M17 3v18" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M17 7.5h4" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/><path d="M17 16.5h4" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" fill="none"/></svg>

Before

Width:  |  Height:  |  Size: 1 KiB

View file

@ -1,133 +0,0 @@
//! Host discovery and startup run on the task pool, away from UI input.
use crate::testpattern;
use makepad_app_asset_server::embed as asset_embed;
use makepad_asset_client::ApiEndpoints;
use makepad_flow::client::{SessionConfig, SessionConnector};
use makepad_flow::embed::{default_root, resolve, EmbedPolicy, Resolved};
use makepad_flow::engine::{FixedGen, HubChat, HubHttp, Seams};
use makepad_flow::host::{FlowServer, FlowServerConfig};
use std::io::{Read, Write};
use std::net::{IpAddr, SocketAddr};
use std::sync::Arc;
use std::time::Duration;
pub struct Bootstrap {
pub host: Option<FlowServer>,
pub testpattern: Option<testpattern::TestpatternService>,
pub session: SessionConnector,
// The store is dropped after everything that can talk to it.
pub store: Option<asset_embed::LocalStore>,
}
fn endpoints(text: &str) -> Option<ApiEndpoints> {
let mut parts = text.trim().rsplitn(3, ':');
let data = parts.next()?.parse().ok()?;
let control = parts.next()?.parse().ok()?;
let ip: IpAddr = parts.next()?.trim_matches(['[', ']']).parse().ok()?;
Some(ApiEndpoints { control: SocketAddr::new(ip, control), data: SocketAddr::new(ip, data) })
}
/// Keep an on-disk listen hint only when it currently speaks Asset Server.
/// The file survives restarts while both ports are ephemeral, so treating a
/// syntactically valid but stale hint as authoritative can strand the flow
/// worker even though discovery can find the new server.
fn health_answers(addr: SocketAddr) -> bool {
let Ok(mut stream) = std::net::TcpStream::connect_timeout(&addr, Duration::from_millis(400))
else {
return false;
};
let _ = stream.set_read_timeout(Some(Duration::from_millis(400)));
let _ = stream.set_write_timeout(Some(Duration::from_millis(400)));
let request = format!(
"GET /v1/health HTTP/1.1\r\nHost: {}\r\nConnection: close\r\n\r\n",
addr
);
if stream.write_all(request.as_bytes()).is_err() {
return false;
}
let mut response = [0u8; 32];
let mut received = 0usize;
while received < 16 {
match stream.read(&mut response[received..]) {
Ok(0) | Err(_) => break,
Ok(count) => received += count,
}
}
response[..received].starts_with(b"HTTP/1.1 200")
}
fn read_token(path: std::path::PathBuf) -> Option<String> {
std::fs::read_to_string(path)
.ok()
.map(|value| value.trim().to_string())
.filter(|value| !value.is_empty())
}
pub fn start() -> Result<Bootstrap, String> {
let root = default_root();
let mut host = None;
let mut store = None;
let mut testpattern = None;
let (hint, token) = match resolve(EmbedPolicy::from_env(), &root, None) {
Resolved::Attach(hint, token, _) => (hint, token),
Resolved::Host => {
let mut config = FlowServerConfig::new(root.clone());
config.asset.archive_outputs = true;
let asset_root = asset_embed::default_store_root("FLOW", "flow-assets");
let pinned = std::env::var("FLOW_ASSET_SERVER").ok().filter(|s| !s.trim().is_empty());
let hinted = if let Some(text) = &pinned {
Some(endpoints(text).ok_or("FLOW_ASSET_SERVER must be ip:control_port:data_port")?)
} else {
std::fs::read_to_string(asset_root.join("listen")).ok().and_then(|s| endpoints(&s))
};
let resolved = asset_embed::resolve("FLOW", "flow-assets", pinned.is_some(), hinted);
eprintln!("[flow-ui] assets: {}", resolved.note);
if let Some(local) = resolved.local {
config.asset.endpoints = Some(local.endpoints());
config.asset.server_id = Some(local.server_id());
config.asset.token = Some(local.token().to_string());
store = Some(local);
} else {
// An explicitly pinned server is authoritative and must use
// the explicitly supplied credential. For automatic attach,
// retain the listen hint only after a live health check;
// otherwise let the asset worker discover the fresh server
// instead of retrying a stale ephemeral port forever.
config.asset.endpoints = if pinned.is_some() {
hinted
} else {
hinted.filter(|value| health_answers(value.control))
};
config.asset.token = if pinned.is_some() {
std::env::var("FLOW_ASSET_TOKEN").ok()
} else {
std::env::var("FLOW_ASSET_TOKEN")
.ok()
.or_else(|| read_token(asset_root.join("admin-token")))
}
.map(|value| value.trim().to_string())
.filter(|value| !value.is_empty());
config.asset.discovery_wait_ms = 3_000;
}
if let Some(value) = std::env::var("FLOW_GEN_BASE_URL").ok().filter(|s| !s.is_empty()) {
let seams = if value == "testpattern" {
let service = testpattern::start_service()?;
let url = service.url.clone();
testpattern = Some(service);
Seams { chat: Arc::new(testpattern::TestpatternChat), gen: Arc::new(FixedGen(url)), http: Arc::new(HubHttp) }
} else {
Seams { chat: Arc::new(HubChat::from_env()), gen: Arc::new(FixedGen(value)), http: Arc::new(HubHttp) }
};
config = config.with_seams(seams);
}
let server = FlowServer::start(config).map_err(|e| format!("Could not host flow server: {e}"))?;
let served = server.endpoints();
let hint = Some(makepad_flow::client::Endpoints { control: served.control, data: served.data });
let token = Some(served.token.clone());
host = Some(server);
(hint, token)
}
};
let session = SessionConnector::start(SessionConfig { hint, root: Some(root), token, ..SessionConfig::default() });
Ok(Bootstrap { host, testpattern, session, store })
}

File diff suppressed because it is too large Load diff

View file

@ -1,271 +0,0 @@
use mod.prelude.widgets.*
use mod.widgets.*
// The face prelude (DESIGN.md §3). Evaluated in the instance's isolate after
// the core + recipe preludes, so `mod.flow.ui.*` become REAL widget
// prototypes and every node type's default `ui` is re-pointed at them.
//
// How a face talks to the flow, read by the host from each widget's own
// object. They are written with `:=` (a typed widget refuses an unknown
// `name:` property, but keeps `name :=` entries beside its children):
// bind := @self design: Input.value graph param; run: locked snapshot input
// bind := "node.port" design graph setting (run faces are inert)
// show := @self a display widget receives this node's first output
// show := "node.port" → that port's value on node.done / node.delta
// param := @name a display widget shows the graph param `name`
// param_bind := @name an input widget edits the graph param `name` (a graph PUT)
// Handlers see `flow` (flow.input / flow.inputs / flow.value / flow.state /
// flow.run / flow.cancel / flow.param) and the face hooks `on_value` and
// `on_state` on the face object; every bridge call is queued to the next frame.
//
// The canvas pads a card's content (14 px) and draws the port strip above
// it. Generator cards contain settings only; OutputFace owns result media.
let Face = View{
width: Fill
height: Fit
flow: Down
spacing: theme.space_2
}
let Meta = Label{
width: Fill
height: Fit
text: ""
draw_text +: {
color: theme.flow_text_muted
text_style: theme.font_regular{font_size: 9}
}
}
let Body = Label{
width: Fill
height: Fit
text: ""
draw_text +: {
color: theme.flow_text_body
text_style: theme.font_regular{font_size: 9.5}
}
}
let FaceInput = TextInput{
width: Fill
height: 96
is_multiline: true
draw_text +: {
text_style: theme.font_regular{font_size: 10.5}
}
}
let Number = mod.widgets.FabValueInput{
width: Fill
height: 24
precision: 0
quantize: true
}
let Strip = View{
width: Fill
height: Fit
flow: Down
spacing: theme.space_2
padding: Inset{left: 14 right: 14 top: 10 bottom: 12}
}
let Row = View{
width: Fill
height: Fit
flow: Right
spacing: theme.space_2
}
mod.flow.ui.NodeFace = Face{}
mod.flow.ui.InputFace = Face{
value := FaceInput{
empty_text: "Type the input value"
bind := @self
}
}
mod.flow.ui.TextFace = mod.flow.ui.InputFace{}
mod.flow.ui.OutputFace = Face{
spacing: 0
value := mod.flow.ui.ValueView{
show := @self
}
}
mod.flow.ui.PublishFace = Face{
params := Strip{
title := TextInput{width: Fill height: 28 empty_text: "default title" param_bind := @title}
namespace := TextInput{width: Fill height: 28 param_bind := @namespace}
tags := TextInput{width: Fill height: 28 empty_text: "flow, tag" param_bind := @tags}
}
published := Meta{show := @self}
}
mod.flow.ui.LlmFace = Face{
model := mod.flow.ui.ModelPicker{
param_bind := @model
}
system := FoldHeader{
header: View{
width: Fill
height: Fit
flow: Right
align: Align{y: 0.5}
spacing: theme.space_1
FoldButton{}
Meta{width: Fit text: "system prompt"}
}
body: View{
width: Fill
height: Fit
flow: Down
padding: Inset{left: 18 top: 2 bottom: 6}
text := Meta{param := @system}
}
animator +: {
active +: {
default: @off
}
}
}
out_scroll := mod.flow.ui.TextScroll{
out := Markdown{
width: Fill
height: Fit
body: ""
show := @self
}
}
}
mod.flow.ui.FnFace = Face{
params := Strip{}
code_scroll := mod.flow.ui.TextScroll{
code := Label{
width: Fill
height: Fit
draw_text +: {
color: theme.flow_text_code
text_style: theme.font_code{font_size: 9}
}
param := @run
}
}
out := mod.flow.ui.ValueText{
show := @self
}
}
mod.flow.ui.HttpFace = Face{
url := Body{param := @url}
status := Meta{show := @meta}
out := mod.flow.ui.ValueView{
show := @self
}
}
mod.flow.ui.AskFace = Face{
question_scroll := mod.flow.ui.TextScroll{
question := Body{param := @question}
}
choice := ComboBox{
width: Fill
height: 26
param := @options
bind := @self
}
answer := TextInput{
width: Fill
height: 32
empty_text: "Answer"
bind := @self
}
answer_button := Button{
width: Fill
text: "Answer"
}
}
mod.flow.ui.ImageFace = Face{
spacing: 0
params := Strip{
summary := Meta{param := @summary}
format := mod.flow.ui.FormatPicker{}
Row{
steps := Number{label: "steps" min: 1 max: 50 step: 0.25 snap: 1 param_bind := @steps}
seed := mod.flow.ui.SeedPicker{param_bind := @seed}
}
model := mod.flow.ui.ModelPicker{
param_bind := @model
}
}
}
mod.flow.ui.UpscaleFace = Face{
spacing: 0
params := Strip{
summary := Meta{param := @summary}
model := mod.flow.ui.ModelPicker{
param_bind := @model
}
}
}
mod.flow.ui.GenFace = Face{
domain := Meta{param := @domain}
params := Strip{
format := mod.flow.ui.FormatPicker{}
seed := mod.flow.ui.SeedPicker{param_bind := @seed}
model := mod.flow.ui.ModelPicker{
param_bind := @model
}
}
}
mod.flow.ui.VideoFace = Face{
spacing: 0
params := Strip{
format := mod.flow.ui.FormatPicker{}
Row{
frames := Number{label: "frames" min: 5 max: 4096 step: 1 snap: 1 param_bind := @frames}
seed := mod.flow.ui.SeedPicker{param_bind := @seed}
}
model := mod.flow.ui.ModelPicker{
param_bind := @model
}
}
}
// The node types keep the objects the core prelude gave them; only their
// default faces are re-pointed at the real widgets.
mod.flow.Node.ui = nil
mod.flow.Input.ui = mod.flow.ui.InputFace
mod.flow.Text.ui = mod.flow.ui.TextFace
mod.flow.Output.ui = mod.flow.ui.OutputFace
mod.flow.Publish.ui = mod.flow.ui.PublishFace
mod.flow.Llm.ui = mod.flow.ui.LlmFace
mod.flow.Fn.ui = mod.flow.ui.FnFace
mod.flow.Http.ui = mod.flow.ui.HttpFace
mod.flow.Ask.ui = mod.flow.ui.AskFace
mod.flow.Gen.ui = mod.flow.ui.GenFace
mod.flow.Image.ui = mod.flow.ui.ImageFace
mod.flow.Upscale.ui = mod.flow.ui.UpscaleFace
mod.flow.ImageFace = mod.flow.ui.ImageFace
mod.flow.VideoFace = mod.flow.ui.VideoFace
mod.flow.InputFace = mod.flow.ui.InputFace
mod.flow.TextFace = mod.flow.ui.TextFace
mod.flow.OutputFace = mod.flow.ui.OutputFace
mod.flow.PublishFace = mod.flow.ui.PublishFace
mod.flow.LlmFace = mod.flow.ui.LlmFace
mod.flow.FnFace = mod.flow.ui.FnFace
mod.flow.HttpFace = mod.flow.ui.HttpFace
mod.flow.AskFace = mod.flow.ui.AskFace
mod.flow.GenFace = mod.flow.ui.GenFace
mod.flow.UpscaleFace = mod.flow.ui.UpscaleFace
nil

View file

@ -1,938 +0,0 @@
//! Pure edits over the wire `Graph`: every canvas gesture builds the next
//! graph here and the app PUTs it. Nothing in this module talks to the
//! server or the widgets, so the helpers are unit-tested as plain data.
use makepad_flow::{
Edge, EdgeRef, Graph, Literal, Node, NodeInput, NodeInputValue, NodeTypeCatalog, Port,
PortType,
};
use makepad_flowgraph::{GraphIndex, FIRST_AT, NODE_WIDTH};
use makepad_widgets::makepad_micro_serde::JsonValue;
use std::collections::{HashMap, HashSet};
/// Canvas geometry shared by auto-placement and the canvas itself.
pub const COLUMN_GAP: f64 = 60.0;
pub const ROW_GAP: f64 = 260.0;
pub fn graph_index(graph: &Graph) -> GraphIndex {
GraphIndex::from_parts(
graph.nodes.iter().map(|node| node.id.as_str()),
graph
.edges
.iter()
.map(|edge| (edge.from_node.as_str(), edge.to_node.as_str())),
)
}
/// A fresh id `<type>_<n>`, lower-cased, that no node in the graph uses.
pub fn fresh_node_id(graph: &Graph, type_name: &str) -> String {
let base: String = type_name
.chars()
.map(|c| c.to_ascii_lowercase())
.filter(|c| c.is_ascii_alphanumeric() || *c == '_')
.collect();
let base = if base.is_empty() || base.starts_with(|c: char| c.is_ascii_digit()) {
format!("node_{base}")
} else {
base
};
let mut n = 1;
loop {
let candidate = format!("{base}_{n}");
if !graph.nodes.iter().any(|node| node.id == candidate) {
return candidate;
}
n += 1;
}
}
fn literal_from_json(value: &JsonValue) -> Literal {
match value {
JsonValue::Null | JsonValue::Undefined => Literal::Null,
JsonValue::Bool(value) => Literal::Bool(*value),
JsonValue::U64(value) => Literal::Num(*value as f64),
JsonValue::U128(value) => Literal::Num(*value as f64),
JsonValue::I64(value) => Literal::Num(*value as f64),
JsonValue::I128(value) => Literal::Num(*value as f64),
JsonValue::F64(value) => Literal::Num(*value),
JsonValue::String(value) => Literal::Str(value.clone()),
JsonValue::BareIdent(value) => Literal::Id(value.clone()),
JsonValue::Char(value) => Literal::Str(value.to_string()),
JsonValue::Array(values) => Literal::Arr(values.iter().map(literal_from_json).collect()),
JsonValue::Object(fields) => {
let mut pairs: Vec<(String, Literal)> = fields
.iter()
.map(|(key, value)| (key.clone(), literal_from_json(value)))
.collect();
pairs.sort_by(|a, b| a.0.cmp(&b.0));
Literal::Obj(pairs)
}
}
}
/// The catalog stores id-typed params (`type: @text`, `method: @get`) as
/// plain strings; the writer must emit them as `@ids`, so the params that
/// the prelude declares as ids are re-typed here.
fn id_param(type_name: &str, name: &str) -> bool {
matches!(
(type_name, name),
("Input" | "Text" | "Output" | "Ask", "type")
| ("Http", "method" | "out")
| (_, "on_fail")
)
}
pub fn port_type_from_name(name: &str) -> Option<PortType> {
Some(match name {
"text" => PortType::Text,
"image" => PortType::Image,
"audio" => PortType::Audio,
"video" => PortType::Video,
"mesh" => PortType::Mesh,
"json" => PortType::Json,
"list" => PortType::List,
"bytes" => PortType::Bytes,
_ => return None,
})
}
fn param_port_type(params: &[(String, Literal)], key: &str) -> Option<PortType> {
params.iter().find_map(|(name, value)| {
if name != key {
return None;
}
match value {
Literal::Id(text) | Literal::Str(text) => port_type_from_name(text),
_ => None,
}
})
}
/// Build a node of `catalog`'s type with the prelude's defaults.
pub fn node_from_catalog(id: String, catalog: &NodeTypeCatalog, at: (f64, f64)) -> Node {
let mut params: Vec<(String, Literal)> = catalog
.params
.iter()
.map(|param| {
let mut literal = literal_from_json(&param.default);
if id_param(&catalog.type_name, &param.name) {
if let Literal::Str(text) = &literal {
literal = Literal::Id(text.clone());
}
}
(param.name.clone(), literal)
})
.collect();
if catalog.type_name == "Fn" && !params.iter().any(|(name, _)| name == "out") {
params.push(("out".to_string(), Literal::Arr(vec![Literal::Id("text".into())])));
}
let mut inputs: Vec<NodeInput> = catalog
.ports
._in
.iter()
.map(|port| NodeInput {
port: port.name.clone(),
ty: port.ty,
value: NodeInputValue::Literal(Literal::Null),
})
.collect();
let mut outputs: Vec<Port> = catalog.ports.out.clone();
match catalog.type_name.as_str() {
"Input" | "Text" | "Ask" => {
let ty = param_port_type(&params, "type").unwrap_or(PortType::Text);
outputs = vec![Port {
name: ty.as_str().to_string(),
ty,
}];
}
"Output" => {
let ty = param_port_type(&params, "type").unwrap_or(PortType::Text);
inputs = vec![NodeInput {
port: "value".to_string(),
ty,
value: NodeInputValue::Literal(Literal::Null),
}];
outputs.clear();
}
"Fn" => {
inputs = vec![NodeInput {
port: "text".to_string(),
ty: PortType::Text,
value: NodeInputValue::Literal(Literal::Null),
}];
outputs = vec![Port {
name: "text".to_string(),
ty: PortType::Text,
}];
}
_ => {}
}
Node {
id,
kind: catalog.kind.clone(),
type_name: catalog.type_name.clone(),
params,
inputs,
outputs,
at: Some(at),
size: None,
flip: false,
loc: Default::default(),
fn_src: (catalog.type_name == "Fn").then(|| "|i| { {text: i.text} }".to_string()),
face_src: None,
on_fail: "fail".to_string(),
label: None,
domain: catalog.domain.clone(),
doc: None,
}
}
/// Add a node of the catalog type at `at`; returns the graph and the new id.
pub fn add_node(graph: &Graph, catalog: &NodeTypeCatalog, at: (f64, f64)) -> (Graph, String) {
let id = fresh_node_id(graph, &catalog.type_name);
let mut next = graph.clone();
next.nodes.push(node_from_catalog(id.clone(), catalog, at));
(next, id)
}
pub fn move_node(graph: &Graph, id: &str, at: (f64, f64)) -> Graph {
let mut next = graph.clone();
if let Some(node) = next.nodes.iter_mut().find(|node| node.id == id) {
node.at = Some(at);
}
next
}
pub fn resize_node(graph: &Graph, id: &str, size: (f64, f64)) -> Graph {
let mut next = graph.clone();
if let Some(node) = next.nodes.iter_mut().find(|node| node.id == id) {
node.size = Some(size);
}
next
}
pub fn flip_node(graph: &Graph, id: &str, flip: bool) -> Graph {
let mut next = graph.clone();
if let Some(node) = next.nodes.iter_mut().find(|node| node.id == id) {
node.flip = flip;
}
next
}
/// Mounted face trees only need rebuilding when their declarations or port
/// shape changed. Position, size, values, docs and run state are refreshed
/// in place and must not tear down an open popup or editor.
pub fn needs_face_remount(old: &Graph, new: &Graph) -> bool {
if old.flow_ui_src != new.flow_ui_src || old.nodes.len() != new.nodes.len() {
return true;
}
old.nodes.iter().any(|old_node| {
let Some(new_node) = new.nodes.iter().find(|node| node.id == old_node.id) else {
return true;
};
old_node.type_name != new_node.type_name
|| old_node.face_src != new_node.face_src
|| old_node
.inputs
.iter()
.map(|input| (&input.port, input.ty))
.ne(new_node.inputs.iter().map(|input| (&input.port, input.ty)))
|| old_node
.outputs
.iter()
.map(|output| (&output.name, output.ty))
.ne(new_node.outputs.iter().map(|output| (&output.name, output.ty)))
})
}
/// Remove a node; edges into and out of it go, dependents' inputs become
/// literal `nil`, and a tool projection that named it drops the name.
pub fn delete_node(graph: &Graph, id: &str) -> Graph {
let mut next = graph.clone();
next.nodes.retain(|node| node.id != id);
next.edges
.retain(|edge| edge.from_node != id && edge.to_node != id);
for node in &mut next.nodes {
for input in &mut node.inputs {
if let NodeInputValue::Edge(edge) = &input.value {
if edge.from_node == id {
input.value = NodeInputValue::Literal(Literal::Null);
}
}
}
}
for tool in &mut next.tools {
tool.inputs.retain(|name| name != id);
tool.outputs.retain(|name| name != id);
tool.nodes.retain(|name| name != id);
}
next
}
pub fn output_port_type(graph: &Graph, node_id: &str, port: &str) -> Option<PortType> {
graph
.nodes
.iter()
.find(|node| node.id == node_id)?
.outputs
.iter()
.find(|output| output.name == port)
.map(|output| output.ty)
}
/// Would an edge `from → to` close a loop? True when `from` already
/// depends on `to` (or they are the same node).
#[cfg(test)]
pub fn would_cycle(graph: &Graph, from: &str, to: &str) -> bool {
let index = graph_index(graph);
let (Some(from), Some(to)) = (index.node(from), index.node(to)) else {
return from == to;
};
index.ancestors(from).contains(&to)
}
/// Every input port an output can legally be wired to: same type, not the
/// same node, no cycle. `Fn` and `Publish.value` inputs are flexible (any
/// type re-types the port) and `Http.body`/`Http.headers` accept anything as
/// well.
pub fn compatible_inputs(graph: &Graph, from_node: &str, from_port: &str) -> Vec<(String, String)> {
let index = graph_index(graph);
let Some(from) = index.node(from_node) else {
return Vec::new();
};
let Some(ty) = graph.nodes[from]
.outputs
.iter()
.find(|output| output.name == from_port)
.map(|output| output.ty)
else {
return Vec::new();
};
let ancestors = index.ancestors(from);
let mut out = Vec::new();
for (node_index, node) in graph.nodes.iter().enumerate() {
if node_index == from || ancestors.contains(&node_index) {
continue;
}
for input in &node.inputs {
let flexible = node.type_name == "Fn"
|| (node.type_name == "Publish" && input.port == "value")
|| (node.type_name == "Http"
&& (input.port == "body" || input.port == "headers"))
|| (node.type_name == "Output" && input.port == "value");
if flexible || input.ty == ty {
out.push((node.id.clone(), input.port.clone()));
}
}
}
out
}
/// Connect an output to an input. Fails with a reason when the types do
/// not match or the edge would loop.
pub fn connect(
graph: &Graph,
from_node: &str,
from_port: &str,
to_node: &str,
to_port: &str,
) -> Result<Graph, String> {
let ty = output_port_type(graph, from_node, from_port)
.ok_or_else(|| format!("{from_node} has no output {from_port}"))?;
if !compatible_inputs(graph, from_node, from_port)
.iter()
.any(|(node, port)| node == to_node && port == to_port)
{
return Err(format!(
"{to_node}.{to_port} does not accept {} from {from_node}.{from_port}",
ty.as_str()
));
}
let mut next = graph.clone();
let node = next
.nodes
.iter_mut()
.find(|node| node.id == to_node)
.ok_or_else(|| format!("no node {to_node}"))?;
let input = node
.inputs
.iter_mut()
.find(|input| input.port == to_port)
.ok_or_else(|| format!("{to_node} has no input {to_port}"))?;
input.value = NodeInputValue::Edge(EdgeRef {
from_node: from_node.to_string(),
from_port: from_port.to_string(),
});
if matches!(node.type_name.as_str(), "Fn" | "Output" | "Publish") {
input.ty = ty;
}
if node.type_name == "Output" {
if let Some((_, value)) = node.params.iter_mut().find(|(name, _)| name == "type") {
*value = Literal::Id(ty.as_str().to_string());
}
}
next.edges
.retain(|edge| !(edge.to_node == to_node && edge.to_port == to_port));
next.edges.push(Edge {
from_node: from_node.to_string(),
from_port: from_port.to_string(),
to_node: to_node.to_string(),
to_port: to_port.to_string(),
});
Ok(next)
}
fn empty_literal(ty: PortType) -> Literal {
match ty {
PortType::Text => Literal::Str(String::new()),
PortType::Json => Literal::Obj(Vec::new()),
PortType::List => Literal::Arr(Vec::new()),
PortType::Image
| PortType::Audio
| PortType::Video
| PortType::Mesh
| PortType::Bytes => Literal::Null,
}
}
fn disconnected_default(type_name: &str, input: &NodeInput) -> Literal {
match (type_name, input.port.as_str()) {
("Llm", "prompt") | ("Http", "url") | ("Image" | "Gen", "prompt") => {
Literal::Str(String::new())
}
("Http", "headers") => Literal::Obj(Vec::new()),
("Fn", _) => empty_literal(input.ty),
_ => Literal::Null,
}
}
/// Drop the edge into an input. Dynamic `Fn.in` fields and built-in inputs
/// regain their empty literal default; an already-literal input is untouched.
pub fn disconnect(graph: &Graph, to_node: &str, to_port: &str) -> Graph {
let mut next = graph.clone();
next.edges
.retain(|edge| !(edge.to_node == to_node && edge.to_port == to_port));
if let Some(node) = next.nodes.iter_mut().find(|node| node.id == to_node) {
let type_name = node.type_name.clone();
if let Some(input) = node.inputs.iter_mut().find(|input| input.port == to_port) {
if matches!(input.value, NodeInputValue::Edge(_)) {
input.value = NodeInputValue::Literal(disconnected_default(&type_name, input));
}
}
}
next
}
/// Set a param, or a literal input of the same name. `type` on an Input /
/// Output / Ask re-types its port so the graph stays consistent.
pub fn set_param(graph: &Graph, node_id: &str, key: &str, value: Literal) -> Graph {
let mut next = graph.clone();
let Some(node) = next.nodes.iter_mut().find(|node| node.id == node_id) else {
return next;
};
let mut is_param = false;
if let Some((_, slot)) = node.params.iter_mut().find(|(name, _)| name == key) {
*slot = value.clone();
is_param = true;
}
if let Some(input) = node.inputs.iter_mut().find(|input| input.port == key) {
if !is_param || matches!(input.value, NodeInputValue::Literal(_)) {
input.value = NodeInputValue::Literal(value.clone());
}
} else if !is_param {
node.params.push((key.to_string(), value.clone()));
}
if key == "type" {
let ty = match &value {
Literal::Id(text) | Literal::Str(text) => port_type_from_name(text),
_ => None,
};
if let Some(ty) = ty {
match node.type_name.as_str() {
"Input" | "Text" | "Ask" => {
node.outputs = vec![Port {
name: ty.as_str().to_string(),
ty,
}];
let id = node.id.clone();
next.edges.retain(|edge| edge.from_node != id);
for other in &mut next.nodes {
for input in &mut other.inputs {
if matches!(&input.value, NodeInputValue::Edge(edge) if edge.from_node == id)
{
input.value = NodeInputValue::Literal(Literal::Null);
}
}
}
}
"Output" => {
if let Some(input) = node.inputs.first_mut() {
input.ty = ty;
}
}
_ => {}
}
}
}
next
}
pub fn set_fn_src(graph: &Graph, node_id: &str, src: &str) -> Graph {
let mut next = graph.clone();
if let Some(node) = next.nodes.iter_mut().find(|node| node.id == node_id) {
node.fn_src = Some(src.trim().to_string());
}
next
}
pub fn set_face_src(graph: &Graph, node_id: &str, src: &str) -> Graph {
let mut next = graph.clone();
if let Some(node) = next.nodes.iter_mut().find(|node| node.id == node_id) {
let src = src.trim();
node.face_src = (!src.is_empty() && src != "nil").then(|| src.to_string());
}
next
}
/// Dependency depth of every node: sources are 0, a node is one past its
/// deepest input.
pub fn depths(graph: &Graph) -> HashMap<String, usize> {
let mut depth: HashMap<String, usize> = HashMap::new();
let mut upstream: HashMap<&str, Vec<&str>> = HashMap::new();
for edge in &graph.edges {
upstream
.entry(edge.to_node.as_str())
.or_default()
.push(edge.from_node.as_str());
}
fn walk<'a>(
id: &'a str,
upstream: &HashMap<&'a str, Vec<&'a str>>,
depth: &mut HashMap<String, usize>,
guard: &mut HashSet<&'a str>,
) -> usize {
if let Some(value) = depth.get(id) {
return *value;
}
if !guard.insert(id) {
return 0;
}
let value = upstream
.get(id)
.map(|sources| {
sources
.iter()
.map(|source| walk(source, upstream, depth, guard) + 1)
.max()
.unwrap_or(0)
})
.unwrap_or(0);
guard.remove(id);
depth.insert(id.to_string(), value);
value
}
for node in &graph.nodes {
let mut guard = HashSet::new();
walk(&node.id, &upstream, &mut depth, &mut guard);
}
depth
}
/// Give every node without an `at` a place. A node with wired inputs sits
/// one column to the right of its right-most upstream node, on its row,
/// using the column pitch the placed nodes already use (else the default);
/// a source sits in the first column, below anything already there. Nodes
/// are placed in dependency order, so every upstream node has a place first.
pub fn auto_place(graph: &mut Graph) {
let depth = depths(graph);
let mut xs: Vec<f64> = graph.nodes.iter().filter_map(|node| node.at.map(|at| at.0)).collect();
xs.sort_by(|a, b| a.total_cmp(b));
xs.dedup_by(|a, b| (*a - *b).abs() < 1.0);
let mut deltas: Vec<f64> = xs
.windows(2)
.map(|pair| pair[1] - pair[0])
.filter(|delta| *delta >= NODE_WIDTH)
.collect();
deltas.sort_by(|a, b| a.total_cmp(b));
let pitch = deltas
.get(deltas.len() / 2)
.copied()
.unwrap_or(NODE_WIDTH + COLUMN_GAP);
let node_index: HashMap<String, usize> = graph
.nodes
.iter()
.enumerate()
.map(|(index, node)| (node.id.clone(), index))
.collect();
let mut upstream = vec![Vec::new(); graph.nodes.len()];
for edge in &graph.edges {
if let (Some(from), Some(to)) = (
node_index.get(&edge.from_node).copied(),
node_index.get(&edge.to_node).copied(),
) {
upstream[to].push(from);
}
}
let mut order: Vec<usize> = (0..graph.nodes.len()).collect();
order.sort_by_key(|index| depth.get(&graph.nodes[*index].id).copied().unwrap_or(0));
let mut first_column_bottom = graph
.nodes
.iter()
.filter_map(|node| node.at)
.filter(|(x, _)| (*x - FIRST_AT.0).abs() < NODE_WIDTH)
.map(|(_, y)| y)
.fold(FIRST_AT.1 - ROW_GAP, f64::max);
let cell_height = ROW_GAP * 0.5;
let mut occupied: HashMap<(i64, i64), Vec<(f64, f64)>> = HashMap::new();
let cell = |at: (f64, f64)| {
(
(at.0 / NODE_WIDTH).floor() as i64,
(at.1 / cell_height).floor() as i64,
)
};
for at in graph.nodes.iter().filter_map(|node| node.at) {
occupied.entry(cell(at)).or_default().push(at);
}
for index in order {
if graph.nodes[index].at.is_some() {
continue;
}
let anchor = upstream
.get(index)
.into_iter()
.flatten()
.filter_map(|from| graph.nodes[*from].at)
.max_by(|a, b| a.0.total_cmp(&b.0));
let mut at = match anchor {
Some((x, y)) => (x + pitch, y),
None => {
first_column_bottom += ROW_GAP;
(FIRST_AT.0, first_column_bottom)
}
};
// Never on top of a placed node: step down a row until clear.
while {
let (cell_x, cell_y) = cell(at);
(-1..=1).any(|dx| {
(-1..=1).any(|dy| {
occupied
.get(&(cell_x + dx, cell_y + dy))
.is_some_and(|points| {
points.iter().any(|(x, y)| {
(*x - at.0).abs() < NODE_WIDTH
&& (*y - at.1).abs() < cell_height
})
})
})
})
} {
at.1 += ROW_GAP;
}
graph.nodes[index].at = Some(at);
occupied.entry(cell(at)).or_default().push(at);
}
}
/// Catalog types with at least one input that accepts `ty`.
pub fn types_with_compatible_input<'a>(
catalog: &'a [NodeTypeCatalog],
ty: PortType,
) -> Vec<&'a NodeTypeCatalog> {
catalog
.iter()
.filter(|entry| {
matches!(entry.type_name.as_str(), "Fn" | "Output" | "Publish")
|| entry.ports._in.iter().any(|port| port.ty == ty)
})
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
use makepad_flow::{NodeParamCatalog, NodePortsCatalog};
fn catalog(type_name: &str, kind: &str, ins: &[(&str, PortType)], outs: &[(&str, PortType)]) -> NodeTypeCatalog {
NodeTypeCatalog {
type_name: type_name.to_string(),
kind: kind.to_string(),
domain: None,
models: Vec::new(),
ports: NodePortsCatalog {
_in: ins
.iter()
.map(|(name, ty)| Port { name: name.to_string(), ty: *ty })
.collect(),
out: outs
.iter()
.map(|(name, ty)| Port { name: name.to_string(), ty: *ty })
.collect(),
},
params: vec![
NodeParamCatalog {
name: "type".to_string(),
default: JsonValue::String("text".to_string()),
doc: String::new(),
range: None,
},
NodeParamCatalog {
name: "steps".to_string(),
default: JsonValue::F64(8.0),
doc: "1..50".to_string(),
range: None,
},
],
face: format!("{type_name}Face"),
doc: String::new(),
}
}
fn empty() -> Graph {
Graph {
revision: 1,
label: "t".into(),
brief: String::new(),
trigger: "manual".into(),
concurrency: 1,
autostart: false,
nodes: Vec::new(),
edges: Vec::new(),
tools: Vec::new(),
flow_ui_src: None,
warnings: Vec::new(),
}
}
#[test]
fn add_node_gets_a_fresh_id_and_prelude_defaults() {
let input = catalog("Input", "input", &[], &[("text", PortType::Text)]);
let (g, id) = add_node(&empty(), &input, (10.0, 20.0));
assert_eq!(id, "input_1");
let (g, id2) = add_node(&g, &input, (10.0, 20.0));
assert_eq!(id2, "input_2");
let node = &g.nodes[0];
assert_eq!(node.at, Some((10.0, 20.0)));
assert_eq!(node.outputs[0].name, "text");
assert!(matches!(
node.params.iter().find(|(n, _)| n == "type").unwrap().1,
Literal::Id(ref t) if t == "text"
));
assert!(matches!(
node.params.iter().find(|(n, _)| n == "steps").unwrap().1,
Literal::Num(v) if v == 8.0
));
}
#[test]
fn face_remount_ignores_layout_but_catches_structure() {
let input = catalog("Input", "input", &[], &[("text", PortType::Text)]);
let (graph, id) = add_node(&empty(), &input, (10.0, 20.0));
let moved = resize_node(&move_node(&graph, &id, (31.5, 47.25)), &id, (440.0, 180.0));
assert!(!needs_face_remount(&graph, &moved));
let mut changed_face = graph.clone();
changed_face.nodes[0].face_src = Some("View{}".into());
assert!(needs_face_remount(&graph, &changed_face));
let mut changed_ports = graph.clone();
changed_ports.nodes[0].outputs[0].ty = PortType::Image;
assert!(needs_face_remount(&graph, &changed_ports));
}
#[test]
fn connect_checks_types_and_writes_the_edge() {
let input = catalog("Input", "input", &[], &[("text", PortType::Text)]);
let image = catalog(
"Image",
"gen",
&[("prompt", PortType::Text), ("image", PortType::Image)],
&[("image", PortType::Image)],
);
let (g, a) = add_node(&empty(), &input, (0.0, 0.0));
let (g, b) = add_node(&g, &image, (0.0, 0.0));
let compatible = compatible_inputs(&g, &a, "text");
assert_eq!(compatible, vec![(b.clone(), "prompt".to_string())]);
assert!(connect(&g, &a, "text", &b, "image").is_err());
let g = connect(&g, &a, "text", &b, "prompt").unwrap();
assert_eq!(g.edges.len(), 1);
let prompt = g.nodes[1].inputs.iter().find(|i| i.port == "prompt").unwrap();
assert!(matches!(&prompt.value, NodeInputValue::Edge(e) if e.from_node == a && e.from_port == "text"));
assert!(would_cycle(&g, &b, &a));
let g = disconnect(&g, &b, "prompt");
assert!(g.edges.is_empty());
assert!(matches!(
g.nodes[1].inputs.iter().find(|i| i.port == "prompt").unwrap().value,
NodeInputValue::Literal(Literal::Str(ref value)) if value.is_empty()
));
}
#[test]
fn publish_value_accepts_and_retypes_non_image_outputs() {
let input = catalog("Input", "input", &[], &[("text", PortType::Text)]);
let publish = catalog(
"Publish",
"publish",
&[("value", PortType::Image)],
&[("asset", PortType::Json)],
);
let (graph, source) = add_node(&empty(), &input, (0.0, 0.0));
let (graph, target) = add_node(&graph, &publish, (300.0, 0.0));
assert!(compatible_inputs(&graph, &source, "text")
.contains(&(target.clone(), "value".to_string())));
let graph = connect(&graph, &source, "text", &target, "value").unwrap();
assert_eq!(graph.nodes[1].inputs[0].ty, PortType::Text);
}
#[test]
fn disconnect_restores_llm_prompt_default() {
let input = catalog("Input", "input", &[], &[("text", PortType::Text)]);
let llm = catalog("Llm", "chat", &[("prompt", PortType::Text)], &[("text", PortType::Text)]);
let (graph, source) = add_node(&empty(), &input, (0.0, 0.0));
let (graph, target) = add_node(&graph, &llm, (300.0, 0.0));
let graph = connect(&graph, &source, "text", &target, "prompt").unwrap();
let graph = disconnect(&graph, &target, "prompt");
assert!(matches!(
graph.nodes[1].inputs[0].value,
NodeInputValue::Literal(Literal::Str(ref value)) if value.is_empty()
));
}
#[test]
fn disconnect_restores_fn_text_input_and_preserves_a_literal() {
let input = catalog("Input", "input", &[], &[("text", PortType::Text)]);
let function = catalog("Fn", "fn", &[("text", PortType::Text)], &[("text", PortType::Text)]);
let (graph, source) = add_node(&empty(), &input, (0.0, 0.0));
let (graph, target) = add_node(&graph, &function, (300.0, 0.0));
let graph = connect(&graph, &source, "text", &target, "text").unwrap();
let graph = disconnect(&graph, &target, "text");
assert!(matches!(
graph.nodes[1].inputs[0].value,
NodeInputValue::Literal(Literal::Str(ref value)) if value.is_empty()
));
let mut literal_graph = graph.clone();
literal_graph.nodes[1].inputs[0].value =
NodeInputValue::Literal(Literal::Str("kept".into()));
let literal_graph = disconnect(&literal_graph, &target, "text");
assert!(matches!(
literal_graph.nodes[1].inputs[0].value,
NodeInputValue::Literal(Literal::Str(ref value)) if value == "kept"
));
}
#[test]
fn output_adopts_the_type_it_is_wired_to() {
let image = catalog("Image", "gen", &[("prompt", PortType::Text)], &[("image", PortType::Image)]);
let output = catalog("Output", "output", &[], &[]);
let (g, a) = add_node(&empty(), &image, (0.0, 0.0));
let (g, b) = add_node(&g, &output, (0.0, 0.0));
let g = connect(&g, &a, "image", &b, "value").unwrap();
let out = &g.nodes[1];
assert_eq!(out.inputs[0].ty, PortType::Image);
assert!(matches!(
out.params.iter().find(|(n, _)| n == "type").unwrap().1,
Literal::Id(ref t) if t == "image"
));
}
#[test]
fn move_and_delete() {
let input = catalog("Input", "input", &[], &[("text", PortType::Text)]);
let llm = catalog("Llm", "chat", &[("prompt", PortType::Text)], &[("text", PortType::Text)]);
let (g, a) = add_node(&empty(), &input, (0.0, 0.0));
let (g, b) = add_node(&g, &llm, (0.0, 0.0));
let g = connect(&g, &a, "text", &b, "prompt").unwrap();
let g = move_node(&g, &b, (300.4, 99.6));
assert_eq!(g.nodes[1].at, Some((300.4, 99.6)));
let g = delete_node(&g, &a);
assert_eq!(g.nodes.len(), 1);
assert!(g.edges.is_empty());
assert!(matches!(g.nodes[0].inputs[0].value, NodeInputValue::Literal(Literal::Null)));
}
#[test]
fn set_param_retypes_an_input_port() {
let input = catalog("Input", "input", &[], &[("text", PortType::Text)]);
let (g, a) = add_node(&empty(), &input, (0.0, 0.0));
let g = set_param(&g, &a, "type", Literal::Id("image".into()));
assert_eq!(g.nodes[0].outputs[0].name, "image");
assert_eq!(g.nodes[0].outputs[0].ty, PortType::Image);
let g = set_param(&g, &a, "value", Literal::Str("x".into()));
assert!(matches!(
g.nodes[0].params.iter().find(|(n, _)| n == "value").unwrap().1,
Literal::Str(ref t) if t == "x"
));
}
#[test]
fn auto_place_walks_left_to_right_by_dependency() {
let input = catalog("Input", "input", &[], &[("text", PortType::Text)]);
let llm = catalog("Llm", "chat", &[("prompt", PortType::Text)], &[("text", PortType::Text)]);
let (g, a) = add_node(&empty(), &input, (0.0, 0.0));
let (g, b) = add_node(&g, &llm, (0.0, 0.0));
let (g, c) = add_node(&g, &llm, (0.0, 0.0));
let mut g = connect(&g, &a, "text", &b, "prompt").unwrap();
g = connect(&g, &b, "text", &c, "prompt").unwrap();
for node in &mut g.nodes {
node.at = None;
}
auto_place(&mut g);
let at = |id: &str| g.nodes.iter().find(|n| n.id == id).unwrap().at.unwrap();
assert_eq!(at(&a), FIRST_AT);
assert_eq!(at(&b).0, FIRST_AT.0 + NODE_WIDTH + COLUMN_GAP);
assert_eq!(at(&c).0, FIRST_AT.0 + 2.0 * (NODE_WIDTH + COLUMN_GAP));
assert_eq!(at(&b).1, FIRST_AT.1);
}
#[test]
fn compatible_types_filter() {
let cats = vec![
catalog("Input", "input", &[], &[("text", PortType::Text)]),
catalog("Image", "gen", &[("prompt", PortType::Text)], &[("image", PortType::Image)]),
catalog("Upscale", "gen", &[("image", PortType::Image)], &[("image", PortType::Image)]),
];
let names: Vec<_> = types_with_compatible_input(&cats, PortType::Image)
.iter()
.map(|c| c.type_name.as_str())
.collect();
assert_eq!(names, vec!["Upscale"]);
}
#[test]
fn graph_index_storage_and_compatibility_scan_are_linear_in_graph_size() {
let input = catalog("Input", "input", &[], &[("text", PortType::Text)]);
let llm = catalog(
"Llm",
"chat",
&[("prompt", PortType::Text)],
&[("text", PortType::Text)],
);
let (seed, _) = add_node(&empty(), &input, (0.0, 0.0));
let (template, _) = add_node(&empty(), &llm, (0.0, 0.0));
let mut graph = empty();
let mut source = seed.nodes[0].clone();
source.id = "n0".into();
graph.nodes.push(source);
for index in 1..512 {
let mut node = template.nodes[0].clone();
node.id = format!("n{index}");
node.inputs[0].value = NodeInputValue::Edge(EdgeRef {
from_node: format!("n{}", index - 1),
from_port: "text".into(),
});
graph.edges.push(Edge {
from_node: format!("n{}", index - 1),
from_port: "text".into(),
to_node: node.id.clone(),
to_port: "prompt".into(),
});
graph.nodes.push(node);
}
let index = graph_index(&graph);
assert_eq!(index.ancestor_indices("n511").len(), graph.nodes.len());
assert_eq!(
compatible_inputs(&graph, "n0", "text").len(),
graph.nodes.len() - 1
);
}
}

View file

@ -1,241 +0,0 @@
//! Thin projection from the editable flow graph into the reusable canvas model.
use crate::graph_edit;
use makepad_flow::{Graph, Literal, Node, NodeInputValue, PortType};
use makepad_flowgraph::{CompatiblePorts, EdgeView, GraphView, NodeView, PortView};
use std::collections::{HashMap, HashSet};
pub(crate) fn declared_output_type(node: &Node) -> Option<PortType> {
if node.type_name != "Output" {
return None;
}
node.params
.iter()
.find_map(|(name, value)| {
if name != "type" {
return None;
}
match value {
Literal::Id(name) | Literal::Str(name) => PortType::from_str(name),
_ => None,
}
})
.or_else(|| node.inputs.first().map(|input| input.ty))
}
#[derive(Clone, Copy, PartialEq, Eq)]
pub(crate) enum PortIcon {
Text,
Image,
Audio,
Video,
Mesh,
Json,
Bytes,
}
impl PortIcon {
pub(crate) fn for_type(ty: PortType) -> Self {
match ty {
PortType::Text => Self::Text,
PortType::Image => Self::Image,
PortType::Audio => Self::Audio,
PortType::Video => Self::Video,
PortType::Mesh => Self::Mesh,
PortType::Json | PortType::List => Self::Json,
PortType::Bytes => Self::Bytes,
}
}
}
fn full_bleed(node: &Node) -> bool {
match node.kind.as_str() {
"output" => declared_output_type(node)
.or_else(|| node.inputs.first().map(|input| input.ty))
.is_some_and(PortType::is_media),
// A generator card shows its settings; the picture lives on the
// Output card it feeds (user, 2026-09-04).
"input" => node
.outputs
.first()
.is_some_and(|port| port.ty.is_media()),
_ => false,
}
}
/// Project the host graph, including its app-owned automatic placement, into
/// the string-keyed data consumed by `FlowCanvas`.
pub fn view_of(graph: &Graph) -> GraphView {
let mut graph = graph.clone();
graph_edit::auto_place(&mut graph);
let connected_outputs: HashSet<(&str, &str)> = graph
.edges
.iter()
.map(|edge| (edge.from_node.as_str(), edge.from_port.as_str()))
.collect();
GraphView {
nodes: graph
.nodes
.iter()
.map(|node| NodeView {
id: node.id.clone(),
title: node.id.clone(),
type_name: node.type_name.clone(),
kind: node.kind.clone(),
at: node.at.unwrap_or(makepad_flowgraph::FIRST_AT),
size: node.size,
flip: node.flip,
inputs: node
.inputs
.iter()
.map(|input| PortView {
name: input.port.clone(),
kind: input.ty.as_str().to_string(),
connected: matches!(input.value, NodeInputValue::Edge(_)),
})
.collect(),
outputs: node
.outputs
.iter()
.map(|output| PortView {
name: output.name.clone(),
kind: output.ty.as_str().to_string(),
connected: connected_outputs
.contains(&(node.id.as_str(), output.name.as_str())),
})
.collect(),
full_bleed: full_bleed(node),
params: node
.params
.iter()
.filter_map(|(name, value)| match value {
Literal::Id(value) | Literal::Str(value) => {
Some((name.clone(), value.clone()))
}
_ => None,
})
.collect(),
})
.collect(),
edges: graph
.edges
.iter()
.map(|edge| EdgeView {
from: edge.from_node.clone(),
from_port: edge.from_port.clone(),
to: edge.to_node.clone(),
to_port: edge.to_port.clone(),
})
.collect(),
}
}
/// Precompute host type/cycle policy for every output. The canvas only turns
/// the selected output's string targets into node/port indices during a drag.
pub fn compatibility_of(graph: &Graph) -> CompatiblePorts {
let mut compatible = HashMap::new();
for node in &graph.nodes {
for output in &node.outputs {
compatible.insert(
(node.id.clone(), output.name.clone()),
graph_edit::compatible_inputs(graph, &node.id, &output.name)
.into_iter()
.collect(),
);
}
}
compatible
}
#[cfg(test)]
mod tests {
use super::*;
use makepad_flow::{Edge, EdgeRef, Loc, NodeInput, Port};
fn node(id: &str, kind: &str, ty: PortType) -> Node {
Node {
id: id.into(),
kind: kind.into(),
type_name: if kind == "output" { "Output" } else { "Input" }.into(),
params: Vec::new(),
inputs: Vec::new(),
outputs: vec![Port {
name: ty.as_str().into(),
ty,
}],
at: Some((10.0, 20.0)),
size: None,
flip: false,
loc: Loc::default(),
fn_src: None,
face_src: None,
on_fail: "fail".into(),
label: None,
domain: None,
doc: None,
}
}
#[test]
fn projects_ports_edges_layout_and_full_bleed() {
let mut source = node("picture", "input", PortType::Image);
source.size = Some((420.0, 260.0));
source.flip = true;
let mut sink = node("result", "output", PortType::Text);
sink.outputs.clear();
sink.params = vec![("type".into(), Literal::Id("image".into()))];
sink.inputs.push(NodeInput {
port: "value".into(),
ty: PortType::Image,
value: NodeInputValue::Edge(EdgeRef {
from_node: "picture".into(),
from_port: "image".into(),
}),
});
let graph = Graph {
revision: 1,
label: "test".into(),
brief: String::new(),
trigger: "manual".into(),
concurrency: 1,
autostart: false,
nodes: vec![source, sink],
edges: vec![Edge {
from_node: "picture".into(),
from_port: "image".into(),
to_node: "result".into(),
to_port: "value".into(),
}],
tools: Vec::new(),
flow_ui_src: None,
warnings: Vec::new(),
};
let view = view_of(&graph);
assert_eq!(view.edges[0].from_port, "image");
assert_eq!(view.nodes[0].at, (10.0, 20.0));
assert_eq!(view.nodes[0].size, Some((420.0, 260.0)));
assert!(view.nodes[0].flip);
assert!(view.nodes[0].full_bleed);
assert!(view.nodes[0].outputs[0].connected);
assert!(view.nodes[1].inputs[0].connected);
assert!(view.nodes[1].full_bleed);
}
#[test]
fn every_media_output_is_full_bleed() {
for ty in [
PortType::Image,
PortType::Video,
PortType::Audio,
PortType::Mesh,
] {
let input = node("generator", "input", ty);
let mut output = node("output", "output", ty);
output.outputs.clear();
output.params = vec![("type".into(), Literal::Id(ty.as_str().into()))];
assert!(full_bleed(&input), "input generator for {ty:?}");
assert!(full_bleed(&output), "Output card for {ty:?}");
}
}
}

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -1,930 +0,0 @@
//! The run queue panel and its event-fed model. Queue rows are keyed by run
//! id; no timer or HTTP poll is needed to advance their state.
use crate::panels::RunBar;
use makepad_flow::{
CreateBatchResponse, Event as FlowEvent, Literal, NodeRowDto, NodeState, RunRowDto, RunState,
ValueRef,
};
use makepad_widgets::*;
use std::collections::{HashMap, HashSet};
script_mod! {
use mod.prelude.widgets_internal.*
use mod.widgets.*
mod.widgets.QueueListBase = #(QueueList::register_widget(vm))
mod.widgets.QueueList = set_type_default() do mod.widgets.QueueListBase{
width: Fill
height: Fill
flow: Down
spacing: 2
tools := View{
width: Fill
height: 20
flow: Right
align: Align{y: 0.5}
padding: Inset{left: 2 right: 0}
title := Label{
width: Fill
height: Fit
text: "QUEUE"
draw_text +: {
color: theme.flow_text_subtle
text_style: theme.font_bold{font_size: 8}
}
}
clear_all := ButtonFlatter{
width: Fit
height: 20
text: "Clear all"
padding: Inset{left: 4 right: 2 top: 0 bottom: 0}
draw_text +: {
color: theme.flow_text_muted
text_style: theme.font_regular{font_size: 8}
}
}
}
hint := Label{
width: Fill
height: Fit
margin: Inset{top: 2}
text: "The queue is empty. Ctrl+Enter adds a batch."
draw_text +: {
color: theme.flow_text_hint
text_style: theme.font_regular{font_size: 8}
}
}
list := PortalList{
width: Fill
height: Fill
scroll_bar: ScrollBar{}
Header := View{
width: Fill
height: 18
flow: Right
align: Align{y: 0.5}
padding: Inset{left: 4 right: 2}
spacing: 4
title := Label{
width: Fill
height: Fit
draw_text +: {
color: theme.flow_text_muted
text_style: theme.font_bold{font_size: 7.5}
}
}
cancel_batch := ButtonFlatter{
width: 18 height: 18 text: ""
padding: Inset{left: 0 right: 0 top: 0 bottom: 0}
icon_walk: Walk{width: 8 height: 8}
draw_icon +: {
svg: crate_resource("self:resources/icons/close.svg")
color: theme.flow_text_muted
}
}
}
// One run: its name, a thin strip in the state's colour, the
// state and time in words, then its x. Everything sits on one
// centre line.
Run := RoundedView{
width: Fill
height: 24
flow: Right
align: Align{y: 0.5}
padding: Inset{left: 4 right: 4}
spacing: 6
cursor: MouseCursor.Hand
capture_overload: true
show_bg: true
draw_bg +: {color: theme.flow_surface border_radius: 6}
select := ButtonFlatter{
width: 56 height: 20 text: "#1"
padding: Inset{left: 2 right: 2 top: 0 bottom: 0}
align: Align{x: 0.0 y: 0.5}
draw_text +: {
color: theme.flow_text
text_style: theme.font_bold{font_size: 8.5}
}
}
progress := RunBar{width: Fill height: Fill thickness: 4}
meta := Label{
width: 80 height: Fit text: "queued"
draw_text +: {
color: theme.flow_text_muted
text_style: theme.font_regular{font_size: 8}
}
}
asset := ButtonFlatter{
width: Fit height: 18 text: "asset" visible: false
padding: Inset{left: 2 right: 2 top: 0 bottom: 0}
draw_text +: {
color: theme.flow_highlight
text_style: theme.font_regular{font_size: 8}
}
}
cancel_run := ButtonFlatter{
width: 18 height: 18 text: ""
padding: Inset{left: 0 right: 0 top: 0 bottom: 0}
icon_walk: Walk{width: 8 height: 8}
draw_icon +: {
svg: crate_resource("self:resources/icons/close.svg")
color: theme.flow_text_muted
}
}
}
}
}
}
#[derive(Clone, Debug, PartialEq)]
pub enum QueueAction {
Select {
run_id: String,
instance: String,
flow: String,
revision: u64,
state: RunState,
planned_nodes: Vec<String>,
started_ms: u64,
finished_ms: Option<u64>,
},
CancelRun {
run_id: String,
instance: String,
/// A batch slice keeps its siblings; a single run is stopped whole.
batch: bool,
},
CancelBatch(String),
ClearAll(ClearPlan),
OpenAsset(String),
}
/// What `Clear all` asks the server for: every batch is cleared as one, and
/// each single run (a Play run) is stopped the way its own `x` stops it.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct ClearPlan {
pub batches: Vec<String>,
/// `(run id, instance)` pairs.
pub singles: Vec<(String, String)>,
}
#[derive(Clone, Debug, PartialEq)]
pub struct QueueRun {
pub run_id: String,
pub instance: String,
pub flow: String,
/// The batch this slice belongs to; a Play run has none.
pub batch: Option<String>,
pub index: u64,
pub revision: u64,
pub state: RunState,
pub planned_nodes: Vec<String>,
pub nodes: HashMap<String, NodeRowDto>,
pub started_ms: u64,
pub finished_ms: Option<u64>,
pub asset: Option<String>,
}
impl QueueRun {
fn from_row(row: RunRowDto) -> Self {
let asset = asset_from_nodes(&row.nodes);
Self {
batch: row.batch.clone(),
run_id: row.run_id,
instance: row.instance,
flow: row.flow,
index: row.batch_index.unwrap_or(1),
revision: row.revision,
state: row.state,
planned_nodes: row.planned_nodes,
nodes: row.nodes,
started_ms: row.started_ms,
finished_ms: row.finished_ms,
asset,
}
}
pub fn permille(&self) -> u16 {
if self.state == RunState::Done {
return 1000;
}
if self.planned_nodes.is_empty() {
return 0;
}
let total: u64 = self
.planned_nodes
.iter()
.map(|node| {
self.nodes.get(node).map_or(0, |row| match row.state {
NodeState::Done | NodeState::Skipped => 1000,
_ => u64::from(row.progress.unwrap_or(0)),
})
})
.sum();
(total / self.planned_nodes.len() as u64).min(1000) as u16
}
fn terminal(&self) -> bool {
matches!(
self.state,
RunState::Done | RunState::Failed | RunState::Cancelled
)
}
}
/// One group of rows: a batch under its header, or a single Play run with
/// no header (`id: None`).
#[derive(Clone, Debug, PartialEq)]
pub struct QueueBatch {
pub id: Option<String>,
pub runs: Vec<QueueRun>,
}
#[derive(Clone, Debug, Default, PartialEq)]
pub struct QueueModel {
pub batches: Vec<QueueBatch>,
pub selected: Option<String>,
/// Single runs the user cleared or stopped: the server keeps their rows
/// for a while, and a snapshot must not bring them back.
hidden: HashSet<String>,
}
impl QueueModel {
pub fn add_batch(
&mut self,
flow: &str,
response: CreateBatchResponse,
planned_nodes: &[String],
revision: u64,
now_ms: u64,
) {
let runs = response
.runs
.into_iter()
.enumerate()
.map(|(offset, run)| QueueRun {
run_id: run.run_id,
instance: run.instance,
flow: flow.to_string(),
batch: Some(response.batch.clone()),
index: offset as u64 + 1,
revision,
state: RunState::Queued,
planned_nodes: planned_nodes.to_vec(),
nodes: HashMap::new(),
started_ms: now_ms,
finished_ms: None,
asset: None,
})
.collect();
self.batches
.retain(|batch| batch.id.as_deref() != Some(response.batch.as_str()));
self.batches.insert(
0,
QueueBatch {
id: Some(response.batch),
runs,
},
);
}
/// Every run the server lists, newest first: batch slices under their
/// header, Play runs on their own.
pub fn set_rows(&mut self, rows: Vec<RunRowDto>) {
let selected = self.selected.clone();
let listed: HashSet<&str> = rows.iter().map(|row| row.run_id.as_str()).collect();
self.hidden.retain(|run_id| listed.contains(run_id.as_str()));
let mut rows: Vec<RunRowDto> = rows
.into_iter()
.filter(|row| !self.hidden.contains(&row.run_id))
.collect();
rows.sort_by(|left, right| {
right
.started_ms
.cmp(&left.started_ms)
.then_with(|| left.batch_index.cmp(&right.batch_index))
});
let mut batches = Vec::<QueueBatch>::new();
for row in rows {
let run = QueueRun::from_row(row);
match run.batch.clone() {
Some(id) => {
if let Some(batch) = batches
.iter_mut()
.find(|batch| batch.id.as_deref() == Some(id.as_str()))
{
batch.runs.push(run);
} else {
batches.push(QueueBatch {
id: Some(id),
runs: vec![run],
});
}
}
None => batches.push(QueueBatch {
id: None,
runs: vec![run],
}),
}
}
for batch in &mut batches {
batch.runs.sort_by_key(|run| run.index);
}
self.batches = batches;
self.selected = selected.filter(|selected| self.run(selected).is_some());
}
pub fn apply_event(&mut self, event: &FlowEvent, now_ms: u64) -> bool {
let Some(run_id) = event.run_id.as_deref() else {
return false;
};
let Some(run) = self.run_mut(run_id) else {
return false;
};
let node = event.node.as_deref().unwrap_or_default();
match event.kind.as_str() {
"run.started" => {
run.state = RunState::Running;
run.started_ms = now_ms;
if let Some(planned) = event.planned_nodes.as_ref() {
run.planned_nodes.clone_from(planned);
}
}
"node.started" => set_node_state(run, node, NodeState::Running, Some(0)),
"node.progress" => set_node_state(
run,
node,
NodeState::Running,
Some(event.permille.unwrap_or(0).min(1000) as u16),
),
"node.waiting" => {
run.state = RunState::Waiting;
set_node_state(run, node, NodeState::Waiting, None);
}
"node.answered" => {
run.state = RunState::Running;
set_node_state(run, node, NodeState::Running, None);
}
"node.done" => {
let outputs = event.output_values();
if let Some(asset) = outputs
.iter()
.find(|(port, _)| port == "asset")
.map(|(_, value)| asset_text(value))
{
run.asset = Some(asset);
}
let row = run.nodes.entry(node.to_string()).or_insert_with(empty_node);
row.state = NodeState::Done;
row.progress = Some(1000);
row.outputs = outputs
.into_iter()
.map(|(port, value)| makepad_flow::PortValueRef { port, value })
.collect();
}
"node.failed" => set_node_state(run, node, NodeState::Failed, None),
"node.skipped" => set_node_state(run, node, NodeState::Skipped, Some(1000)),
"run.finished" => {
run.state = event
.state_text()
.as_deref()
.and_then(parse_run_state)
.unwrap_or(RunState::Done);
run.finished_ms = Some(now_ms);
}
_ => return false,
}
true
}
pub fn select(&mut self, run_id: &str) -> bool {
if self.run(run_id).is_none() {
return false;
}
self.selected = Some(run_id.to_string());
true
}
/// A stopped single run leaves the list now and stays out of later
/// snapshots.
pub fn hide_run(&mut self, run_id: &str) {
self.hidden.insert(run_id.to_string());
for batch in &mut self.batches {
batch.runs.retain(|run| run.run_id != run_id);
}
self.batches.retain(|batch| !batch.runs.is_empty());
if self.selected.as_deref() == Some(run_id) {
self.selected = None;
}
}
pub fn clear_all(&mut self) -> ClearPlan {
let mut plan = ClearPlan::default();
for batch in self.batches.drain(..) {
match batch.id {
Some(id) => plan.batches.push(id),
None => {
for run in batch.runs {
self.hidden.insert(run.run_id.clone());
plan.singles.push((run.run_id, run.instance));
}
}
}
}
self.selected = None;
plan
}
pub fn run(&self, run_id: &str) -> Option<&QueueRun> {
self.batches
.iter()
.flat_map(|batch| &batch.runs)
.find(|run| run.run_id == run_id)
}
fn run_mut(&mut self, run_id: &str) -> Option<&mut QueueRun> {
self.batches
.iter_mut()
.flat_map(|batch| &mut batch.runs)
.find(|run| run.run_id == run_id)
}
fn items(&self) -> Vec<QueueItem> {
let mut items = Vec::new();
for batch in &self.batches {
if let Some(id) = batch.id.as_ref() {
items.push(QueueItem::Header {
id: id.clone(),
runs: batch.runs.len(),
done: batch.runs.iter().filter(|run| run.terminal()).count(),
});
}
items.extend(batch.runs.iter().cloned().map(QueueItem::Run));
}
items
}
}
#[derive(Clone, Debug)]
enum QueueItem {
Header { id: String, runs: usize, done: usize },
Run(QueueRun),
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum RunRowHit {
Select,
Cancel,
Asset,
}
fn resolve_run_row_hit(cancel: bool, asset: bool, select: bool) -> Option<RunRowHit> {
if cancel {
Some(RunRowHit::Cancel)
} else if asset {
Some(RunRowHit::Asset)
} else {
select.then_some(RunRowHit::Select)
}
}
#[derive(Script, ScriptHook, Widget)]
pub struct QueueList {
#[deref]
view: View,
#[rust]
model: QueueModel,
#[rust]
now_ms: u64,
/// Instance → label (`run #3`), the name a single run's row shows.
#[rust]
labels: HashMap<String, String>,
}
impl QueueList {
pub fn add_batch(
&mut self,
cx: &mut Cx,
flow: &str,
response: CreateBatchResponse,
planned_nodes: &[String],
revision: u64,
now_ms: u64,
) {
self.now_ms = now_ms;
self.model
.add_batch(flow, response, planned_nodes, revision, now_ms);
self.sync_empty(cx);
}
pub fn set_rows(&mut self, cx: &mut Cx, rows: Vec<RunRowDto>) {
self.now_ms = wall_clock_ms();
self.model.set_rows(rows);
self.sync_empty(cx);
}
pub fn set_now(&mut self, cx: &mut Cx, now_ms: u64) {
self.now_ms = now_ms;
if self
.model
.batches
.iter()
.flat_map(|batch| &batch.runs)
.any(|run| !run.terminal())
{
self.redraw(cx);
}
}
pub fn set_labels(&mut self, cx: &mut Cx, labels: HashMap<String, String>) {
if self.labels != labels {
self.labels = labels;
self.redraw(cx);
}
}
/// Feed one event to its row; returns whether the queue knew the run
/// (an unknown run's row is the caller's to fetch).
pub fn apply_event(&mut self, cx: &mut Cx, event: &FlowEvent, now_ms: u64) -> bool {
self.now_ms = now_ms;
let known = event
.run_id
.as_deref()
.is_none_or(|run_id| self.model.run(run_id).is_some());
if self.model.apply_event(event, now_ms) {
self.redraw(cx);
}
known
}
pub fn select(&mut self, cx: &mut Cx, run_id: &str) {
if self.model.select(run_id) {
self.redraw(cx);
}
}
/// No row is the shown run any more (the canvas went back to design).
pub fn deselect(&mut self, cx: &mut Cx) {
if self.model.selected.take().is_some() {
self.redraw(cx);
}
}
pub fn actions(&mut self, cx: &mut Cx, actions: &Actions) -> Vec<QueueAction> {
let mut out = Vec::new();
if self.view.button(cx, ids!(clear_all)).clicked(actions) {
out.push(QueueAction::ClearAll(self.model.clear_all()));
self.sync_empty(cx);
}
let items = self.model.items();
let list = self.view.portal_list(cx, ids!(list));
for (index, item) in list.items_with_actions(actions) {
let Some(row) = items.get(index) else {
continue;
};
match row {
QueueItem::Header { id, .. } => {
if item.button(cx, ids!(cancel_batch)).clicked(actions) {
out.push(QueueAction::CancelBatch(id.clone()));
}
}
QueueItem::Run(run) => {
let row_clicked = item.as_view().finger_up(actions).is_some_and(|up| {
up.is_primary_hit() && up.is_over && up.was_tap()
});
let hit = resolve_run_row_hit(
item.button(cx, ids!(cancel_run)).clicked(actions),
item.button(cx, ids!(asset)).clicked(actions),
row_clicked,
);
match hit {
Some(RunRowHit::Select) => {
self.model.select(&run.run_id);
out.push(QueueAction::Select {
run_id: run.run_id.clone(),
instance: run.instance.clone(),
flow: run.flow.clone(),
revision: run.revision,
state: run.state,
planned_nodes: run.planned_nodes.clone(),
started_ms: run.started_ms,
finished_ms: run.finished_ms,
});
}
Some(RunRowHit::Cancel) => {
if run.batch.is_none() {
self.model.hide_run(&run.run_id);
self.sync_empty(cx);
}
out.push(QueueAction::CancelRun {
run_id: run.run_id.clone(),
instance: run.instance.clone(),
batch: run.batch.is_some(),
});
}
Some(RunRowHit::Asset) => {
if let Some(asset) = run.asset.clone() {
out.push(QueueAction::OpenAsset(asset));
}
}
None => {}
}
}
}
}
out
}
fn sync_empty(&mut self, cx: &mut Cx) {
self.view
.label(cx, ids!(hint))
.set_visible(cx, self.model.batches.is_empty());
self.view
.button(cx, ids!(clear_all))
.set_enabled(cx, !self.model.batches.is_empty());
self.redraw(cx);
}
}
impl Widget for QueueList {
fn draw_walk(&mut self, cx: &mut Cx2d, scope: &mut Scope, walk: Walk) -> DrawStep {
let items = self.model.items();
while let Some(step) = self.view.draw_walk(cx, scope, walk).step() {
let list_ref = step.as_portal_list();
let Some(mut list) = list_ref.borrow_mut() else {
continue;
};
list.set_item_range(cx, 0, items.len());
while let Some(index) = list.next_visible_item(cx) {
let Some(row) = items.get(index) else {
continue;
};
match row {
QueueItem::Header { id, runs, done } => {
let item = list.item(cx, index, id!(Header));
item.label(cx, ids!(title)).set_text(
cx,
&format!("batch {} · {runs} runs · {done} done", short(id)),
);
item.draw_all_unscoped(cx);
}
QueueItem::Run(run) => {
let item = list.item(cx, index, id!(Run));
let selected = self.model.selected.as_deref() == Some(run.run_id.as_str());
let name = match run.batch {
Some(_) => format!("#{}", run.index),
None => self
.labels
.get(&run.instance)
.cloned()
.unwrap_or_else(|| "run".to_string()),
};
item.button(cx, ids!(select)).set_text(
cx,
&format!("{}{name}", if selected { " " } else { "" }),
);
let state = state_name(run.state);
if let Some(mut bar) = item.widget(cx, ids!(progress)).borrow_mut::<RunBar>() {
bar.set_progress(cx, f64::from(run.permille()) / 1000.0, state);
}
let end = run.finished_ms.unwrap_or(self.now_ms);
let elapsed = format_elapsed(end.saturating_sub(run.started_ms));
let meta = match run.state {
RunState::Queued | RunState::Waiting => state.to_string(),
_ => format!("{state} · {elapsed}"),
};
item.label(cx, ids!(meta)).set_text(cx, &meta);
item.button(cx, ids!(asset))
.set_visible(cx, run.asset.is_some() && run.terminal());
// A batch slice's x cancels it, so it goes quiet once
// the slice is over; a single run's x is Stop, which
// also removes a finished run.
item.button(cx, ids!(cancel_run))
.set_enabled(cx, run.batch.is_none() || !run.terminal());
item.draw_all_unscoped(cx);
}
}
}
}
DrawStep::done()
}
fn handle_event(&mut self, cx: &mut Cx, event: &Event, scope: &mut Scope) {
self.view.handle_event(cx, event, scope);
}
}
fn empty_node() -> NodeRowDto {
NodeRowDto {
state: NodeState::Pending,
progress: None,
stage: None,
outputs: Vec::new(),
error: None,
text: None,
}
}
fn set_node_state(run: &mut QueueRun, node: &str, state: NodeState, progress: Option<u16>) {
let row = run.nodes.entry(node.to_string()).or_insert_with(empty_node);
row.state = state;
if progress.is_some() {
row.progress = progress;
}
}
fn asset_from_nodes(nodes: &HashMap<String, NodeRowDto>) -> Option<String> {
nodes.values().find_map(|node| {
node.outputs
.iter()
.find(|output| output.port == "asset")
.map(|output| asset_text(&output.value))
})
}
fn asset_text(value: &ValueRef) -> String {
match value.preview.as_ref() {
Some(Literal::Str(text)) => text.clone(),
_ => value.digest.clone(),
}
}
fn parse_run_state(value: &str) -> Option<RunState> {
Some(match value {
"queued" => RunState::Queued,
"running" => RunState::Running,
"waiting" => RunState::Waiting,
"done" => RunState::Done,
"failed" => RunState::Failed,
"cancelled" => RunState::Cancelled,
_ => return None,
})
}
fn state_name(state: RunState) -> &'static str {
match state {
RunState::Queued => "queued",
RunState::Running => "running",
RunState::Waiting => "waiting",
RunState::Done => "done",
RunState::Failed => "failed",
RunState::Cancelled => "cancelled",
}
}
fn short(value: &str) -> &str {
value.get(..value.len().min(8)).unwrap_or(value)
}
fn format_elapsed(ms: u64) -> String {
let seconds = ms / 1000;
if seconds < 60 {
format!("{seconds}s")
} else {
format!("{}m", seconds / 60)
}
}
fn wall_clock_ms() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_millis() as u64
}
#[cfg(test)]
mod tests {
use super::*;
use makepad_flow::{BatchRunDto, CreateBatchResponse};
fn batch() -> CreateBatchResponse {
CreateBatchResponse {
batch: "a3f2".into(),
runs: vec![
BatchRunDto {
run_id: "run-1".into(),
instance: "instance-1".into(),
},
BatchRunDto {
run_id: "run-2".into(),
instance: "instance-2".into(),
},
],
}
}
fn event(run: &str, kind: &str) -> FlowEvent {
FlowEvent {
topic: "run".into(),
kind: kind.into(),
run_id: Some(run.into()),
..FlowEvent::default()
}
}
#[test]
fn events_update_only_the_keyed_row_and_selection_sticks() {
let mut model = QueueModel::default();
model.add_batch("demo", batch(), &["gen".into()], 7, 1_000);
assert!(model.select("run-1"));
let mut progress = event("run-2", "node.progress");
progress.node = Some("gen".into());
progress.permille = Some(420);
assert!(model.apply_event(&progress, 2_000));
assert_eq!(model.run("run-2").unwrap().permille(), 420);
assert_eq!(model.run("run-1").unwrap().permille(), 0);
assert_eq!(model.selected.as_deref(), Some("run-1"));
let mut finished = event("run-2", "run.finished");
finished.state = Some(makepad_widgets::makepad_micro_serde::JsonValue::String(
"done".into(),
));
model.apply_event(&finished, 3_000);
assert_eq!(model.run("run-2").unwrap().state, RunState::Done);
assert_eq!(model.selected.as_deref(), Some("run-1"));
}
#[test]
fn grouping_and_clear_all_preserve_batch_boundaries() {
let mut model = QueueModel::default();
model.add_batch("demo", batch(), &[], 7, 1_000);
let second = CreateBatchResponse {
batch: "beef".into(),
runs: vec![BatchRunDto {
run_id: "run-3".into(),
instance: "instance-3".into(),
}],
};
model.add_batch("demo", second, &[], 7, 2_000);
assert_eq!(model.batches.len(), 2);
assert_eq!(model.items().len(), 5);
assert!(model.select("run-1"));
assert_eq!(model.clear_all().batches, vec!["beef", "a3f2"]);
assert!(model.batches.is_empty());
assert_eq!(model.selected, None);
}
fn play_row(run_id: &str, instance: &str, started_ms: u64) -> RunRowDto {
RunRowDto {
run_id: run_id.into(),
instance: instance.into(),
flow: "demo".into(),
batch: None,
batch_index: None,
revision: 7,
state: RunState::Done,
planned_nodes: Vec::new(),
nodes: HashMap::new(),
outputs: HashMap::new(),
http_log: Vec::new(),
started_ms,
finished_ms: Some(started_ms + 1),
}
}
#[test]
fn a_play_run_lists_without_a_header_and_stays_hidden_once_cleared() {
let mut model = QueueModel::default();
model.set_rows(vec![play_row("run-p", "instance-p", 5_000)]);
assert_eq!(model.items().len(), 1);
assert!(matches!(model.items()[0], QueueItem::Run(_)));
assert!(model.batches[0].id.is_none());
let plan = model.clear_all();
assert!(plan.batches.is_empty());
assert_eq!(plan.singles, vec![("run-p".to_string(), "instance-p".to_string())]);
// The server still lists the run for a while; the snapshot must not
// bring it back, and it is forgotten once the server drops it.
model.set_rows(vec![play_row("run-p", "instance-p", 5_000)]);
assert!(model.items().is_empty());
model.set_rows(Vec::new());
assert!(model.hidden.is_empty());
}
#[test]
fn cancelling_one_row_does_not_remove_its_siblings() {
let mut model = QueueModel::default();
model.add_batch("demo", batch(), &[], 7, 1_000);
model.hide_run("run-1");
assert!(model.run("run-1").is_none());
assert!(model.run("run-2").is_some());
assert_eq!(model.batches.len(), 1);
}
#[test]
fn run_row_children_take_precedence_over_selection() {
assert_eq!(
resolve_run_row_hit(true, false, true),
Some(RunRowHit::Cancel)
);
assert_eq!(
resolve_run_row_hit(false, true, true),
Some(RunRowHit::Asset)
);
assert_eq!(
resolve_run_row_hit(false, false, true),
Some(RunRowHit::Select)
);
}
}

File diff suppressed because it is too large Load diff

View file

@ -1,167 +0,0 @@
//! The dev-only `testpattern` path (`FLOW_GEN_BASE_URL=testpattern`): the
//! hub's `testpattern` image model served in-process on a private fleet
//! name, and a chat seam that streams a deterministic paragraph — so the
//! whole picture (tokens, progress, the picture landing) can be exercised
//! with no fleet on the LAN. Never used unless the knob is set.
use makepad_ai_hub::download::Downloader;
use makepad_ai_hub::peer_serve::PeerOptions;
use makepad_ai_hub::registry::{Domain, ModelSpec, Registry};
use makepad_ai_hub::server::{start_service as start_hub_service, ServiceConfig, ServiceHandle};
use makepad_flow::engine::executors::chat::ChatSeam;
use makepad_flow::engine::{ChatEvent, ChatTurn};
use std::time::Instant;
/// A fleet name no frontend listens for, so the LAN beacon this service
/// sends never reaches the user's `gen` fleet pickers.
const FLEET: &str = "flow-testpattern";
const WORDS_PER_SECOND: f64 = 18.0;
const FIRST_TOKEN_SECS: f64 = 0.4;
/// Owned dev service. Dropping it releases the service handles and removes
/// the process-private cache rather than leaking both for the app lifetime.
pub struct TestpatternService {
pub url: String,
handle: Option<ServiceHandle>,
cache_dir: std::path::PathBuf,
}
impl Drop for TestpatternService {
fn drop(&mut self) {
drop(self.handle.take());
let _ = std::fs::remove_dir_all(&self.cache_dir);
}
}
/// Start the hub service with the `testpattern` image model on a loopback
/// port and return an owner the app retains until shutdown.
pub fn start_service() -> Result<TestpatternService, String> {
let cache_dir = std::env::temp_dir().join(format!(
"makepad-flow-ui-testpattern-{}",
std::process::id()
));
let downloader = Downloader::new("http://127.0.0.1:1", None).map_err(|error| error.to_string())?;
let handle = start_hub_service(ServiceConfig {
host: "127.0.0.1".to_string(),
port: 0,
cache_dir: cache_dir.clone(),
registry: Registry {
models: vec![ModelSpec {
id: "testpattern".to_string(),
domain: Domain::Image,
backend: "testpattern".to_string(),
available: true,
gated: false,
vram_gb: Some(0.0),
min_vram_gb: None,
min_compute_cap: None,
note: None,
license: None,
files: Vec::new(),
}],
},
downloader,
peer: PeerOptions {
serve: Some(false),
sources: Some(Vec::new()),
..Default::default()
},
fleet: FLEET.to_string(),
})
.map_err(|error| error.to_string())?;
Ok(TestpatternService {
url: format!("http://{}", handle.addr),
handle: Some(handle),
cache_dir,
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn owned_service_removes_its_process_cache_on_drop() {
let cache_dir = std::env::temp_dir().join(format!(
"makepad-flow-ui-testpattern-drop-test-{}",
std::process::id()
));
std::fs::create_dir_all(&cache_dir).unwrap();
std::fs::write(cache_dir.join("sentinel"), b"owned").unwrap();
drop(TestpatternService {
url: String::new(),
handle: None,
cache_dir: cache_dir.clone(),
});
assert!(!cache_dir.exists());
}
}
/// Streams one vivid paragraph built from the prompt, word by word.
pub struct TestpatternChat;
impl ChatSeam for TestpatternChat {
fn start_turn(
&self,
_system: &str,
prompt: &str,
_model: &str,
_max_tokens: Option<u32>,
_thinking: Option<bool>,
) -> Result<Box<dyn ChatTurn>, String> {
let subject = prompt.trim().trim_end_matches('.');
let subject = if subject.is_empty() { "an empty scene" } else { subject };
let text = format!(
"{subject}. Late light rakes across the scene from low on the left, warm and long, \
while the sky above cools to violet; a 35 mm lens sits close and wide, so the \
foreground looms and the horizon falls away. Surfaces keep their grain wet stone, \
brushed metal, worn paint and a thin haze softens the far edges. The mood is quiet \
and expectant, a held breath before the last of the light goes."
);
Ok(Box::new(Turn {
words: text.split_inclusive(' ').map(str::to_string).collect(),
next: 0,
started: Instant::now(),
done: false,
cancelled: false,
}))
}
}
struct Turn {
words: Vec<String>,
next: usize,
started: Instant,
done: bool,
cancelled: bool,
}
impl ChatTurn for Turn {
fn poll(&mut self) -> Vec<ChatEvent> {
if self.done {
return Vec::new();
}
if self.cancelled {
self.done = true;
return vec![ChatEvent::Failed("cancelled".to_string())];
}
let elapsed = (self.started.elapsed().as_secs_f64() - FIRST_TOKEN_SECS).max(0.0);
let due = ((elapsed * WORDS_PER_SECOND) as usize).min(self.words.len());
let mut out = Vec::new();
while self.next < due {
out.push(ChatEvent::Delta(self.words[self.next].clone()));
self.next += 1;
}
if self.next >= self.words.len() {
self.done = true;
out.push(ChatEvent::Done {
text: self.words.concat(),
});
}
out
}
fn cancel(&mut self) {
self.cancelled = true;
}
}

View file

@ -1,80 +0,0 @@
//! Flow's application palette. Keeping the design-pass colours here lets
//! every panel, face and canvas shader share one named set without changing
//! the approved appearance.
use makepad_widgets::*;
pub fn state_color(state: &str) -> Vec4f {
match state {
"running" | "ready" | "queued" => vec4(0.35, 0.62, 1.0, 1.0),
"done" | "ok" | "idle" => vec4(0.30, 0.77, 0.42, 1.0),
"failed" | "error" => vec4(0.95, 0.43, 0.43, 1.0),
"waiting" => vec4(0.95, 0.76, 0.3, 1.0),
"cancelled" | "skipped" => vec4(0.55, 0.55, 0.58, 1.0),
_ => vec4(0.45, 0.45, 0.5, 1.0),
}
}
script_mod! {
use mod.prelude.widgets_internal.*
mod.theme.flow_window = #x0f0f10
mod.theme.flow_grid_a = #x111111
mod.theme.flow_grid_b = #x161616
mod.theme.flow_surface = #x1c1c1f
mod.theme.flow_surface_deep = #x151517
mod.theme.flow_surface_translucent = #x161618e8
mod.theme.flow_surface_hover = #x232327
mod.theme.flow_surface_raised = #x2a2a30
mod.theme.flow_surface_input = #x3a3a40
mod.theme.flow_edge = #x2b2b30
mod.theme.flow_edge_soft = #x33333a
mod.theme.flow_divider = #x26262c
mod.theme.flow_shadow = #0005
mod.theme.flow_clear = #0000
mod.theme.flow_scrim = #000c
mod.theme.flow_text = #xe8e8ec
mod.theme.flow_text_body = #xd0d0d4
mod.theme.flow_text_code = #xc8c8cc
mod.theme.flow_text_chip = #xdddddd
mod.theme.flow_text_port = #x9a9aa2
mod.theme.flow_text_muted = #x8a8a92
mod.theme.flow_text_subtle = #x6e6e76
mod.theme.flow_text_empty = #x6a6a72
mod.theme.flow_text_hint = #x5e5e66
mod.theme.flow_text_grip = #x4a4a52
mod.theme.flow_text_port_connected = #xc7c7d1
mod.theme.flow_text_port_open = #x80808c
mod.theme.flow_text_white = #xffffff
mod.theme.flow_accent = #xff5c39
mod.theme.flow_accent_hover = #x6b5148
mod.theme.flow_highlight = #x5a9cff
mod.theme.flow_success = #x4cc46a
mod.theme.flow_error = #xf26d6d
mod.theme.flow_waiting = #xf2c14e
mod.theme.flow_chat = #x8b7cf6
mod.theme.flow_generation = #xf2994a
mod.theme.flow_function = #xe6c04a
mod.theme.flow_http = #x4ac2e6
mod.theme.flow_input = #x3fb9a8
mod.theme.flow_port_text = #xd8e6ff
mod.theme.flow_port_image = #xffe0c8
mod.theme.flow_port_audio = #xe6d8ff
mod.theme.flow_port_video = #xffd8e6
mod.theme.flow_port_mesh = #xd8f2d8
mod.theme.flow_port_json = #xfff2c8
mod.theme.flow_port_list = #xcce680
mod.theme.flow_port_bytes = #xd0d0d0
mod.theme.flow_state_running = #x599eff
mod.theme.flow_state_idle = #x737380
mod.theme.flow_badge_input = #x1f3a37
mod.theme.flow_badge_output = #x1f3a26
mod.theme.flow_badge_chat = #x2b2748
mod.theme.flow_badge_generation = #x40301e
mod.theme.flow_badge_http = #x1e363d
mod.theme.flow_badge_waiting = #x3d3620
}

View file

@ -1,198 +0,0 @@
//! The value cache: bytes by digest, in RAM under an LRU budget, fetched over
//! the data plane on worker threads and posted back through a channel the
//! UI drains per frame. Nothing here touches the disk (thin-client law).
use makepad_flow::client::{ClientError, FlowClient};
use makepad_flow::ValueBytes;
pub use makepad_media_view::MediaKind;
use makepad_widgets::makepad_platform::thread::SignalToUI;
use std::collections::{HashMap, HashSet};
use std::sync::mpsc::{channel, Receiver, Sender, TryRecvError};
use std::sync::{Arc, Mutex};
pub const DEFAULT_BUDGET: usize = 64 * 1024 * 1024;
/// One content-type seam for cards and the modal viewer. Magic bytes recover
/// mesh/splat values when a server had to label them as octet-stream.
pub fn media_kind(value: &ValueBytes) -> MediaKind {
makepad_media_view::media_kind(&value.content_type, &value.bytes)
}
pub enum ValueArrival {
Ready(ValueBytes),
Failed { digest: String, error: ClientError },
}
pub struct ValueCache {
budget: usize,
entries: HashMap<String, (ValueBytes, u64)>,
bytes: usize,
tick: u64,
pending: HashSet<String>,
sender: Sender<ValueArrival>,
receiver: Receiver<ValueArrival>,
}
impl Default for ValueCache {
fn default() -> Self {
Self::new(DEFAULT_BUDGET)
}
}
impl ValueCache {
pub fn new(budget: usize) -> Self {
let (sender, receiver) = channel();
Self {
budget,
entries: HashMap::new(),
bytes: 0,
tick: 0,
pending: HashSet::new(),
sender,
receiver,
}
}
pub fn get(&mut self, digest: &str) -> Option<ValueBytes> {
self.tick += 1;
let tick = self.tick;
let entry = self.entries.get_mut(digest)?;
entry.1 = tick;
Some(entry.0.clone())
}
pub fn contains(&self, digest: &str) -> bool {
self.entries.contains_key(digest)
}
pub fn insert(&mut self, value: ValueBytes) {
self.pending.remove(&value.digest);
if self.entries.contains_key(&value.digest) {
return;
}
self.tick += 1;
self.bytes = self.bytes.saturating_add(value.bytes.len());
self.entries
.insert(value.digest.clone(), (value, self.tick));
while self.bytes > self.budget && self.entries.len() > 1 {
let Some(oldest) = self
.entries
.iter()
.min_by_key(|(_, (_, touched))| *touched)
.map(|(digest, _)| digest.clone())
else {
break;
};
if let Some((value, _)) = self.entries.remove(&oldest) {
self.bytes = self.bytes.saturating_sub(value.bytes.len());
}
}
}
/// Fetch a value on a worker thread unless it is cached or in flight.
/// Returns whether a fetch was started.
pub fn request(&mut self, digest: &str, client: Arc<Mutex<FlowClient>>) -> bool {
if self.entries.contains_key(digest) || !self.pending.insert(digest.to_string()) {
return false;
}
let digest = digest.to_string();
let sender = self.sender.clone();
std::thread::Builder::new()
.name("flow-ui-value".into())
.spawn(move || {
let result = client
.lock()
.map_err(|_| ClientError::Protocol("flow client lock poisoned".into()))
.and_then(|client| client.value(&digest));
let arrival = match result {
Ok(value) => ValueArrival::Ready(value),
Err(error) => ValueArrival::Failed { digest, error },
};
let _ = sender.send(arrival);
SignalToUI::set_ui_signal();
})
.is_ok()
}
/// Everything the workers delivered since the last drain; arrivals are
/// stored before they are returned, so callers only need the digests.
#[cfg(test)]
pub fn bytes(&self) -> usize {
self.bytes
}
pub fn drain(&mut self) -> Vec<Result<String, (String, ClientError)>> {
let mut out = Vec::new();
loop {
match self.receiver.try_recv() {
Ok(ValueArrival::Ready(value)) => {
let digest = value.digest.clone();
self.insert(value);
out.push(Ok(digest));
}
Ok(ValueArrival::Failed { digest, error }) => {
self.pending.remove(&digest);
out.push(Err((digest, error)));
}
Err(TryRecvError::Empty | TryRecvError::Disconnected) => return out,
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn value(digest: &str, size: usize) -> ValueBytes {
ValueBytes {
digest: digest.to_string(),
content_type: "application/octet-stream".into(),
bytes: vec![0u8; size].into(),
}
}
#[test]
fn value_media_kind_maps_every_viewer() {
for (content_type, bytes, expected) in [
("image/png", b"png".as_slice(), MediaKind::Image),
("video/mp4", b"mp4".as_slice(), MediaKind::Video),
("audio/wav", b"wav".as_slice(), MediaKind::Audio),
("model/gltf-binary", b"glb".as_slice(), MediaKind::Mesh),
("application/x-ply", b"ply".as_slice(), MediaKind::Splat),
("text/plain", b"hello".as_slice(), MediaKind::Text),
] {
let value = ValueBytes {
digest: "kind".into(),
content_type: content_type.into(),
bytes: bytes.to_vec().into(),
};
assert_eq!(media_kind(&value), expected, "{content_type}");
}
}
#[test]
fn value_media_kind_uses_glb_and_ply_magic() {
let mut value = ValueBytes {
digest: "magic".into(),
content_type: "application/octet-stream".into(),
bytes: b"glTF\x02\0\0\0\x10\0\0\0".to_vec().into(),
};
assert_eq!(media_kind(&value), MediaKind::Mesh);
value.bytes = b"ply\nformat ascii 1.0\n".to_vec().into();
assert_eq!(media_kind(&value), MediaKind::Splat);
}
#[test]
fn lru_evicts_the_least_recently_touched() {
let mut cache = ValueCache::new(10);
cache.insert(value("a", 4));
cache.insert(value("b", 4));
assert!(cache.get("a").is_some());
cache.insert(value("c", 4));
assert!(cache.contains("a"));
assert!(!cache.contains("b"));
assert!(cache.contains("c"));
assert_eq!(cache.bytes(), 8);
}
}

File diff suppressed because it is too large Load diff

Some files were not shown because too many files have changed in this diff Show more