Compare commits

..

574 commits

Author SHA1 Message Date
66cc4f15f5 fix(widgets): restore enter_isolate/leave_isolate dropped by upstream merge
The fe5b75d92 merge took upstream's widget_async.rs and dropped the
wm isolate-entry API (IsolateEntry, enter_isolate, leave_isolate)
while lib.rs and apps/flow-ui + apps/wm still use it, breaking
makepad-widgets. Restored verbatim from 77d91385f; structs unchanged
so the code applies as-is. makepad-widgets, makepad-app-flow-ui and
makepad-wm all check clean.
2026-09-10 19:01:52 +03:00
b9a083c26f fix(android): bundle robius secondary dex + wrap app_main binaries
- bundle_crate_secondary_dex: merge OUT_DIR/classes.dex from dependency
  build scripts (robius-sms/trigger/ussd javac+d8 Java for manifest
  receivers/services) into the APK as classesN.dex. Without it the
  manifest declares BootReceiver/SmsReceiver/AccessibilityService that
  are missing at runtime (ClassNotFoundException on boot).
- has_explicit_lib_target: a crate with src/main.rs containing app_main!
  is an app, not a lib — generate the android wrapper ([lib] path =
  src/main.rs) so the JNI entry (activityOnCreate) is linked. Fixes
  UnsatisfiedLinkError for app crates that also ship src/lib.rs.
2026-09-10 16:26:35 +03:00
Arena Agent
fe21c07d84 skills: add Makepad game skills pack ported from threejs-game-skills
Nine agent skills for building Makepad/Rust games, ported from
majidmanzarpour/threejs-game-skills. Same director-routed workflow and
premium bar; runtime rewritten for this fork's game crates.

- makepad-game-director        entrypoint, routing, continuity, asset probe
- makepad-gameplay-systems     loop, movers vs rigid bodies, input, camera, netplay
- makepad-aaa-graphics-builder lighting, shaders, budget, visual scorecard
- makepad-game-ui-designer     HUD, menus, touch and XR UI
- makepad-debug-profiler       defect bisection and profiling
- makepad-qa-release           verification ladder, evidence, packaging
- makepad-3d-generator         CC0 model search, casts, procedural geometry
- makepad-image-generator      texgen textures, palettes, sky, icons
- makepad-audio-generator      sample bank, mixer, material impacts, 3D audio

Written against the real APIs in libs/game/*, libs/sim and apps/arcade:
the game.* verb table, GameRenderer adaptive quality, the packed 6-float
GameMeshVertex layout, script_mod! splash styling, makepad-test driving,
and the BUDGETS.md numbers. No paid generation API is required - the CC0
library plus seeded makepad-game-gen replaces them.

Includes install.sh (Codex/Claude), validate-skills.sh and a repo-aware
probe_assets.sh; both scripts verified against this checkout.
2026-09-05 21:18:58 +00:00
Arena Agent
fe5b75d92d Merge latest makepad work while preserving fork game crates and apps 2026-09-05 20:55:35 +00:00
Admin
a75fe914ff layout: extend turtle sizing and add Grid 2026-09-05 17:21:48 +02:00
Admin
b005c6e9b2 Preserve complete Route geometry within measured web memory budgets 2026-09-05 17:07:06 +02:00
Admin
9e61553c96 Keep Route 3D buildings under bounded web memory pressure 2026-09-05 15:21:31 +02:00
Admin
09fa1f080c Restore WebGL text with complete fallback samplers
Use cached transparent 2D and cube textures for unallocated optional samplers without bypassing invalid-resource, target, or framebuffer-feedback guards.

Require actual glyph draws in release smoke tests and keep shader compilation asynchronous in probe instrumentation.
2026-09-05 14:39:06 +02:00
Admin
1801e38453 Harden web renderer and make Route location opt-in
Bound drawable, geometry, texture, image, map and radar work; validate WebGL submissions and retire GPU resources safely. Treat context loss as terminal without re-entering Wasm after worker termination.

Add explicit Route location consent and regression coverage, including software-WebGL release probes for six deployed demos.
2026-09-05 12:13:36 +02:00
435135ed50 feat(arcade): port makepad game libs + arcade from makepad-internal onto the fork
Reintroduce apps/arcade and the ten game crates that were ripped out of
the fork (assets, audio, blocks, coedit, gen, net, pkg, render, script,
session), from makepad-internal/dev. Reconcile against the fork's layout:

- makepad-game-sim/math come from the fork's existing libs/sim + sim/math;
  the copied internal libs/game/{sim,math} are dropped and the kept game
  crates plus arcade point at ../../sim and ../../sim/math.
- arcade AI is rewritten off the removed makepad-ai agent API onto the
  makepad-ai-hub headless ChatProvider worker (example/cad pattern):
  AiWorker{send,cancel,poll}, ai_worker_loop driving
  ClaudeApiChatProvider(ClaudeCli), AiWorkerEvent availability/delta/
  done/error mapped into the chat feed and authoring land_edit, with
  main.rs send_message/cancel_request/event-drain rebuilt around it.
- the game-script sandbox is adapted to the fork's jailed splash storage:
  every isolate gets mod.fs = splash_storage jail; splaes splash_storage
  set_root_for_heap is now public so the ScriptHost (or Splash) can grant
  a per-game jail root. ENT font: build.rs/dispatch.rs entities fill new
  fork sim Entity/Part fields via ..Default::default(); world_raycast now
  borrows mutably and returns the material id.

cargo check -p makepad-arcade and the game + arcade test suites pass.
2026-09-05 05:49:12 +03:00
Admin
83a00d2801 flow: archive generated assets and add live source editing
Host or attach the asset service, archive generated and terminal outputs, and provide image/video/audio/text/3D viewers with fullscreen and gallery navigation. Load only visible bounded previews, generate video posters and text excerpts, and normalize legacy thumbnails off the UI thread.

Expose video frame counts and Sandbox chat providers, make failed queue cleanup idempotent, compact the panels, and correct short-wire direction. Replace App view with a movable, resizable CodeEditor overlay with debounced source saves, live graph updates, invalid-source preservation and close/reopen geometry.

Validation: release Flow, Flow UI and flowgraph suites passed 335 tests (two ignored). Updated Flow UI suite passed 107 tests; asset route suites passed nine tests after legacy preview fixes. The rebuilt release app passed remote editing, dragging, resizing, invalid-source, close/reopen and App removal checks; final All assets scrolling measured 16.5 ms median and 18.3 ms maximum over 12 inputs. Test instances were closed through the remote protocol.
2026-09-05 01:47:53 +02:00
Admin
75b295ce9a vj: synchronize decks and queue remote stem processing
Update deck and mixer synchronization, apply matched deck rates to splats, and queue remote stem extraction through the fleet. Refresh catalog and creator pipeline integration.

Validation: release VJ suite reports 890 passed, 10 failed, nine ignored. Eight mixer failures reproduce in serial execution (gain, transitions, sample alignment and video fades); two asynchronous media tests also failed in the full run. The new matched-rate splat test passes. These remaining failures are not claimed as fixed.
2026-09-05 01:45:43 +02:00
Admin
b49e2e5060 piano-model: calibrate partial gains and decay against recordings
Apply immutable pitch/velocity-interpolated modal calibration while preserving an explicit uncalibrated constructor. Add offline acoustic tooling, reference measurements and regression coverage.

Validation: 16 calibration/acoustic release tests passed, one ignored.
2026-09-05 01:42:39 +02:00
Admin
3ce27922a4 sim: use deck geometry for collision and sensing
Use deck surfaces consistently in dynamics, floor normals, sensing and voxel handling.

Validation: release makepad-game-sim tests passed.
2026-09-05 01:42:39 +02:00
Admin
3dfe5aaed3 show-control: restore sixteen DMX scenes with local save overlays
Bundle the recovered numbered scene bank, support sixteen slots, and keep editable current state and overrides in the local overlay. Validate legacy control mapping.

Validation: 27 release tests passed.
2026-09-05 01:42:39 +02:00
Admin
2f44d20b51 apple: avoid blocking video clocks and release native players once
Use output/presentation timestamps instead of AVPlayerItem.currentTime on the UI thread. Consume only fresh frames and stop polling paused players after their poster arrives. Balance Objective-C ownership and make native cleanup idempotent.

Validation: release Flow playback smoke and process sampling; the previous currentTime mutex hotspot is absent in the updated sample.
2026-09-05 01:42:39 +02:00
Admin
f8b9a670b7 widgets: preserve numeric edit completion and menu focus
Find numeric completion actions reliably and restore menu focus using tracked areas. Handle touch dismissal and focus changes without leaving menus open.
2026-09-05 01:42:39 +02:00
Admin
88048aa784 assets: extend authoring pipelines and typed asset search
Add sandbox authoring effects and character/composite creation, improve creator submission handling, and extend publishing/search metadata. Update importer conversions and the asset UI integration.

Validation: 31 store search tests, ten composite tests and two author-policy tests passed in release mode.
2026-09-05 01:42:39 +02:00
Admin
97912796a8 render: support rigged models and custom materials across viewers
Add declarative CSG joints, bindings and clips, custom render materials, and skinning/lightmap updates. Update shared media and XR viewers, including splat visibility and optional mesh HUD.

Validation: 400 release render tests and seven CSG rig tests passed.
2026-09-05 01:42:39 +02:00
Admin
1be1e2153c ai-hub: gate downloads by disk capacity and recover fleet admission
Account for missing and partial model files per volume, reserve headroom, and reject disk-constrained workers before accepting a job. Preserve typed admission failures so callers can choose another peer, and make activity gating and cancellation recover cleanly.

Validation: 592 release hub tests passed, one ignored; required-CUDA builds deployed to six idle Windows workers.
2026-09-05 01:42:38 +02:00
Admin
39a1b742ac flow-ui: a locked run's pictures and clips still take clicks — the viewer opens from the Output card again; only the inputs stay inert
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 21:17:11 +02:00
Admin
6f2fdfa861 flowgraph: a wire between nearly level ports takes the short run instead of a trip around the card (the fillet rule rejected any interior segment under two radii); fewer bends win only among routes of comparable length; six screenshot fixtures and the routing review
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 21:07:02 +02:00
Admin
1181a3bae4 flow + flow-ui + widgets: every creator pipeline is a template — 55 templates in six groups (Image, Video, Audio, 3D, Vision & text, Utilities), all evaluated and engine-exercised in tests; the New picker, the flows.templates tool and the palette group the same way; the Templates menu shows them under group headings, and a menu taller than the window scrolls
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 21:03:54 +02:00
Admin
2b4bff9a38 widgets + flow-ui: menu entries fire again — the bar's Closed action no longer hides the Selected one behind it, and an open menu takes pointer and key input before the panels beneath its popup
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 20:54:18 +02:00
Admin
30575f082d flow: route generation by request workload
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-04 20:27:00 +02:00
Admin
b0b0e41da0 flow-ui: select runs from the whole queue row
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-04 20:15:05 +02:00
Admin
f3b6c2f36c Merge branch 'work' into webdemos 2026-09-04 20:13:43 +02:00
Admin
778dbb8fc7 flow: accept bodyless 204 responses
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-04 20:12:49 +02:00
Admin
8f51704697 flow-ui: a Templates menu on the bar lists every template the server serves; choosing one creates and opens a flow from it
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 20:09:13 +02:00
Admin
f45793a8fa vj: batch live video publishes into grid rows 2026-09-04 17:47:47 +02:00
Admin
28016ca897 vj: keep generated clips at the head of the grid 2026-09-04 17:41:18 +02:00
Admin
2a064d3928 workspace: add loader, haptics, voice, and runtime fixes 2026-09-04 17:39:41 +02:00
Admin
acd23289c6 flow + flow-ui: a cancelled run clears its cards and bars at once and its live nodes read cancelled; layout edits (move, size, flip) work while a run is shown; the queue row is one centre line — name, strip, state · time, x
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 17:24:17 +02:00
Admin
a6c5f15bf7 widgets: FabValueInput honours visible, so the seed picker's random mode hides the number field
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 15:04:50 +02:00
Admin
767e260b97 flow + flow-ui: parallel runs — random seeds with a die, a fleet parallelism estimate (generation nodes bound it; the chat box never caps the image fleet), batch routes, the Run dropdown (1 / max · N), Ctrl+Enter queues a run, and the Running panel becomes a Queue of every run with per-row and per-batch cancel and Clear all
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 15:01:15 +02:00
Admin
bf9418e2bc vj: ironfish / synth rack, program mix, and a clean release check
Clocked piano/ironfish/drum rack, program-bus mix, splat/mixer/music
updates. Silence the unused warnings that show up in `cargo check -p
makepad-vj --release`.
2026-09-04 14:54:07 +02:00
Admin
b37b1168fd media_view: the video, audio, mesh and splat viewers leave asset-ui for one crate; flow-ui shows every media type on the Output card and in the modal viewer
ViewSplat gains a typed set_scene_bytes so a viewer inside a face isolate never needs a script apply; MeshView gains visible.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 13:49:32 +02:00
Admin
a7774885a1 flow + flow-ui + flowgraph: the loaded flow is the editable design (Input.value lives in the file); Play clones it into a numbered locked run instance with a Design button back; inspector text rows follow the graph
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 13:37:38 +02:00
Admin
c71d6392f9 flow + flow-ui + flowgraph: a stale keep-alive is replayed on a fresh socket; a palette drop rewrites the file text so the new card mounts; the progress bar is centred in the header and inset past the corners
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 13:27:21 +02:00
Admin
7e397cf31e flow + flow-ui: Publish end node writes to the asset library; Assets tab in the Inspector; prompt-to-library template
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 13:27:21 +02:00
Admin
9a31d2104a flow-ui + widgets: a chosen model shows no node list, and a closed ComboBox shows the start of a long label
The picker's own label counts the ready nodes, so the GPU list under a
chosen model was clutter; it stays only for a model no node can serve,
where it names why. A closed ComboBox reset its text field to the tail of
the label after set_text; the cursor now sits at the start.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 12:03:50 +02:00
Admin
24c927ad7e flow-ui + widgets: every dropdown is the searchable ComboBox
Model and format pickers, the inspector's one-of rows, the Ask face's
choice and face-declared pickers all render as ComboBox; the popup and
its input map through the canvas zoom. ComboBoxRef gains changed_label
and set_selected_by_label so the bindings did not have to change shape.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 12:01:42 +02:00
Admin
d20a20871a flowgraph: a routed wire turns right after its port and hugs its own card
The straight run out of a port is 10 px (8 px minimum) with the endpoint
fillets shrinking to fit, and the first and last vertical runs keep only
the narrow 6 px envelope from the wire's own card; other cards keep the
two-tier clearance and parallel wires keep their cable spacing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 11:48:58 +02:00
Admin
30ef92597f flow-ui: the image generator card shows only its settings; the picture lives on the Output card it feeds
The generator's own preview was clutter next to the Output card's
picture. The Image face drops its preview and generator cards are no
longer full-bleed; the two picture tests follow the picture to the
Output card.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 11:35:05 +02:00
Admin
5863c43eb1 flow-ui: an input card is one text area filling the card, with no name beside it
The host wrapped every bound face control in a labelled row, so the
prompt card showed a "value" label beside a fixed-height box. A
multi-line text area is now its own row and follows the card's size.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 10:48:03 +02:00
Admin
cff15ac045 widgets: a fab number field drops a label that cannot fit instead of crushing it to a dot
A 54 px width chip left the "w" label one pixel column, which rendered
as a stray dot beside the number.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 10:43:12 +02:00
Admin
2a5172465c flowgraph + flow-ui: two wire modes never mixed — bezier or routed — and routed wires use narrow gaps
Routed mode never falls back to a curve: a wire with no corridor takes the
least-collision orthogonal path. Clearance is two-tier: the comfortable
12 px card clearance with 16 px fillets first, then 6 px per side with
8 px fillets, so a routed wire passes a 20 px gap. Bezier mode is a pure
cubic per wire. View → Wires switches the mode (in-process until the app
has a settings store).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 10:43:12 +02:00
Admin
27f02f2443 flowgraph: the canvas's style table resolves its registered type, so node and port icons draw again
The lib's script named FlowCanvasStyles unqualified before it was in
scope; the styles object never applied and every icon lookup came back
empty (the colours survived through the per-kind colour properties). The
style types are registered as components and the default references
mod.widgets.FlowCanvasStyles.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 10:33:33 +02:00
Admin
c7241e00be hub: a node that evicted every resident releases its cached allocator pool before refusing a load or publishing usable VRAM
The 5090 kept about 1.5 GB of CUDA allocator pool after evicting every
model, so a card that fits flux2-dev refused it (30510 MB free reported,
32090 MB in a fresh process). Admission now trims the pool when the last
resident is gone and re-measures before refusing; usable VRAM is measured
after the same trim; the refusal names the pool it released.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 10:22:57 +02:00
Admin
c6890fd47a flowgraph: the flow canvas and wire router become libs/flowgraph, a widgets-only crate any UI can host; flow-ui is its first consumer
makepad-flowgraph carries the canvas (camera, cards, ports, wires,
selection, edits out), the router, a string-keyed view model
(GraphView/NodeView/PortView/EdgeView), script-registered node and port
styles keyed by kind, and a three-method NodeFaces trait for per-node
face widgets. It depends on makepad-widgets only. flow-ui projects its
splash-evaluated Graph through a 147-line adapter (graph_view.rs), keeps
its faces, file writer, panels and services, and renders pixel-identical
to before (hidden before/after grabs, zero differing pixels).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 10:22:21 +02:00
Admin
cf3b6488de flow-ui: oval port discs with a small input notch, the icon visually centred, and wires that start at the output's tip
The input notch was as wide as the output point; it is now a small notch.
The disc is a slight oval along the flow axis and the type icon shifts a
little toward the point (outputs) or away from the notch (inputs) so it
reads centred. Wires anchor at the tip of an output's point and at the
apex of an input's notch instead of the disc centre, so a wire leaves the
point along its axis.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 08:08:20 +02:00
Admin
f5ec84e671 flow-ui: the port disc's shape carries the flow direction — outputs end in a point, inputs have the matching dimple
Replaces the loose chevron beside the label. The cable side of an output
disc is drawn as a sharp > point, the cable side of an input disc as a >
dimple the point would fit into; both follow the card's flip. The type
icon stays centred and the labels sit close to the disc again.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 07:58:15 +02:00
Admin
8f23e3b6ee flow-ui: the port chevron sits between the disc and its label, never inside the disc
The direction arrow shared the disc with the type icon and crowded it.
It now centres in the gap between the disc edge and the port name, so an
input reads "disc > label" and an output "label > disc"; the type icon
returns to the disc centre and the labels move out to make the room.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 07:52:02 +02:00
Admin
ad5e98bbad hub registry: flux2-dev's VRAM estimate is its measured peak, 30 GB
Measured on the RTX 5090 (2026-09-04, 1024x1024, 8 steps, nvidia-smi 250 ms
samples): 30.5 GB used at peak, the run completed in 52 s. The old 29 GB
was a pre-measurement guess that under-reported the model by 1.5 GB.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 07:48:01 +02:00
Admin
75c34413be hub: the PRO 6000 serves image as well as chat and text
flux2-dev cannot fit on the 5090 at the default reserve, and the role
table barred the only card that can hold it. The user opened image on
10.0.0.165 ("let the rtx serve images too"); the role test and the
flow's role-aware listing test follow.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 07:41:59 +02:00
Admin
c90df1fe0a flow + hub: a node advertises only the models it can admit, the flow picks admitted nodes for the named model, retries on refusal, and names every node's reason when none can take it
The 5090 listed flux2-dev ready (total VRAM passed the gate) then refused
every job (29696 + 2048 reserve > ~30510 usable); the flow picked it by
domain ETA and never retried. Nodes now publish vram_usable_mb and mark
un-admittable models too_small; the fleet gate uses usable VRAM; the
flow's gen executor picks admitted nodes for the requested model (ready
first), retries up to three nodes after an admission refusal, and when no
node can take the model its error says why per node (role, too small with
the numbers, waiting for VRAM). The flow's model listing drops (model,
node) pairs the node's fleet role bars, so the picker no longer counts the
chat-only PRO 6000 as ready for image; its label reads ready/absent/too
small with the GPUs named.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 00:45:42 +02:00
Admin
5a9eed960c Merge branch 'work' into webdemos 2026-09-04 00:18:58 +02:00
Admin
303458d07a webgl: a 2D texture pass builds its camera through set_ortho_matrix like every other backend, so the exploded z-layer view's camera reaches the GPU — the hand-built ortho branch uploaded an identity view and clipped every exploded draw, leaving the tweaker's layers view a bare window on the web; the Y flip for render-to-texture is one helper shared with the keep-camera branch
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-04 00:18:58 +02:00
Admin
4c370488e9 flow-ui: wires are stable and selectable — a route depends only on its own obstacles and keeps its kind unless a new one is clearly better, with a fixed tie-break; a click within six pixels selects a cable, the inspector shows the connection, Delete removes it
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 00:00:26 +02:00
Admin
2e869f84ea route: the ocean-high overlay comes from the re-sharded archive (58 shards of at most 16 MB, one small leaf directory each) instead of two shards whose directories decoded to 407 MiB
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 23:58:08 +02:00
Admin
8c40d28a47 flow-ui: a close button at the top-left of the image viewer, fixed to the screen; the bar and the button fade with the viewer
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 23:51:47 +02:00
Admin
73886d1664 Merge branch 'work' into webdemos 2026-09-03 23:45:22 +02:00
Admin
3c2530d000 web memory diet: the 805 MiB at load was the ocean-high archive's 13 M-entry leaf directory decoded to 407 MiB per lookup round and evicted at once — leaves now parse streaming into a window around the waiting tile ids (LeafParseLimits); a phone policy on the web (deviceMemory, UA, touch + short side) caps wasm at 512 MiB with a 320 MiB budget; archive leaf/range caches, reads and bakes in flight follow the budget; packed tile bytes stay packed until the bake decodes them; terrain scratch sized to the viewport and dropped with the layer; Cx::memory_report by owner. Phone viewport 1334 → 308 MiB after a minute of pans, desktop 1208 → 588
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 23:45:22 +02:00
Admin
2ed495f2ed flow-ui: the flip decision scores length, crossings, bends and loops instead of length alone, so a crossed pair unflips; ports and wire midpoints carry direction chevrons that hide below half zoom
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 23:44:42 +02:00
Admin
4781378985 flow-ui: the image viewer is modal for input — wheel, pointer and keys stop at it, and the canvas ignores a wheel under any open overlay
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 23:43:19 +02:00
Admin
b41fab482f flow-ui: a wire takes the simplest shape the geometry allows — the clear S-curve wins over any corridor route, tangents scale with the hop, level ports get a straight line, and no route has more bends than an obstacle forces
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 23:20:27 +02:00
Admin
6139a7386f flow + flow-ui: a card can face right-to-left — flip: true in the file mirrors its ports; the router is directional; a card auto-flips when that cuts its cable length below 80 percent, a hand flip pins it; the flip animates
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 23:13:39 +02:00
Admin
b7d95d823e flow-ui: the image viewer opens fitted and centred — the picture syncs after the stage has drawn instead of reading its cleared area as empty; a sampling-mode field replaces the abuse of the image shader's rotation; pan clamps at the picture's edge, 1:1 means one image pixel per device pixel, the checker stays fixed to the screen
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 23:00:51 +02:00
Admin
8ff1d1c41b flow-ui: wires never pass behind a card — endpoint cards are obstacles except at their own port stub, targets below or below-left route through a clear corridor; the selection outline is a stroke on the card's own rounded body under the port discs; a press on a picture box drags the card and a still click opens the viewer
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 22:43:49 +02:00
Admin
f95e34b4c1 flow-ui: the inspector has one purpose — the selected node's header and note, the settings its card does not show, its connections as chips, its result, the raw face source folded under Advanced; a full-window image viewer with cursor-anchored zoom, pan, fit and 1:1, arrows through the instance's pictures, save and copy digest
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 22:38:44 +02:00
Admin
c79e84efca map: a tilt is two fingers moving together up or down — same vertical direction for both, the pair's stroke within a fifth of vertical; spread and angle no longer matter, so real fingers trigger it
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 22:38:16 +02:00
Admin
4aec005469 flow-ui: wires route around cards with fillets and a pulse that travels the cable when a value lands; Clear drops an instance's generated state and keeps its inputs, as a route, a toolbar button, a menu entry and a shortcut
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 22:32:05 +02:00
Admin
4ee4f4cb79 flow-ui: the resize path sets walks and fits through typed setters, never a script apply from the main VM on an isolate's widget — a failed apply had left a freed script object behind and wedged every frame; a resized card fills its picture box, clips its face and lets the last flexible element take the height
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 22:15:37 +02:00
Admin
018ce7370d map: labels hold still through a gesture and never pop — the settled placement rides the camera delta while anything moves (pan now part of the motion signature; re-place only beyond the pan/zoom law), every draw uses the rect-centre fold pivot so CPU placement and the GPU warp agree (a fresh place drew about the screen corner: the giant space-warp labels), and a re-place cross-fades: survivors keep their birth, newcomers fade in, dropped labels retire from their own camera over 250 ms
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 22:10:03 +02:00
Admin
8eaa6ec3df map: two-finger tilt follows the phone convention — fingers up tilt into 3D, down flattens
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 22:06:35 +02:00
Admin
8b05496651 flow + flow-ui: width and height snap to the type's step (16 for images, the doc is the authority) in the fields, the inspector, the presets and the evaluator with a warning, so a fleet backend never sees an illegal size; text in cards and the inspector scrolls inside a bounded area that follows streaming until you scroll up
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 22:01:09 +02:00
Admin
2f3e393c59 web_server: a directory path without its trailing slash (/score) redirects to /score/ instead of 404, query preserved
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 22:00:31 +02:00
Admin
ddc4709b32 map: touch gestures — one-finger pan and double-tap zoom, two-finger pinch zoom around the midpoint, rotate with a 5° dead zone, and a parallel vertical slide for tilt, one state machine native and web; the web page keeps browser pinch-zoom out (non-passive touch listeners, touch-action none, maximum-scale 1)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 21:58:36 +02:00
Admin
4cd24c4905 route demo: the first location fix flies the map to the user at zoom 14, as native does; Amsterdam stays the default until a fix arrives
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 21:54:21 +02:00
Admin
bc6c620ede hub + flow: what the chat review found — the in-process route retries an empty completion too, a node says whether its prefill opened thinking so a brief-mode answer is never discarded, a preferred model falls back to normal election when no node has it, discovery keeps looking for the preferred model until patience runs out
Verified on the LAN fleet: a new prompt → paragraph → picture.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 21:50:59 +02:00
Admin
ca92ff9da4 route: the open sea on the web — the ocean-low/ocean-high overlays are an always-on group in the shared hosted overlay table, read through the archive plane (makepad.nl, cached); native keeps the local ocean files only as a dev override; the native-only OCEAN_MBTILES path is gone
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 21:44:36 +02:00
Admin
dbb46be699 Merge branch 'work' into webdemos 2026-09-03 21:43:15 +02:00
Admin
a53d9b6b40 route: no clock-driven night theme — the theme is the user's toggle, remembered in cx.storage, default day (the hour rule fired on the web for the first time once wasm had a clock and darkened the map after 19:00)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 21:43:15 +02:00
Admin
43302a63ed flow-ui: every card owns a draw list and draws in z order, selection brings it to the front; and the design review's findings — every event kind remapped through the camera, run events keyed by run id, no remount mid-run, an input journal that survives a failed PUT, terminal states reconciled, the total bar from the planned node set, isolate ownership on instance change, popups retired before an isolate is freed, the Ask face answers on a button, the menu bar navigates by keyboard, no per-frame allocation in the canvas draw
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 21:39:05 +02:00
Admin
dbc7838094 route: the maps-folder field and its save button are gone from the settings panel; the maps root is automatic
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 21:31:09 +02:00
Admin
635c3a3063 map: ready tiles are inserted at most two per frame (byte budget kept), queued visible ring first then margin ring, the stale tile drawn until its replacement lands — a restyle burst of 4-9 refined tiles no longer stalls a frame for 70-120 ms
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 21:23:34 +02:00
Admin
aa271c36f6 map + route + geodata: the Terrain layer through the archive plane — TerrainSource (hosted .mkmap elevation shards fetched by range through the shared reader, a local MBTiles only as a dev override), the hillshade rendered on the pool's heavy lane with reused TerrainScratch buffers (web capped at 2048×1536 at DPR 1, native DPR-aware), one request in flight; the demo checkbox now renders terrain like native
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 21:20:28 +02:00
Admin
d641cbf995 map: labels no longer vanish — the gesture label budget is charged from the placement loop, not from candidate collection (4-7 ms on the web at z15-16, up to 6000 candidates in space warp), so a place can no longer commit an empty cache; a truncated pass is never a strict cache hit, arms its own settle wake, and the at-rest follow-up chain is capped at 4
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 21:17:47 +02:00
Admin
cd33943a7a web_server: radar and weather run on KNMI's documented anonymous key when no --knmi-key-file is given; without --live-cache the pollers keep an in-memory cache and say so once
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 21:10:11 +02:00
Admin
01e77d99c0 map: the shimmer clock is one pass-level uniform (draw_pass.shiny_time, a map-owned slot, never draw_pass.time) written once per heartbeat tick instead of patched into every retained draw call's block — the settle tail stops re-uploading 600 uniform blocks per tick
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 21:09:44 +02:00
Admin
b04333265e webgl: uniform blocks upload only when their generation moved — one global monotonic counter on Cx hands out every generation (draw call create/dirty/zbias, draw list allocate/transform/re-record, pass allocate/time/dpi/ortho/camera and the web flipped copy, shader scope writes), so a reused pool slot can never match a cached generation; clear_draw_items, pool reuse and VAO recreation reset the caches; the JS caches key on the generation alone. Proven on a local build: the pan screenshot keeps every tile, the settle tail drops 20.5 → 12.8 MiB/s
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 20:59:55 +02:00
Admin
179124f085 Merge branch 'work' into webdemos 2026-09-03 20:46:27 +02:00
Admin
e2a37c97f7 flow-ui: a format picker with common image sizes and a swap beside width and height, the picture clipped inside the card under the selection outline, a face-declared control mounts as a labelled row on the node it sets with a hint when it binds elsewhere, the Fn face gets a strip for them
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 20:46:21 +02:00
Admin
e689aea8f5 web_server + geodata + route: live radar, wind and weather on makepad.nl/api — one bounded poller per feed, hourly, disk-backed cache served from an Arc snapshot (a restart never re-polls early), 503 warming until the first result, health reports ok/warming/unavailable with timestamps; KNMI key from --knmi-key-file, the documented anonymous open-data key otherwise; libs/geodata fetches through the platform HTTP client instead of shelling out to curl; the client retries 503 after 30 s and disables a layer only on 404
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 20:46:04 +02:00
Admin
a50750fda6 flow-ui: Flows, Running and Palette as their own rounded panels with splitters, the inspector and source pane likewise, columns resizable; gaussian frame shadows; keys and IME reach the focused face field through the canvas transform
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 20:42:45 +02:00
Admin
199be9eb1d map + route: the six overlays (EV chargers, transit, nature, districts, building age, population) read through the same archive plane as the base tiles — OverlaySource {name, TileSourceConfig}, hosted .mkmap archives on makepad.nl fetched by range through the shared archive reader and disk cache, a local .mbtiles only as a dev override; one layer table (apps/route/src/overlays.rs) for the native and demo builds, the demo checkboxes now set overlays like native
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 20:41:27 +02:00
Admin
e48b056df1 Revert "webgl: a uniform block is uploaded only when its generation moved — per draw call (uniforms_gen), per draw list (uniforms_gen, recording_gen), per pass (pass_uniforms_gen) and per shader scope block; the JS side caches (ptr, len, gen) per uniform buffer and re-records/recompiles reset it; direct camera writers (vj effects, render scene, the web flipped copy) bump the pass generation"
This reverts commit 0d7ddee3f7.
2026-09-03 20:37:35 +02:00
Admin
df9f4498b6 flow: the style picker lives on the node whose input it sets — the add_style Fn, not the image
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 20:35:25 +02:00
Admin
2dbd1b7dca flow + hub: a flow's LLM turn asks the node not to think and sends no token cap for zero, so a fleet Qwen node answers with a visible paragraph; the empty-state placeholder draws one icon
Verified on the LAN fleet: prompt → paragraph → picture in twenty seconds.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 20:34:02 +02:00
Admin
bb6d48fc6a Merge branch 'work' into webdemos 2026-09-03 20:29:04 +02:00
Admin
1f244282b1 map bake: a face-band triangle is a face only when all three records pack (a mixed triangle stays on the road path); a panicking pool job no longer takes the worker down — caught, reported once, the tile fails; the bake panic that blacked the native map and killed the web workers
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 20:29:04 +02:00
Admin
075e1a74d4 flow-ui: pickers filled from the hub for image and text nodes, popups anchored through the canvas transform, labelled face controls, add_style explains itself, cards resize from a grip with size: vec2 kept in the file, full-bleed pictures, a click anywhere on a card selects it and still reaches the face, no remount on layout-only edits, panels float over the canvas
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 20:26:13 +02:00
Admin
d852699d08 route: no tile-source dropdown — makepad.nl is the tile source through the local range cache; a world.mkmap in the saved maps folder is the only override; the stale preference file is removed on start
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 20:23:07 +02:00
Admin
7db06e8c09 Merge branch 'work' into webdemos 2026-09-03 20:22:57 +02:00
Admin
0d7ddee3f7 webgl: a uniform block is uploaded only when its generation moved — per draw call (uniforms_gen), per draw list (uniforms_gen, recording_gen), per pass (pass_uniforms_gen) and per shader scope block; the JS side caches (ptr, len, gen) per uniform buffer and re-records/recompiles reset it; direct camera writers (vj effects, render scene, the web flipped copy) bump the pass generation
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 20:22:56 +02:00
Admin
a8377920b7 hub + flow: a whitespace-only chat completion is retried once and then fails instead of passing as an answer; a flow's model is a fleet model id unless it names a weight file on disk; chat models show under the text domain in /v1/models
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 20:17:54 +02:00
Admin
fe23e067dd AGENTS.md: the execution policy — zero locking on the UI thread as one mechanism for native and wasm, no temporary threads (the pool), the standard operating flow (Codex codes, Fable designs and reviews, Grok tests), and the tweaker on Shift+F10
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 20:11:28 +02:00
Admin
7f2cc52e57 Merge branch 'work' into webdemos 2026-09-03 20:04:18 +02:00
Admin
bb49fe6989 map + platform: retained per-tile draw lists — each resident tile owns one DrawList2d per carto pass (fill, casing, stroke, icon, icon-high, shadow) and the label glyph batches are retained the same way, recorded when the bake, LOD ring, fringe/icon gates, flat/tilted or clip change and re-attached otherwise; a pan/zoom/tilt frame pushes this frame's uniforms onto the retained calls (DrawVars::update_uniforms_on_area, resolved slot table) and uploads zero instance bytes; the tilted per-pass depth is a pass_depth uniform; a held list's zbias resolves at entry (zbias_hold in every backend); the shimmer heartbeat patches shiny_time in place without a redraw; a freed/reused sub-list id is skipped by every draw-tree walker and the mask list re-records empty on the flat transition (contract test). Web pan tail 1,098 → 20 MiB/s, flat pan 2.4 → 0.43 MiB/frame; Metal grabs within the run-to-run noise floor
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 20:04:18 +02:00
Admin
64f6d41731 flow: pin the exact chat hand-off the Llm node makes (system, user turn, model) as a regression test
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 20:01:52 +02:00
Admin
662a1216e5 flow: the gen executor renews the hub lease every two seconds and says bye on shutdown, so fleet boxes stop reaping our jobs; flow-ui's bridge publishes run and instance events as messages on the bus, long runs return early with a subscription, and the AI gets templates, models and create tools
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 19:53:01 +02:00
Admin
a40ad74a63 flow-ui: the empty-state icon is the node's output type — a picture card waits with the picture icon, not the text glyph
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 19:47:41 +02:00
Admin
15c3b11da1 ai-services: what the pubsub review found — modules get a subscription seam and a publish sink through both WM paths, lease end flushes unsubscribes to hosted services, the subscription cap counts closing rows, an endpoint's queue is dropped after Unregister, the prompt drops a subscription on final
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 19:39:10 +02:00
Admin
4e8e4cd45f flow-ui: the design pass — menu bar and toolbar with the total run bar, continuous zoom through the draw-list view transform with pointer remapping, dark checker canvas with grid steps, shadowed cards with icon labels and port icons, glowing wires, per-node progress bars, full-bleed image cards, palette cards you drag out, the fab edit controls in the inspector, a template picker behind New, model pickers from the hub; MenuBar widget in the shared crate
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 19:38:21 +02:00
Admin
102ffc5fa5 ai-services: messages on the bus — a manifest declares topics, the engine subscribes on a tool's behalf or by ToolResult.subscribe, a service publishes Message frames, an idle conversation wakes on a message as an event turn under rate laws; the WM bus forwards the new frames; every app that matches the wire gets its arm
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 19:27:45 +02:00
Admin
14b194bed1 flow: the contact-sheet acceptance and an instance/run/value battery over real sockets — Http read, Fn, two images in parallel, an Ask that parks and resumes the run, a third image, an Http POST of the result
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 19:00:06 +02:00
Admin
7b10d61fc7 flow: the hub's models on the wire — GET /v1/models snapshots the discovered fleet (health + model list per node, refreshed on demand, fleet hints for tests) and /v1/nodes carries the model ids per domain
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 18:56:08 +02:00
Admin
f615054dab tweaker + webgl: click-to-climb continues only from the widget the climb started on, so a press on a sibling picks that sibling (the empty draw_bg was a bare View being pinned); the Shader tab says when a layer has no live draw call; the WebGL paint walk no longer resets every draw list's view_transform — the magnified material well drew at the window origin on the web
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 18:55:42 +02:00
Admin
de9aba3d83 vj + web: the Layer button works on the web — a second Window is a queried capability (OsType::is_single_window; the web creates none and reports it once), so the output becomes an in-page full-canvas layer with browser fullscreen (Esc, the browser's own fullscreen exit, or a double-click leave it); a pass without a draw list settles its dirty flag instead of erroring every frame; exitFullscreen fixed; fullscreenchange feeds the window geometry
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 18:53:02 +02:00
Admin
93e7dae15d Merge branch 'work' into webdemos 2026-09-03 18:38:26 +02:00
Admin
976ea0ed04 tweaker: Shift+F10 toggles it on every platform (KeyEvent::is_tweaker_toggle, one call site; the web page swallows exactly Shift+F10 so the browser never sees it); the exploded z-layer view has no keyboard shortcut any more — it is a button in the tweaker
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 18:38:26 +02:00
Admin
263d6f0ae8 flow-ui: the canvas — node frames with faces evaluated in an isolate per instance, typed port dots, bezier wires, drag-move and drag-connect, palette, inspector, source pane, App view, Running list — integrated with the chat bridge and wired to the run routes
Zoom is three discrete sizes with a pan-only camera: pointer hit-testing
ignores the draw-list view transform. Faces bind with bind := "node.port"
because typed widgets refuse unknown properties.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 18:34:29 +02:00
Admin
dd51a028aa vj: a go-to-start transport button per deck — seeks to 0:00 through the normal seek path, keeps the playing state, leaves the loop alone; skip_start.svg icon; decks test
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 18:31:18 +02:00
Admin
ec40fdb050 vj + widgets: double-click a knob or fader to reset it to its default — the Slider handles tap_count 2 and emits its normal Slide action; the deck controls carry their neutral defaults (pitch 0, gain/EQ/stems 1, filter centre, crossfader centre)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 18:27:49 +02:00
Admin
32bea89a92 vj: a playing slice keeps the big waveform of the whole sample — the active slot is an overlay on it (accent band, bracket edges, wrapped playhead, slot number) while the lane's viewport stays fixed; the grid cells are selectors only, their mini waveforms are gone
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 18:27:40 +02:00
Admin
8d542feea7 platform: a child pass orphaned by its attaching draw list is no longer painted — the window's gauss_scene pass stayed a live_with_parent child after the map went flat and re-encoded a frozen 400-item list every pan frame with stale geometry ids (new tile meshes × old instance counts, tens of millions of triangles into a texture nobody read); make_child_pass records the recording list + redraw id, stale passes are skipped, a cached View re-attaches on a cache hit; tests for the orphan gate and the pool generation contract
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 18:22:02 +02:00
Admin
ccf818313d Merge branch 'work' into webdemos 2026-09-03 18:20:07 +02:00
Admin
e3abf4dbbf map: the hosted-tile cache keeps the same ~2x-visible margin as the local one — a budget pinned at the visible set evicted the trailing edge of every pan on the next frame and refetched it a moment later
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 18:20:07 +02:00
Admin
09e4bf1e1e flow: what the batteries pinned — 3xx refused, a node object where a port reference is required is an error, undeclared Fn ports refuse edges, parameter errors point at the field, the omitted node is named, allowed HTTP methods listed, a 192 KiB source cap, an unterminated Flow{ is a parse error
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 18:15:39 +02:00
Admin
a2e0c43e4c flow: the old pipelines ship as templates — GET /v1/templates lists them with label, brief, inputs and outputs; POST /v1/flows/{name} {template} creates a definition from one
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 18:13:53 +02:00
Admin
e878c4dbb6 Merge branch 'work' into webdemos 2026-09-03 18:04:35 +02:00
Admin
0d3a9b84d6 map: nothing animates at rest — the water/foliage shimmer heartbeat runs only during interaction or a camera animation, plus a 1 s settle tail, then freezes at its last phase; flat and tilted views alike
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 18:04:35 +02:00
Admin
c36f1e78bc flow-ui: the AI service bridge — a flows service (authoring + what is running) and one service per definition on the aichat bus, per-turn context, tools answered off worker threads
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 18:04:14 +02:00
Admin
1980c24973 platform: a draw call whose geometry id went stale is skipped, not drawn with whatever mesh now sits in the reused slot — the runaway triangle count that took a web map pan to 1 fps; reported with a power-of-ten backoff, never per frame
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 18:01:27 +02:00
Admin
58f0700df3 flow: media inputs route by (domain, port) — inpaint takes image and mask as named inputs, control's image is the primary — and every gen node's declared port types are authoritative, including derived Image and Upscale; ocr template typed; empty legacy port arrays warn
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 17:58:36 +02:00
Admin
2a096e50c3 Merge branch 'work' into webdemos 2026-09-03 17:57:37 +02:00
Admin
5a9fa62eed vj: no timed std waits on wasm workers — blocking channels, one tagged deck+background inbox for the stems/lyrics workers, a lost-wakeup fix in the thumb queue, native-only gates on the video decode loops; the DJ web app stops losing a worker at start-up
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 17:57:37 +02:00
Admin
428d23e8ed flow: instances, runs and values on the wire — instance CRUD and inputs (answers a parked Ask), runs with cancel, values on the data plane with ETag/Range and PUT upload, run events fanned into the ring by topic, autostart, janitor, bounded shutdown; real seams by default with with_seams for tests
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 17:57:03 +02:00
Admin
098ba7737a flow: three hostile test batteries — the server routes over real sockets, the language, the engine — with the findings pinned as ignored tests naming the bug
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 17:54:25 +02:00
Admin
a8b0382dea map: the water/foliage shimmer read draw_pass.time, which flags every DrawMap* shader as animated and repaints the whole map at display rate forever — a Rust-stamped shiny_time uniform instead; the 20 Hz heartbeat drives it
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 17:52:44 +02:00
Admin
6691420e25 Merge branch 'work' into webdemos 2026-09-03 17:49:38 +02:00
Admin
2aa0780d0a webgl: BGRA uploads become RGBA at upload and sample_as_bgra is a plain sample on the web, as on every native backend — the tilt-shift's sharp band showed red/blue-swapped water
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 17:49:38 +02:00
Admin
b3da0c5712 flow: ports declare their types — ports: {in: {name: @type}, out: {…}} — so a second image port (mask, last_frame, reference) is as typed as the first; Inpaint, edit references, Music.lyrics, Paint.reference_image, Control.control restored; DREAM closes its loop again; the node catalog walks the prelude
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 17:39:41 +02:00
Admin
90c4f59357 asset-store: bump the static-export golden snapshot for the new rights fields
44b3a7f02 added artist/artist_url/album/source_url/license/license_url to
the exported manifest.json search block, which is part of the hashed
snapshot payload -- the contract changed on purpose, so the deterministic
golden in deterministic_golden_rewrites_graph_and_indexes_every_route was
stale (expected 78d089..., actual 7331ed62...). Verified the export is
still deterministic (same hash across repeated runs) and updated the
golden constant, with a failure message that names the printed `left`
value as the new golden so the next bump is a one-liner.
2026-09-03 17:37:46 +02:00
Admin
639887dda0 webgl: texture passes get their depth target — the tilted map (and every 3D scene drawn into a texture) was draw-order only: hollow buildings, no roofs, landmarks buried
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 17:32:55 +02:00
Admin
2b98a1f048 flow: the run engine — instances with an inputs table, one VM per run with live Fn closures, a dataflow scheduler, five executors (chat over the hub session, gen over the fleet, fn, http under an egress policy, ask that parks a run), content-addressed values with a RAM budget and spill
Real adapters by default (HubChat, FleetGen, HubHttp); seams for tests; the
gen path is proven end to end against the in-process testpattern hub service.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 17:01:53 +02:00
Admin
d74875327f wasm bridge: the page environment carries js_worker_wait so the module links — the pool landing added the import for workers only
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 17:01:52 +02:00
Admin
13f935911a flow: the server host (state thread, two planes over bounded_http, definition routes, events long-poll, lock/listen/token files, watcher) and the client (fail-closed connect, session, subscriber, embed policy); apps/flow-server bin and the flow-ui shell that hosts the server in-process
The two lanes were written blind against the same wire spec; the seams were
reconciled by hand: one FlowSummary, an optional graph on a definition, the
epoch-stamped string event cursor, and strict_json::parse_depth for bodies
that nest deeper than the default cap.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 16:55:59 +02:00
Admin
42213f95b1 Merge branch 'work' into webdemos 2026-09-03 16:41:35 +02:00
Admin
862f3bd298 asset widgets + chat ui + render: fan-outs and jobs on the runtime pool, the transcript read without a lock on draw
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 16:41:34 +02:00
Admin
2a902c646d flow: the recipe prelude loads behind the core one and every pipeline template evaluates, round-trips and is canonical — a test per template
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 16:41:34 +02:00
Admin
285bc9866f asset-ui: workers from the runtime pool, the video player owns its frames and audio state
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 16:38:48 +02:00
Admin
cb498c9c5c Merge branch 'work' into webdemos 2026-09-03 16:38:24 +02:00
Admin
9e2d65b50b pdf + photos + task + wm + video + automate + widgets: no per-job threads — pool jobs and start-up workers
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 16:38:24 +02:00
Admin
90dfcb630b flow: the language — splash prelude for nodes, evaluate a flow file into a graph with typed ports and located errors, tool schema from Input/Output, canonical writer
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 16:29:16 +02:00
Admin
72ab9b0f31 Merge branch 'work' into webdemos 2026-09-03 16:21:17 +02:00
Admin
697215e1d9 route + converse + example-map: every worker comes from the runtime pool or a start-up worker
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 16:21:17 +02:00
Admin
2d4037d141 flow: every asset-creator generation kind as a prelude prototype, and every pipeline as a flow template — inventory and gaps alongside
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 16:17:41 +02:00
Admin
b1a92b73c6 Merge branch 'work' into webdemos 2026-09-03 16:15:08 +02:00
Admin
6dcca0088b workspace: no timed std waits on web-reachable paths — the clippy gate covers every web demo's dependency set
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 16:15:08 +02:00
Admin
78a1681a49 bounded_http: the store's HTTP/1.1 server parser is its own crate — the store keeps only its JSON response shim; flow-server is the second consumer
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 16:13:00 +02:00
Admin
5184340565 platform + vj + map + files + widgets + image_tiles: the runtime owns one warm two-lane task pool — jobs never spawn threads
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 16:06:44 +02:00
Admin
c7639f0841 clippy: timed std waits (sleep, recv_timeout, wait_timeout, park_timeout) are disallowed — they read the std clock and panic on wasm workers
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 16:02:19 +02:00
Admin
dc9cce6c94 workspace: no std clock on the web in any crate the web apps link — the last start-up worker death is gone
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 15:33:31 +02:00
Admin
cb75d367d6 asset client + store: no std clock on the web — the catalog worker no longer dies at start-up
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 14:58:38 +02:00
Admin
35ac364eea vj: the catalog runtime pump no longer logs every poll
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 14:51:26 +02:00
Admin
71c0e1eedf vj: a paused deck is never moved by sync or the loop grid, a loop cell plays exactly its bars with the view held, and the stems banner clears
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 14:44:11 +02:00
Admin
4e61e44fca route + map: the desktop app streams the makepad.nl archive through a persistent on-disk range cache, chosen in settings
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 13:47:14 +02:00
Admin
707d7029e9 vj: every worker thread comes from the platform spawner — the loop ½ button no longer panics the web app
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 13:46:15 +02:00
Admin
acb8689ad7 vj: the loop grid is built and refined on the web too — one pool + channel path for both targets
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 13:33:39 +02:00
Admin
b0ec2a4d0a vj: the audio engine owns the mix state — the UI sends commands over a wait-free ring and reads snapshots, no lock on either thread
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 12:36:57 +02:00
Admin
569b4a7f7c dj-pack: every CC BY and CC BY-SA version and the public domain mark are redistributable licences
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 11:29:43 +02:00
Admin
44b3a7f02e dj-pack + vj: every track carries its artist, licence and source, and the DJ UI shows them
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 10:50:50 +02:00
Admin
83a8d4f69d fonts: the web demos start with the Latin set — CJK and emoji faces load on the first glyph that needs them
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 10:40:48 +02:00
531114e841 nigig: fix makepad-ai-hub ServiceHandle http_thread type (join network HttpServerHandle) 2026-09-03 10:55:00 +03:00
5ec5b65e8d nigig: carry workspace.dependencies into android wrapper manifest
The generated android wrapper re-creates a standalone workspace and only
forwarded [patch.*] sections from the workspace root manifest, so deps
declared via [workspace.dependencies] + workspace = true failed to inherit
in wrapped crate builds. Extract the [workspace.dependencies] section the
same way patches are handled and inject it into the wrapper manifest.
2026-09-03 10:55:00 +03:00
477421e314 nigig: NIGIG test-mode forwarding, custom manifest hook, ortho camera support
- makepad_test/runtime.rs: forward NIGIG_TEST_MODE from host env to the
  Android app via 'am start' intent extra; add wait_timeout (60s) used by
  wait_visible/wait_hidden/wait_count; make query_widgets tolerant of
  snapshot timeouts; grant READ_CONTACTS during adb setup
- makepad-platform android_jni.rs: read makepad.NIGIG_TEST_MODE intent
  extra and surface it as the NIGIG_TEST_MODE env var via apply_studio_env
- cargo_makepad compile.rs: support verbatim custom AndroidManifest.xml in
  addition to the templated variant
- makepad-xr xr_root.rs: add ortho camera controls (ortho, ortho_height,
  min/max), derive Debug on XrCamera
- docs: ANDROID.md and DESKTOP_VISIBLE.md for makepad_test
2026-09-03 10:55:00 +03:00
aeca2e551a makepad_test: grant runtime permissions after install to prevent dialog overlay
The GrantPermissionsActivity pops up during navigation and blocks the
app's event loop, preventing hub responses. Pre-grant all runtime
permissions after APK install to avoid this.
2026-09-03 10:55:00 +03:00
454d60897e makepad_test: force-stop interfering Robrix app during tests
PID 28203 (rs.robius.robrix) was the actual zombie reclaiming foreground
and killing our test app - not our own package. Force-stop both the
target package and known interfering Makepad apps (Robrix) during test
setup to prevent cross-app foreground competition.

Also remove the pm disable-user approach as it doesn't help against
a different package's zombie process.
2026-09-03 10:54:59 +03:00
5c99c410b1 makepad_test: use pm disable-user to prevent Samsung zombie resurrection
Samsung devices keep killed app processes alive and bring them back to
the foreground ~15s later, killing our fresh test instance. force-stop
and kill -9 don't prevent this. pm disable-user fully prevents the
zombie from being restarted. Re-enable before launching the new instance.
2026-09-03 10:54:59 +03:00
bc55ff94aa android: send BeforeStartup/AfterStartup over websocket
The Android platform never sent BeforeStartup or AfterStartup messages
via the studio websocket. Desktop platforms send these through their
stdin event loops, but Android uses websockets instead of stdin.

Without AfterStartup, the hub never broadcasts AppStarted to UI
clients, causing makepad-test to time out waiting for app startup.
2026-09-03 10:54:59 +03:00
dae59f1422 feat(test): extend protocol for touch, long-press, paste, IME composition
Add RemoteTouchState, RemoteTouchPoint, RemoteTouchUpdate, RemoteLongPress,
RemoteTextPaste, RemoteIMEComposition wire structs to StudioToApp enum.

Dispatch new events through cx_shared.rs (TouchUpdate→Event::TouchUpdate,
LongPress→MouseUp+MouseDown, TextPaste/IMEComposition→Event::TextInput).

TestApp: touch_down/move/up, long_press, paste_text, ime_composition.
Locator: touch_down/move/up, long_press, paste, ime_composition.
2026-09-03 10:54:59 +03:00
3e6c5fd57e feat(widgets): reexport optional Makepad sibling crates
Adds feature-gated optional deps and re-exports (makepad-test, makepad-csg,
makepad-gltf, makepad-mbtile-reader, makepad-fast-inflate) so a downstream
workspace can depend on makepad-widgets as its sole Makepad source.
2026-09-03 10:54:59 +03:00
4c94531456 feat(test): Android makepad_test via adb + in-process hub (legacy Java path)
Adds the Android test runtime to makepad_test: builds the APK with
cargo-makepad's standard Java path, installs and launches via adb with
makepad.STUDIO_* intent extras (incl. STUDIO_BUILD), connects the app to an
in-process hub over adb reverse, and waits for startup + responsiveness.
Adds clean in-process hub shutdown (HttpServerHandle + GatewayHandle Drop)
and the STUDIO_BUILD intent parsing on the app side. No native-activity or
NDK APK compilation code is included.
2026-09-03 10:54:58 +03:00
Admin
c3d9314f8b vj: each deck's transport sits on two larger rows under its mixer — every button visible on both decks
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 09:39:07 +02:00
Admin
daf88e10cb route demo: the tiles come from the repacked world archive at its own never-cached path
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 09:31:20 +02:00
Admin
645de42e52 vj: a deck plays and draws while it decodes, and the stems swap in sample-aligned — one streaming path for native and web
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 09:26:24 +02:00
Admin
a3248d1c51 web audio: the worklet gets the audio access pointer as its context — the thread-stack call gained a request id and the audio start was still passing the pointer in its place
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 08:56:04 +02:00
Admin
410acd0eaa web + vj + route: the console carries failures and one-line summaries, nothing per request, per tile or per hiccup
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 08:54:21 +02:00
Admin
58d3fd5d5d web audio: the output is created inside the first gesture, and a stalled worklet module load is retried on a fresh context
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 08:34:59 +02:00
Admin
2db9d4812c mkmap: a root record may decode to 512 MiB — the densest world shards list over five million tiles
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 05:20:46 +02:00
Admin
8ae7c3c396 map: every typed stream index stays u16 — streams chunk under 65,536 vertices
Measured first: the tilted start view carried 15.1 MiB of u16 and
11.6 MiB of u32 indices against 77.3 MiB of vertices, the u32 premium
5.7 MiB, all on the big tiles (fill up to 149 k and casing up to 98 k
vertices). Streams now split at bake time into triangle-safe chunks of
fewer than 65,536 vertices, one geometry per chunk (at most four on any
tile, 191 duplicated vertices in 10.9 M), so u32 leaves the typed
streams. Amsterdam start view: tilted 114.3 -> 108.6 MiB (fill 28.4 ->
24.7, casing 40.4 -> 38.4), flat 311.7 -> 287.7 (fill 83.9 -> 67.3,
fringe 57.1 -> 51.6); the harness prints vertex/index counts and widths
per stream. Same triangles in the same order; Metal hidden grab within
label noise of the head without it, web GPU gate clean (IDX1 lane).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 04:13:46 +02:00
Admin
d6d324144e map: round road caps are a fragment SDF, dead cap rows gone
A round cap was a hard disc fan of up to 32 vertices per road end with no
anti-aliasing; a Butt/Square start pushed a feather pair it never
indexed. The cap is now carried by the last body pair extended by half a
width, with uv.y = 1 + cap_axis, and the fragment computes the circle's
coverage with the same derivative-based AA as the band edge. Amsterdam
start view: casing 42.5 -> 40.4 MiB (shape 100 17.1 -> 16.5, shape 110
15.2 -> 14.5), tilted total 116.4 -> 114.3 MiB, flat 313.9 -> 311.7.
Cap edges gain AA; Metal hidden grab vs the head without it 1.23 % /
0.27 % start (label placement in the crops), web GPU gate clean
(CAP1 lane).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 01:04:31 +02:00
Admin
662d141d19 map: building wall instances on a 20-byte record
A wall edge was eleven f32 (44 B): both ends, base, height, normal, AO,
colour, z-bias. It is now i16 ends, u16 centimetre heights (max error
8 um in the fixtures), f16 AO and z-bias ticks and a UNorm8x4 colour; the
normal is derived from the edge in the shader with the courtyard winding
preserved. Both the wall draw and the projected wall-shadow draw read the
new record. Amsterdam start view: wall_inst 19.3 -> 8.8 MiB, tilted total
127.0 -> 116.4 MiB, flat unchanged. Metal hidden grab within noise of the
head without it (0.99 % / 0.27 %); web GPU gate clean (WALL2 lane).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 01:04:31 +02:00
Admin
13c2500678 Merge branch 'work' into webdemos 2026-09-03 00:29:15 +02:00
Admin
66617cdd7a vj: stem separation is a setting — hub by default, local only on purpose, never auto-started on the operator's machine
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 00:29:15 +02:00
Admin
aa907cab9c platform: a frame trace on both desktop backends, and a window that cannot present stops ticking at 600 Hz
MAKEPAD_TRACE=frames prints, every two seconds per window, what woke
each tick (the waitable beat, a timeout, a message, a timer, a drain,
the display link), the NextFrame gap histogram, the present gap
histogram, the flip lead and refused presents — one bool check per
tick when off, readable over --remote. It was written to chase the DJ
beat wave's judder on Windows and found the measurable defect on the
way: when no window could present (minimised, hidden, or a disconnected
RDP session where the compositor refuses every present), the paint
tick slept a millisecond and re-ran, dispatching NextFrame and
redrawing at about 600 Hz for frames nothing would show. That case is
paced at 8 ms now, the same cadence as video pacing, the idle beat and
a hidden macOS window; a visible window that drops a frame keeps its
1 ms retry.

The judder itself could not be measured hidden — a disconnected RDP
session presents nothing at all — and the reading is that a connected
RDP session's bursty ~30 fps frame delivery is what the person saw;
the trace settles it in one visible run. platform 153 tests; both
desktop targets check.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 00:18:25 +02:00
Admin
97e0701ff8 vj + platform: the web audio thread never waits on a lock — a loaded deck plays
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 00:17:00 +02:00
Admin
0ef0311812 web audio: a throw inside the worklet's process() is reported with its real text instead of a bare ErrorEvent
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 00:10:10 +02:00
Admin
1e2dcd0da8 web: a pass without a draw list is skipped instead of taking the app down (F12 layers overlay)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-03 00:03:23 +02:00
Admin
b39d301ee2 web audio: the worklet links the whole module — every platform import the audio thread does not serve is stubbed, clocks and the UI wake are real
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 23:55:07 +02:00
Kevin Boos
493d23a763
Put the F10/F12/Shift+F12 dev overlays behind an opt-in (#1209)
* remote: honor MAKEPAD_REMOTE in requested(), and hush the close notices when the bridge is off

`requested()` only scanned argv, while `requested_bind()` also reads
MAKEPAD_REMOTE. So `MAKEPAD_REMOTE=1` started the bridge but everything
keyed off `requested()` still said no. Just delegate, so there's one
answer to "did this process ask for the remote bridge".

Also stop printing `[makepad-remote] user closed window ...` to stdout
from every app on every window close -- that line is for the agent
driving the app, so only print it when the bridge is actually up. The
log ring still gets it either way, for /log.

* devtools: put the F10/F12/Shift+F12 overlays behind an opt-in

Three dev tools are currently wired into every app with no way to turn
them off, each on a bare function key:

  F10        the exploded draw-list view. Intercepted in
             Cx::call_event_handler *before* the app's handler, and once
             it's up it also eats Escape, the arrows, +/-/0, I and H --
             no modifier needed -- plus every drag outside the flat band.
  F12        the design tweaker, a child of every Window. Once it's up it
             swallows every pointer event over the body, so the app looks
             frozen to the mouse.
  Shift+F12  the screen recorder, which starts writing mp4s to disk.

None of that is something a shipped app wants a user to find by accident,
and there was no flag, env var or property to stop it.

So: one gate, platform/src/devtools.rs. `--devtools`, or
MAKEPAD_DEVTOOLS=1, and --remote implies it since the /snap + /click loop
drives the tweaker. An explicit MAKEPAD_DEVTOOLS=0 wins over all of it,
which also keeps the off path testable under --remote.

Only the hotkeys are gated, not the tools. Cx::sploded_toggle,
set_tweak_on and ScreenCap::toggle are untouched and still public, so an
app that wants any of this puts it on a key of its own choosing -- that's
the app deciding, rather than a key nobody knew was bound.

Gating F10 and F12 is enough to reach all of it: everything else these
two claim sits behind `sploded.active` / `tweak_is_on()`, and with the
hotkeys gated the only remaining ways in are the /tweak routes (already
--remote, which implies devtools) and an app's own call.

* text_input: drop the Ctrl+Enter submit clause again

`|| mods.control` made Ctrl+Enter submit a multiline input. On
Linux/Windows that's already what is_primary() means, so it changed
nothing; on macOS it turned Ctrl+Enter from "insert a newline" into
"send", which is a surprise in the middle of a chat app's composer.

The comment right above it already described the old behavior, so this
puts the code back in line with it.
2026-09-02 23:41:39 +02:00
Admin
fef80585cb map: marker stalks and stoplights are instances
Every floating POI or charger marker in 3D mode drew two crossed wall
quads on the 48-byte generic layout, and every signal a pole plus three
light quads; together they were the whole 17.1 MiB fill_3d_misc stream.
They are now one 24-byte instance each (anchor, arm/height, colour,
z-bias) over one shared stalk template and one shared stoplight template
per tile, drawn by a prop draw struct on the same lighting path.
Amsterdam start view: fill_3d_misc 17.1 -> 0.0 MiB, stalk_inst 0.1 MiB
(4,887), stoplight_inst 0.2 MiB (1,866); tilted total 143.7 -> 127.0 MiB,
flat unchanged (313.9). Metal hidden grab within the label-noise band of
the head without it (1.78 % / 0.42 % start); web GPU gate clean
(STALK1 lane).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 23:39:20 +02:00
Admin
7a5bf69272 ai-hub registry: the Salamander drumkit samples come from the makepad.nl mirror — the GitHub repo only carries the .sfz files
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 23:37:48 +02:00
Admin
fad1c49827 web server: audio and text files are served, and models/ is immutable like maps/
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 23:37:33 +02:00
Admin
dc85eb0ed3 web + vj: render-to-texture passes keep their 3D camera on WebGL, thumbnails wait for shader compile, bundled tiles re-ask — effect thumbnails match native
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 23:36:53 +02:00
Admin
0fdfba76a3 files: the web's makepad home is a fixed virtual path — std's temp_dir panics there
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 23:26:01 +02:00
Admin
eec516ee06 cef: a Windows backend — the browser's page renders on Windows
The crate had a macOS backend and a stub for every other OS whose
Browser::new refused, so on Windows the app drew its chrome and the
page stayed black. The macOS file becomes native.rs with its islands
gated (the Metal blit, the IOSurface target, the app bundle and
re-exec) and Windows islands beside them: libcef.dll loaded with the
dist's Release dir on the search path, the process handle as the main
args, the exe as its own subprocess, the resources hard-linked flat
beside libcef where it looks for them (the dist keeps icudtl and the
paks in Resources, which crashes libcef), the app's own timer as the
pump, software paint into our texture, and a per-process profile when
a sibling browser holds the profile lock (the unix singleton check
does not exist there). The API hash is pinned to 13800, the layout the
bindings were written for — asking each dist for its newest misaligned
the browser settings struct on 144 and 151. Linux keeps the stub.

Verified on the Windows box with CEF 151: google.com renders in a
standalone browser and in a WM tile (a warm instance adopted in 88 ms,
the next standby up with its own profile); on this Mac with CEF 138 the
accelerated path still renders. cef 4 tests; the browser checks for
windows-msvc. Windows paints in software for now; the D3D11 shared
texture is the next lane.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 23:04:01 +02:00
Admin
bc87a3d359 Merge branch 'work' into webdemos 2026-09-02 22:48:59 +02:00
Admin
4f98c73be2 vj: a deck shows what it is loading — fetch, decode, stems — with real progress
vj: a deck shows what it is loading — fetch, decode, stems — with real progress

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 22:48:58 +02:00
Admin
a2cbe38330 map, platform: the tile budgets follow the platform's one memory number
Cx::memory_budget_bytes() is set once at startup from the device
(ProcessInfo on macOS/iOS, ActivityManager on Android, GlobalMemoryStatusEx
on Windows, /proc/meminfo on Linux, the wasm maximum on the web) through
one policy: desktop keeps 1536 MiB unless RAM is under 8 GiB (then RAM/4),
mobile takes RAM/4 clamped to 384..1536 MiB. The map's four byte budgets
are fractions of it (upload 1/64, pending 1/4, tile cache 25/32, HTTP
cache 5/32; the tile cache caps at 1/2 on web heaps under 1 GiB) instead
of constants. At 1536 MiB they are the binary sizes 24/384/1200/240 MiB,
about 5 % above the old decimal constants. Harness totals unchanged; Metal
grab and web GPU gate within noise (BUDGET1 lane).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:48:11 +02:00
Admin
7a2c09a873 map: road-union faces on a 16-byte vertex
Shape-0 Boolean union faces (plazas, road bodies, tunnel and bridge
faces) leave the 28-byte road vertex for FaceVertexTyped: i16 anchor,
UNorm8x4 colour, F16x2 class/material + depth tick, F16x2 deck lift +
coverage. The road shader draws them through a second vertex format on
the same pass; face streams use u16 indices. Amsterdam start view:
casing 73.6 -> 58.7 MiB (42.5 casing + 16.2 face), tilted total
158.6 -> 143.7 MiB, flat 328.8 -> 313.9, 433 ms/tile. Metal hidden grab
within the noise floor of the head without it (1.26 % / 0.23 %, label
placement); web GPU gate clean. Faces now draw as one batch per tile
before the stroke bands, so a translucent shadowed face over a crossing
casing blends ground instead of stroke at that pixel — not visible in
the crops (FACE1 lane).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:48:11 +02:00
Admin
cc1f25622a vj: cached thumbnails appear within a second
vj: cached thumbnails appear within a second and the pipeline never hangs

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 22:47:39 +02:00
Admin
afdffd71b9 vj: clicking an effect tile puts it in a channel on the web
vj: clicking an effect tile puts it in a channel on the web

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 22:43:45 +02:00
Admin
7b33f7bec6 vj + platform: a loaded deck actually plays on the web
vj + platform: a loaded deck actually plays on the web

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 22:43:14 +02:00
Admin
3b1a8c264f platform: an in-app drag works without an OS drag session (web, Linux) — effect tiles drop into the channels on the web
platform: an in-app drag works without an OS drag session, so effect tiles drop into the channels on the web

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 22:41:46 +02:00
Admin
e7be0e31dc vj + platform: the browser UI thread never waits on a lock, and the web hot paths log only errors and summaries
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 22:13:26 +02:00
Admin
362ac59407 wm: the checkout is found from the working directory too, and a sibling binary is an .exe on Windows
A wm started with cargo run from the repo root but with its target dir
elsewhere (CARGO_TARGET_DIR) found no checkout above its exe and fell
back to sibling binaries — which never exist as bare names on Windows —
so the launcher listed only the linked modules. The checkout is now
looked for above the exe and then above the current directory, and a
sibling binary carries the .exe extension on Windows. Every app is one
cargo run away again.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:12:10 +02:00
Admin
d8539d89e1 map: the detail parse and merge stop allocating
Per-layer tag arenas and views replace a HashMap per feature, the tag
whitelist is applied while parsing, and the worker's output scratch
buffers are reused. Release, Amsterdam: mvt-parse 377,074 -> 3,071 and
detail-merge 2,713,553 -> 19,817 allocations (135x fewer); parse median
5.9 -> 1.8 ms, merge 63.6 -> 9.7 ms; bake 429 -> 344 ms/tile, totals
unchanged (158.6 MiB tilted). Buffers, labels and icons byte-identical
across the 25 fixtures; Metal grab within the noise floor (0.9 % / 0.1 %)
of the head without it; web GPU gate clean (ALLOC1 lane).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
463de64b33 web: shaders compile per draw list, link in parallel
The first WebGL frame compiled every registered program synchronously;
on ANGLE-Metal that is 8.6 s of from_wasm dispatch for the DJ app (35 ms
of wasm), during which no timer, fetch completion or animation frame can
run. SwiftShader hid it by compiling at first draw. Programs are now
queued only when a draw list references them and linked with
KHR_parallel_shader_compile; the first draw waits only for the programs
it needs. DJ app on the GPU: localhost fetches 8.7 s -> 0.2 s, 39
programs ready in 300 ms; route first draw at 224 ms (STARTUP1 lane).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
9b4a3cac3e network: every completion raises the UI signal
A runtime the platform did not create (the asset client builds its own)
has no wake fn, so on the web its responses sat until an unrelated timer
pumped the event loop. EventSink::emit now sets the UI signal as well;
the DJ app polls its store session on Event::Signal instead of waiting
for the next poll tick.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
7e66991246 map: finished bakes reach the screen on the next frame
The worker drain runs once per frame and uploads until a 6 ms / 24 MB
budget is spent (nearest tiles first). Its continuation was a redraw,
and a Draw never re-enters handle_event, so the rest of the queue waited
for an unrelated timer (2-16 s in the web demo). When tiles remain the
drain now also asks for a next frame; that event arrives after the pass
and drains again. Real GPU, no probe, no input: all 25 start-view tiles
inserted between 2 and 4 s at 60 fps (DRAIN1 lane + continuation fix).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
62f38a5b33 route: --hour=N pins the theme hour for harness grabs
The route theme follows the wall clock (night from 19:00), so a grab taken
in the evening compares dark against a daytime baseline. The flag is a
harness override only; without it the clock still rules.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
30fc13bba0 map-tiles: repack runs on all cores, resumes per shard, reports --status
Mac-side only: no in-place rewrite, no push, no fallback source. Shards are
written atomically and skipped on resume; root.mkidx lands once at the end.
Ranged --verify streams the output back. 16 jobs: 4.2 tiles/s on the
500-tile sample (REPACK2b lane).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
cb572aee0f map-tiles: makepad-map-repack rewrites an archive to what the renderer reads
A new binary in tools/map_tiles streams an .mkmap shard by shard into a
new archive: every tile's protobuf is decoded, the osm_* tag tables are
rewritten to the reader's key whitelist, the field-101 shadow sections are
stubbed (regions and building groups stay byte-identical), field 100 is
kept, and the tile is re-brotli'd with the archive's own codec and
dictionary; leaf directories and root.mkidx are rebuilt, output is
deterministic and resumable per shard with a sidecar manifest, --tiles
limits a run to a Hilbert range or a z/x/y list, --dry-run reports and
--verify decodes both archives and checks the policy. On the 25 Amsterdam
start-view tiles: 95.1 -> 47.6 MB decoded, 30.6 -> 16.7 MB compressed.
A label sort tie-break makes the bake byte-deterministic for the parity
test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
f987651778 map: the analytic road fringe is baked only when the view is flat
The fringe stream (AA skirts around the road-union faces) is drawn only
below 25 degrees of tilt, yet every tile baked it: 57 MiB of the Amsterdam
start view that the tilted demo never drew. The bake now takes a fringe
flag the way it takes the 3D flag: wanted while the tilt is under 30
degrees, dropped only above 40, decided in the frame loop next to the
2D/3D flag and re-baked through the same restyle path with a full road
core rebuild; the harness prints both the tilted and the flat totals.

Harness: 215.5 -> 158.6 MiB for the 25 start-view tiles tilted (fringe 0),
328.8 MiB flat (fringe 188.3), 429 ms/tile.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
5a5eda2979 map: road, fill and roof streams on the typed vertex formats, u16 indices
Roads move from eight f32 lanes to a 28-byte RoadVertexTyped (i16 anchor
at 1/64 tile unit, f16 offset / depth / uv pairs, unorm8 colour, f16
params, exact f32 deck), ground fills and roofs to 16 bytes (i16 anchor,
unorm8 colour, f16 params; u16 depth ticks for fills, exact f32 height for
roofs); indices are u16 whenever a stream has fewer than 65,536 vertices.
The three shaders read the typed POD fields directly (the fetch converts),
uploads go through Geometry::update_typed with the shader's layout, the
space-warp subdivision decodes / interpolates / re-encodes the typed
records, and a tile that lands before its shader has drawn waits in the
pending queue instead of panicking. The shader compiler types a packed
field access as vec2f / vec4f, and the Metal name table gains the packed
short / ushort / uchar / char vector names.

Harness: 249.6 -> 215.5 MiB for the 25 start-view tiles, 384 ms/tile.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
5b3378164c platform: thread-caching allocator for the threaded wasm build
std's wasm allocator is dlmalloc behind one global spin lock taken by every
thread on every allocation, so eight workers baking map tiles serialised on
it: a tile that bakes in 0.4 s natively took 15-20 s (mvt-parse 0.7-4.5 s,
detail-merge 2.3-12.5 s), and the whole start view needed 150 s. This
installs a #[global_allocator] for wasm32+atomics: per-thread size-class
free lists (16 B .. 32 KiB) carved from 64 KiB chunks of the dlmalloc heap,
remote frees pushed onto per-chunk Treiber stacks and drained by the owning
thread, exit-safe reclamation, the main thread's caches pre-filled at
startup so its hot path never touches the global lock.

Measured on the same capture with 8 workers: mvt-parse 7-68 ms, detail-merge
27-415 ms per tile (native 4-17 / 23-70), zero traps, zero Atomics.wait,
the full start view. The two-worker stop-gap in worker_count goes with it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
55a38104aa platform: typed compact vertex formats and u16 indices
Draw shaders can declare geometry POD fields as F16x2/F16x4, U16x2/I16x2
(plain or normalised) and UNorm8x4/SNorm8x4; the fetch converts them to
vec2f/vec4f on Metal and WebGL2 (per-attribute vertexAttribPointer with
type, normalised flag, byte stride and offset), the headless JIT decodes
the same formats, and Vulkan/OpenGL/D3D11 refuse compact layouts with a
logged skip rather than drawing garbage. Attribute packing computes
physical offsets with natural alignment for the whole record; the all-F32
path stays byte-identical (stride = slots * 4, packed_geometry_N on GL).

Geometry::update_typed / update_typed_with_recycled_buffers take
IndexData::{U16,U32} plus a byte vector and the shader's input layout;
updates are validated (stride, whole vertices, index range) and every
geometry carries its index width and a layout signature that survive
releasing the CPU staging, so a draw checks layout and index type against
the resident buffers, never the staging enum. Compact fields are rejected
in instance PODs and in nested aggregates at shader validation; SNORM
minima clamp to -1 like WebGL2.

On Metal the decoded (logical) vertex is reached through the shader
context like the instance is, so every shader function that reads
geometry compiles.

No map stream is converted yet; that is the next cut.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
99d2180aad map: roofs on a 20-byte vertex, contact shadows as instances
Lifted shape-0 roofs leave the 48-byte universal vertex for a five-slot
RoofVertexPacked (anchor, unorm8 colour, exact metre height, f16
material | zbias ticks); DrawMapRoof reconstructs the fixed vector
channels and shares the colour pass, the fade, the LOD band and the
shadow-mask projection / footprint cut-out. Roof records that need more
(parapet AO depth variants, marker stalks) stay on the generic layout in
a fill_3d_misc stream. The tree and signal contact shadows are no longer
tessellated ten-segment discs in a vertex stream but four floats per disc
drawn as an instanced unit quad into the shadow mask, with the same
radial ramp.

Harness: 281.5 -> 249.6 MiB for the 25 start-view tiles, 387 ms/tile.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
5b49fee084 platform, map: the UI thread never futex-waits on wasm
std's wasm allocator spins, so the Atomics.wait that killed one WebGL2
run in four came from std Mutex / Once contended between the browser main
thread and a tile worker. Every lock the UI thread shares with a worker
(task pool state, cancellation generation, task results, the scheduler,
the web request table, the archive read states and shard file cache) now
goes through lock_from_ui, which spins on try_lock on wasm32 and is a
plain lock elsewhere. The icon mesh cache, icon slot table and tag-key
whitelist are warmed on the UI thread before the tile pool starts so no
Once can wait either.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
9d664ca967 map: roads on a 32-byte vertex
Casing, stroke and fringe records leave the 48-byte universal vertex for
an 8-slot road layout: full-precision anchor and deck height, f16 offset /
depth / tessellator uv pairs, unorm8 colour, and one f16 integer that
packs class, material, dash id, kind (stroke / union face / fringe) and an
explicit expanded flag. DrawMapRoad reproduces DrawMapVector's expansion,
terrain, tilt, space-warp and shadow-mask behaviour for those streams;
casing/stroke draw through it in the colour passes, the fade cross-fade
and the shadow-mask projection, and the space-warp subdivision has a
road-stride midpoint.

Harness: 339.0 -> 281.5 MiB for the 25 start-view tiles, 437 ms/tile.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
ef25495540 map: building shadows derived at draw time in a screen-space mask pass
The bake no longer unions every building's projected roof and silhouette
into a material-6 ground fill per tile (122.5 MiB of the Amsterdam start
view, plus the b-shadow bake laps). MapView renders one child pass per
frame: wall records swept along the sun as instanced quads, roof/deck
projections of the lifted geometry, footprint cut-outs, and the tree/signal
contact discs (split from the icon stream); every ground shader samples the
mask at its screen position with the same alpha the decals used.

Metal returns the pass texture bottom-up, so the sample is flipped per OS
(WebGL2 flips its render-to-texture projection instead). The draws inside
the mask pass unbind the mask sampler (feedback loop). The compact fill
shader sets the shadow varyings so land and water are shadowed too.

Harness: 475.5 -> 339.0 MiB for the 25 start-view tiles, 411 ms/tile.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:02:33 +02:00
Admin
02b58fc931 map: ground fills on a 20-byte vertex — position, colour, variant + AA coverage, depth ticks; DrawMapFill is the fill path of the map shader
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 22:02:33 +02:00
Admin
57995b6c92 map: uploaded staging is freed on a pool worker — a large free on the UI thread of the threaded web build contends the allocator lock, and a contended lock there is an Atomics.wait the main thread may not make
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 22:02:33 +02:00
Admin
9047600062 map: the bake report can dump the raw and decoded tiles it measured
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 22:02:33 +02:00
Admin
a6f495dfd3 map: street trees are instances of one template per tile; the CPU staging of every uploaded stream is dropped; the memory diet keeps its platform half only
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 22:02:33 +02:00
Admin
72b60b6d0c map: the memory diet folded in — CPU staging freed after upload, budgets from one platform number, bakes capped by bytes in flight, per-zoom bake profiles, a memory report; instance records count as retained CPU
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 22:02:33 +02:00
Admin
2d4ff16b93 map: POI symbols and building walls are instances — one shared mesh per symbol slot, one record per footprint edge, extruded and placed in the vertex shader
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 22:02:33 +02:00
Admin
383b83e783 map: the Amsterdam bake report — the real archive through the real bake path, bytes per stream and milliseconds per tile
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 22:02:33 +02:00
Admin
c2c7f51400 platform: a hosted window on Windows draws again — the depth buffer matches the shared target's allocation
Under the window manager a child renders into a shared texture the
host allocates at the next power of two, while the Window pass carried
a depth texture sized to the pass rect; D3D11 refuses render targets
whose colour and depth views differ in size, so the clear landed (it
goes to the view directly) and every draw was dropped — each hosted
tile on Windows showed only its clear colour, the terminal's blank
rectangle among them. The pass setup now sizes the depth buffer from
the target's allocation when it draws to a texture, from the pass rect
otherwise (a pure helper with tests). The hidden-windows switch is
honoured by the Windows backend on the first show, so a test run stays
off the desktop.

Verified on the box: a hosted terminal's first frame 720 ms after
launch with the prompt in it, a typed dir listed. platform 132 tests;
the windows-msvc target checks.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 22:01:54 +02:00
Admin
21fce80db6 vj: stale-recipe thumbnails are swept from storage on startup
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 21:57:24 +02:00
Admin
5c78ac04a3 vj: web effect thumbnails show the effect, not a black frame
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 21:48:05 +02:00
Admin
c527cd8e88 vj: the web thumbnail pipeline never blocks the main thread
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 21:27:34 +02:00
Admin
6fddf9e285 vj: loading a track onto a deck fetches and decodes its audio on the web
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 21:25:46 +02:00
Admin
0b5bff02cc Merge branch 'work' into webdemos 2026-09-02 21:11:47 +02:00
Admin
f35dd1dd8c vj: effect thumbnails actually render and persist on the web
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 21:11:47 +02:00
Admin
ff6cfec0fc wm: minimise, maximise and close in the bar's top right on Windows and Linux
The omarchy bar is the caption strip and the frame is fully
client-sized there, so a WM window had no window controls at all. The
bar's right cluster now ends in three of our SVG buttons after the
status modules — minimise, maximise (restore once maximised), close —
wired to the window's own minimize, maximize, restore and close;
hover washes the slot, the close carries the accent, tooltips name
them, and the drag query answers Client over them. macOS keeps its
traffic lights on the left. The gallery shows them on every platform.
wm 155; the WM checks for windows-msvc, linux-gnu and wasm32. A real
click on Windows is the box's to prove.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 21:11:05 +02:00
Admin
5c4076168c cef: the build script downloads its CEF distribution itself, on every host, with no shell
The browser could not build from a clean clone on Windows: the fetch
was a bash script needing python3 and tar. It is Rust in build.rs now:
the spotify index is read by a strict scanner, the newest stable
standard build for the platform is picked (or MAKEPAD_CEF_VERSION pins
one), the archive is downloaded to a .part file with its size and sha1
checked, extracted beside it, and a current-<platform>.txt pointer file
names the dist — read before the old symlink, so the Mac keeps its
pinned 138 while a fresh box gets the current build; a platform that
already has a dist is never bumped. MAKEPAD_CEF_OFFLINE refuses with the
dir and pointer named; MAKEPAD_CEF_DRY_RUN prints the plan (with
MAKEPAD_CEF_PLATFORM to resolve another host's). Build-time deps only:
ureq (rustls), bzip2 (bundled), tar, sha1_smol. The shell script is gone.

Proven on this Mac: a pinned re-download of the 138 macosarm64 archive
(255 MB, sha1 verified) differs in nothing from the existing dist over
1,193 files; dry runs for linux64 and windows64 pick the archives already
on disk; the browser checks with no download; the cef crate's build deps
compile for the windows-msvc and linux-gnu hosts. Picker and pointer
tests 5. An actual download on Windows or Linux is not yet run.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 20:56:48 +02:00
Admin
1dcca41ec4 wm: a theme with no wallpapers fetches them on its own
The bundled default theme ships its colours, not its pictures, so a
fresh install had nothing behind the desk until someone ran the
importer. The desk now fetches the current theme's wallpapers from the
omarchy repo on a thread when its backgrounds folder is empty and shows
the first one the moment it lands; the importer shares the fetch. No
network or no pictures means nothing changes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 20:49:58 +02:00
Admin
453197da4d dj-pack: analyse produces the beat grid, overview and loop-splat caches the demo cache ships
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 20:32:01 +02:00
Admin
c1f7b53dca asset-client + dj-pack: a long description never rejects a snapshot; the packer writes one bounded line
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 20:26:56 +02:00
Admin
ffd62da242 dj-pack + vj: the demo cache carries every per-track analysis — slices, beat grid, overviews — and the web app reads instead of computes
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 20:22:09 +02:00
Admin
c7fe851c2e vj: effect thumbnails render, encode and persist in browser storage on the web
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 20:15:47 +02:00
Admin
a9b2185759 Merge branch 'work' into webdemos
# Conflicts:
#	apps/vj/src/main.rs
2026-09-02 20:06:45 +02:00
Admin
8625076661 aichat + photos + image_tiles: make me a picture and it lands on the wall; the wall filters as you type, tiles flying to their places
gen.image{prompt} is a service inside the assistant: on a worker it
runs the creator pipeline for the image domain against the node the
runner picks (LAN discovery, ETA-ranked, the resident flux model wins),
streams the node's progress into the tool card, and writes the picture
under the makepad home's gen/ dir; the creator library gains
generate_bytes — the request, poll and fetch that generate_and_publish
now shares — so no second pipeline exists. photos.add{path} bakes that
one file into the open library (only from the gen dir or the person's
home), re-opens the wall and glides onto the new picture; the module
executor answers it later through the host's reply sink. The model
chains them, launching photos in between. A registry seam found on the
way: a call dispatched in the same event a link was adopted reached the
port before its Registered, so the registry now answers a leading
Register at once.

The wall gained a search box: every keystroke re-cuts the packing over
the matches (every word in a title or link; title beats link, a whole
word beats a substring), each picture flies from its drawn rect to its
target over 450 ms, dropped pictures shrink and fade, returning ones
grow back; Esc clears; ⌘F or a bare slash focuses the box; the status
reads "35 of 293 · ro". photos.filter{query} sets the same from the
pane; instances are rebuilt only while a flight runs, and the clock is
the platform's.

Proven on the fleet: "make me a picture of a red bicycle on the moon
and put it in photos" → the file in 40 s, photos launched, added and
shown among the comics; typing r, ro, rob re-packs the wall live; the
filter through the pane in a process tile and as a module. ai-services
39, aichat 5, photos 9, image_tiles 12, creator 9; aichat and photos
check for wasm32.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 20:03:54 +02:00
Admin
53f0e965f8 vj: the DJ/VJ app exposes itself to the desktop assistant
A service over the engine the UI already drives: status (what is live
and next, the fader, overlay, autopilot, bpm), search over the catalog,
cue (the UI's content-click path), fader, next, autopilot with a style,
overlay, and the decks — deck_play, deck_stop, deck_load, crossfade —
through the same functions the console's controls call. A context line
follows the live item, fader and autopilot. Written by the delegate from
the sheets/files shapes; vj's ai tests 4, the binary suite 821 (the two
media decode-pool timing tests fail on this Mac before and after).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 20:03:54 +02:00
Admin
40e483f1ff vj: the web music browse actually fires — proven on the live path
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 20:02:06 +02:00
Admin
93232a2b2e platform: two pool workers on wasm until the allocator is per-thread
std's wasm allocator is one global spin lock, so workers that allocate
serialise on it and per-job throughput collapses with the worker count
while the browser main thread starves behind them. Measured on the map
bake in the route web demo: a tile takes 15-20 s with 8 workers and
0.1-0.5 s with 2; the start view completes in 80 s instead of 150 s.
Every web app's pools get the cap through worker_count. It goes when the
thread-caching allocator (ALLOC2) lands.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 59096700d56cd8f3f15724e24129c0b428bf6ae7)
2026-09-02 19:36:48 +02:00
Admin
64d2d3acee map-build: an unfinished bake resumes or restarts itself, and the maps root does not depend on the cwd
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 19:25:33 +02:00
Admin
cc8ee5c9f5 vj: the web music browse fires when the store is ready, and startup never stalls the UI thread
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 19:14:21 +02:00
Admin
04a36ded31 map-build: the bake produces only what the renderer reads; a bad tile is skipped and logged
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 19:05:11 +02:00
Admin
017dc69faf Merge branch 'work' into webdemos 2026-09-02 18:52:08 +02:00
Admin
78bfd5c27c vj: the web explorer shows the store's tracks — the listing reaches the rows
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 18:52:08 +02:00
Admin
3691875775 wm: launcher icons — a photo for Photos, a shirt for Fabric, a globe for Route, a play badge for Video, a pulse for the task manager
Five 16 px icons in the set's own style (our SVGs, stroke 1.2,
currentColor), wired into the shell's icon set and the launcher map.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 18:47:34 +02:00
Admin
0950ed7b4e Merge branch 'work' into webdemos 2026-09-02 18:46:52 +02:00
Admin
4b529a1203 map: tile ranges fetch once, centre-out, and never time out while queued
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 18:46:52 +02:00
Admin
37686534b0 image_tiles: the wall is box-packed — justified rows keep every picture's own shape and fill the width
A pure packer (pack.rs) lays pictures in rank order into rows of one
target height and scales each full row to span the wall exactly, so
tall strips sit beside wide panels with no gaps and no overlap; the
last row keeps the target height instead of stretching two pictures
into giants; a 4 % gutter is split per cell. TileGrid gains the packed
wall as its default (packed: false gives the old uniform grid), frames
its camera on the packing's bounds, hit-tests through the row
partition, and picks each atlas page's level of detail from the
largest visible tile in it rather than one unit cell. photos needs no
change. image_tiles 10 (5 new), photos 7; the photos lib checks for
wasm32. Driven hidden: the 293 comics in justified rows, zoom, pan, a
click on a comic naming it, and photos.show gliding onto one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 18:38:48 +02:00
Admin
95fd7d6c82 platform: a hosted window with a dpi override lays out in its own points and gets the host's pointer remapped
A stdin-loop child stored the host's WindowGeomChange raw and never
recorded the host dpi as the window's native dpi, while the native
path converts every OS geometry through the app's dpi_override and
remaps every pointer. VJ is the app that sets an override (its console
scale shrinks the dpi to fit a tile's height), so under the window
manager it laid out in larger points than the pointer it was handed —
every click landed short — and each host resize re-set the raw host
size under the override, so consoles of the wrong physical size were
drawn into the shared framebuffer over the previous full one: the
stacked stale consoles the user saw. Terminals never showed it; they
set no override.

The stdin arms of every platform now apply a host geometry the way the
native path does (the host dpi recorded, the size converted through
the override) and remap pointer positions with the override scale;
window hit-tests compare in native points. No host-side change: the
crop is per presented frame and the swapchain handoff only bridges a
resize.

Proven hidden under the WM: the explorer tab answers a click, typing
filters the music list, a split and its close re-lay the console out
with no ghosts. platform 128 tests; Linux and Windows targets check.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 18:37:49 +02:00
Admin
0e5e007f03 Merge branch 'work' into webdemos 2026-09-02 18:30:27 +02:00
Admin
5fdd5d5f47 vj: on the web the explorer lists the site store's music and a deck loads it with its stems
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 18:30:27 +02:00
Admin
190062e4a1 route: the maps app exposes its tool table to the desktop assistant
The same tools route's own assistant has — plan, add and remove stops,
status, along-the-way search, map fly-to and trip framing, layers,
theme, markers, geocoding, weather, trip history, navigation — go on
the AI bus from one table (the definitions local_agent already maps),
dispatched through the one execute_tool; reads are Read, anything that
moves the map or changes the trip is Act. The context line carries the
map view and the trip; the desktop pane opening closes route's own
assistant panel. Written by the delegate from the sheets shape; route
25 tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 18:29:43 +02:00
Admin
a772ec8f40 terminal + browser: a warm-pool standby joins the AI bus only once adopted
Dormant instances registered their services, so the assistant could
type into a shell or steer a page nobody could see. The port opens at
startup for a real launch and on WmEvent::Adopted for a standby, the
way files already did.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 18:25:40 +02:00
Admin
432121ebe5 aichat engine + wm: launch, then use — the assistant continues in the same turn once the app it started is on the bus
An ok os.launch result is held while the launched app is not on the
bus and delivered when it registers (within 8 s), rewritten as "<App>
is running now. Its tools: … Call them directly." with the tool table
rebound mid-turn where the model can (the local model prepends the
update to the tool result; Claude keeps its native table until the next
turn and gets the names in the text); past the patience the result says
the app has not connected yet. An assistant entry that never got
visible text is dropped before a tool call and at the end of a turn,
and the panel draws an empty one as nothing — the blank block above
the first card is gone. The model's context now opens with the running
apps and their tool names and lists the others as os.launch candidates;
the WM's launch description says a running app's tools are already
there, and its answer for one that is already running says so and
brings it to the front.

Proven on the fleet model: "open the photos app and find the pictures
about dogs" → launch (running 4 s later) → search, summary, search →
the answer, one turn, the tile beside the pane. ai-services 38, wm 153,
aichat 2; aichat checks for wasm32.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 18:23:15 +02:00
Admin
73ffe87e8b Merge branch 'work' into webdemos 2026-09-02 18:11:05 +02:00
Admin
d1910b8fe4 web-server: a store snapshot's extensionless routes are served with the types the exporter recorded
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 18:11:05 +02:00
Admin
8b46ce0279 aichat: the composer's hint is a dark grey Ask AI, not the typed colour
The placeholder used the focused-text colour, and the composer is
always focused, so the hint looked like an entry.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 18:09:14 +02:00
Admin
4ca36c1d92 ai hub + services: the assistant's model comes from wherever it is resident — the fleet chat box, with tools, then the local weights
The hub chat session's election no longer stops at this machine: a
co-located serving holder first, then a fleet chat node heard by
discovery and role-allowed for chat (a short patience for beacons),
then the weights on this machine — and an honest answer naming both
misses and where weights may be put when none of those exists. Tool
packs ride every route: the system text carries the tool table as the
node's chat_system, one splitter strips thinking and collects
<tool_call> bodies across deltas, the in-process worker's parser is the
one parser, and tool results go back as tool turns. A node that fails
mid-turn ends that turn with the node named; the person's next line
re-elects, served first on the new route. The tools-only guard on the
proxy is gone. The session exposes its route, and the panel's chip
shows it.

The local lookup is independent of the working directory: the env
override, then the makepad home's weights (Qwen preferred, largest),
then the checkout the binary came from, then the cwd — so an assistant
launched from a binary copy finds the same weights as one run from the
checkout, and the Local provider always builds even with none.

Proven live with no local weights: from the sheets overlay a plain
line reached the fleet's 27B (the node with the model resident won the
pick), the model called sheets.summary and answered in 12 s; the app
stayed at 254 MB. hub_chat + local_llm 11, services 34.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 18:06:00 +02:00
Admin
82ac76882e photos: the picture wall as an app — the SMBC archive on the tile engine, a module from its first line; fabric in the launcher
apps/photos (lib + bin) puts libs/image_tiles' TileGrid in a tile: it
opens a baked library found by name (IMAGE_TILES_HOME, the cwd's
local/image-tiles, the checkout the binary was built in — the smbc
collection first), shows a status line with the bake command when
there is none, and exposes photos.search (every query word in a title
or link), photos.show (the camera glides onto one picture) and
photos.summary — the same answer for the port and for the module's
executor. PhotosModule mints the view inside an isolate with an open
schema that takes a collection name, never a path; the WM links it
behind app-photos and lists it in the menu. image_tiles learned to
decode GIF and WebP and to read a manifest's pictures from disk, so
the bake can take local/smbc's mirror: the last 300 comics are baked
under local/image-tiles/smbc (293; seven of the mirror's files are
saved error pages). fabric joins the launcher with a title and a
polite close.

Driven hidden: the wall in a process tile beside the left pane, panned
and zoomed; search and show cards; the same wall as an in-process
module; fabric launched and closed; standalone photos with the F10
overlay on the left. photos 7, image_tiles 5, wm 152; the photos lib
checks for wasm32.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 17:59:12 +02:00
Admin
6105b61e6e sheets + files: the assistant reads and writes cells, makes folders, renames and trashes
sheets: read_range (one cell or an A1 rectangle, up to 2000 cells, rows
as tab-separated display values), find (case-insensitive over the used
range, 50 hits), write_cell and set_range (Act) — edits go through the
same commit path as paste and direct entry, so recalculation, undo and
the chrome all see them; the module executor answers the same tools.
files: mkdir, rename (a bare name, no separators) and trash (into the
app's own trash dir, never a delete) — Destructive, so the router shows
a confirm card first; every path through the jail, on the demo VFS as
on disk. Written by the delegate from the existing shapes; sheets 88,
files 158 tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 17:58:35 +02:00
Admin
43c36c1a36 Merge branch 'work' into webdemos 2026-09-02 17:57:19 +02:00
Admin
3c033977f8 geodata: a data library keeps its own clock — the GUI platform is not a dependency
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 17:57:18 +02:00
Admin
7f1fefd56d terminal + browser: the assistant can read the screen, run a command, read and steer the tabs
Two services on the AI bus. terminal: read_screen (the visible grid as
text, cursor and cwd in the note), read_scrollback (up to 2000 recent
lines) and run — which types the line into the live, unsandboxed shell
exactly as the person would, refuses control characters and anything
over 4 kB, and answers at once so the model reads the outcome with
read_screen next. browser: page (the active tab's title and url — CEF
exposes no page text, so none is promised), tabs, navigate and new_tab,
http(s) or about:blank only. Context lines: the shell's cwd, the active
tab. Written by the delegate from the sheets/files shapes; terminal
331 + 5 tests, browser checks.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 17:55:54 +02:00
Admin
4e58ab9dec vj: the output window exists only once opened — never on the web
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 17:52:32 +02:00
Admin
f80895accd Merge branch 'work' into webdemos 2026-09-02 17:47:46 +02:00
Admin
7deb052ad1 web-server: ETags from size and mtime, never from content; upgrades only where a socket route exists
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 17:47:46 +02:00
Admin
4317f58804 sim + render + chat: walk decks as a surface, the filmed body is no obstruction, the brief never asks
- libs/sim: DeckStrip (deck.rs) — every laid corridor registers its walk
  deck; movers pin to it exactly instead of climbing 2.5 m slab kerbs;
  feet inside a static box lift onto its top; the world composes floor,
  terrain, materials and voxel in one sampler
- libs/sim + libs/render: camera_path_limit takes the filmed body and never
  treats it as an obstruction (the coaster's ride boom was clamped to 1 m);
  the shadow gate's settle test gets a hair of float slack
- libs/asset/chat: the in-game builder never asks before building; the
  brief teaches game.driver and the race countdown, under budget

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 17:41:08 +02:00
Admin
dbe43bd327 wm + widgets: the AI panel on the left, pushing the body in
Decision 17: the chat never sits over the app. Under the WM the pane is
a left column of the desk row — its walk width is the reserved strip,
eased from nothing to the card plus a gap as it slides, the card
right-aligned in the strip so it comes in from off the desk's left edge,
its edge on the right — and the desk's layout takes what is left, tiles
snapping to it while the strip moves the way they follow a drag. In a
standalone Window the F10 slot does the same: it reserves the strip as
the body's left inset every frame, lays the body out beside it, and
reads the app's own ground colour after its retint. The background
switcher's index lives on App, not in a static.

Verified by hidden grabs: a terminal full width; F10 → the pane on the
left and the desk narrowed by 450 px with the terminal filling it; F10
→ full width again. sheets standalone: F10 → the grid starts right of
the card, nothing over cells; F10 → back. wm 152, widgets 138 (+ the
two known widget_tree failures), sheets 84; the WM checks for wasm32.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 17:38:39 +02:00
Admin
0b2be94917 Merge branch 'work' into webdemos 2026-09-02 17:35:00 +02:00
Admin
e6a305c7e7 ai-hub + dj-pack: a whole track fits a stems job; long tracks split into spans
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 17:35:00 +02:00
Admin
82def6b25b wm: every app under the desk — the chat keeps the keyboard through an automatic refocus, a pool that gives up, polite closes for browser, sheets and aichat
Driven native, one app at a time, with the assistant up at boot:
terminal, files, task and browser warm-adopted; sheets, image, video
and pdf launched cold; sheets also as an in-process module; the tools
answering through the pane (os.list/launch/close/open, files' listing,
a refused jail escape, a treemap, sheets' summary both ways).

What broke and is fixed: when a window closed or a client died the
layout's automatic refocus took the keyboard from the OPEN chat, so an
os.close typed in the pane sent the next console line into a sheet —
the refocus now leaves the keys with the pane while it is open (a click
on a tile still moves focus, through focus_client). A warm spawn that
cannot start (an installed layout without the binary) counts as a crash
so the pool gives that app up instead of retrying every tick. browser
and sheets link the WM API: a title in the bar and a polite close on
CloseRequested; aichat quits on CloseRequested (the desktop going down).

Measured (WM log, launched → first flip): warm-pooled apps 25–50 ms;
cold launches through cargo's freshness check 0.7–3.7 s (up to 14 s
under a parallel launch's target lock); the same binaries run plain
0.3–1.5 s after macOS has scanned them. wm 148, sheets 84, aichat 2.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 17:24:54 +02:00
Admin
6d47c3f5ac wm: the omarchy themes moved to omacom/omarchy — the importer follows
basecamp/omarchy answers 301 on the API and the importer fetched no
wallpapers; the repo is omacom/omarchy now, same quattro branch.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 17:11:26 +02:00
Admin
341f66d10a Merge branch 'work' into webdemos 2026-09-02 17:06:28 +02:00
Admin
d9f04fcabf dj-pack: pack reads caches, never creates them; dry-run writes nothing
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 17:06:28 +02:00
Admin
d3cd233e23 wm: one desk for every target — the host seam, the assistant seated in-process, the web profile, and the assistant up at boot
The desk (layout, tiles, the shell, the AI pane, the module host, the
bus) is one program everywhere; what differs around it is asked through
host.rs: a monotonic clock in platform seconds (std::time traps on
wasm), whether child processes exist at all, a child's environment, the
theme choice kept in the desk's own storage namespace. Every Instant in
the WM is gone; the hub, the warm pool and process launches are skipped
where there are no processes, and the bar's clock falls back to the
platform epoch when it cannot fork date.

The pane has a second body: where the chat is linked in as a module —
the web build, or a desktop that switched aichat to module hosting — the
pane instantiates mod.widgets.AiChatOverlay by name, exactly as a
Window's F10 slot does, and the WM's own os service and every module
instance reach it as in-process links (pane_links.rs), never as frames.
The registry's web profile makes every linked module a Module and
answers os.launch of anything else with unavailable. sheets' Open and
Save go to the instance's storage jail, asynchronously, on every
platform.

The assistant starts with the desktop (decision 15): the aichat child
(native) or the overlay (in-process) comes up at startup with the pane
closed and no focus taken, drawn just off the desk's edge so the child
is configured and presents before any F10 — which now only slides it
in. The gallery fixture leaves the desk's DSL for a by-name host (its
construction overflowed the wasm stack).

Platform bug found by the web drive and fixed in libs/wasm_bridge: the
ToWasm reader added a block's absolute end to a relative base, so two
JS→wasm messages in one buffer decoded garbage and trapped; it seeks to
the stored end, with a batched-buffer test.

Verified natively by a hidden drive: the child's first frame 1002 ms
after boot, before any F10; F10 → the pane at once with the composer
focused; F10 → gone. On the web (threaded build, 1.5 MB brotli, COOP/
COEP): desk, pane with No model, os.list, an in-process Sheets tile and
its summary, terminal answered unavailable, zero traps. wm 148, sheets
84, app-module 3, wasm-bridge 1; wasm32 checks and the std::time clippy
gate clean for wm, sheets, app-module, aichat, wasm-bridge.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 16:57:45 +02:00
Admin
6857d86461 files: the projected treemap clips boxes at the near plane and keeps the camera above them
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 16:51:13 +02:00
Admin
94b8726ffb dj-pack: tracks in, stems through the hub, a site store snapshot out
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 16:50:11 +02:00
Admin
0684b4123d asset-store: export-static reads every content schema the server serves
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 16:49:57 +02:00
Admin
5d007e0373 files: an unset projection preference means 2.5D
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 16:44:42 +02:00
Admin
0105fa5cd8 files: the space view opens in 2.5D
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 16:41:53 +02:00
Admin
9630337538 asset-store: export-static selects what the namespace holds and says why when it does not
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 16:39:46 +02:00
Admin
511048ca85 vj: no filesystem paths on the web startup path — the store is the disk there
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 16:34:27 +02:00
Admin
412b3c920e files: the space view rescans through the virtual filesystem — the web has no other disk
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 16:32:06 +02:00
Admin
106b38cede wasm bridge: the imported memory honours the module's declared limits
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 16:28:44 +02:00
Admin
aa7345d8c5 vj: the whole app on the web — site store for content, embedded store for local imports, native-only seams Unavailable
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 16:23:52 +02:00
Admin
6be1118b2d vj: the whole app on the web — site store for content, embedded store for local imports, native-only seams Unavailable
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 16:12:34 +02:00
Admin
58ad22e94b Merge branch 'work' into webdemos 2026-09-02 16:00:18 +02:00
Admin
7613f3ef2f web-server: one deadline per response from its size, 404 before 405 for unknown API paths, one Earth radius
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 16:00:18 +02:00
Admin
6b77318209 chat: the in-game builder never asks before building
"make me a giant forest with roads" answered with a plan and "Want me to
go ahead and build it now?" — the player is holding a controller. The
agentic doctrine and the game brief now say it outright: a request is an
instruction, decide the open details, build in this turn, report in a
sentence; a turn that ends in a question is a failed turn.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 15:45:03 +02:00
Admin
6f8c08f4a6 web-server: POST /api/crash stores crash reports in a rotating log on both servers
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 15:44:46 +02:00
Admin
640602cc84 Merge branch 'work' into webdemos 2026-09-02 15:42:27 +02:00
Admin
6c32c88d1a web-server: ETags from size and mtime, deadlines from size, Allow per resource
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 15:42:27 +02:00
Admin
4946c9eb0d sim + render: a repaint is not a world edit — colour and glow restyles never rebake the lightmap
Crossroads rebaked its lightmap on every stoplight phase (70 bakes a
minute): the traffic kit's lamp repaint bumped the one static revision
the renderer keys its slabs, occlusion, shadow receivers AND the GPU
lightmap kick on. The world now carries a paint revision beside the
geometry revision; the static slabs repack on either, the bake kick
only on geometry. Tests pin both.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 15:41:21 +02:00
Admin
9c9c72ba8d web-server: bodies land before workers, client keys normalized, verbs fail closed
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 15:36:09 +02:00
Admin
441d604e5f vj: dropped or picked audio publishes into the connected store
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 15:32:39 +02:00
Admin
22faab2c8b Merge branch 'work' into webdemos 2026-09-02 15:30:29 +02:00
Admin
cf2b8ca4bd platform: no std::time on web — clippy guard and the platform clock everywhere a web build runs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 15:30:29 +02:00
Admin
d3472ebd35 tsdf: the clock-taking XR helpers are native-only — the browser has no depth camera and no Instant
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 15:29:00 +02:00
Admin
faf8112816 web path: the tessellator's lap timer, the trace span and the fusion cycle timer have no clock on the web
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 15:27:30 +02:00
Admin
82d0cfaab9 web path: the trace helper keeps its doc, the journal nonce steps a counter where there is no clock or pid
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 15:24:00 +02:00
Admin
0f967ce382 web path: the stream trace and the sqlite pager never wait on a clock the browser does not have
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 15:23:10 +02:00
Admin
942ff86df3 sqlite: the browser store has one owner — its locks never wait on a clock
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 15:21:02 +02:00
Admin
670cc76977 asset-store: the platform storage API is a hard dependency on the web, where the embedded store is the only store
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 15:17:16 +02:00
Admin
77d91385fc wm: the module contract and the first in-process app — sheets in a tile, in an isolate of its own
libs/app_module (makepad-app-module) is the contract (aicontrol §3):
AppModule (id, label, register into an isolate, an OpenSchema, create
into InstanceParts, capabilities), InstanceHandles (an InstanceScope
owner token, the storage jail as a cx.storage namespace, the viewport,
a ReplySink for calls that finish later), a ServiceExecutor addressed
apart from the root, and an OpenSchema that refuses raw paths — a file
is a handle the host issued or nothing. Only types live there.

The WM hosts one (module_host.rs): allocate the isolate, retint its
theme from the palette, register, create — one trusted entry into the
isolate, never a second &mut Cx beside the VM; teardown drops the root
first, runs shutdown in the isolate, frees it. MpModuleView is the tile
for an instance's root, drawing and dispatching with the isolate
INSTALLED on Cx (enter_isolate/leave_isolate, new in widgets) and
gating keys on the WM's focus; a TileHost trait covers both tile kinds
so the desk and the focus logic never ask which. The registry overlay
(apps.rs) links modules per app-* feature; the desktop default stays
Process, switched per app in ~/.makepad/wm/apps.splash or by a dev
--module flag. The bus gains its in-process leg: an instance is an
m<id> endpoint the pane addresses like any other — the leg the web
superbuild runs everything on.

sheets is lib + bin: SheetsModule mints MpSheets{} in the isolate and
answers sheets.summary on its executor; the standalone binary keeps its
Window and F10 overlay behind the standalone feature.

The drive found a platform bug: a shader compiled on a widget minted in
an isolate hit the draw-shader object cache by object index alone, so
an isolate's Button reused the main heap's entry and drew with zero
instance slots. The cache is keyed by (heap, object) in every backend,
and add_instance refuses a zero-slot draw call instead of dividing by
it. Isolate cost, measured (the P2 entry gate): 12 ms and ~8 MB per
isolate, flat to 32 — no pre-warming needed.

Verified by hidden grabs under the WM: /os.launch sheets → an
in-process Sheets tile (no child process), the apps row Desktop ·
Sheets, /sheets.summary answered through the in-process leg, a warm
terminal beside it, F10, close → the instance torn down and its isolate
freed. wm 145, sheets 84, app-module 3, services 33; platform and draw
check for windows-msvc, linux-gnu and wasm32.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 15:16:42 +02:00
Admin
9942e26f48 asset-store: the embedded module lives in its directory now (the browser-durability landing's deletion)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 15:16:05 +02:00
Admin
372bfd7960 asset-store: browser durability — generation extents over cx.storage, chunked CAS, quota and GC
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 15:14:48 +02:00
Admin
22b2bf9fc3 sim + chat: the composed world surface takes a map floor; the chat gets plan tools
- libs/sim: meshfloor.rs FloorRaster (one storey per column from a map's nav
  graph); composed_surface_sample_at(floor, terrain, materials, voxel) is the
  one seam nav, corridors and lots read; in-eval voxel ops no longer bump the
  history revision
- libs/asset/chat: world.get_plan / world.set_plan (shape-validated, stale
  revisions refused), the authoring state prefixed to every turn, map turns
  routed to the plan tools; game.md: the foundation line, the capability kinds
  verbatim, no order-independence claims

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 15:02:46 +02:00
Admin
0ecb81cf56 ai models: the model-crates env-var cleanup — 172 research knobs gone, the unset default is the code
The per-family model crates (flux, h3, paint, music, speech, stems,
vision, sfx, rife, trellis, beats, common) and the libs/diffusion
research binaries read some 190 environment variables that were
research knobs: tensor dumps, per-stage timing, oracle-parity and
fixture rigs, experiment toggles. The path taken with none of them set
is the one that ships; every such knob is deleted with the code it
gated, and every losing branch of an experiment toggle goes with its
toggle — dead kernels, fields and functions included. What remains are
the real configuration variables (the FLUX_*_MODE family, FLUX_GRAPH,
the VAE pool cap, the FLUX2 text-encoder residency, H3_VAE_BATCH, the
music3 caches and official modes, the stems/beats f16 switches, the
weight and data roots) and the build-script variables.

Rebased on the runtime cleanup: precision stays explicit everywhere
(GemmPrecision, f16_attention_operands, the H3 text precision, DA3's
StrictF32 in code); no act16, no H3_ACT_F16, no FLUX_ATTN_F16 or
FLUX_VAE_CONV_GEMM reads survive.

Reviewed by the delegate reviewer (APPLY, no findings) and gated on the
Windows CUDA box: all seventeen model crates check, motion and vision
tests, the hub and the diffusion bins — the gate caught one CUDA-only
tap marker the Mac never compiles, removed here. On this Mac: the same
checks plus motion 24, paint 158 and vision 23 tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 14:50:06 +02:00
Admin
f51ca0795e workspace: the wasm interpreter's tests build at opt-level 1 — its own profile setting is ignored inside a workspace, and opt-level 0 overflowed the script eval stack
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:45:08 +02:00
Admin
1b7089962a files: the space view is where a tab starts
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:42:22 +02:00
Admin
fd5d70c108 cargo-makepad: the app's own resources are packaged under its bin name, which is how self:// resolves
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:42:22 +02:00
Admin
04a94ef780 realtime: one service-log line when a live session opens and one when it closes
A headless box's service log showed only the startup banner for a whole
session; now it records the model and wire encodings on open, and on close the
elapsed time, frames in/out, fps, and the dropped/undecodable/unencoded counts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 14:41:09 +02:00
Admin
6b661bf4c5 web: crashes report themselves — panic text, breadcrumbs, memory, workers; a dead instance stops pumping
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:37:51 +02:00
Admin
dfe2087399 web-server: O(1) report admission, one connection deadline, shared route sampler, static fallbacks, cache policy
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:37:36 +02:00
Admin
fb1416d8c0 cargo-makepad: the threaded wasm module is linked with the 4 GiB wasm32 memory ceiling
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:37:26 +02:00
Admin
f6bbee90ef wasm bridge: shared memory asks for the 4 GiB wasm32 ceiling and steps down where the engine refuses
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:33:35 +02:00
Admin
b0646d0714 Merge branch 'work' into webdemos 2026-09-02 14:31:16 +02:00
Admin
232909d16a script: the VM reaches std and its slot through one host — no aliased references
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:31:15 +02:00
Admin
aa816ed8a6 windows h264 decoder: pictures come out one access unit later — rewrite the SPS level so the DPB is one picture deep
Measured on the body node: the Microsoft H.264 decoder accepts MF_LOW_LATENCY and
CODECAPI_AVLowLatencyMode and ignores both; it holds pictures until the DPB it
derives from level_idc is full (six access units in, nothing out until DRAIN), and
appending access unit delimiters changes nothing. With level_idc rewritten to 1.0
in every SPS the DPB is one picture and each picture is emitted as the next access
unit arrives (5 of 6 while streaming, the last on flush). flush() now sends
COMMAND_DRAIN. The captured-stream test replays dumped access units and requires
the pictures while streaming; the keyframe-request check is its own test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 14:30:44 +02:00
Admin
2b035ff33c route: the navigation session keeps time on the platform clock — the std clock traps on wasm
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:28:20 +02:00
Admin
ff62db3c5e ai libs: the runtime env-var cleanup — precision is a per-caller policy, not an environment side channel
The AI runtime crates read their precision, activation and kernel
choices from environment variables; the code path taken with none set
is the one that ships, so every knob that selected it is now an explicit
argument and every losing path is gone. GemmPrecision { f16_accumulate,
f16_activations } is passed by each caller: the default {true, true} is
the old unset Flux route; H3's DiT and text encoder pass {false, false}
(H3's >1e4 activations saturate f16 — the policy H3 used to set through
FLUX_GEMM_F16ACC=0 on itself), its VAE {true, false}; DA3's StrictF32
selects f32 packed attention in code; Hy-Motion carries an explicit
f16_attention_operands flag through its text refiner, its double and
single blocks and the CUDA backend (true in production, false only in
its full validator). The libs/diffusion bins — a separate workspace —
are migrated to the same shapes.

Benches and validators no longer set variables on themselves: llama's
skip-logits is a session option (the CUDA bench turns it on), OCR takes
explicit use_f16_gemm and tiled_roformer options, the lane speculative
probe reads its CLI. The live gates the first cut had deleted are back
as explicit-config tests: MMQ M=129, the strided-f32 MMV path, the
RMS+MUL CPU oracle. The loader's THREADS and CHUNK_MB stay real settings.
The Metal quantized-matmul experiment (metal_qmm and its vendored MLX
kernels) was reachable only through a knob and goes with it.

Reviewed in three rounds by the delegate reviewer (the last round
accepted everything but one Hy-Motion call site, fixed in round four
and reviewed here), and gated on the Windows CUDA box: lib checks of
common/paint/loader/cuda/llm/motion/vision, motion 24 and vision 23
tests, the hub check, the diffusion bins, llm 253 passed / 1 ignored.
On this Mac: the same checks plus the motion and vision tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 14:26:48 +02:00
Admin
c24c2064fd aichat: the Window overlay — F10 in every standalone app, the in-process port, the /ai bridge routes, sheets as the pilot
Every Window now carries an AiChatSlot beside the tweaker: zero cost
while off, inert when the window manager hosts the process (the WM's
pane is the chat then), and on the first bare F10 it instantiates the
chat's module root BY NAME — mod.widgets.AiChatOverlay — which exists
when the app links makepad-aichat and calls its script_mod; an app that
does not gets one log line. The overlay slides in from the right over
the body with the WM pane's motion, owns the pointer inside its rect and
the keyboard through its composer, keeps ticking while hidden, and draws
in the window's retained overlay list so it composites over the deferred
body. Requests from outside the tree ride Cx globals, never statics.

An app exposes itself with one call: AiServicePort::open(cx, manifest)
is the hosted transport under the WM and, standalone, an in-process link
parked on Cx (PendingServiceLinks) that whichever chat root is up adopts
into its registry — the overlay today, the superbuild's pane later. The
registry wakes the UI when it sends, so an in-process call is answered
without waiting for a pointer event. NoModel leaves the engine feature
so a build without a model runtime still answers honestly and keeps the
tool console; aichat's engine is a default-on feature.

The bridge: Cx::ai_callback beside tweak_callback, /ai?on=1|0, /ai?say=…
and /ai/transcript (the overlay publishes the transcript as JSON after
each state change), installed by makepad_aichat::script_mod.

sheets links aichat and exposes sheets.summary (name, used range, header
row, selection) through the port: standalone the overlay answers it,
under the WM the bus does.

The local model no longer loads on a registration: its session starts
on the first user line, so an app joining the bus costs nothing.

Verified by hidden grabs: sheets standalone /ai?on=1 → the overlay with
the summary card and no model; F10 closes it; under the WM the sheets
tile shows no overlay and F10 toggles the WM pane. widgets 135 + 4
(two widget_tree find_within tests fail before and after this change),
aichat 33, sheets 83 + 2, services 29; widgets and aichat check for
wasm32.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 14:24:41 +02:00
Admin
e2a262b05a asset-store: the portable core — embedded feature, page-store and CAS seams, a synchronous in-process API
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:14:27 +02:00
Admin
19cf377238 platform: the thread runtime contract — spawner, tasks, pools, scheduler, UI waker
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:14:15 +02:00
Admin
d76abf387e Merge branch 'work' into webdemos 2026-09-02 14:14:14 +02:00
Admin
ab8febc66d cargo-makepad: fonts packaged from the app's font manifest
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:14:14 +02:00
Admin
d1e7a6fea8 windows h264 decoder: AVLowLatencyMode is a VT_UI4; the round-trip test tolerates the MF encoder's access unit delimiters
The Microsoft H.264 decoder answers a VT_BOOL CODECAPI_AVLowLatencyMode with
"VT_UI4 != pValue->vt" (seen on the body node), so send the number. The
Media Foundation encoder opens each access unit with NAL 9, which the
keyframe check now skips.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 14:13:54 +02:00
Admin
05fd7e2dae Merge branch 'work' into webdemos 2026-09-02 14:12:33 +02:00
Admin
611c9ebd8c cargo-makepad: production packaging stays off fat LTO; script VM under LTO investigated
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:12:33 +02:00
Admin
2e48c5bfb7 asset-client: the static-store read mode and polled runtime
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:10:50 +02:00
Admin
acad401218 sim: an agent with no route holds and retries instead of walking into the wall
A partial route to the nearest reachable cell, a hold at its end, and
re-planning with backoff (15 to 240 ticks) replace the old head-
straight-at-it fallback; cell derivation prefilters entities per chunk.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 14:10:23 +02:00
Admin
dcf7d21060 windows h264 decoder: ICodecAPI low-latency, per-pump trace, access-unit dumps, and the stream tests run on Windows
Trace every ProcessOutput HRESULT/status, keep the first access units beside the
trace so a stream can be replayed offline, ask for CODECAPI_AVLowLatencyMode through
ICodecAPI as well as the MF_LOW_LATENCY attribute, and let the round-trip test plus a
captured-stream replay test run on a Windows box.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 14:08:57 +02:00
Admin
60ee97858c cargo-makepad: package artifacts carry a content hash so a re-upload is a new URL
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 14:03:59 +02:00
Admin
d6cc49ad22 windows mft: PROVIDES_SAMPLES is bit 0x100, not bit 0
The Microsoft H.264 decoder reports output stream flags 0x107 (whole samples,
single sample per buffer, fixed size); reading bit 0 as "the MFT allocates its
own samples" handed ProcessOutput no buffer and every call failed with
E_INVALIDARG — seen in the MAKEPAD_H264_DEBUG trace on the body node.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 14:00:58 +02:00
Admin
c1febc7985 windows h264 stream decoder: low-latency mode, output type before first ProcessOutput, real MF_E codes, trace file
The live decoder never produced a frame on the body node: MF_E_TRANSFORM_STREAM_CHANGE
was hand-derived as 0xC00D6D60, which is MF_E_TRANSFORM_TYPE_NOT_SET; the real stream
change (0xC00D6D61) was treated as a hard error, the output type was only negotiated
lazily, and without MF_LOW_LATENCY the decoder holds a reorder window a 2-3 frame live
pipeline never fills. Set MF_LOW_LATENCY on the transform, commit NV12 before the first
ProcessOutput, re-negotiate on TYPE_NOT_SET/STREAM_CHANGE/BUFFERTOOSMALL, drain on
NOTACCEPTING, and stamp packets with monotonic 100 ns timestamps. MAKEPAD_H264_DEBUG=<file>
traces packets, HRESULTs, negotiations and frames for headless services.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 13:55:28 +02:00
Admin
4081e48c35 Merge branch 'work' into webdemos 2026-09-02 13:54:36 +02:00
Admin
e5d37e0b0b platform: a web file picker and file drop that hand apps bytes
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 13:54:36 +02:00
Admin
2f5a9849c6 files: the file browser on the AI bus — four read-only tools through a correlated, cancellable runner
Hosted by the window manager, files opens one AiServicePort with a
manifest built from the same table the app's own panel reads its tool
specs from, so the two can never drift: list_dir, read_file, stat,
treemap_summary, all Read. The calls run on a worker of their own
(ai_service.rs): every job carries the engine's call id and a cancel
flag the folder walk checks on every entry, a queued job cancelled
before its turn never runs, treemap_summary reports a permille per
direct child, and results come back by id whatever the order. The old
panel's order-only runner stays beside it for the old chat, behind the
chat feature it belongs to.

The port opens at startup unless the process is a warm-pool standby,
which waits for Adopted so a dormant instance never shows as a running
Files; a context line (folder, view, selection) follows every change;
ChatOpen from the desktop closes the app's own panel; the jail resolves
every path exactly as before.

Verified by hidden grabs under the WM: /os.launch files → the pane's
apps row reads Desktop · Files; list_dir, a refused jail escape, and a
treemap summary answered as cards. 151 tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 13:45:13 +02:00
Admin
de937ac01c route: the web build is a one-to-one recompile — native UI, service seams behind it
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 13:43:19 +02:00
Admin
84b46c11a8 mbtile reader: file-backed readers are native-only so the map stack builds for wasm
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 13:42:30 +02:00
Admin
17437a2f53 Merge branch 'work' into webdemos 2026-09-02 13:35:43 +02:00
Admin
4d75e65b13 web server: hardening after security review — worker-only parsing, FD-relative static opens, strict framing, bounded work, panic recovery, Cloudflare-aware limits
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 13:34:33 +02:00
Admin
b0e34c2f57 wm: the AI pane and its bus — the aichat child seated in the slot, F10, the os service with the typed open
The chat is a child app (apps/aichat) seated in the WM's right-side pane
instead of a tile. The WM gives it the slot (shell/ai_pane.rs: 440 px
clamped to 40 % of the desk, easeOutQuint in/out, opaque card with a
2 px edge, one MpRunView that keeps ticking the child while hidden), the
bus (ai_bus.rs: up-frames stamped with the WM's own endpoint ids, every
registration remembered and replayed when the pane connects, down-frames
routed by endpoint, a synthetic Unregister when a client dies, ChatOpen
broadcast on every slide) and its own os service on that bus: list (text
+ JSON rows), launch/focus/close by registry id, and open through the
same typed OpenRequest a file browser's double-click takes.

Bare F10 toggles the pane before the keymap and before any tile
(decision 8); the pane owns the pointer inside its rect and the keyboard
while open, claimed after its run view's next draw; the child's Close
means hide; its death empties the slot; shutdown kills it. The pane's
child is invisible to everything the desk enumerates. The WM declares
its font set (International, with the emoji family it uses).

aichat: the composer takes the keyboard after its first draw and keeps
it across sends; the running card's bar is a pixel width.

Verified by hidden --remote grabs: F10 → the pane with the os
registration in the apps row; /os.list, /os.launch terminal (tile beside
the pane, keys stay on the chat), /os.focus; F10 again → pane gone.
142 tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 13:29:04 +02:00
Admin
1940f3da25 fonts: leave apps/wm untouched — its font-set declaration waits for the aichat lane to land
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 13:23:12 +02:00
Admin
6981e9052b fonts: manifest generated from the chains, app-level font assets, a symbol fallback, deprecated i18n aliases
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 13:22:42 +02:00
Admin
942476f68c vj: the whole app type-checks for wasm32 — portable hub discovery and typed-unavailable client facades
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 13:21:14 +02:00
Admin
f233756d22 game brief: races, rings, mocap and biomes are one call each
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 13:18:36 +02:00
Admin
89fe45355b platform: wheels and flight sticks are game inputs, with an output-report handle
IOHIDManager claims known wheels and joystick-class devices, parses
them by HID usage (steering, pedals, twist, throttle, hat, buttons) and
hands out an IOHIDDeviceSetReport handle so an app can drive force
feedback from userspace — no kext, no DriverKit. The studio wire learns
RemoteJoystick.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 13:18:36 +02:00
Admin
c3f631d4e3 livepipe: one reusable pipe from a camera to a fleet node and back
Source (webcam by device, a still for tests), one VideoToolbox encoder
session per run for the H.264 wire with raw as the fallback, the hub
realtime session on a node chosen by domain, typed events (status,
connected, aux, rate, ended) through a mailbox to any consumer. The
sandbox's mocap is the first; fabric is next.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 13:18:36 +02:00
Admin
730b5b8e08 video: the Windows H.264 stream decoder pulls output before it knows the format
The decode loop returned before ProcessOutput until the output type was
negotiated — but the MFT only reports MF_E_TRANSFORM_STREAM_CHANGE, the
event that starts negotiation, FROM ProcessOutput. A node fed Annex-B
access units never yielded a frame. Now the pump always pulls, a stream
change picks NV12, sets the type and refreshes size and stride, need-
more-input ends the pump, and rows are de-strided on the way out. The
decision is a pure function with a regression test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 13:18:36 +02:00
Admin
79e7526b1e sqlite: a page-store seam — the file backend as before, an in-memory backend, and open_memory / open_with
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 13:18:18 +02:00
Admin
a6d0338ad2 widgets: popup menu items run in the popup owner's script VM; app_main! releases its borrow after a trap
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 13:15:40 +02:00
Admin
d931f9d764 map: tile archive third pass — per-pass pruning, validated watcher metadata, shared blob bytes, timer watchdog, atomic cancellation
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 13:14:57 +02:00
Admin
52251b77c2 platform: a namespaced async key/value storage API on Cx — files natively, IndexedDB on the web
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 13:06:35 +02:00
Admin
e21db96d50 route: demo review fixes — route origin, rain lifecycle, hosted route validation, request context, provisioning
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 13:06:09 +02:00
Admin
81ef71c38c network: native backends honour the response body cap
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 12:52:06 +02:00
Admin
a7eae82e41 map: the tile build keeps time with the platform clock
std::time::Instant trapped on every wasm tile worker; the profilers and stage timers
now use Cx::monotonic_now, and the native watcher, writer and their tests are gated
to native targets.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 12:43:31 +02:00
Admin
38fe712728 Merge branch 'work' into webdemos 2026-09-02 12:41:06 +02:00
Admin
09b41a8dc9 fonts: FontSet and FontPolicy — one application choice, selected-only loading, a font-assets manifest
Latin (IBM Plex) is the web default, International (adds the CJK and emoji faces) the
desktop default; app_main! records the choice before widgets register, the theme only
registers the selected chains, a glyph miss never starts a load, and the resolved font
resources are emitted as the makepad.font-assets.v1 section for the packager.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 12:40:50 +02:00
Admin
e0c6e74bf2 aichat: the progress bar and system lines use the theme's highlight colour
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 12:37:33 +02:00
Admin
7b9ed2c3de aichat: the assistant as an app — the panel owns the engine, the bus client, settings with the local-only lock
apps/aichat is a lib and a bin. The panel widget owns the registry, the
model and the transcript, draws EngineState as user lines, markdown
answers, tool cards with progress and confirm buttons, and system lines
over a composer; every event pumps the engine and a busy turn asks for
the next frame. The bus client turns the window manager's stamped
studio frames into registry links under the WM's own endpoint ids
(registry.register_as). Settings persist under ~/.makepad/aichat with
the cloud lock kept in code. The binary is both the standalone window
and the WM's special child.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 12:33:32 +02:00
Admin
fde3f398d9 Merge branch 'work' into webdemos 2026-09-02 12:29:56 +02:00
Admin
2308736a2a ai-services: the real models behind the engine feature — local through the hub, Claude, and none
LocalModel drives the hub chat session; a changed tool table or system
prompt rides the next turn as an update block instead of restarting an
append-only context. ClaudeModel owns the history and rebuilds the
per-turn provider with the new native tool array, dots mapped to the
double underscore Anthropic allows. NoModel answers nothing but says so,
so the web demo's pane, services and tool console work without a
backend. Provider rows report honest availability, the local model file
policy is one function, and the crate no longer needs the converse seam.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 12:25:56 +02:00
Admin
14d0723872 cargo-makepad wasm: production packaging — strip, small profile LTO, optional binaryen, size report
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 12:24:41 +02:00
Admin
8ae0ffb951 ai-services: the engine core — registry, router and conversation, tested against a scripted model
The registry issues endpoints, answers each port's registration, keeps
where every instance lives, builds the tool table (each app once,
canonical names) and the briefs, pumps the links and forgets dead ones.
The core drives a Cx-free Model seam: routes each tool call to the
instance the model named (or the focused one), stripping the selector;
holds destructive calls for a confirm card, with host risk floors that a
service can raise but never lower; times out silent services, extended
by progress up to a hard cap; answers unknown names with the real ones;
reconfigures the model when the registry moves and restarts it when it
cannot rebind; honours result dispositions (end the turn, reset the
conversation); caps tool rounds; and offers a local tool console that
drives services without a model. Time is host-supplied seconds, never
Instant, so the same core runs on the web.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 12:23:07 +02:00
Admin
5cdcaedb8e Merge branch 'work' into webdemos 2026-09-02 12:20:14 +02:00
Admin
ba1010d7c3 asset-client: review fixes — additive web feature, shared transport contract, guarded cache and base URL
The web feature no longer hides the native API; both transport adapters validate
responses through one upper layer with framing headers refused and redirects off;
the platform transport passes a body cap to makepad-network (honoured by the web
backend); the filesystem cache verifies before deduplicating; BaseUrl canonicalizes
and rejects dot segments; ClientLocation is authoritative.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 12:19:48 +02:00
Admin
2acb7983ab ai-services: wire v2 — endpoints, receiver-side caps, result disposition
An app id says what a service is; a host-issued EndpointId says which
instance. Ports register with a nonce, learn their endpoint from the
host's Registered answer, take only frames addressed to it, and never
stamp themselves on the way up. Hosted frames are refused over a size
cap before deserialization and every variant is validated on receipt;
tool schemas must describe an argument object. Results carry bounded
structured data and a disposition (continue, end the turn, reset the
conversation); Cancelled and TimedOut are outcomes of their own.
InstanceMeta records where an instance lives for the model.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 12:14:52 +02:00
Admin
d292ec7fe4 Merge branch 'work' into webdemos 2026-09-02 12:04:26 +02:00
Admin
126d8c9168 route: the demo profile runs in the browser — merged panel draw, platform clock, unavailable backends tolerated
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 12:04:08 +02:00
Admin
cf4c84c22f keys: F10 is the assistant — the exploded-view debugger moves to Shift+F10, the screen recorder to Ctrl+F10
The platform's exploded z-layer viewer consumed F10 before any app saw
it. F10 now belongs to the AI chat (the WM pane and the Window overlay);
the debugger is Shift+F10 and the recorder Ctrl+F10, beside the
designer's F12. The recorder widget gains a hotkey_ctrl setting.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 12:01:37 +02:00
Admin
c55a315515 platform: monotonic clock beside the wall clock, trap-safe dispatch, studio worker only with a studio
Cx::monotonic_now backs every elapsed and deadline measurement (Instant natively,
performance.now on the web, reachable from workers); dispatch keeps its handler
installed across traps and rejects re-entry without panicking; the studio websocket
worker spawns only when a studio is configured and waits with park_timeout instead of
spinning.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 11:56:54 +02:00
Admin
9633ded6f4 web server: site static serving plus the first nav backends
tools/web_server becomes the deployable site server on our own HTTP stack: Range, HEAD,
brotli negotiation, ETags, cache and cross-origin headers, error reporting, and
/api/healthz, /api/search, /api/route and /api/along over map_nav with bounded query
and route workers and a data directory outside the public root. makepad-network learns
HEAD and exposes raw header lines.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 11:42:36 +02:00
Admin
0fd1ea2e92 route: demo profile — native/demo features, side-panel and provisioning seams, hosted tiles, HTTP nav client
The demo build has no AI panel, no first-run bake and no local files: tiles come from the
hosted .mkmap archive, and search, routing, along-route lookups, weather, radar and wind
go through the site's API with client-side trip state.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 11:41:14 +02:00
Admin
5d28ed551a map: tile archive review fixes — legacy MBTiles path restored, decode off the UI thread, retry, cancellation, blob dedup, validation
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 11:34:56 +02:00
Admin
4144dc414c files: the spy test filesystem implements the clock
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 11:31:46 +02:00
Admin
9ae91aca3c asset-store: static export contract — versioned static index and a sanitized deterministic export-static tool
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 11:28:31 +02:00
Admin
fe920c8988 files: review fixes — demo scan exclusions, Zipf sizes, trash root guarded, depth bound, tests
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 11:28:30 +02:00
Admin
47ac158a42 asset-client: portable primitives — completion transport, cache store, static-site config
A completion-based Transport with the native TCP adapter and a platform-HTTP adapter over a
client-owned network runtime, a CacheStore seam with the filesystem cache and a memory
LRU, BlobContent / ClientLocation / Unavailable types, and native/web features so the
client builds for wasm32; static-site connect is a typed Unavailable until the runtime lane.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 11:26:03 +02:00
Admin
68fb182bf4 Merge work into webdemos: the unified trace switch
The startup trace keeps its topic gate and runs on the platform clock so it stays
wasm-safe.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 11:25:03 +02:00
Admin
3c7566a565 Merge work into webdemos: the mp prefix rename
apps/files, apps/sheets, libs/wm_api, libs/wm_theme and the renamed env vars and script
namespaces meet the demo seams; added demo files move to the renamed paths; the files app
keeps its own makepad_home so demo builds without the chat feature do not link the AI hub.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 11:23:11 +02:00
Admin
3389475d33 trace: one switch — MAKEPAD_TRACE=<topics>, trace!(topic, …), and /trace on the bridge
The per-crate *_TRACE / *_DEBUG / *_DUMP environment switches in
platform, widgets and cef become topics of one process-wide set:
startup, frame, present, timer, studio, gpu.*, shader.*, gl.*,
runview.dpi, wayland, tweak, map.*, cef. Hierarchical matching with
negation and all; read once at init; settable on a running instance
through GET /trace?topics=…; an atomic short-circuit keeps the
no-tracing case off the lock in per-frame code; spans are absent on
wasm32 where Instant panics.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 11:23:08 +02:00
Admin
debd8c1075 rename: the mp prefix goes — apps/wm, files, terminal, browser, task, sheets, image, video, pdf; libs/wm_api and wm_theme
Packages are makepad-<name> with the short name as the binary. Env vars
follow (MAKEPAD_WM_*, MAKEPAD_FILES_*, MAKEPAD_TERMINAL_*), config moves
under ~/.makepad/<app>/, the theme namespaces are mod.wm_theme and
mod.browser_theme, the hosted AI envelope key is wm_ai. platform/video
becomes makepad-platform-video so the video app can be makepad-video.
Carries the Score entries that were pending in the WM's curated table.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 11:17:12 +02:00
Admin
a7af3ea99a platform + mpfiles: platform clock instead of std::time on the web, unwind-safe event dispatch
std::time is unsupported on wasm32-unknown-unknown; the demo paths now use Cx::time_now
and the pinned demo clock, and Cx::call_event_handler restores its handler slot on unwind
with an explicit re-entry diagnostic instead of cascading into a second panic.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 11:11:42 +02:00
Admin
0537687b96 mpfiles: distinct repo-owned pictures for the demo thumbnail pool
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 11:04:28 +02:00
Admin
3203e32f4a finance: review fixes — id remapping on persist, structural determinism tests, one cfg seam
Persisting the generated ledger remaps every id through an explicit map so category
parents and scheduled entries resolve in a fresh native database; determinism is proven
by a structural fingerprint plus reference-integrity and transfer-pair tests; native
persistence and import live behind one cfg-gated module interface with a runtime
pinned day (2026-08-28 in demo builds) used by all shared range code.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 10:58:42 +02:00
Admin
aa96dbd625 cargo-makepad wasm: package the bin target's wasm and create dirs before minifying
A package whose bin name differs from the package name (apps/finance: finance) no longer
fails at packaging; the bin is resolved from cargo metadata, default-run or --bin, and the
minified JS copies are written after their directories exist.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 10:53:12 +02:00
Admin
58e1f72d72 mpsheets: review fixes for the demo seam
Visibility wrappers around the picker and disk controls, initial document and demo
loading owned by SheetDocs, a clipboard seam instead of a platform cfg in the view,
loading a demo resets history, selection and scroll, ASCII help text, stronger
bundled-document tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 10:50:51 +02:00
Admin
ad24534b94 map: persistent asynchronous tile archive over a completion-based byte source
libs/mbtile_reader gains pure root/leaf/blob parsing; widgets/map gains archive.rs with
a ByteSource trait, a file backend on the worker pool and an HTTP-Range backend over
cx.http_request, a TileArchive that fetches the root once, caches leaf directories and
dedups in-flight reads, and MapView::set_source_config so local archives and hosted
archives share one request path into the existing tile build and ready-tile delivery.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 10:50:15 +02:00
Admin
13b4c485e5 mpfiles: demo build — procedural fake filesystem, still-image thumbnails, chat feature
The chat feature (default on) owns the AI hub dependency so a demo graph never names it;
demo builds install a seeded procedural DemoVfs of ~38k files with Zipf sizes and deep
paths, read text through Vfs::read_bytes, take thumbnails from an embedded picture pool
with one still per video, run listing/scan/thumbs inline, and keep prefs in memory.
Process spawning is confined to the native seam.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 10:48:41 +02:00
Admin
0ccd384a73 platform web: focus dispatch, window-zero geometry and generation-correct ids after the startup deferral
Review fixes: focus/lost-focus events dispatch unconditionally again, only window
zero's creation updates the shared browser geometry, the guarded sites resolve the
generation-correct id of physical slot zero, and four regression tests cover the
deferred-geometry state machine.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 10:47:35 +02:00
Admin
1e93309385 AGENTS.md: designs stay local; no OS screenshots; focus and hidden-window laws
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 10:36:02 +02:00
Admin
daff390fc6 finance: demo feature — generated ledger, SQLite target-gated, import hidden
seed::generate(years, today) is the pure deterministic generator; native first-run
persists it through the unchanged db path, demo builds keep the ledger in memory with a
pinned day so screenshots are stable. makepad-sqlite is a non-wasm dependency and the db
module only exists outside demo builds; the CSV import screen is absent in demo builds.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 10:33:55 +02:00
Admin
c17480faf1 render: a non-rigid body may carry its own orientation
A coaster car upside down at the apex of a loop is a body whose
quaternion is not the default; the renderer now honours it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 10:33:30 +02:00
Admin
5b309b311f vj: 32-bit usize constants no longer overflow on wasm32
The 4-16 GiB cache budgets are computed in u64 and saturate to usize::MAX on 32-bit
targets, same as the shared libs.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 10:31:59 +02:00
Admin
83d8eac9cc mpsheets: demo feature and the SheetDocs seam
A demo build carries three bundled formula sheets instead of disk CSV open/save; the
seam is one trait with a native fs impl and a bundled impl chosen at start, the
Save/path chrome hides behind can_save, and the wasm clipboard write is skipped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 10:23:47 +02:00
Admin
ed5b2fa509 web: wasm32 portability in libs and web startup geometry deferral
libs/ai/loader, libs/sqlite_query, libs/video_flow: 32-bit const overflows and unix-only
file locking/bulk reads gated so the app crates type-check for wasm32-unknown-unknown.
platform web: ToWasmInit arrived before the app's Window widget allocated window zero and
indexed an empty pool; the browser geometry is now stored and applied once the window
exists, and the other id_zero index sites are guarded.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
2026-09-02 10:23:47 +02:00
Admin
876e3c4e18 game brief: buildings have insides — ask for a program at the door
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 09:50:47 +02:00
Admin
d7f9813443 game brief: to change a map, edit the plan you were given; vegetation is a layer
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 09:16:57 +02:00
Admin
b94bc58a7f ai-services: the wire, the app port and the panel state — one conversation, many apps
An app exposes an AI service (manifest, tools with a risk class, a port
that receives calls and answers them later); a host aggregates every
connected service into one chat. Hosted by mpwm the port rides the
studio protocol's Custom frames under the mpwm_ai envelope; embedded, it
is a channel pair. The engine behind the feature flag and the panel that
draws EngineState land in the next lanes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 09:15:53 +02:00
Admin
adbb078b68 render: a water volume can be physics without a picture
The water shader blends premultiplied, so an alpha-zero sheet meant to
hide the river's buoyancy boxes became additive and painted bright
stepped rectangles over the ribbon and the meadow — every claim-level
invariant green while the screen was wrong. WaterVolume::draw_sheet
splits physical from visual: river volumes keep floating things and
never draw; declared pools still do.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 04:42:34 +02:00
Admin
e18cc694fc chat: a tool call the model wrapped inside its own think block still executes
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 04:42:34 +02:00
Admin
1ef6f59c3e game brief: on a diet and world.plan-first
33.7k bytes to 16.4k: the plan doctrine at the top, the laws, the one-
call list, the build order for a new game, how to read assists and
refusals; per-verb prose left to the verb docs. Back under the context
budget the crate's own test enforces.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 04:11:26 +02:00
Admin
c86a750258 chat: a game session reads its doctrine first and is not the generation assistant
The system prompt chose the generation persona and the <<tool>> JSON
guidance whenever image.generate was among the tools — which a game
session legitimately carries for missing art — and appended the game
doctrine 19k characters later. Against the exact live prompt the model
answered "I can build that for you." and stopped; the same request
with the doctrine first emitted the call. Agentic is now decided by the
presence of world.* tools: doctrine first, an in-game builder persona,
the model's trained tool template, and no duplicate capability text.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 03:37:57 +02:00
Admin
f2b02531b7 drumkit: a sample-based kit for the score preview; the physical model parked as drumkit_phys
The kit that plays drum scores is now a sample player over the Salamander Drumkit (velocity layers with round-robins, derived toms, a synthesised clap), loaded off the audio thread and swapped in atomically. The physically modelled kit stays as makepad-drumkit-phys for a later pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 02:52:12 +02:00
Admin
066c78292e piano model: the lower register is hammered, not plucked
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 02:52:12 +02:00
Admin
79882b5fb6 fabric: a photo or a live camera to a fitted sewing pattern
apps/fabric runs SAM 3D Body in-process (install + licence through the hub panel), measures the rest-pose body mesh (libs/fabric/measure: 25 tape measurements from plane slices and landmarks) and drafts parametric patterns (libs/fabric/draft: T-shirt, A-line skirt, easy trousers; nesting, true-scale SVG, tiled A4 PDF). Live mode streams the webcam at a few frames a second, shows the posed body with the measurement rings riding on it, and re-drafts when the numbers settle. Measurements sit in a DataGrid that copies as tab-separated text.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 02:51:45 +02:00
Admin
8ea3684250 widgets: EventOrder reachable from the DSL; DataGrid hit-tests where it was drawn
A View can now say event_order: EventOrder.Down in script. The DataGrid cached its geometry at draw time, before a Fill sibling could shift it, so clicks in a right-hand column landed outside; it re-anchors to its drawn rect on every event.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 02:51:45 +02:00
Admin
cb97fc569f vj: the loop splat — a song sliced into per-stem, beat-quantized loops on the APC40 grid
Eight song sections by drums, bass, vocals, other and mix; launch and swap on the bar, right-click halves and quarters, per-cell waveforms with score blocks, a full-width score popup (drums, bass tab, melody with lyrics) played back through the piano and the sample kit, an NMF drum transcriber, Basic Pitch for pitched stems and Beat This! grid refinement when installed, beat jump and phase-flip buttons, a loading overlay for the grid, and the shared model install panel in INSTALL MODELS.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 02:51:44 +02:00
Admin
120b7e23ae score view: the engraver as a shared library, with drum and pitched score builders
libs/score_view carries the engraving, spacing, font and document code out of the score app so any app can show notation: a ScoreView widget with width/page/content fits, pan and zoom, a playhead, a dark palette, drum-key labels, lyrics under melody notes, and builders for drum, pitched and bass-tab scores. The score app uses it. Vector glyphs no longer write depth for transparent fragments, and the Metal screenshot staging sizes itself from the source texture.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 02:51:44 +02:00
Admin
31e5faaff5 ai-hub: local model runner, licence acknowledgements, a shared install panel; Beat This!, Basic Pitch and the Salamander drum-kit entries
LocalModels runs registry models in-process (install state, resumable downloads, recorded licence acknowledgements at $MAKEPAD_HOME/license_acks.json, weight paths by file role) and libs/ai/hub_ui is the install panel + licence modal every app can embed. New native ports: Beat This! (beats + downbeats) and Basic Pitch (note transcription) with their registry entries; the Salamander Drumkit samples (CC BY-SA 3.0, 37 files pinned by size and sha256) as a sample bank the downloader fetches like a model.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 02:51:44 +02:00
Admin
f4af6df5a0 sim: terrain knows who changed it — a plan layer over player history
Chunks keep the player's bytes apart from the composed density: plan
presses land in an owner-keyed PlanTerrainPatch that is retracted with
its feature and never touches history, so removing a railway takes its
bed with it and a human fill under it survives. Surface queries answer
Surface | Hole | Outside instead of pretending the heightfield continues
over a hole or past the border; landforms issued inside a plan eval are
plan, outside it history. Seven tests pin the layering, including
same-inputs-twice equality.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 02:38:40 +02:00
Admin
c189b035af game brief: agents follow the rules by themselves — you never script a stop
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 02:29:06 +02:00
Kevin Boos
79f938c09d
Splash isolate host: keep inserted subtrees reachable, survive foreign-heap values, contain isolate panics (#1208)
* widget_tree: keep manually inserted children linked across a refresh

A container that owns a child outside its own child vec inserts it with
`insert_child_deep` and never reports it from `children()`, so a refresh
unlinked it from every top-down search and the removal pass then deleted
its subtree. Everything under it went quiet, and invisibly: an empty
`WidgetRef` is a silent no-op.

Mark such children `manual` and keep the live ones linked, clearing the
flag once the parent reports the child itself.

* script: survive a foreign-heap value in the GC mark walk

`GenVec`'s `Index` bounds-checks the raw Vec before the generation check,
and `len` never shrinks, so an index past the end belongs to a different
heap. Aborting the process over one takes the whole app down for a fault
confined to a single script heap.

Add bounds-checked accessors and use them in `mark_value_fields!`, so
marking skips and reports a foreign value; name the table where an index
does still panic.

* splash: contain isolate panics, and report the silent failures

A Splash isolate runs user script on the UI thread, and the VM swap in
`with_isolate_installed` was not panic-safe: a panic could leave the app
VM swapped out, with later script resolving against the wrong heap.
Restore through the unwind, and contain panics at each entry point (pump
arms, timers, host callbacks, isolate GC, body eval, hook calls).

Report what used to fail silently too: callback errors, a script->widget
call whose target is gone, and wrong-VM routing. Adds three headless
regression tests.
2026-09-02 02:26:52 +02:00
Admin
7ff875a33e ai-hub: keep a peer's in-flight beats/notes/local work out of the body commits
The last two hub commits staged whole files and carried uncommitted hunks
of another lane (beats-native, notes-native, the local runner, new
domains and license keys) that reference files not yet in the tree. This
restores those files to the body changes only; the other lane's edits
stay in its working tree.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 01:47:49 +02:00
Admin
8c568dfcb0 ai-hub: drop the SAM 3D Body reference worker backend
The native port covers everything the Python reference worker did (body,
hands, mask prompt, multi-person), so the subprocess backend, its fake
worker harness, the sam3dbody-ref registry entry and the
MAKEPAD_SAM3DBODY_* environment go. The packet validator moves to the
native backend.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 01:45:22 +02:00
Admin
a648cf8808 ai-hub: body session options — hands, detect, persons=N
The sam3dbody backend reads its options from the request's prompt string
(`hands`, `detect`, `persons=N`): `hands` runs the full mode and the
packet carries which hands were fused and their boxes; `detect` finds up
to N persons with SAM 3.1 (an optional native-segment role on the body
entry, the same artifact the segment entry pins) and runs one body pass
per person with its box and mask, so the packet's people array grows.
The body crate shares one body pass between the packet, mask and hands
entry points, and infer_full takes the mask prompt too.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 01:36:25 +02:00
Admin
62dff26092 ai-body: the mask prompt — a person's segmentation mask conditions the body pass
BodyModel::infer_masked takes a provided person box and its full-frame 0/1
mask: the mask is warped with the crop's own affine (bilinear, rounded back
to 0/1 like the reference's uint8 warp), encoded by the prompt encoder's
mask CNN (four stride-2 convolutions with channel LayerNorm and erf GELU,
then a 1x1 to 1280) and added to the backbone tokens before the ray
conditioning, replacing the folded no-mask term. Oracle parity on the
mask fixture: warp exact, every CNN stage within f32 accumulation noise,
conditioned context 8e-4, end to end kp3d 2.2 mm / kp2d 0.25 px. 41 tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 01:29:15 +02:00
Admin
c9e6d88bf1 ai-body: the hands pass — hand crops, the hand decoder, the hand-mode rig and the wrist fusion
BodyModel::infer_full runs the reference's full mode natively: the body
pass's hand boxes become two 512 crops (the left one cut from the mirrored
image, padding 0.9), each goes through the backbone, its own ray
conditioning and the hand decoder (the body decoder with the *_hand
tensors), the hand-mode rig pre-transform (local_to_world_wrist, wrist and
root offsets, non-hand parameters zeroed) and the hand camera head's
scale factor of 10; the left result is un-mirrored. The fusion gates each
hand (wrist angle, box size, keypoint spread, wrist distance), re-prompts
the body decoder with the trusted wrists and elbows as point prompts (the
decoder now takes N prompt tokens and a previous estimate), and writes the
fused wrist angles, hand parameters and hand scale/shape back before the
final rig pass. pose.rs gains the roma xyz (extrinsic) and XZY (intrinsic)
euler pairs; mhr.rs returns joint global rotations.

Oracle parity on both full-mode fixtures: every hand-decoder stage per
step (tokens 7e-3, heads 4e-4, rig params 1e-4), fusion reports as the
reference (one hand trusted on the standing photo, both on the crop), and
end to end kp3d 1.6 mm / kp2d 0.85 px. 36 tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 01:26:21 +02:00
Admin
0e94bbac42 game brief: rivers, highways, junctions and gates are one call
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 01:23:58 +02:00
Admin
525ba1c2f4 metal: a device-resident two-way decoder layer, and the body decoder rides it
One backend call per SAM-style decoder layer (PE norms, token
self-attention, token-to-image cross-attention, erf-GELU feed-forward,
final norm) inside one command buffer, with the layer's f32 weights cached
on the device under their content identity and pooled transients.
gpu_two_way_layer_resident in the common backend; CUDA declines and the
per-op path stays. Body decoder loop on the M3 Max: layers 100 -> 9 ms,
frame 405 -> 266 ms, oracle parity unchanged (kp3d 1.5 mm).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 00:25:59 +02:00
Admin
d006d0a1bc metal: resident f32 linears keep their weight on the device
gpu_linear_f32_resident promised a resident weight but the Metal tensor
backend re-uploaded it every call, which for the MHR rig's 664 MB pose
corrective matrix cost 50 ms a step. Metal tensors now carry a content
identity (fresh at creation and after every in-place write, so a cache
keyed by it can never serve a stale weight), and the resident linear caches
the device buffer under it. Body frame on the M3 Max: 705 -> 405 ms (rig
302 -> 20 ms, refine 41 -> 13 ms, heads 10 -> 6 ms).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 00:20:46 +02:00
Admin
d53c77d4a5 metal: a device-resident ViT stack, and the body backbone rides it
The Metal tensor backend is host-Vec based: every op copied its inputs up
and its result back, which put the DINOv3 backbone at 2.1 s a frame on an
M3 Max. try_vit_backbone_resident_f32 runs a whole pre-norm ViT stack
(LayerNorm, q/k/v, rotate-half rope from tables, flash attention, out,
LayerNorm, SwiGLU, down; residuals in place) inside one command buffer
against cached bf16 weights, with a new kernel_makepad_rope_half_tables_f32
kernel for the rope. The precompiled metallib now carries the bf16 GEMM
kernels the runtime source compile already enabled on bfloat devices.

The common backend exposes it as gpu_vit_backbone_resident (CUDA declines;
its per-op path is already resident) and the body backbone tries it first:
233 ms a frame on the M3 Max, same oracle parity (kp3d 1.5 mm).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 00:16:31 +02:00
Admin
a2aaa8f778 ai-body: the FP8 bias rides a column-broadcast add on the device
The FP8 backbone mode added each linear's bias by downloading, tiling and
re-uploading it per call, which cost more than the FP8 GEMM saved. A
gpu_add_cols_broadcast op (CUDA kernel; host loop on the Metal tensor
backend) adds a cols-wide bias to every row on the device, and each linear
keeps its bias resident after the first upload.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 00:05:03 +02:00
Admin
8964ba6313 ai-body: an FP8 backbone mode, off by default, measured against the oracle
BodyModel::set_backbone_fp8 quantises every backbone weight to E4M3 with
a per-tensor absmax scale and runs the tensor-core FP8 GEMM (bias
broadcast after). A backend without FP8 turns it off per layer on the
first refusal, so Metal keeps bf16. The oracle test reports its
accuracy and timing next to bf16; the default stays bf16 until the
numbers say otherwise.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:54:43 +02:00
Admin
a9ce596e07 ai-body: the crop warp runs across cores
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:52:13 +02:00
Admin
7598346ff5 ai-body: tensor-core GEMMs for the backbone, and the rig's correctives only where they count
The backbone's linears now take the cuBLASLt bias-epilogue / bf16 mm
paths (bf16 operands, f32 accumulation, bf16 output — the reference's
precision) with the f32-accumulating GEMM as the fallback. The rig's
pose correctives run on the final refinement step only by default: the
intermediate steps only feed keypoints back into the decoder, and the
oracle shows the difference is 0.4 mm (2.1 vs 1.7 mm against the
reference) for half the loop's rig time; `correctives_every_step` keeps
the exact mode.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:49:46 +02:00
Admin
4be6d19695 ai-body: the test modules import the grid constants they still use
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:43:53 +02:00
Admin
45b5b98612 ai-body: the crop size is a runtime knob, and the loop reports where its time goes
The backbone, conditioning and decoder now take any square crop whose
side is a multiple of 16 (512 is the trained size); BodyModel::set_crop_size
selects it and caches the dense positional grid per size. Measured on the
oracle image against the reference: 512 gives 1.7 mm mean keypoint error,
384 and 256 about 2 cm mean (6 to 7 cm worst joint), 192 falls apart —
the knob is a real accuracy trade, not free speed. The decoder loop
reports its split (layer chain, heads, rig+camera, refinement) so the
next optimisation is chosen on numbers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:33:26 +02:00
Admin
97e9572f42 speech: one STT/TTS API on every platform, through the ai-hub
Apps ask the hub for a recognizer or a voice and get one; where it runs is
the hub's decision. AiHub::start_stt / start_tts return poll-driven
sessions shaped like the chat session. The Auto ladder is Whisper/Kokoro in
this process (weights present, machine election), on the machine node over
loopback, on a LAN node, else the OS engine; SpeechReach::Local is the
"don't reach out" knob. Audio always comes back as PCM: the app owns the
device.

Three layers:
- makepad-ai-speech is the whole speech model family, engines only.
  libs/voice (Whisper + Silero VAD) folds in as the `whisper` and `vad`
  modules next to kokoro and indextts, each a cargo feature; the Apple
  bridges and the Speaker/VoiceTranscriber selection leave it.
- makepad-system-speech (new) is the OS speech services as blocking fns:
  Apple SpeechAnalyzer/AVSpeechSynthesizer via Swift, Windows.Media.Speech*
  on the vendored bindings, Android SpeechRecognizer/TextToSpeech through
  MakepadSpeech.java (API 26 floor), espeak-ng on Linux. It models the two
  STT shapes honestly: PCM in (Whisper, Apple) versus an engine that owns
  the microphone (Android, Windows), with capabilities the caller reads.
- the hub grows speech sessions, in-process Whisper/Kokoro workers with the
  residency election, a `whisper` wire backend (stt domain, registry entry
  pinned to ggerganov/whisper.cpp) so a Mac can serve a Quest, and a
  `language` field on the generate request.

Consumers: the Window voice input runs on an STT session and switches to
engine-mic mode when the recognizer owns the microphone; converse's
SpeechOutput is a lazily started TTS session plus a pump thread; route
drops its private speech copy for converse; vj's lyrics fallback and the
alignment bakes call the engines directly.

Verified here: speech-roundtrip through the real sessions (Apple voice in,
in-process Whisper on Metal out, 4.3% WER); system-speech-test TTS->STT
verbatim; hub/converse/system-speech unit tests; msvc, aarch64-android and
linux-gnu cross-checks; Java against android-34. Windows, Android and Linux
bridges are compile-checked only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:32:35 +02:00
Admin
0fd356dbf0 windows: the vendored bindings are generated from a checked-in filter
libs/windows/windows-rs/src/Windows/mod.rs was a hand-pruned snapshot with
no recipe behind it; adding a WinRT namespace meant hand-copying vtables.
Now tools/windows_bindgen (standalone, windows-bindgen 0.62.1) regenerates
it from filter.txt: package mode with the COM `_Impl` traits, Win32 imports
linked through windows_core, upstream's 800-column rustfmt (the repo-root
rustfmt.toml disables formatting and has to be overridden), the filter
closed over every dependency the generator would otherwise skip a member
for, the namespace tree folded into one file with every feature gate kept,
and the two spots where the published 0.62.1 generator predates the vendored
core 0.62.2 normalized (Error::from_thread, imp::array_proxy).

The filter is the old file's item list plus Windows.Globalization.Language,
Windows.Media.SpeechRecognition and Windows.Media.SpeechSynthesis for the
OS speech engines. The generated `deprecated` gate is declared as a feature.
Checked on x86_64-pc-windows-msvc: platform, video, network, mpterm,
system-speech.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:32:35 +02:00
Admin
346f31f8c4 ai-body: flash attention for the head-dim-64 blocks
The backbone (20 heads) and the decoder (8 heads) both have 64-wide
heads, which the stack's FA2 kernel covers: f16 operands with f32
softmax and accumulation, the reference's own precision class, instead
of the composite path that materialises the 1029 x 1029 scores per head.
The composite path stays as the fallback where a backend lacks the
kernel. Oracle parity unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:15:56 +02:00
Admin
b22259b542 ai-body: the context stays on the GPU; only the pose token leaves the loop
The ray-conditioning conv used to download the 1024 x 1280 embedding,
concatenate the 99 ray features on the host and upload the result; it is
now two resident linears over the two column blocks with the no-mask
term folded into the bias, plus an add (32 ms -> a few on the 4090).
The decoder loop downloaded the whole normalised token block every
layer for its one pose row; it now slices that row and fetches the
block once at the end (or per layer under a trace). Same numbers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:13:46 +02:00
Admin
d78411a63b ai-body: the per-step work moves to the GPU
On the 4090 the frame was 885 ms of which 840 were host scalar loops in
the decoder loop: the three 70-row refinement FFNs (about 240M
multiply-adds a step), the two heads, and the rig's 55k x 3000
corrective output layer plus its identity blendshape sum. Those are now
GPU-resident linears (heads.rs GpuStepHeads, MhrRig::prepare_gpu), the
skinning computes one transform per joint instead of one per influence,
and the results are identical (all 29 oracle tests unchanged). The rig
still runs entirely on the host when no GPU side was prepared.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:12:05 +02:00
Admin
6a1c16b2cb ai-body: third-party notices — what the port is implemented after, and what it is not
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:05:43 +02:00
Admin
9ff44e87a6 ai-hub: the body-native wiring, this time only the lane's hunks
The recut in a6341981d applied its patch against the wrong directory
and restored the six shared files to the previous tree without the
body-native hunks. This commit adds exactly those: the `body-native`
feature and optional dependency, the pinned `sam3dbody` registry entry
and its test, the backend arms and the module declaration. Working
tree untouched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:03:47 +02:00
Admin
a6341981d8 ai-hub: the body-native commit carried a peer's in-flight hub hunks; put them back where they were
66e5e2f11 committed the working tree of the shared hub files and with it
another lane's uncommitted edits (a new domain, request fields, a
backend arm, manifest lines). This commit restores those files to the
previous tree plus only the body-native hunks. The working tree is
untouched: the peer's edits stay on disk as their uncommitted work,
exactly as before.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:03:20 +02:00
Admin
66e5e2f117 ai-hub: SAM 3D Body runs natively — sam3dbody on the body domain, oracle-matched end to end
model.rs closes the loop: crop -> backbone -> ray-conditioned context ->
six decoder steps, each through the pose head, the rig, the camera and
the projection, then the packet the sandbox already reads (kp3d/kp2d in
camera axes, the 204 rig parameters, global rotation, camera translation,
joint positions). Against the reference on the oracle image, on Metal:
3D keypoints within 1.7 mm, 2D within 0.4 px, rig parameters, camera
and rotation within 2e-3. packet.rs writes the JSON by hand with the
reference worker's rounding and field order.

The hub gains the `body-native` feature (default on): registry entry
`sam3dbody` pinned to the Comfy-Org repack by revision, size and sha,
body_native_backend.rs beside the subprocess reference backend with the
same live_step contract, the `body` capability advertised when the
feature is compiled, and a stubbed test double for the CPU-only tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:01:49 +02:00
Admin
69d842c400 ai-body: the promptable pose decoder and its refinement loop, oracle-matched on Metal
decoder.rs builds the 145-row token set (pose, previous, prompt, two
hand-box rows, 70 keypoint rows, 70 3D-keypoint rows), runs the six
layers with the SAM-style repeated positional encoding (none on the
first layer's self-attention), cross-attends against the ray-conditioned
context, and after each step hands the normalised pose token to the
heads and updates the keypoint rows from the caller's feedback: the
2D-keypoint positional FFN and the bilinearly sampled context features
on the valid rows, the pelvis-centred 3D positional FFN on the rest.
heads.rs holds the host-side ReLU FFN heads, the refinement FFNs, the
hand-box MLP and the hand classifier. Against the oracle on Metal, every
layer's residual stream is within 6e-3, every step's pose head within
8e-4 and camera head within 2e-5; token assembly and the host heads
match to 1e-4.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 22:58:29 +02:00
Admin
9e343a8f20 ai-body: the DINOv3 ViT-H+/16 backbone, crop and ray conditioning; Metal gains rope-half and affine layer norm
dino.rs is the 32-block ViT-H+/16 on the gpu_* surface after the TRELLIS
conditioner: bf16-resident linears with f32 accumulation, layer scale
folded into the output projections, rotate-half rope, SwiGLU, the
model's own final norm. preprocess.rs is the crop (box -> 1.25 pad ->
3:4 -> square -> 512 bilinear warp, ImageNet normalisation), the CLIFF
condition vector and the patch rays; condition.rs is the dense
positional encoding and the ray-conditioned decoder context. Against the
reference oracle on Metal: backbone 1.1% mean relative (bf16 noise),
ray-conditioned context 8e-5, crop within one u8 rounding step, rays and
dense PE 1e-7. One reference detail the paper does not state: its shrink
of the ray field is an antialiased filter whose taps clip at the image
edge, so the two edge patches sample inside their block centre (9.03 and
501.97 rather than 7.5 and 503.5); block centres left 0.1 of error.

The Metal tensor backend was missing rope_half (it aliased the
interleaved layout) and layer_norm_mul_add; both now exist with the
CUDA contract, which is what lets this backbone run on Apple silicon.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 22:56:46 +02:00
Admin
8211ae6d85 ai-body: the MHR rig and the pose head's parameter decoding, oracle-exact
mhr.rs is the Momentum Human Rig on the CPU: blendshapes, the 889x249
parameter transform, parents-first similarity kinematics (x,y,z,w
quaternions, Rz Ry Rx eulers, 2^s scales), the sparse-then-dense pose
corrective MLP over joints 2.., linear blend skinning and the 308-row
keypoint regression. pose.rs decodes the 519-wide head output: 6d
rotations, the 23 ball / 58 hinge / 6 translation body layout, the
mixed-dof hand layout, scale and hand component bases, the camera
translation and the perspective projection. Against the reference
oracle: vertices within 1e-4 cm with correctives on, keypoints within
1e-6 m, rig parameters within 1e-7. Two conventions the spec could not
settle on paper are now settled by the fixture: the head's global
rotation triple arrives Z,Y,X-ordered, and the corrective features start
at joint 2 (750 wide). Fixture tests skip cleanly without the oracle
directory or the weights.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 22:51:19 +02:00
Admin
f51b5f3bbf ai-body: the crate for the native SAM 3D Body port, with its weights reader
makepad-ai-body joins the AI model workspace: the constants of the
architecture (DINOv3 ViT-H+/16 at 512, the 6-layer promptable decoder,
the 519-wide pose head, the 127-joint / 18439-vertex MHR rig) and the
single-file safetensors reader for the Comfy-Org repack, which fails
closed on a Meta checkpoint-style header and checks the shapes the port
is written for at load. The backbone, decoder and rig modules follow
in their own lanes against the spec under local/agent_state/sam3dbody.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 22:31:12 +02:00
Admin
294cb9ea47 video: the single-frame mp4 is written on every platform, not stubbed
encode_intra_frame_mp4 on Windows and Linux now goes through the
platform file encoder (Media Foundation sink writer, GStreamer): one
keyframe-only stream, one frame, a scratch file read back and removed.
It pays the container machinery the Apple session path avoids, but a
tile tape baked on any machine reads on any other, which is what the
image-tiles library sits on. A round-trip test encodes a frame and
decodes it through the platform decoder on whichever platform runs it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 22:24:40 +02:00
Admin
929f822ca5 video: the single-frame mp4 encode exists on every platform
encode_intra_frame_mp4 was macOS-only, and libs/image_tiles imports it
unconditionally, so image-tiles, its example and source-library did not
build for Windows (found by a Windows-target cargo check sweep of the
workspace; every other crate checks clean). Other platforms now get the
same signature answering with an error, the way the rest of this file
stubs what it cannot do yet.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 22:20:43 +02:00
Admin
b7fe32cf07 game brief: a destructible world, path-edited railways, and models that load now
The doctrine catches up with the engine: landforms and tunnels are one
call and digging works anywhere; railways and roads are edited by their
path points (crossings generate, styles switch, any model drives);
freshly modeled aliases are live immediately — never park a display
substitute.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 22:15:45 +02:00
Admin
5692fabbdf sim: the landform world proves itself — walker through the tunnel included
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 22:15:45 +02:00
Admin
3fcccf3b47 sim: the whole world is implicitly editable, and one seam says where the ground is
The voxel layer's volume gate is gone — any edit materializes chunks
anywhere on the map (volumes remain as styling), with the lazy-chunk
economics proven by test: zero storage until the first op. The composed
surface seam (GameWorld::surface_height_at: heightfield where untouched,
voxel surface where a chunk owns it) is what ground queries, senses and
the drape corridor read, so a mountain raised mid-game and a dug crater
are the same truth the vehicles drive on. landform.rs adds the macro
shapes (mountain/hill/ridge/valley/crater/plateau, fbm detail, slope
rock + snow recolor) and the tunnel op rides the capsule-filtered
materialization.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 22:15:38 +02:00
Admin
f7093bf82f map_nav: the search db's positioned read builds on Windows
searchdb imported std::os::unix::fs::FileExt for read_exact_at, which
does not exist on Windows (reported from a Windows checkout of work).
A small ReadExactAt trait now wraps unix pread and Windows seek_read,
both cursor-free, so the shared reader keeps serving lookups from
several threads without a lock. Checked on x86_64-pc-windows-msvc.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 22:13:13 +02:00
Admin
914471fa31 ai-hub: a feed session whose last socket left ends on its idle timeout; skin: parent, skinned centroid and a nodes-only rig for retargets
A realtime feed session parked in its wait-for-a-frame loop forever when
the client died without sending stop (a sandbox quit left job-2 live on
.123 for five minutes holding the GPU slot). The wait loop now returns
to the top of the session loop once no socket is left, where the idle
timeout counts a socketless session down. Test covers it.

SkinnedModel gains node_parent/node_count, joint_skinned_centroid (the
direction a leaf limb actually runs, from the flesh it skins) and
from_nodes (a mesh-less rig for hierarchy-maths tests) — what the
sandbox's webcam mocap retarget needs.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B626urtY1Xo4hQdLzvSK6F
2026-09-01 22:06:29 +02:00
Admin
895b9a71c3 particles: an emitter can ride a body's own frame
EmitterAnchor::EntityLocal carries an offset that yaws with the entity —
a locomotive's chimney keeps its smoke over the funnel through every
corner. The step lookup resolves (position, yaw) instead of position.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 20:51:32 +02:00
Admin
ea50c77970 chat_ui: the feed's session gets its profile brief back
Same orphaning as the sandbox: context::assemble had no caller post-P8,
so gen/vj sessions ran brief-less. The executor's capability doc leads
with the assembled profile layer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 20:36:55 +02:00
Admin
c18b7f4169 game brief: railways are one call too
The verbs existed with the right doc strings, but the model reads the
brief, not the verb table — and the brief's ONE CALL list never said
railway, so it hand-placed track pieces, horrendously. Now it says it,
with the two-line essential shape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 20:20:04 +02:00
Admin
b0380bac34 image-tiles: the picture-wall engine as a library — tape atlases, a baker CLI, and the TileGrid widget
The Source Library wall's pixel engine, extracted for anyone's pictures:
libs/image_tiles carries the NV12 slot pyramid, the 32x32 shard atlas
geometry and the one-HEVC-intra-frame-per-file tape codec (tape.rs), a
small SQLite index (db.rs), a priority decode pool (store.rs), and the
TileGrid widget — instanced tiles batched per atlas page, anchor-locked
wheel zoom with log-space glide, continuous per-shard LOD with
crossfades, full-resolution promotion under byte budgets with LRU
eviction, and uniform-only re-presents inside the pad of the last build.
image-tiles-bake downloads a manifest of URLs (fetch pool free to be
wide, encode pool hard-capped — concurrent VT session churn has panicked
the encoder's IOMMU) and bakes a library; examples/image_tiles views one.
No JPEGs on disk; decode and encode are the hardware's.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 20:19:56 +02:00
Admin
0d7185e0de asset chat: catalog SQL answers in-process, and the game brief stops teaching the y mistake
Post-P8 the game session still advertised assets.query/assets.schema but
the in-process executor answered both Unavailable — the model's main
alias-search tool was dead, and every "find me X" turn gave up on the
store. Both now execute over the server's bounded /v1/assets/query route
(schema via sqlite_master plus the usage notes). The game brief gains the
new-game-vs-edit rule, interior design law, generate-after-search, and
explicit y semantics so player_pos().y stops being pasted into
ground-relative fields.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 20:18:31 +02:00
Admin
237da90a36 render: the sprite lane hands the screen draw back the way it found it
One DrawSceneScreen serves both the in-world video screen and every
sprite billboard, and the sprite loop left its last pose, size and atlas
behind — so the next frame's "is there a screen" gate (zero screen_size
draws nothing) read the leftover and drew the whole spritemap as one
opaque quad: under the last-drawn unit in C&C, and hanging in the sky of
any level that draws no sprites at all. Snapshot the host's pos/size/
texture before the lane and restore after.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 19:54:29 +02:00
Admin
6251f7c615 ai-hub: body domain — live pose packets ride the realtime session
New `body` backend (sam3dbody-ref): a persistent length-prefixed-PNG /
JSON-lines worker subprocess seam with ready handshake, per-frame timeout,
bounded restarts. LiveFrameOut grows aux_json — structured per-frame JSON
sent to the client before the frame — and output_encoding "none" makes a
session pose-only (refused with loop_mode feedback, also on control flips,
which upgraded apply_control to Result). Worker code+model stay
box-provisioned via MAKEPAD_SAM3DBODY_WORKER; the repo carries only the
MIT seam. Codex lane + Fable review (ready handshake, spawn timeout).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0165w1ZL1f1TruX5u2qC7mSX
2026-09-01 19:05:34 +02:00
Admin
176433a56e tweaker: click-again climbs the pick — the container under the children is reachable
The deepest-wins pick could never land on a container its children fully
cover (the chat panel, any rounded-bg pane). Now clicking inside the
pinned widget again walks the pin up one ancestor per click, skipping
design-transparent views, zero-rect wrappers and ancestors whose area
misses the click (a splitter's grab bar); at the window the climb wraps
back to the deepest pick. The window itself is never borrowed — it is
mid-dispatch when the handler runs (tree lookups decide before any
widget borrow).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 18:18:30 +02:00
Admin
915ce7c4e2 asset-server: the embed module — fully-local mode becomes shared
The VJ's attach-or-host logic (health probe, beacon listen, the main
store's server.lock as proof-of-life; loopback-only Host with the library
publisher, rooted in the main library when one exists) moves into
makepad-app-asset-server as embed::resolve, steered by <PREFIX>_ASSET_
EMBED/ROOT/PORT. vj/local_store.rs becomes the thin VJ-flavoured wrapper.
First second consumer: the sandbox.
2026-09-01 18:01:21 +02:00
Admin
0131ba49ed mpwm: --test-action launch-<app> — put one registered app on the desk, deterministically
the existing startup test hook learns the launcher's verb, which is how a
hidden instance hosts a real child for framebuffer verification.
2026-09-01 17:43:47 +02:00
Admin
df265f7a55 vj: a short console draws smaller — height joins the scale chain
console_scale defended the width and rearranged the very short, but
between the fold thresholds (~820 points) and the height the console
actually needs (~1000) lay a dead zone where the lower region simply
clipped — an mpwm tile at 844 points lost its rail from MAP down and half
a grid row. TARGET_HEIGHT_POINTS closes it: console_dpi now takes the
window's physical height and a short console shrinks exactly as a narrow
one does, to the same MIN_SCALE floor, before the folds take over —
measured in the shrunk points. A wide, short window still prefers the
lists-beside arrangement (judged at native scale, so the chain stays a
pure function of the physical window). The whole rule chain threads the
height; tests cover the tile shape that exposed this and the new ladder.
2026-09-01 17:43:47 +02:00
Admin
d5a6d64d1b vj: LIGHTS and ARCHIVE are rail chips under MAP — any content chip is the way back to the grid
The tiny icon page-tabs are gone. The special pages join the lane rail as
first-class chips: when one is up no content lane reads selected, and
clicking VIDEO (or any lane) flips the lower region back to the grid — the
persisted ARCHIVE page can no longer read as an empty library.
2026-09-01 17:21:35 +02:00
Admin
1d97d63e2f rustc 1.98 sweep: script module glob imports nothing uses, and strlen declared as libc has it
1.98's sharper macro-use tracking flags the live-id macro globs five script
modules kept without using; windows-strings' strlen extern takes *const
c_char with a cast at the two call sites instead of tripping
suspicious_runtime_symbol_definitions (PCSTR is transparent over *const u8,
so the ABI never changed).
2026-09-01 17:12:15 +02:00
Admin
e4a2e75d66 ai-hub: backend imports and helpers follow their feature gates
the vision and ocr backends' ContentBackend machinery lives behind the
llm feature; the imports and the lane-refill helpers now do too, so a
client crate compiling the hub with a leaner feature set (vj, chat-ui)
builds without warnings. pipeline_submit_collect stays feature-free for
its tests and says so.
2026-09-01 17:09:26 +02:00
Admin
89fe23fb68 vj: the self-hosted store is the full host — the library publisher rides along, loopback only, no beacon
Attach still wins whenever discovery finds a server (health probe, UDP
beacon, main-store lock). When nothing answers, the VJ now boots the
standalone asset-server's Host instead of a bare AssetServer: catalog +
CAS plus the ai-content library publisher over local/ai_content_library,
so a self-hosted VJ sees the same rows asset-ui publishes — not an empty
seed store. The fallback root is the user's main library when one holds a
catalog (server.lock arbitrates the race; losing it means attach). Both
planes bind 127.0.0.1 and no beacon runs: a private host is not a thing
the network can discover. HostConfig grows a blob_refs knob so the VJ
keeps its reference-import policy; the store decision is logged at boot.
2026-09-01 17:09:26 +02:00
Admin
e6c84d9b6a zune: the crate-root macro_use attrs go — rustc 1.98 phases the inner form out
zune-core's was load-bearing (alloc macros, no_std) and becomes the outer
form on its extern crate; bmp, jpeg and qoi never used it — deleted.
2026-09-01 17:08:31 +02:00
Admin
46cd1735d0 chore: workspace members and remaining sources
Squashed from work:
- frametween: the VJ's in-betweener becomes a library — the whole mode set, one definition
- rtsmap: one seeded generator for tiled strategy maps
- workspace: register the score crates, archive_org and mp4_index
- map: bake a runnable Amsterdam test map from inside the route app
- platform: file dialogs on Android and iOS, and unbreak the Android build
- ai-hub: makepad-asset-ai becomes makepad-ai-hub at libs/ai/hub, the chat pane becomes makepad-chat-ui, the service bin
- strict-json: the dependency-free JSON module gets its own crate; asset-client re-exports it so nothing downstream move
- asset-creator: the pipeline library is born — specs, the deps gate, and the derived-state law (aicore §9)
- makepad_ai is deleted — every backend is a hub pipe, the agent seam lives with its consumers (aicore §14, decided 2026
2026-09-01 16:46:35 +02:00
Admin
0257d6c6dc engine libs: render and sim for a strategy round, the per-unit decision hook, the mp4 sample index, stitch and xatlas quieted, example refresh
Squashed from work:
- docs: button shader annotations — widgets/button.rs complete + splash demo buttons
- tweaker: the Shader tab shows the pinned widget's ANIMATOR STATES as little posed swatches under the well — one per tr
- widgets: the glass stops reading draw_pass.time, so glass apps idle again
- render, sim, platform: what a strategy round needs underneath
- draw, platform: overlays now composite above content that uses draw_depth
- asset: mp4 sample index for range-streaming, chat tools, import profiles
- platform: native file and save dialogs, in-house on all three desktops
- sim: a per-unit decision hook — one commandable unit can think for itself, before the kit steers
- makepad_ai is deleted — every backend is a hub pipe, the agent seam lives with its consumers (aicore §14, decided 2026
- libs: the zero-warning sweep — stitch casts say what they mean, xatlas keeps upstream's surface quietly
- zero-warning sweep, round two — the first full-workspace pass
- zero-warning sweep, round three — the model lanes and the deep examples
- zero-warning sweep, round four — the last stragglers
2026-09-01 16:46:35 +02:00
Admin
87474bd68d finance: a personal ledger, budgets and reports on the DataGrid
Squashed from work:
- finance: a personal ledger, budgets and reports on the DataGrid
- zero-warning sweep, round two — the first full-workspace pass
2026-09-01 16:46:35 +02:00
Admin
79ccb10e81 mp*: the treemap camera era — canon-space packing, 2.5D and orbit views, streaming walks, the qwen sidepanel; mpwm linux desktop integration; pdf_parse recovery
Squashed from work:
- pdf_parse: indirect /Length, and recover the glyph names a symbol font uses
- mpwm: make the Linux desktop integration usable (#1204)
- mpfiles: the size map goes all the way down, and remembers what it found
- mpfiles: the treemap walk streams every directory at every depth — no more opaque growing blocks over deep heavy subtr
- mpfiles: local qwen chat sidepanel over the treemap selection
- mpfiles: a camera over the treemap — scroll zooms at the cursor with real re-layout detail, drag pans, double-click fi
- mpfiles: a 2.5D treemap — the map extrudes by nesting depth under a 3d toolbar mode
- mpfiles: a real orbit camera over the block view — left-drag orbits, right-drag pans, click stays a click; the project
- mpfiles: the wheel glides, the filter stops paying per frame and per keystroke — measured weights cached against the t
- mpfiles: the 3d view wears no name tags — labels only on hover, the tooltip carries the numbers
- mpfiles: a camera gesture never re-flows the map — mid-motion the picture rides one rigid remap of the last layout, an
- mpfiles: a small zoom keeps the map still — the layout is only remade when the camera truly leaves it (a 1.3x detail b
- mpfiles: a camera reveal is not an arrival — cells the cull swings into view appear instantly at full strength, cells
- mpfiles: the treemap packs in canon space — the arrangement is zoom-invariant by construction; zoom only refines the t
- mpfiles: the map stops breathing — insets are fractions of map space, names reserve no room and float on scrims with a
- mpfiles: a camera settle is calm — zoom detail materialises in place, no arrival fade; the ceremony stays with filter
- mpfiles: camera motion animates nothing but the camera — flat labels freeze per relayout and ride the remap; a calm se
- mpfiles: a fast zoom-out never outruns the map — cull escape relayouts the same frame, the glide's destination is laid
- mpfiles: wheel zoom in the raised modes anchors on the tile top under the cursor, not the ground hiding behind it
- mpfiles: first consumer of the ai-hub — the ask panel rides AiHub::start_local_chat, its private engine is gone (aicor
- ai-hub: chats run the machine election — route to a serving holder, wait on a loading one, claim and publish when open
- zero-warning sweep, round two — the first full-workspace pass
- zero-warning sweep, round three — the model lanes and the deep examples

Co-authored-by: Alex <blackanger.z@gmail.com>
2026-09-01 16:46:34 +02:00
Admin
517308e676 vj: deck sync as a held lock, archive.org as a content source, frametween becomes a library, runs execute in the app
Squashed from work:
- vj decks: sync is a held lock — pinned master, per-pump rate servo
- vj: the deck explorer opens on music
- frametween: the VJ's in-betweener becomes a library — the whole mode set, one definition
- frametween: the gate says which tier moved the picture, from pixels
- frametween: BGRA words go in as they are, and a host may own the clock
- vj: archive.org as a content source
- platform: native file and save dialogs, in-house on all three desktops
- vj: rounder slider caps, and the cap body reads as a blob
- ai-hub: makepad-asset-ai becomes makepad-ai-hub at libs/ai/hub, the chat pane becomes makepad-chat-ui, the service bin
- vj: DREAM runs execute in the app — pipelines.rs becomes the run it used to watch (aicore §9 / F1)
- vj: plain generations execute in the app too — the store's job queue loses its last vj client (aicore §9 / F2)
- asset-creator: the runner — generate one thing and put it in the catalog, one implementation for every surface (aicore
- importer + asset-server host: the coordination era ends (aicore P7)
- client + chat dispatcher: the dead wire comes out (aicore P7/P8)
- zero-warning sweep, round five — vj and chat-ui
- zero-warning sweep, round six — three cascades
- zero-warning sweep, round seven — the last two
2026-09-01 16:46:34 +02:00
Admin
c460905e2a maps + route: the self-baking Amsterdam test map, reversible mkmap weave, the rtsmap generator, road faces, guarded first-run
Squashed from work:
- rtsmap: one seeded generator for tiled strategy maps
- map: bake a runnable Amsterdam test map from inside the route app
- route: the first-run test map starts itself, and its buttons are guarded
- map: the test map bakes its road faces too, from the same shared pass
- map_tiles: mkmap-extract — the weave is reversible, world-cells reconstructible from world.mkmap
- route: the local dispatcher rides the hub — its copied engine is deleted, limits preserved (aicore P1)
- route + converse: off makepad_ai — the Agent seam moves to converse, route's cloud dispatcher rides the hub's Claude p
- ai-hub: chats run the machine election — route to a serving holder, wait on a loading one, claim and publish when open
- libs: the zero-warning sweep — stitch casts say what they mean, xatlas keeps upstream's surface quietly
2026-09-01 16:46:33 +02:00
Admin
145d0b1fe2 score: the notation suite — engraving, layout, playback, midi and musicxml import, the physical piano model, and the score app
Squashed from work:
- score: a headless music engraving, playback and notation engine
- score: the notation app — pianist mode, editing, playback
- piano_model: it was a plucked string by construction, and 20 voicings
- score: one document you can pan, zoom and navigate
- piano_model: the body tap was a click, and the objective was rewarding noise
- score: add the sound panel and library modules
- piano_model: a second engine, and the attack that finally sounded right
- score: two instruments, reverb and brightness — and the rest of the panel gone
- score model: a note remembers how it was struck, and the score remembers the pedal
- score import: keep the velocities and the pedal the file was carrying
- score playback: play the performance, not a flattened copy of it
- score ui: the music list moves to the sidebar, and the view stops fighting itself
- score: the application ships its font and eight performances
- piano_model: a limiter that rides the music, so the knee stops shaping chords
- piano_model: the forte bell was the treble's dynamic slope, and the bass was dying at its own prompt rate
- piano_model: the bridge decides each partial's decay, and a fixed multiplier cannot say that
- piano_model: each partial gets its own two coupled modes, from the eigen algebra
- piano_model: a median that fell between the peaks made every bass partial a drain
- score-ai: LocalBroker — the seam's in-process implementation over the session engine (aicore P8)
- client + chat dispatcher: the dead wire comes out (aicore P7/P8)
- score_pdf: the score model grew a pedal map — the pdf importer initialises it
- libs: the zero-warning sweep — stitch casts say what they mean, xatlas keeps upstream's surface quietly
- score app: the shipped-piece test speaks the PERFORMANCES table
- zero-warning sweep, round three — the model lanes and the deep examples
2026-09-01 16:46:32 +02:00
Admin
e37c263b9c ai backbone: the hub era — makepad_ai deleted, every backend is a hub pipe; machine residency elections, job leases, ETA placement; the store only stores; creator pipelines run in the app (aicore)
Squashed from work:
- asset-ai: FastH3 4-step fast video backend; clip keyframes on the wire
- asset-ui: loop video chains — text→image→video that ends where it began
- h3: safetensors -> pruned-Q4_K GGUF quantizer for the 24GB DiT tiers
- h3_quant_gguf verify: row-error gates calibrated to the measured Q4_K floor
- asset-ai realtime: the feedback loop — the source anchors, the drifted frame inits
- asset-ai realtime: a feedback loop survives a resize and travels by default
- asset-ai realtime: the feedback loop frees itself from the feed handshake and pauses for its listener
- asset-ai realtime: the outbound encode leaves the loop's critical path
- asset-ai ocr: the ocr domain — Chandra 2 at page resolution, and the tower goes planner-owned
- llm slots: a lane can hold an image span — embedding prefill and a rope cursor of its own
- vision tower on CUDA: the encode leg gets its two missing kernels
- llm/ocr: one M-RoPE grid encoder for both image paths, and a livelock made an error
- vision tower on CUDA: the f16 GEMM keeps the precision it was throwing away
- live: a feed that moves box takes its trip with it — one seed image
- vision tower on CUDA: the tiled attention becomes bit-exact, and tensor cores go
- llm prefill on CUDA: the MMA attention kernel gets the tile a 4-to-1 model needs
- asset-ai ocr: the CUDA encode lane joins the integration — vision-parity sits beside run's three arms, and the kernels
- Merge branch 'ocr-perf-integration' into work
- asset-ai: the live anchor can follow the trip, and text leaves the 5090
- asset-ai: the camera moves the world, and the world starts still
- asset-import: the EA strategy classics, in the one 2D contract
- rtsmap: one seeded generator for tiled strategy maps
- asset-ui: one card for the strategy classics, with a pack dropdown
- asset-ai: music3 reference-audio path, ocr/h3 backends, registry
- asset: mp4 sample index for range-streaming, chat tools, import profiles
- cnc: tiberium is twelve growth frames, not twelve empty variants
- platform: native file and save dialogs, in-house on all three desktops
- chat: the scan holds out for a lane home
- chat: a full home queues you — take the free lane
- chat: the preload has a percentage, and the boundless cap stops showing
- llm cuda: the 32x2 attention tile — even GQA ratios stay on MMA
- sa3 gets a bake path: the sfx model's tables precomputed by a diffusion-side bin
- sqlite_query: anti-join regression test
- td import: HARV's second frame block is its harvesting cycle, not a turret
- asset-ui: sprite enhancement runs on the 32B dev DiT — distillation, not the prompt, was the ceiling
- ai-hub: makepad-asset-ai becomes makepad-ai-hub at libs/ai/hub, the chat pane becomes makepad-chat-ui, the service bin
- asset-ui: test health fixtures grow the realtime field they were born without
- ai-hub: one home at ~/.makepad — weights/ run/ cache/ logs/, the service cache migrates from ai_content by a single re
- ai-hub: subprocess workers die with the node — process groups everywhere, PDEATHSIG on linux, one KILL_ON_JOB_CLOSE Jo
- ai-hub: the hub object — AiHub::in_process, pipes vocabulary, and the local LLM engine generalized out of mpfiles (aic
- strict-json: the dependency-free JSON module gets its own crate; asset-client re-exports it so nothing downstream move
- ai-hub: the machine layer — node entries, the 0600 machine token, and the residency election that IS the lock (aicore
- ai-hub: MPHUB1 — the fabric beacon only dedicated nodes can send (aicore §4)
- ai-hub: job leases — work lives only while it is renewed (aicore §8)
- asset-creator: the pipeline library is born — specs, the deps gate, and the derived-state law (aicore §9)
- ai-hub: RAM residency facts — the CPU-side twin of residency.rs (aicore §3)
- ai-hub: ETA placement primitives — relative GPU throughput, the four-term estimate, and an observable breakdown (aicor
- ai-hub: leases go live on the wire — origin fields on submit, /job/<id>/keepalive, /bye, and the reaper that cancels w
- ai-hub: the chat providers move in — fleet qwen, openai, grok, claude/codex/grok CLIs, the responses driver, and the w
- asset-creator: the engine — one pipeline run against the hub, deps-gated, spliced, cancellable, resumable-by-construct
- ai-hub: the machine node mode — --machine binds loopback, registers in ~/.makepad/run, and exits on its own once idle
- asset-creator: makepad-creator-run — the detached client for runs that must outlive a window (aicore §9)
- ai-hub: a native Claude Messages-API provider — API-key or Claude Code OAuth, bounded SSE streaming, injected tools (a
- route + converse: off makepad_ai — the Agent seam moves to converse, route's cloud dispatcher rides the hub's Claude p
- asset-creator: the preset tables move in — fifteen chain-policy constants shared by every creator app (aicore §9 / P6)
- makepad_ai is deleted — every backend is a hub pipe, the agent seam lives with its consumers (aicore §14, decided 2026
- ai-hub: loads hold the machine residency election — set_model_state claims on Loaded and publishes the service port (a
- ai-hub: chats run the machine election — route to a serving holder, wait on a loading one, claim and publish when open
- ai-hub: pick_for_domain_eta — ETA-ranked placement over the shared hard-filter core (aicore §6 / P4)
- asset-creator: the engine picks a provider per stage at dispatch time — a chain's later stages see fresh fleet state (
- ai-hub: the fabric secret gates the service HTTP surface — bearer on everything but /health and the ticketed peer path
- vj: DREAM runs execute in the app — pipelines.rs becomes the run it used to watch (aicore §9 / F1)
- asset-creator: the runner — generate one thing and put it in the catalog, one implementation for every surface (aicore
- chat-ui: the session runs in the app — no broker anywhere on the chat path (aicore P8 / F5)
- asset-store: assets.query is a first-class query endpoint — the bounded SQL surface outlives the broker (aicore P8 / F
- asset-creator: CreatorTools — the chat tool pack for a store that only stores (aicore §9 / P8)
- asset-store: the shrink — the store stores (aicore P7)
- importer + asset-server host: the coordination era ends (aicore P7)
- store config purge + asset-ui goes fleet-direct; the derive protocol gets its route proof (aicore P7)
- client + chat dispatcher: the dead wire comes out (aicore P7/P8)
- ai-hub: 0.3.0 — the health version says which era a node runs
- ai-hub: the default fleet is 'gen' — apps hear the LAN without env plumbing
- ai-hub: the preload note percents the prefill, not the job bar
- ai-hub: conversations keep their KV — the wire mirror, the lane identity, the in-turn dynamic context (aicore §7)
- ai-hub: an open-think model is thinking from its first token
- libs: the zero-warning sweep — stitch casts say what they mean, xatlas keeps upstream's surface quietly
- zero-warning sweep, round two — the first full-workspace pass
- zero-warning sweep, round three — the model lanes and the deep examples
- zero-warning sweep, round four — the last stragglers
- zero-warning sweep, round five — vj and chat-ui
- zero-warning sweep, round six — three cascades

Co-authored-by: Claude <info@makepad.nl>
2026-09-01 16:46:31 +02:00
Admin
dbb82b8b61 chore: the zero-warning sweeps — every target of every workspace crate compiles clean on macos, windows, linux, ios and android
Squashed from work:
- platform, draw, widgets: the cross-target zero-warning sweep
- zero-warning sweep, round two — the first full-workspace pass
2026-09-01 16:46:29 +02:00
Admin
bada23dda2 frame witnesses: presents, uploads, encode and gpu time, a pinned display link, SHIFT+F12 screen capture, MPINPUT latency
Squashed from work:
- every window can record itself: SHIFT+F12 writes picture and sound to local/screencap
- the frame has witnesses now: presents, uploads, encode time, gpu time, and a pinned display link
- the upload counter names its kind: instances and textures split, and any single item over half a megabyte logs itself
- MPINPUT: input-to-glass latency in the present pulse
2026-09-01 16:46:29 +02:00
Admin
7e6e87fa1a platform: native file and save dialogs, in-house on all three desktops plus Android and iOS
Squashed from work:
- platform: native file and save dialogs, in-house on all three desktops
- platform: file dialogs on Android and iOS, and unbreak the Android build
2026-09-01 16:46:28 +02:00
Admin
db3864b0bd map rendering: overlays and labels, per-view clip, the world-clamped centre, carto's road ladder, the @cam gate
Squashed from work:
- map: overlay and label rendering
- map: bake a runnable Amsterdam test map from inside the route app
- map: gate the @cam readout; a .mkmap archive is a directory
- map: the view's own draw list carries its clip
- the map's centre is clamped to the world, not wrapped past it
- a country is drawn in hairlines: carto's road ladder joins by zoom
2026-09-01 16:46:28 +02:00
Admin
3b0b16e8ee draw: overlays composite above draw_depth content, per-session DrawVector geometry, one uniform into a whole draw list
Squashed from work:
- draw: DrawVector reused one geometry slot for every session in a frame
- draw, platform: overlays now composite above content that uses draw_depth
- DrawVars::set_uniform_on_draw_list — one uniform into every retained call of a shader in a list, pass repainted
2026-09-01 16:46:28 +02:00
Admin
149d19fcbb platform: midi pitch-bend byte order, many http connects at once, what a strategy round needs underneath
Squashed from work:
- render, sim, platform: what a strategy round needs underneath
- platform: midi pitch bend was sent with its bytes the wrong way round
- http: the connect slot is a gate, not a turnstile — many connects at once
2026-09-01 16:46:27 +02:00
Admin
e0530b061b glass + widgets: realtime parent blur, glass button icons, idling without draw_pass.time, fold_header opens honestly
Squashed from work:
- glass: a glass button can carry an icon
- widgets: the fab palette cell's colour is a field, not an object
- widgets: the glass stops reading draw_pass.time, so glass apps idle again
- live-with-parent passes: the glass blurs the world in realtime again
- fold_header: visually open is open
- fold_header: the ease snaps to its ends
- fold_header: settled state is the truth, the pane edge is the law, the area is the fold
2026-09-01 16:46:27 +02:00
Admin
b408131172 video: hardware first-frame decode from RAM, one encoder-transform report, stills without an AVAssetWriter
Squashed from work:
- video: hardware first-frame decode straight from RAM — no temp files
- video: the encoder transform is reported once, not once per encoder
- video: a single still does not need a whole AVAssetWriter
2026-09-01 16:46:27 +02:00
Admin
b3dd79978b tweaker + shader const-table: the design-feedback campaign — annotated props, hot-patchable shader constants, animator-state wells, the theme tab, typed editors, undo through anonymous paths
Squashed from work:
- widget_tree: skip-search nodes bound path-cache invalidation
- tweaker: fold the shader source view; plain TextInput, not CodeView
- docs: button shader annotations — widgets/button.rs complete + splash demo buttons
- tweaker: the Shader tab shows the pinned widget's ANIMATOR STATES as little posed swatches under the well — one per tr
- tweaker: the animator-state swatches are full-size wells stacked vertically under the main material well (same width,
- tweaker: typed editors for structured values — a reflected Vec2/3/4 is fused x/y/z/w scrub fields (the whole vector re
- shader: const-table mode — /** name min..max step s */ float literals inside fn bodies compile to hot-patchable scope-
- tweaker: the Props tab opens with TWEAKABLES — every annotated value (a /** */ doc on the key) hot-first, each with it
- tweaker: the sidebar row templates are hoisted into shared let bindings (one source of truth for the Props list, the S
- docs: shader-code constants annotated in the six core widgets
- tweaker: SHADER CONSTANTS — the Props tab opens with the annotated literals inside the pinned widget's draw-layer fn b
- tweaker: undo/redo resolve the pinned widget when a history step's path runs through anonymous segments (a '-' or a li
- shader const table: whole-number literal operands lift too — the parser packs `x + 6.`, `y - 2. - b`, `w * 4.`, `n - 1
- tweaker: the Shader tab reads well stack → SHADER CONSTANTS for the mirrored layer → INPUTS (its uniforms/instances, a
- tweaker: the panel cleaned up to the user's list — the filter (with the exploded-view and note buttons) is the top row
- tweaker: three panel polish nits — a section whose rows are all secondary leads with its first three instead of an emp
- tweaker: the VJ session's fix batch — (1) mirrored material wells clip to their scroll viewport: the well is its own d
- tweaker: theme colours, chunk 1 — the panel reads the app's palette from mod.theme's cascade once per session (every c
- tweaker: the theme hover pulse — hovering a colour chip pulses that theme colour live across the whole app, and grabs
- tweaker: the theme palette strip in the colour popover — hover names and pulses a theme colour, a click binds the prop
- tweaker: the Theme tab — every theme colour and number in a fourth panel tab, colours edited live app-wide, all of it
- tweaker: F12 works without the bridge
- tweaker: the selection wears viewfinder corners, not a box
- tweaker: radius handles come to the hand, not the eye
2026-09-01 16:46:26 +02:00
Kevin Boos
0e19a65eac
x11/opengl/wayland/metal: implement what these backends silently skipped (#1201)
* windows: publish shader-cache entries atomically, and unwind a cancelled resize

Two ways a second instance, or an interrupted drag, leaves a window permanently
degraded. Both were found auditing the backend rather than reported, and both
are cheap.

The DXBC cache is a plain per-user directory that every makepad process on the
machine reads and writes, and entries were written straight to their final path.
`fs::write` truncates first, so a second instance starting at the same moment
could read the prefix of a shader the first was still writing -- the file exists
and the read succeeds, so nothing notices until `CreateVertexShader` is handed a
truncated blob. `shader_bytes_cached` even documented the read path as catching
this, which it did not. Entries are now written to a temporary file and renamed
into place, which is atomic, and a blob that is not a complete DXBC container
(the 32-byte header and its magic) is recompiled instead of used. The temporary
name carries the process id so two writers cannot collide on it either.

`is_in_resize` and the 8 ms resize timer are armed by WM_ENTERSIZEMOVE and were
disarmed only by WM_EXITSIZEMOVE. Win32 does not guarantee that pairing, and a
drag that ends without it leaves the timer forcing repaints forever and the
window presenting unpaced for the rest of its life -- it never returns to vsync.
WM_CANCELMODE now unwinds the loop, gated on having actually been in it, since
that message also arrives for menus and capture changes and unwinding a resize
that was not running would cost a needless ResizeBuffers each time.

Verified against a release build: two instances launched simultaneously on a
cold cache both come up correctly with no panics and no temporary files left
behind, and window resizing is unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* d3d11: validate a cached shader against the size its own header declares

Checking the magic and a 32-byte minimum accepts a blob that was truncated
after its header, which is exactly the shape a half-written cache entry takes.
The DXBC header carries the container's own total size at offset 24, so
comparing it to the length rejects those too.

Publishing entries atomically stops this backend from writing a partial one,
but the cache is a plain shared directory that older builds have already
written to, so the read path still has to be able to tell.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* windows: stop double-scaling a popup window's position

`CxOsOp::CreatePopupWindow` arrives with a position the caller has already put
in physical screen pixels -- it adds the parent window's physical origin to an
offset it scaled by the parent's per-monitor DPI -- and `new_popup` then
multiplied it by the DPI again. The popup landed at `dpi` times its intended
screen coordinates: exact at 100%, and progressively further away above it.

The size was scaled by the *system* DPI, which is the primary display's and
goes stale after a live scale change, so on a second display of a different
scale it was the wrong number twice over. It is now passed through unscaled,
because it is provisional either way: `init` runs `set_inner_size` immediately
afterwards, which scales by the window's own per-monitor DPI once the HWND
exists on its target display.

Note for reviewers: this is unreachable today. `WindowHandle::new_popup` is the
only producer of the op and nothing in the tree calls it, so there is no symptom
to reproduce -- which is also why the arithmetic was free to drift.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* x11: implement window resizing, and stop a protocol error killing the app

Two things the X11 backend did not do, both of which it silently pretended to.

`set_inner_size` and `set_outer_size` were empty function bodies, so
`CxOsOp::ResizeWindow` dispatched to nothing: an app calling `WindowHandle`'s
resize on X11 got no error, no log and no resize. They now call
`XResizeWindow`, clamped to the CARD16 range the protocol encodes an extent in,
since a zero is a BadValue. `set_outer_size` delegates rather than pretending:
the window manager owns the decoration frame, so a client can only ask for its
own extent.

Xlib's default error handler prints to stderr and calls `exit(1)`, and makepad
installed none, so a single rejected request killed the process outright -- a
bad geometry, a race against a window the WM has already destroyed, a missing
extension. Protocol errors are asynchronous and frequently not caused by the
code that happens to be running, so terminating is never the proportionate
response. A handler is now installed before the first request and logs the
error, request and minor codes plus the resource id.

`XResizeWindow` and `XSetErrorHandler` were not in the hand-written Xlib
bindings; both are added, along with the `XErrorHandler` callback type.

Untested: the author has no X11 machine. Both changes are small, local and fail
closed -- an unusable size is clamped rather than sent, and the handler only
turns an existing hard exit into a log line -- but neither has been run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* opengl/wayland/metal: name three failures the backends currently swallow

None of these is device-loss recovery -- that stays a D3D11-only feature. They
are the cases where a backend already fails and says nothing useful, so a bug
report arrives as "the window went black" with no cause attached.

EGL: `eglMakeCurrent` and `eglSwapBuffers` failures were logged unlatched on
paths that run every frame, so a persistent failure emitted thousands of lines a
second. Both are latched once per outage and cleared on the next success, and
`EGL_CONTEXT_LOST` (0x300E) is now named explicitly, since that is EGL saying
the GPU reset and every GL object is dead -- which this backend cannot yet
recover from, and should at least say so. `OpenglCx::make_current` returned `()`
while discarding the result of a call that can fail; it returns `bool` now, so a
caller can skip GL work that would otherwise run with no current context and be
silently dropped.

Wayland: `wp_viewport.set_destination` raises the `bad_value` protocol error --
which disconnects the client -- on a zero or negative extent, and a float-to-int
cast turns both a negative and a NaN into zero. `WaylandWindow::new` clamps the
size for the EGL call but stores it unclamped into the geometry, so an app
created at a degenerate size reached that request. The two destination writes
are floored the way the EGL extent already is.

Wayland `CxOsOp::ResizeWindow` was an empty arm. A client has no "set my size"
request, but window geometry defaults to whatever the surface commits, and the
paint path derives the EGL extent and viewport destination from
`window_geom.inner_size` -- so writing it is the whole operation. Refused for a
maximized or fullscreen toplevel, which must keep the configured geometry or the
compositor raises `invalid_surface_state`. `RepositionWindow` stays a no-op and
now says why: `move` is interactive and serial-gated, and `reposition` is an
xdg_popup request needing a protocol version this backend does not bind.

Metal: a command buffer that ends in `MTLCommandBufferStatusError` produced no
pixels, and nothing noticed -- the completion handler runs either way, so the
in-flight queue drains and the hang watchdog stays quiet over a stale window.
The status is now checked in the handler that already exists and the error named,
capped at 32 reports.

Untested: none of these three backends can be run here. All three are
type-checked for their targets. Deliberately NOT written: anything for Vulkan
(vulkan.rs is not compiled by any target available here, so it cannot even be
type-checked) or the web backend (no JS runtime available to syntax-check the
shim), and any Metal device-loss latch or recovery, because `MTLCreateSystemDefaultDevice`
returns the non-removable SoC GPU and loss in the D3D11 sense does not occur there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* x11/wayland: cap a window extent, and refuse a resize that says nothing

Running the branch on Linux turned up four things it and #1197 let through.

A saved size is floored but never capped, and Wayland is the one backend with
no displays to fit against, so `{"inner_size":[1e9,1e9]}` reached
`eglCreateWindowSurface` and tripped its `assert!`. The app then died before it
could rewrite the state file that was killing it, so every later launch died the
same way -- the exact failure #1197 exists to prevent, one line further down.
`sanitize_window_geom` now caps as well as floors, and both Wayland EGL surface
creations fall back to the default size rather than taking the process with them.

`CxOsOp::ResizeWindow` had the same hole and no cap at all on Wayland:
`resize(100000, 100000)` went straight into the window geometry, which left the
surface EGL_BAD_SURFACE and corrupted the size the toplevel restores to -- a
maximize/restore round trip came back 1259x1259, a fullscreen one 100000x100000.
X11 had the opposite problem: it clamped a zero or a negative up to the CARD16
floor and produced a one-pixel window, silently, where Wayland refused the same
request and said why. Both now go through one `sanitize_resize`, so they answer
a bad request identically.

Last, the new X11 error handler is an `extern "C"` frame and logging panics on a
closed stdout -- `app | head` is enough -- so a protocol error aborted the
process instead of the `exit(1)` the handler exists to prevent. Reporting is
wrapped in `catch_unwind`.

Verified on Ubuntu 25.10 / GNOME on both backends: the 1e9 file now starts and
repairs itself, the resize matrix caps at 16384 with no EGL error and restores
correctly, and the two backends log the same refusal. 76 unit tests.

Still open, deliberately not fixed here: a Wayland self-resize dispatches no
WindowGeomChange. handle_platform_ops holds the Cx borrow for its whole loop, so
sending one needs a deferred-event path that does not exist yet.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 21:50:37 +02:00
Kevin Boos
fd6f307a2b
windows: publish shader-cache entries atomically, and unwind a cancelled resize (#1200)
* windows: publish shader-cache entries atomically, and unwind a cancelled resize

Two ways a second instance, or an interrupted drag, leaves a window permanently
degraded. Both were found auditing the backend rather than reported, and both
are cheap.

The DXBC cache is a plain per-user directory that every makepad process on the
machine reads and writes, and entries were written straight to their final path.
`fs::write` truncates first, so a second instance starting at the same moment
could read the prefix of a shader the first was still writing -- the file exists
and the read succeeds, so nothing notices until `CreateVertexShader` is handed a
truncated blob. `shader_bytes_cached` even documented the read path as catching
this, which it did not. Entries are now written to a temporary file and renamed
into place, which is atomic, and a blob that is not a complete DXBC container
(the 32-byte header and its magic) is recompiled instead of used. The temporary
name carries the process id so two writers cannot collide on it either.

`is_in_resize` and the 8 ms resize timer are armed by WM_ENTERSIZEMOVE and were
disarmed only by WM_EXITSIZEMOVE. Win32 does not guarantee that pairing, and a
drag that ends without it leaves the timer forcing repaints forever and the
window presenting unpaced for the rest of its life -- it never returns to vsync.
WM_CANCELMODE now unwinds the loop, gated on having actually been in it, since
that message also arrives for menus and capture changes and unwinding a resize
that was not running would cost a needless ResizeBuffers each time.

Verified against a release build: two instances launched simultaneously on a
cold cache both come up correctly with no panics and no temporary files left
behind, and window resizing is unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* d3d11: validate a cached shader against the size its own header declares

Checking the magic and a 32-byte minimum accepts a blob that was truncated
after its header, which is exactly the shape a half-written cache entry takes.
The DXBC header carries the container's own total size at offset 24, so
comparing it to the length rejects those too.

Publishing entries atomically stops this backend from writing a partial one,
but the cache is a plain shared directory that older builds have already
written to, so the read path still has to be able to tell.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* windows: stop double-scaling a popup window's position

`CxOsOp::CreatePopupWindow` arrives with a position the caller has already put
in physical screen pixels -- it adds the parent window's physical origin to an
offset it scaled by the parent's per-monitor DPI -- and `new_popup` then
multiplied it by the DPI again. The popup landed at `dpi` times its intended
screen coordinates: exact at 100%, and progressively further away above it.

The size was scaled by the *system* DPI, which is the primary display's and
goes stale after a live scale change, so on a second display of a different
scale it was the wrong number twice over. It is now passed through unscaled,
because it is provisional either way: `init` runs `set_inner_size` immediately
afterwards, which scales by the window's own per-monitor DPI once the HWND
exists on its target display.

Note for reviewers: this is unreachable today. `WindowHandle::new_popup` is the
only producer of the op and nothing in the tree calls it, so there is no symptom
to reproduce -- which is also why the arithmetic was free to drift.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 21:50:26 +02:00
Admin
fd0ea7e264 chore: workspace members and remaining sources
Squashed from work; the fine-grained history is under tag archive/work-2026-08-29:
- mp* wave: mpwm window manager + the mp app family, WM API, theme bridge, PDF engine fix
- Modal claims no layout slot: the DJ page fills its window again
2026-08-29 09:26:27 +02:00
Admin
4ddba62d4b engine libs: csg document, stitch simd, sim providers, teamtalk LAN voice, gltf writer, render, route and maps, bench hygiene
Squashed from work; the fine-grained history is under tag archive/work-2026-08-29:
- mp* wave: mpwm window manager + the mp app family, WM API, theme bridge, PDF engine fix
- mpwm polish wave: terminal key focus, focus-history close order, pop-back-to-origin, occupied-workspace cycling, demo
- work: land the sources the last commits reference
- route: the assistant icon the committed UI references
- fast_inflate: the benches name their dev-deps
- gif/weezl: drop the vendored benches nobody can run
- weezl: the decode tests generate their own LZW fixture
2026-08-29 09:26:27 +02:00
Admin
d25cce9ae6 mp*: the window-manager app family — quick-look, warm-instance pool, pty teardown, theme bridge
Squashed from work; the fine-grained history is under tag archive/work-2026-08-29:
- mp* wave: mpwm window manager + the mp app family, WM API, theme bridge, PDF engine fix
- mpwm polish wave: terminal key focus, focus-history close order, pop-back-to-origin, occupied-workspace cycling, demo
- mpwm: ghost-launch/menu-flash root cause, scroll z-gate, browser binds, path-free status, viewers delisted, Fab opens
- mpwm: Hyprland-exact close animation over a frozen snapshot
- mpwm: warm-instance pool, flat-luminance opens, flicker-free CEF resize
- work: land the sources the last commits reference
- mpwm: quick-look gap fixes; image cache eviction on preview unload
- tweaker: material thumbnails + vibecode popup + ctrl-space notes, undo/redo over the edit ledger, capture-semantics pi
- tweaker: tabbed side panel (Props/Shader/Tree) - shader tab with checkerboard material well + prompt, complete widget-
- mpwm+mpterm: text quick-look — pty teardown deadlock fix, in-place retarget verified, debug probes stripped
2026-08-29 09:26:26 +02:00
Admin
19097335b6 vj: the responsive DJ console (PR #1199 by vjroger), the DREAM expand-flux-video pipeline as one store graph, 16:9 image particles, XF defaults
Squashed from work; the fine-grained history is under tag archive/work-2026-08-29:
- mp* wave: mpwm window manager + the mp app family, WM API, theme bridge, PDF engine fix
- mpwm polish wave: terminal key focus, focus-history close order, pop-back-to-origin, occupied-workspace cycling, demo
- work: land the sources the last commits reference
- vj: responsive DJ mixer + Windows drag-and-drop, cherry-picked from PR #1199 (vjroger)
- vj particles: image mode homes on a 16:9 picture plane
- vj: DREAM pipeline — expand -> flux1 still -> minimax i2v loop, with a canvas picker
- vj: the row shows the real prompt, a flux picker, XF default, and new clips stop vanishing
- a DREAM run is one declared graph the store runs without us
2026-08-29 09:26:26 +02:00
Admin
d24cbee60d asset platform: store pipelines with honest banded progress, vision annotation through the job queue, fleet scheduling (roles, spread, peer provisioning), the music contract, synonym search, the runs card
Squashed from work; the fine-grained history is under tag archive/work-2026-08-29:
- mp* wave: mpwm window manager + the mp app family, WM API, theme bridge, PDF engine fix
- work: land the sources the last commits reference
- kenney: catalogue all 50 free 3D kits; Modal dismissed() never fired
- asset store: central vision-annotation queue; Kenney donate prompt
- asset-ai: vision domain — image + prompt -> text on every fleet node
- annotation runs the fleet's vision services through the normal job queue
- vj: responsive DJ mixer + Windows drag-and-drop, cherry-picked from PR #1199 (vjroger)
- store search: WordNet synonym expansion, query-side; per-term seeks
- video scheduling: a cold model pin never downloads past a warm one, and a dedicated box only serves its role
- an expansion can never lose a run — and `expand: true` finally means something
- fleet panel: a slow box is not a missing box
- h3: first+last keyframe conditioning (the weights were always FL2VA)
- registry: the three H3 FL2VA tiers name their real conditioning
- fleet scheduling: spread before stacking, and a stuck job moves
- a download never steals a job from a box that has the weights
- the faster GPU takes the tie: 6000 > 5090 > 4090
- an evicted flux model gives the card its VRAM back
- a job now records what each of its stages was handed
- open a stage in RUNS and read what it sent
- every expanded song was an instrumental: the lyrics had nowhere to go
- the writer never overwrites words the person wrote
- store: a dependent job's body is spliced from its deps' results at claim
- a fleet box's job row says what that job was asked for
- the store runs a whole pipeline, and one record says how far it got
- a client can declare a whole run, watch it, and stop it
- 100% now means published, and the expander is a job you can queue
- one card says what a spawned task is doing, everywhere
- a run that ends says so, instead of being noticed later
- when every box holding the model is busy, buy another copy
2026-08-29 09:26:25 +02:00
Admin
3b4d6a4ff7 platform+widgets: sploded 3D inspect view, the tweaker design-feedback suite, script math-AOT and docs channel, pointer-pin and pixel-probe, modal layout, map warp
Squashed from work; the fine-grained history is under tag archive/work-2026-08-29:
- mp* wave: mpwm window manager + the mp app family, WM API, theme bridge, PDF engine fix
- mpwm polish wave: terminal key focus, focus-history close order, pop-back-to-origin, occupied-workspace cycling, demo
- mpwm: warm-instance pool, flat-luminance opens, flicker-free CEF resize
- work: land the sources the last commits reference
- kenney: catalogue all 50 free 3D kits; Modal dismissed() never fired
- platform: windows check green again — SetWindowTextW binding
- map: exact warp-aware inverse projections — pointer ops work folded
- mpwm: quick-look gap fixes; image cache eviction on preview unload
- tweaker: material thumbnails + vibecode popup + ctrl-space notes, undo/redo over the edit ledger, capture-semantics pi
- tweaker: vibe popup card chrome + dispatch order, ctrl-space notes verified, sploded design v2 chapter
- tweaker: tabbed side panel (Props/Shader/Tree) - shader tab with checkerboard material well + prompt, complete widget-
- sploded v2: nesting-depth z, hairline scope frames, body pass
- sploded: pin the depth convention with a test, kill the draw_depth residue
- sploded: real body-pass split (scene-pass capture, panel flat) + y-convention source of truth with anti-flip gate test
- sploded: hollow outlines, flat-band input, ray-pick unprojection
- tweaker: shader tab defaults to the selection's first draw layer, stale hint trimmed
- sploded: outlines become clipped, antialiased strips; tighter deck
- sploded: merge the lane's v2 (nesting-depth z, clipped AA strip outlines, flat-band input, unproject, SplodedStack bod
- sploded: the exploded view is a LIVE view — pointer events route through the inverse explode transform (ray -> plane -
- tweaker: tabs are real widgets (uid, tree node under the dock, own plane in 3D) and pickable; navigation-class clicks
- sploded: pinned/hover outlines render on the widget's own plane in 3D — per-widget nesting depth lives on the platform
- tweaker: the material well renders the pinned widget's actual shader — the swatch byte-copies the widget's live draw c
- tweaker: the Shader tab shows the shader as written — the layer's pixel/vertex fn source (nearest definition up the co
- sploded: I = true isometric preset (yaw 45°, pitch atan(1/√2))
- tweaker: eyedropper — the colour popover's pick button arms a pixel probe; the next press in the app samples that devi
- tweaker: the shader loop closes — /tweak/apply resolves the pinned widget by uid (anonymous path segments never round-
- vj: responsive DJ mixer + Windows drag-and-drop, cherry-picked from PR #1199 (vjroger)
- tweaker: the material well is a magnifier — the mirrored instance draws at the widget's native size in the well's own
- tweaker: per-layer material thumbnails — the Widget derive emits WidgetNode::layer_areas() (every #[live] Draw… field
- tweaker: the shader source view is the real CodeView (syntax highlighting, selection, editing) when the app registers
- tweaker: Ctrl+Enter sends on every platform (TextInput treated only Cmd as primary on macOS, so Ctrl+Enter inserted a
- Modal claims no layout slot: the DJ page fills its window again
- tweaker: every fn apply recompiles (eval_chunk ran every chunk under ONE synthetic callsite, so the script body — and
- tweaker: an apply whose draw shader fails to compile is rejected — the layer goes back (last live fns / the fn as writ
- tweaker: the Shader tab's source view owns its scrolling (the ScrollYView around the CodeView double-scrolled the care
- widgets: set_visible belongs to every widget, not just View (#1194)
- script: a dead heap's resource handles must not outlive it (#1195)
- Resources: search the executable's directory, not only the working directory (#1196)
- Windows: fit a restored window to the displays that are actually attached (#1197)
- d3d11: a failing GPU call reports the loss instead of killing the process (#1198)

Co-authored-by: Kevin Boos <1139460+kevinaboos@users.noreply.github.com>
2026-08-29 09:26:24 +02:00
Kevin Boos
7a342be4d4
d3d11: a failing GPU call reports the loss instead of killing the process (#1198)
* d3d11: a failing GPU call reports the loss instead of killing the process

The backend already notices a removed device when `Present` returns
DXGI_ERROR_DEVICE_REMOVED, but a device rarely dies at a moment as convenient
as a present. It dies between frames, and the next thing that touches it is a
resource creation or a buffer map -- of which this file had 76 unwrapped, plus
three `std::process::exit(1)`. So the usual outcome of a GPU driver reset,
a TDR or a hybrid-GPU transition across suspend/resume was a panic, and the
graceful path was unreachable.

Route the calls a dead device actually reaches through `D3d11Cx::note_error`,
which asks `GetDeviceRemovedReason` rather than pattern-matching the HRESULT the
failing call happened to return -- creation calls do not reliably return the two
DXGI device-lost codes, while the device itself always knows and keeps saying
so. That answer sets a process-wide `device_lost` latch and logs once.

The softened sites are the ones a dead device lands on first: draw-list and pass
uniform buffers, which upload every frame with no dirty gate; texture and render
target creation; and shader object creation, whose `CxOsDrawShader::new` already
returned `Option` with both callers handling `None`.

Three latent bugs fall out of auditing them, each of which loses content
permanently rather than noisily:

  - `update_vec_texture` consumed the dirty flag with `take_updated()` and then
    returned early when the pixel buffer was out on loan, so a texture that hit
    that window was never uploaded again. For the glyph atlas that means all
    text disappears for the life of the process. The Metal backend already
    guards this; D3D11 did not.
  - `hlsl_compile_shaders` drains `compile_set` destructively, so a shader whose
    object creation failed was dropped from the queue forever and everything
    drawn with it silently stopped rendering. Failed creations go back in the
    queue.
  - `render_view` unwrapped the geometry index buffer but passed the vertex
    buffer through as an `Option`, so a missing one bound null and drew nothing
    with no error anywhere. Both are now checked together, and a draw call with
    either missing is skipped under a `debug_assert!` that it only happens on a
    lost device.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit cc980805b5774253e592e183f577c5ba08ce85db)

* d3d11: rebuild the device and every GPU resource after a device loss

Detection landed already: present() sets a per-window device_lost and the paint
loop stops re-dirtying the pass. Nothing rebuilt anything, so the window stayed
frozen until the app was restarted.

Recovery runs at the top of win32_event_callback, the one place holding both
&mut D3d11Cx and &mut Vec<D3d11Window> while nothing is mid-render. It releases
each window's swap chain, back buffer, view and paint-beat registration (DXGI
allows one flip-model chain per HWND, so the dead one must be gone first),
recreates the device tier when the device really is gone, drops every GPU handle
the Cx holds, and rebuilds each swap chain against the same HWND.

Clearing handles is only half of a sweep. Geometry and instance uploads are
gated on dirty flags cleared unconditionally once the upload runs, and textures
on a dirty rect consumed by take_updated, so every gate is re-armed or the empty
slot is never refilled. The CPU-side sources all survive: texture pixels live in
TextureFormat::Vec*, geometry in CxGeometry, and shaders keep their compiled
DXBC plus the on-disk cache, so recovery recreates shader objects without
compiling any HLSL.

Retries are spaced 250ms to 4s and driven by the existing signal heartbeat
rather than a new timer, because the GPU can stay absent for a long time. While
lost, the loop is forced to Wait at both EventFlow decisions -- every condition
that would otherwise choose Poll is unsatisfiable when nothing can paint, so it
would spin for the whole outage -- and pending screenshot requests are failed,
both to release their requester and because a non-empty queue is itself one of
those conditions.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit d603d3e7996cd71b5da6ed3372c9dc75eebb4a77)

* d3d11: bind the hot-path buffers by reference, not by clone

The device-loss bails introduced an AddRef/Release pair per uniform-buffer
upload and per draw call, on paths that run for every draw list, pass and
geometry every frame. Borrowing reads the same Option without touching the
refcount; only IASetVertexBuffers genuinely needs an owned Option, which is what
the code built before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit ecfab9a9355d50754a7117de8303d66f8919b2ee)

* d3d11: make the device-loss recovery actually work, and add a way to exercise it

Three defects, each of which stopped recovery dead, and none of which is visible
without running it. `MAKEPAD_D3D11_TEST_DEVICE_LOSS=<seconds>` forces a full
device recreation on a timer so the path can be exercised without a driver
reset; a real removal cannot be provoked from inside the process. It is a
stronger test than merely setting the latch, because the device really is
replaced, so any GPU object the sweep fails to rebuild still belongs to the old
device and cannot render against the new one.

  - The sweep called `set_updated` on every texture, which panics for anything
    that is not a `Vec*` format. The first render target it reached took the app
    down. Only vec textures carry a dirty rect; render targets, depth buffers
    and shared textures have no CPU-side contents and get their alloc record
    cleared instead.

  - Every rebuilt swap chain failed with `E_ACCESSDENIED`. DXGI allows one
    flip-model swap chain per HWND at a time and D3D11 destroys lazily, so the
    immediate context's own reference to the back-buffer view kept the old chain
    -- and its claim on the window -- alive after the application had dropped
    every handle it held. `ClearState` + `Flush` once, after all the windows have
    released and before any rebuild.

  - The post-recovery redraw marked every pass slot dirty, including ones
    nothing had drawn into, and `draw_pass_to_texture` unwraps
    `main_draw_list_id` immediately. Only passes that have one are marked.

Verified against a release build: six consecutive forced device recreations,
no panics, the UI rendering correctly after each (text included, so the glyph
atlas re-uploads from its retained pixels), `ResizeBuffers` working on a rebuilt
chain, handle count flat across recoveries, and the loop idle afterwards.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit bb7d1c019612a4fb2668640f47b8e16c3886e1cf)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 09:41:33 +02:00
Kevin Boos
e40a5318f7
Windows: fit a restored window to the displays that are actually attached (#1197)
An app that persists its window geometry has to restore it into a display
arrangement that may have changed completely since it was saved: the display
the window sat on can be gone, a docked laptop can be back on its built-in
panel, and the file can hold values no display ever had. Nothing validated any
of it -- `configure_window` stored the numbers and each backend passed them
straight to `CreateWindowExW` / `initWithContentRect:` / `XCreateWindow` -- so a
window could come back off-screen or too small to grab, with no way back except
deleting the state file.

Windows also manufactured those values. Win32 reports a minimized window at
`(-32000, -32000)` with a zero-sized client rect, `WM_SIZE` published that as
the window's authoritative geometry, and an app saving on shutdown wrote it
down. `WM_MOVE` meanwhile published nothing, so a window that was dragged but
not resized persisted its pre-drag position (X11's `ConfigureNotify` and macOS's
`windowDidMove:` both already published).

Add `platform/src/screen.rs`, holding the policy in one place:

  - `sanitize_window_geom` needs no display knowledge and every backend reaches
    it through `CxWindow::create_geom`. It is what protects the backends a fit
    cannot help: Wayland enumerates no displays for a client and hands the size
    to `wl_egl_window_create`, which rejects a non-positive one -- a persisted
    `0` or `NaN` panicked the app at startup -- and X11 encodes extents as
    unsigned 16-bit and answers a zero with a protocol error that, with no error
    handler installed, terminates the process.
  - `clamp_point_to_screens` pins the origin BEFORE the window is created. The
    fit alone is too late: `set_inner_size` runs in between and works relative
    to wherever the window landed.
  - `fit_window_rect_to_screens` corrects the finished rectangle. A window
    already wholly on the desktop is returned untouched, including one
    deliberately spanning two adjacent displays; anything else moves to the
    display it overlaps most, or nearest by centre, capped to that work area.

Displays come from `EnumDisplayMonitors` + `GetMonitorInfoW` on Windows (both
absent from the vendored bindings, so linked here), `NSScreen.screens` on macOS,
and the root geometry plus EWMH `_NET_WORKAREA` on X11. Wayland is unaffected by
the class of bug: a client there cannot know or choose where its windows go.

Report a minimized window from `GetWindowPlacement().rcNormalPosition`,
converted out of workspace coordinates, so what an app persists is the geometry
the window actually returns to; skip publishing on `SIZE_MINIMIZED`; publish on
`WM_MOVE`, deferred to `WM_EXITSIZEMOVE` during a user drag because the Cx
handler redraws on every geometry event.

Positions are now documented and implemented as physical screen pixels on
Windows and X11 (points on macOS) and are never DPI-scaled, matching
`get_position`, `create_position` and the platform calls. `set_position` alone
had been scaling its argument, so `set_position(get_position())` moved a window
to twice its coordinates on a 200% display.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 07:44:21 +02:00
Kevin Boos
dd4c8309d9
Resources: search the executable's directory, not only the working directory (#1196)
A packaged desktop build addresses its resources through a relative package root
-- `cargo packager` and `robius-packaging-commands` both use `.`, with the
resource trees sitting beside the executable -- and a relative `File::open`
resolves against the process working directory. Any launcher that sets no
working directory therefore starts the app somewhere unrelated and every font,
icon and image open fails: a URL-protocol handler (`HKCR\<scheme>\shell\open\
command` carries no working directory), a file association, a service, a
shortcut created without one.

The result is not a clean failure. The window comes up and lays out correctly,
shader-drawn shapes and buttons render, and network-loaded images appear, but
every glyph and every bundled icon is missing, because those are the parts that
need a file. It reads as a renderer bug rather than a missing directory.

macOS avoids this through `apple_bundle_load_dependencies`, and a Linux `deb`
package uses an absolute `/usr/lib/<name>`, so Windows is the only desktop
target whose resource lookup depends on where it was started from.

Retry a failed open against the directory holding the executable. This is
purely additive -- a path that resolves today resolves identically, and only an
open that would have failed reaches the fallback -- so a dev build's
workspace-relative dependency paths keep working unchanged.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 07:44:10 +02:00
Kevin Boos
5aad53afbc
script: a dead heap's resource handles must not outlive it (#1195)
* widgets: set_visible belongs to every widget, not just View

`ui.value_lg.set_visible(w >= 150)` logged "widget method set_visible not
found for uid WidgetUid(1236)" once per resize, and the widget simply never
reflowed. Label, Button and every other leaf refused a method that View
alone implemented — even though visibility is a Widget-trait property that
`#[visible]` derives for exactly those widgets.

it went unnoticed because the fixed-slot list pattern wraps its rows in
Views. what it breaks is the widget tiles, where a bare Label is toggled by
an on_widget_resize, and the failure reads as a layout that just doesn't
respond to its size.

handled once now, in WidgetRef::script_call, after the widget's own
script_call declines the method — so set_visible (and a visible() getter)
work on anything, and View's copy is gone. a bad argument still keeps the
current visibility and returns an error instead of guessing true.

* script: a dead heap's resource handles must not outlive it

the launcher died about one run in five, always inside the GC and never
anywhere near what caused it:

  gc.rs:300: index out of bounds: the len is 21 but the index is 22

only after an isolate had been torn down and another started — closing a
widget preview, or granting `network` (alloc-time, so the app's isolates
restart).

CxScriptResources caches (heap_key, abs_path) -> that heap's LOCAL handle,
and a heap_key is an allocation ADDRESS. the only cleanup was
CxScriptResourceGc, which runs when the owning heap's own GC sweeps that
handle — and a heap that is dropped wholesale, as a Splash isolate's is,
never sweeps anything. so the entries outlived the heap, and the next
isolate whose root_objects landed on that freed address asked for the same
font and was handed the dead heap's handle index. it stored it in its own
FontMember{res, asc, desc}, where 22 means nothing in a table of 21 —
and nothing noticed until that heap's next collection walked the font
object it had every right to walk.

gc_heaps() drops a dead heap's entries, and detaches handles no surviving
heap still maps to (handle values are heap-local, so two heaps' handles can
be equal). called from gc_dead_splash_isolates beside the storage and
bridge purges, which already runs before a new isolate can allocate.

anything keyed by heap_key needs to be in that function, for this reason.

the same hunt turned up a second crossing, fixed here too:
View::script_call(render) built its `me` object in whatever VM happened to
be calling, protoed off the SOURCE view's heap, and forwarded the caller's
args object into the target VM. render a view whose isolate has since been
torn down and that object stays behind in the CALLER's heap holding a dead
heap's index. it refuses now when the two heaps differ.
2026-08-26 10:43:58 +02:00
Kevin Boos
9c62043f9a
widgets: set_visible belongs to every widget, not just View (#1194)
`ui.value_lg.set_visible(w >= 150)` logged "widget method set_visible not
found for uid WidgetUid(1236)" once per resize, and the widget simply never
reflowed. Label, Button and every other leaf refused a method that View
alone implemented — even though visibility is a Widget-trait property that
`#[visible]` derives for exactly those widgets.

it went unnoticed because the fixed-slot list pattern wraps its rows in
Views. what it breaks is the widget tiles, where a bare Label is toggled by
an on_widget_resize, and the failure reads as a layout that just doesn't
respond to its size.

handled once now, in WidgetRef::script_call, after the widget's own
script_call declines the method — so set_visible (and a visible() getter)
work on anything, and View's copy is gone. a bad argument still keeps the
current visibility and returns an error instead of guessing true.
2026-08-26 10:43:42 +02:00
Admin
93f27f802e chore: workspace members for the new crates, makepad.splash, the naming and loader check scripts, and error_log
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- vj: the thumbnail pipeline becomes one honest machine, and effects go livecodable
- widgets: boxed labels center on their ink, not on the font's line box
- metal: a fresh texture forgets nothing it never had — reallocated vec textures upload whole
- fab: a 3D creation shell and the viewer built on it
- texcomp: the block codec and the container every texture will travel in
- mixer: a live LR-Mix surface for the XR18 — strips paired the way the desk is run, auto-connect, EQ that bends its own curve, and a sweep paced so the console drops nothing
2026-08-26 08:49:50 +02:00
Admin
6c8d301da9 asset-ui: the classic import surface follows what the importer now produces, and the chat tells the truth while it works
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- vj: the thumbnail pipeline becomes one honest machine, and effects go livecodable
- asset-ai: the chat tells the truth while it works
- audio: a FLAC decoder from the specification, beside the MP3 and Vorbis ones
- asset-ui: the classic import surface follows what the importer now produces
2026-08-26 08:49:50 +02:00
Admin
4c5228d9ca mixer: a live LR-Mix surface for the XR18
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- mixer: a live LR-Mix surface for the XR18 — strips paired the way the desk is run, auto-connect, EQ that bends its own curve, and a sweep paced so the console drops nothing
2026-08-26 08:49:49 +02:00
Admin
705d6c73ed vj: the console — real transports and reverse, the music explorer's IMPORT, local store, lyrics and model plumbing, DJ-tab scroll knobs, title-click reset, crossfade fix, stem headroom, and music/sfx browse models
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- vj: the music explorer gets IMPORT, and catalog controls fold away in local mode
- vjfx: three lanes land — engine hooks + hold stage, the videomesh engine, and the audio picture
- vj: the thumbnail pipeline becomes one honest machine, and effects go livecodable
- vj: thumbnails become mp4 — hardware-coded sheets at measured-4K cells, and the bake stops racing the GPU
- store: the ceremony dies — batch publish, one transaction, and the engine stops re-reading its own log
- store: the ceremony dies — batch publish, one transaction, and the engine stops re-reading its own log
- vj: the console grows real transports, and the deck stops lying about reverse
- vj: reverse earns a memory, and the effects stop aging
- vj: video goes NV12 end to end, and the GPU does the unpacking
- vj: windows hands the main thread one megabyte, and the script tree wants two
- vj: the GPU learns to see motion — realtime frame tweening on every deck
- vj: the tweener learns — RIFE runs on the Mac and feeds the warp
- vj: the classical tweener grows up, and every deck gets a tween chip
- models: an interrupted install can never load broken
- vj: the transport becomes a platter — velocity in, position out, one map
- vj: the tween presenter reads the platter — one clock per deck, cued once at the frame on screen
- vj: the presenter switch lands without its scaffolding
- vj: the next pair's fields are fetched ahead of the change under the capacity law — a pair change costs an ordinary beat; macos: the layer's own display link paces the frame when the system offers it, the old path stays as fallback
- vj: AI3 subdivides adaptively — one, three or seven neural frames per pair, chosen from measured synth time against the pair's own period, with classical flow between them and a 7-3-1-FL fallback; the deck shows the depth
- vj: local store, lyrics and model plumbing, and the frame-interpolator's device parity check
- vj: DJ-tab scroll knobs, title-click reset, crossfade fix, stem headroom — plus the pre-Ampere CUDA fix (#1193)
- vj: the deck explorer lists music-tagged audio, the sfx surface filters by tag not category, the track list fills the window, and the modal host has no layout footprint

Co-authored-by: Rogier de Leeuw <vjroger@gmail.com>
2026-08-26 08:49:49 +02:00
Admin
dc389d71a1 vj: the transport becomes a platter — one clock per deck, the producer contract, GPU frame tweening with RIFE, NV12 end to end, AI3 adaptive subdivision, and bit-identical decks
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- vj: thumbnails become mp4 — hardware-coded sheets at measured-4K cells, and the bake stops racing the GPU
- vj: the console grows real transports, and the deck stops lying about reverse
- vj: reverse earns a memory, and the effects stop aging
- vj: video goes NV12 end to end, and the GPU does the unpacking
- vj: the GPU learns to see motion — realtime frame tweening on every deck
- vj: the tweener learns — RIFE runs on the Mac and feeds the warp
- vj: the classical tweener grows up, and every deck gets a tween chip
- vj: the tween clock tells presented time, not producer time
- vj: the transport becomes a platter — velocity in, position out, one map
- vj: the tween presenter reads the platter — one clock per deck, cued once at the frame on screen
- vj: the OFF tier joins the platter — a resident clip's picture is cache[nearest(pos)]
- vj: the media thread loses its second clock — resident clips park the decoder
- vj: the producer gets a contract — keyed ladders, deadlines, and a capacity law
- vj: two decks, one law — identical inputs are bit-identical, and the warp agrees to the byte
- vj: the presenter switch lands without its scaffolding
- vj: the next pair's fields are fetched ahead of the change under the capacity law — a pair change costs an ordinary beat; macos: the layer's own display link paces the frame when the system offers it, the old path stays as fallback
- vj: AI3 subdivides adaptively — one, three or seven neural frames per pair, chosen from measured synth time against the pair's own period, with classical flow between them and a 7-3-1-FL fallback; the deck shows the depth
- video_flow: the flow debug bins, declared behind the convert feature so --no-default-features skips them instead of failing
2026-08-26 08:49:48 +02:00
Admin
cb49b032df vjfx: 104 presets, engine hooks + hold stage, the videomesh engine, the audio picture, livecodable effects, and mp4 thumbnail sheets
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- vjfx: 104 new presets — the transition lane fills out and the screen family goes wide
- vjfx: three lanes land — engine hooks + hold stage, the videomesh engine, and the audio picture
- vj: the thumbnail pipeline becomes one honest machine, and effects go livecodable
- vj: thumbnails become mp4 — hardware-coded sheets at measured-4K cells, and the bake stops racing the GPU
- store: the ceremony dies — batch publish, one transaction, and the engine stops re-reading its own log
- store: the ceremony dies — batch publish, one transaction, and the engine stops re-reading its own log
- vj: the console grows real transports, and the deck stops lying about reverse
- vj: reverse earns a memory, and the effects stop aging
- fab: a 3D creation shell and the viewer built on it
2026-08-26 08:49:48 +02:00
Admin
9821263b2c fab: a 3D creation shell and the viewer built on it — colour picker, material textures, dials that move the scene while they drag, the glTF loader, the tour, and the probes
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- fab: a 3D creation shell and the viewer built on it
- raytrace: the traced pane starts coarse and doubles to native, with the raster underneath
- fab: a colour picker, a material's textures, and dials that move the scene while they drag
- texcomp: the block codec and the container every texture will travel in
- fab: FAB_PROBE_MAT — per-material triangle counts, texture presence and uv spread in the roof probe
2026-08-26 08:49:47 +02:00
Admin
8f3c826265 csg: deterministic boolean bench + fuzz harnesses
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- csg: add deterministic boolean bench + fuzz harnesses (examples)
2026-08-26 08:49:47 +02:00
Admin
d72cffd0f6 raytrace: the traced pane starts coarse and doubles to native, with the raster underneath
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- fab: a 3D creation shell and the viewer built on it
- raytrace: the traced pane starts coarse and doubles to native, with the raster underneath
- texcomp: the block codec and the container every texture will travel in
2026-08-26 08:49:47 +02:00
Admin
87e0ce82e9 texcomp: the block codec and the container every texture will travel in
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- texcomp: the block codec and the container every texture will travel in
2026-08-26 08:49:47 +02:00
Admin
f788946065 render+sim+gltf: a HUD, the receiver shader with sun and cascaded shadows, metered exposure and haze, two shaders that never compiled, map facing becomes heading, and the viewer hooks
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- render+sim: the two hooks the model viewer already relies on
- viewport: Realtime gets the engine's cascaded shadow pass, the NOAA sun, metered exposure with sky ambient, a haze knob, a time-of-day slider in the header, and no grid — the building itself still waits for its casters and direct light
- render: the engine's receiver shader takes the sun with two-sided normals and the shadow term for the model batches
- fab: a 3D creation shell and the viewer built on it
- render: two shaders that never compiled — a let is not assignable, a var is
- sim: a declared map facing becomes a body's heading through one rule
- render: a HUD that already reads as a game's before anyone styles it
- asset+sim: the two modules their own commits already declared
2026-08-26 08:49:46 +02:00
Admin
9c3b2298ea audio: a FLAC decoder from the specification
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- audio: a FLAC decoder from the specification, beside the MP3 and Vorbis ones
2026-08-26 08:49:46 +02:00
Admin
229741662c libs/ai: the step cost model per device, the cold-turn speculative path, and RIFE on Metal with its device-parity check
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- vj: the tweener learns — RIFE runs on the Mac and feeds the warp
- asset-ai: the chat tells the truth while it works
- llm: a cold turn on the solo slot takes the session-native speculative path — think-mode turns no longer re-ingest the whole conversation through the draft head (66 → 122 tok/s on the four-lane box)
- llm: the step cost model is chosen per device — the RTX PRO 6000's measured verify curve (13.7 + 3.17·B ms) beside the 5090's; the bench warms every tail shape and times two windows
- vj: local store, lyrics and model plumbing, and the frame-interpolator's device parity check
2026-08-26 08:49:46 +02:00
Admin
69385ed1be libs/asset: batch publish in one transaction and its route with the hostile cases, hardware sha256 proved against the oracle, classic worlds stop being mirror images, sounds and sprites publish a picture, an interrupted model install can never load broken, ActorDef::scaled, and the asset-ai chat tells the truth
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- vj: the thumbnail pipeline becomes one honest machine, and effects go livecodable
- repo: context_ladder scratch bin stays local, not shipped
- vj: thumbnails become mp4 — hardware-coded sheets at measured-4K cells, and the bake stops racing the GPU
- store: the ceremony dies — batch publish, one transaction, and the engine stops re-reading its own log
- vj: the console grows real transports, and the deck stops lying about reverse
- models: an interrupted install can never load broken
- importer: the classic worlds stop being mirror images
- asset-ai: the chat tells the truth while it works
- sqlite: derived tables get their real names, their predicates, and all their arms
- llm: the step cost model is chosen per device — the RTX PRO 6000's measured verify curve (13.7 + 3.17·B ms) beside the 5090's; the bench warms every tail shape and times two windows
- importer: a sound and a single-tile sprite publish a picture like everything else
- asset: hardware sha256 kernels, proved against the software oracle before they run
- sim: a declared map facing becomes a body's heading through one rule
- asset: the batch publish route, with the hostile cases it has to refuse
- asset: an example that asks a live store which assets carry a thumbnail
- asset+sim: the two modules their own commits already declared
- asset: ActorDef::scaled — every linear quantity follows the map's person height — plus the place-dump and retire-stale store examples, and the game chat context stops reporting work it did not do
2026-08-26 08:49:46 +02:00
Admin
ed5de46749 sqlite_query: derived tables get their real names, their predicates, and all their arms
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- store: the ceremony dies — batch publish, one transaction, and the engine stops re-reading its own log
- sqlite: derived tables get their real names, their predicates, and all their arms
2026-08-26 08:49:45 +02:00
Admin
45e65e69db draw+widgets: a Splash keeps its host's walk across a body rebuild, a dock redraws its panels, boxed labels center on their ink, hsv2rgb compiles, and slider/tip/dropdown catch up
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- vj: the console grows real transports, and the deck stops lying about reverse
- vj: video goes NV12 end to end, and the GPU does the unpacking
- widgets: boxed labels center on their ink, not on the font's line box
- metal: a fresh texture forgets nothing it never had — reallocated vec textures upload whole
- fab: a 3D creation shell and the viewer built on it
- widgets: a dock redraws its panels, not just its own frame
- draw: hsv2rgb takes vector bounds, so it compiles
- texcomp: the block codec and the container every texture will travel in
- widgets: a Splash keeps the walk its host declared across a body rebuild, and logs a body that fails to evaluate instead of drawing nothing
- vj: DJ-tab scroll knobs, title-click reset, crossfade fix, stem headroom — plus the pre-Ampere CUDA fix (#1193)

Co-authored-by: Rogier de Leeuw <vjroger@gmail.com>
2026-08-26 08:49:45 +02:00
Admin
fd22db91ff script: the parser fix with the case that caught it, and shader calls
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- vj: video goes NV12 end to end, and the GPU does the unpacking
- script: parser, with the case that caught it in the test suite
2026-08-26 08:49:44 +02:00
Admin
68d80b69e5 windows: the paint beat becomes the swapchain's own beat, and /g learns to see
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- vj: thumbnails become mp4 — hardware-coded sheets at measured-4K cells, and the bake stops racing the GPU
- windows: the paint beat becomes the swapchain's own beat, and /g learns to see
2026-08-26 08:49:44 +02:00
Admin
d5b89df37d macos+metal: the paint beat becomes the display's own beat — link-paced frames, drawables presented plainly, instance buffers and vec textures safe under a live draw, a stalled-command-buffer watchdog, a shielded resetCursorRects
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- vj: thumbnails become mp4 — hardware-coded sheets at measured-4K cells, and the bake stops racing the GPU
- vj: the console grows real transports, and the deck stops lying about reverse
- vj: the GPU learns to see motion — realtime frame tweening on every deck
- macos: the paint beat becomes the display's own beat
- windows: the paint beat becomes the swapchain's own beat, and /g learns to see
- metal: vec textures ride the command stream — the CPU stops overwriting what the GPU is still reading
- metal: instance buffers stop being rewritten under a live draw
- macos: next frames and draws are stamped with the flip they aim at
- metal: a fresh texture forgets nothing it never had — reallocated vec textures upload whole
- metal: a watchdog for stalled command buffers — it names the pass, and only aborts when asked
- macos: resetCursorRects no longer aborts the app when AppKit re-enters it
- macos: nothing panics across resetCursorRects — the callback is shielded and its cursors are retained
- vj: the next pair's fields are fetched ahead of the change under the capacity law — a pair change costs an ordinary beat; macos: the layer's own display link paces the frame when the system offers it, the old path stays as fallback
- macos: a drawable from the layer's display link is presented plainly — presenting it at a time is forbidden and raised in every visible window
- macos: a window paced by the layer's display link never asks the layer for a drawable — the beat waits for the link's update; ObjC exceptions are logged with their reason before they unwind
- macos: the layer's display link is opt-in (MAKEPAD_METAL_DISPLAY_LINK=1) until it paces at the display's rate — 11 fps visible against 62 on the proven path
- macos: the layer's display link asks for the screen's maximum rate, consumes every drawable it hands out, and traces updates/consumed/presented per second — still 75 ms per present under a drag, so it stays opt-in
- fab: a 3D creation shell and the viewer built on it
- raytrace: the traced pane starts coarse and doubles to native, with the raster underneath
- macos: a link-paced beat blocks till the next flip, and an armed paint clock means wait — the main thread no longer polls at 100% CPU between frames
2026-08-26 08:49:44 +02:00
Admin
d1a0eb1cb8 platform: the paint clock contract — a beat per backend, and the time repaint stops resurrecting passes
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- vj: thumbnails become mp4 — hardware-coded sheets at measured-4K cells, and the bake stops racing the GPU
- vj: reverse earns a memory, and the effects stop aging
- vj: video goes NV12 end to end, and the GPU does the unpacking
- windows: the paint beat becomes the swapchain's own beat, and /g learns to see
- platform: the time repaint stops resurrecting passes their owner left behind
- metal: a fresh texture forgets nothing it never had — reallocated vec textures upload whole
- fab: a 3D creation shell and the viewer built on it
- raytrace: the traced pane starts coarse and doubles to native, with the raster underneath
2026-08-26 08:49:43 +02:00
Rogier de Leeuw
413709b565
ai-cuda: pre-Ampere machines get their CUDA store back (#1192)
A Turing box (RTX 2080 Ti, sm_75) lost ALL of CUDA because two kernel
files refused to compile for it, and one failed kernel build means the
stub store — surfaced in the VJ as "stems: model error: no compiled-graph
device" on the DJ tab.

diffusion_ops.cu used three sm_80-only pieces unguarded: bf16 wmma
fragments (the type itself is incomplete before Ampere), cp.async, and
the m16n8k16 mma shapes. The cp.async helpers now fall back to
synchronous copies below sm_80 — the f16 wmma flash/sdpa kernels lose
their prefetch overlap on Turing, not their contents — while the bf16
and FA2 kernels are compiled out and their launchers refuse pre-sm_80
devices with cudaErrorNotSupported instead of returning a buffer the
kernel never wrote.

fattn/common.cuh made mkllm_unused_vars constexpr: the no-cp.async
branch of ggml_cuda_fattn_mma_get_nstages calls it, and a non-constexpr
callee poisoned the constexpr config chain on exactly the pre-Ampere
device pass — the arch nobody had compiled for.

Stems verified on the 2080 Ti: stems-ops-check all green (SNR 137-147 dB
against the CPU reference), two tracks separated end to end, output
confirmed clean by ear.

Co-authored-by: vjroger <r.deleeuw@qogni.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 22:15:00 +02:00
Kevin Boos
6cf59e1509
StackNavigation: expose the view a transition is heading toward (#1191)
`current_view` only advances once a push or pop transition finishes, so
mid-transition it still reports the outgoing view. Callers that want to
address the view the app considers current, e.g. to retitle it, had to
either duplicate the id themselves or special-case `is_transitioning`.

* Add `destination_view()`, which reports the incoming view as soon as a
  transition starts and falls back to `current_view` when settled.
2026-08-25 06:14:32 +02:00
Admin
8b5caf41e1 video: the sample-attachments call takes a CoreFoundation Boolean, so it builds on x86_64 macOS too
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 06:03:00 +02:00
Admin
152b11f20a vj: the deck models install themselves, and Windows gets its waveform back
- INSTALL MODELS under the music explorer: a download dialog naming both
  MIT weight sets (BS-RoFormer splitter 527MB, whisper large-v3-turbo
  1.6GB), where they land and their licenses; resumable sha256-pinned
  downloads through the asset-ai downloader (featureless dep — the same
  slice the asset UI links); cancel mid-flight (the button flips to
  CANCEL, .part resumes later), MB progress, and the row disappears on a
  provisioned machine. When the last model lands the loaded decks
  separate immediately: the stems worker now re-probes the checkpoint
  per job instead of latching its absence, and the lyrics transcriber
  unlatches too (the Apple fallback yields to whisper mid-session).
- DrawWaveLane's stem palette moves from instance inputs to uniforms:
  36 vertex inputs blew D3D11's vs_5_0 limit of 32 (error X4506), which
  left the music decks with NO waveform at all on Windows.
- --remote HOST:PORT binds a named interface so another machine can
  drive an app over the LAN (fleet-box testing); bare --remote stays
  loopback.
- queue chip: the + glyph centres in its 26x18 box.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-23 10:16:08 +02:00
Admin
ec07fe6519 readme: the VJ build section stops pointing at the work branch
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-23 08:58:21 +02:00
Admin
f44616b2a7 makepad-vj effects: the effect renderstack, with the shaders inside the documents
The VJ's EFFECT surface is a small renderer of its own. An effect is a
`.splash` DOCUMENT — engine choice, stages, fields, parameters and now the
SHADERS THEMSELVES live in the document rather than in Rust. That is the
shape this commit introduces (dev has never seen an intermediate one): a
document is the whole effect, and the Rust side is the engine families that
documents draw with.

The engine families: particles and emitters, GPU sim swarms and fluid, static
meshes (firefly synchrony, harmonograph loom, domino liturgy), tiles, flock,
clouds, city, pipes, stock charts, an SDF raymarcher with a subclassable
`scene_sdf`, and a mountain-jet endless range with a beat-pulsed fighter.

Three things make them a system rather than a demo reel:

  - SIM FIELDS — a float simulation-texture primitive, so an engine can carry
    GPU state across frames (wind, particles, fluid) instead of being a pure
    function of the clock.
  - CONTENT COUPLING — `content:` in a document and `input0` on every engine's
    tex0, so an effect can consume the program video: drapes, backdrops,
    mirrors, billboards, chrome, frescoes, canopy, glass, silk, wall, swarm,
    pen and mosaic families all take the picture playing behind them.
  - MUSIC BINDING — shaders read the beat, so the whole library moves with the
    track rather than on its own clock.

Roughly a hundred seeded preset documents ship with it, each with a lazily
rendered animated thumbnail (rendered 4x and box-downscaled, 16-tap SSAA);
the thumbnail pass went from four minutes to eleven seconds. CONTRACT.md is
the document contract and its verify recipe; IDEAS.md is the campaign tracker.
2026-08-23 01:35:44 +02:00
Admin
17b8a658fc makepad-vj: the live performance console — sweep-law transport, slots, and VFR import
A VJ console that is its own asset store: five surfaces (VIDEO program tiles,
MUSIC DJ decks, SFX, EFFECT slots, LIGHTS) over the asset server, driven by
APC40 hardware or the screen.

The transport is the centre of it. The SWEEP LAW: one direction, one beat
step, and a pacer that never hiccups — a loop wrap is never fast-forwarded and
the grid keeps no holes. On top of that: bracket-to-bracket scrub math so the
playhead learns the user's trim, SCRATCH as a sprung shuttle on every deck
transport, ping-pong and per-slot mute, hot-standby decks, one rate authority,
and beat sync that fits a whole clip into N bars at a musical rate while
cached loops keep real wrapping pts so the phase survives.

VARIABLE FRAMERATE VIDEO IMPORT measures the flow field a clip needs without a
model (libs/video_flow), which is what makes free-rate bounce-looping GPU
playback possible; the enhance pipe uprezes and tweens a deck clip in one
decode and one encode, with the motion vectors inside the mp4.

The console itself: makepad orange and the brand lockup, Blender-style BPM,
system tooltips, popover sliders, bracket trim handles, one button family and
a no-push band, doc-labeled slot dials with MIDI learn and strict typing, a
BEATS dropdown and slow-biased jog, an IMPORT panel that arms, runs and stops,
and BLAST — one press invents a visual for every parallel pipe. Generation
runs six jobs in flight across the fleet, and a job row names what it made.

The library grid fills as a relay — page, detail, manifest, blob, decode,
texture — and every hand-off used to be picked up by the 20Hz poll timer, so
each hop cost a tick however fast the store answered. The hand-offs now run on
the frame while anything is owed, manifests jump the queue like the details
that produced them, and resolve/decode/worker width all widen while nobody is
on stage and narrow together the moment the program window opens or a deck
plays. The visible page resolves first. Measured on the same warm cache: a
48-tile page 4.7s -> 1.9s, a tab's first visit 4.3s -> 1.1s, decode wait 48ms
-> 9ms, a second visit painted in the same frame with zero decodes, and no
UI hitches across the session.

Also here: wave analysis (tempo, grid, downbeat, tiles) with a judge suite
that scores the detector against click tracks and a human drummer, stem
separation, karaoke lyric display, Art-Net/DMX light control, and an output
window that projects without janking on Windows mouse movement.

466 tests, zero warnings.
2026-08-23 01:35:44 +02:00
Admin
9d2e17e3b9 chore: README, workspace, agent docs, and the box-driving scripts
- README: a build quick-start for macOS and Windows, the honest Linux
    story, what CUDA is for and how to install the separation model.
  - AGENTS.md: the `--remote` control surface protocol, so the harness it
    documents is usable without reading platform/src/remote.rs.
  - Cargo.toml: workspace membership for the crates this series adds and
    removes.
  - tools/: the Windows box scripts (wincmd, winps and friends, winrun) and
    remote_smoke.sh — how a build gets driven on a remote machine.
  - apps/asset-server: the standalone server binary and its README.
  - Small follow-ups in libs/{windows,apple_sys,makepad_test,mbtile_reader,
    converse} and apps/route, plus .gitignore and makepad.splash.
2026-08-23 01:34:36 +02:00
Admin
d1fb9ad8ad examples: datagrid, portallist_hit, render_to_texture, splat_bench
Four new examples, each of which is also the test surface for the thing it
demonstrates:

  - datagrid — the DataGrid widget across five tabs (sheets with a formula
    engine, big data, charts, pixels, widgets).
  - portallist_hit — PortalList hit-testing, with a UI test suite.
  - render_to_texture — the offscreen pass, with a UI test suite.
  - splat_bench — Gaussian splat sorting and drawing under load.

uizoo gains a dropdown tab; the splash example's UI test follows the splash
host-services change.
2026-08-23 01:34:36 +02:00
Admin
e2616d2d45 asset-ui: analysis, masks, music, webcam, and a library that keeps up with the store
The asset browser becomes the front end for everything the AI stack can do:

  - analysis — a content analysis pass over imported assets.
  - mask_paint — paint a mask over an image for inpainting.
  - music_page — the music generation surface.
  - webcam — live capture as a generation input.
  - fast_presets, store_content — preset and store-content plumbing.

Around them the existing surfaces get the corrections the store and the video
widget forced: sprite-split and child-pass thumbnail fixes, one video widget
everywhere (opens stop fighting each other and the rail never reflows),
grouped classic/Duke import that lands 102 assets instead of a card flood,
chat that greets with silence rather than a banner, and a pipeline that
survives restarts.
2026-08-23 01:34:36 +02:00
Admin
0f67c0b593 libs/render + xr: levels, player navigation, and Gaussian splats that sort on the GPU
libs/render picks up the two biggest new modules in the group: `level.rs`
(the imported-world runtime) and `player_nav.rs` (walkable-surface planning
with clearance bands). The renderer, shader set, GPU lightmap and skinning
paths all grow with them — a chart-edge texel is no longer trusted with a
skirt's light, a lamp only receives what the sky is not already delivering,
and lamp photometry comes from the fixture rather than the mesh scale. A
`walk_probe` example drives the navigation directly.

xr gets splat packing and a GPU splat sort (`splat_pack.rs`, `splat_sort.rs`,
with tests), and view_splat is largely rewritten on top of them.

remesh, xatlas, gltf, splat and sim carry the supporting work: the xatlas
unwrap hang is fixed and the pass is several times faster, the glTF writer
emits the rig and vehicle contracts, and sim grows the entity layer the
imported worlds drive.
2026-08-23 01:34:35 +02:00
Admin
016a171a35 libs/audio_*: MP3, Vorbis and Ogg of our own, plus lyric alignment and audio imaging
Another dependency the app should not be asking the platform for:

  - audio_decode — MP3 (layer 3, LSF tables, synthesis) and Ogg Vorbis
    (codebooks, floor, residue, MDCT) decoders, with tag reading. Both are
    checked against oracle fixtures rather than against our own expectations.
  - audio_encode — an Ogg Vorbis encoder: MDCT, psychoacoustics, floor and
    Huffman coding, setup tables, plus `oggenc` and `audiobench` binaries.
  - audio_picture — waveform and spectrogram rendering, and compositing.
  - audio_lyrics — word-level lyric alignment (DTW plus a DP snap) and the
    baked schema behind karaoke timing.
  - audio_sidechannels — the side-channel plumbing between them.

libs/voice grows a CUDA backend and an alignment path beside its CPU decoder,
with a `whisper_parity` binary to keep the two honest.
2026-08-23 01:34:35 +02:00
Admin
7f59912916 libs/ai: one AI stack, replacing libs/ggml, llama, mlx, cuda, tts, voice2 and pbr_paint
The model code was spread across eight crates that had grown into each other:
ggml and cuda and mlx each owned part of a tensor runtime, llama and tts and
voice2 each owned part of a model, and libs/diffusion owned everything else.
They are now one tree with an explicit shape:

  libs/ai/cuda     — kernels and launch surface
  libs/ai/metal    — Metal shaders and the shim
  libs/ai/llm      — the language-model runtime (sessions, lanes, contexts,
                     the CUDA and Metal executors, the compiled Metal path)
  libs/ai/models/  — common, flux, h3, music, paint, speech, stems, vision

libs/diffusion is not deleted but demoted: what remains is the VALIDATOR
crate — several dozen `*_validate.rs` oracles that check a native
implementation against a reference, which is where they belong now that the
implementations live next door.

The functional work inside the move is mostly in the LLM runtime: N lanes that
draft while one verify batch serves all of them, per-slot prefill over a shared
folded attention arena, speculation that survives batching, and a scheduler
that reports rather than publishes. And in the CUDA build: a machine without
usable CUDA must still LINK (and say so), the default kernel arch is the
building machine's GPU, `NO_CUDA` forces the stub even where the toolkit
exists, and kernels compile in parallel with progress.

libs/video_flow is new here: classical optical flow estimation and the `mkfl`
motion-field payload — a flow field measured from a clip without a model,
which is what drives free-rate bounce-looping playback and the uprez/tween
enhance pipe.
2026-08-23 01:34:35 +02:00
Admin
2d23dba736 libs/asset: the store runs on our own SQLite, and the importers learn the whole map contract
The asset store now uses libs/sqlite_query as its ONLY engine — not a feature
flag, not a fallback. That closes the Windows gap (the embedded store starts
there now, and a SHARED->EXCLUSIVE upgrade is handled rather than assumed
free) and takes the C dependency out of the build everywhere else.

Around it:

  - store: a garbage collector, catalogued content that is referenced in place
    instead of copied, the `vjeffect` kind, and host/chat routes that keep up
    with the chat wire below.
  - importer: the unified map contract reaches quake2, quake3, doom and duke —
    world placement, nav, welding, prelit maps and glTF node handling shared
    rather than reimplemented per game. Music import, billboards and stateful
    props move to the data crate so readers stop linking the importer.
  - ai: the serving side of multi-lane chat — per-lane conversations, honest
    progress and acceptance reporting, penalties and a watchdog, context as a
    per-box number that compacts instead of erupting, a realtime session mode,
    and inpaint/flux2 backends. `chat_bench` measures the rate the way the
    client meter computes it.
  - client / chat / chat_ui: a publication can NAME a file instead of carrying
    it; the wire says whether a turn is warm and whether it is thinking, so a
    client stops guessing; transcript and feed widgets render history the way
    the model wrote it. `SessionConfig::catalog_runtime` lets a host size the
    catalog runtime's lanes itself — a browsing UI puts every listing, every
    per-tile resolve and every thumbnail blob through that one runtime and
    wants a wider fast lane than the shared default, while media lanes keep
    it (a few big transfers, not a thousand small ones).
  - widgets: the shared asset widgets — one video view (knobbed seek,
    transport, bracket trim, rail playback) used everywhere, plus thumb,
    preview, scene view, walk-world and the lyric reader.
2026-08-23 01:34:34 +02:00
Admin
19c37a3df8 libs/sqlite_query: an SQLite engine of our own
A from-scratch, dependency-free SQLite implementation: file format reader and
writer (b-tree read and write paths, pager, journal, WAL), a SQL lexer,
parser and AST, a planner, and an executor — plus locking, integrity checking
and a `sqlq` CLI.

It exists because the asset store needs a database on every platform the app
ships to, without a C toolchain in the build and without a system library
whose version is somebody else's decision. The test suite is the argument:
DML, DDL, concurrency, crash recovery, a query corpus and a DML fuzzer, all
checked against real SQLite behaviour rather than against our own reading of
the spec.
2026-08-23 00:43:20 +02:00
Admin
7e929df04d widgets: DataGrid, ComboBox, Tip, ValueInput, DropSlider, and a widget tree that keeps up
Six new widgets, all built for the console-density end of the spectrum:

  - DataGrid — a 2D-virtualised grid (rows AND columns), the table/spreadsheet
    counterpart to PortalList. Cells host arbitrary widgets from templates.
  - Chart — trend and sparkline drawing that composes inside docks and grids,
    which the old DrawVector-based chart could not.
  - ComboBox — a text input with a filtered, keyboard-navigable popup list;
    type-to-filter rather than pick-from-a-menu.
  - Tip / TipLayer — system tooltips: a shared overlay layer, hover timing and
    placement handled once instead of per widget.
  - ValueInput — a numeric field you can also drag, Blender-style.
  - DropSlider — a slider that lives in a popover, for consoles with no room
    for a permanent one.

widget_tree gets the larger share of the changed lines: observation and
patching paths reworked so a structural rebuild is not the answer to every
change. PortalList picks up the scroll-distance readout and the dead-isolate
guard from upstream; window.rs grows maximize/restore forwarders; splitter
exposes a color hook; fold_header, scroll_bar, text_flow and drop_down2 get
follow-ups.
2026-08-23 00:43:20 +02:00
Admin
5dc8ef5256 platform: a remote control surface, streaming video codecs, and float render targets
An app built with `--remote` now serves a localhost HTTP control surface:
window list, per-window PNG grabs, real mouse/key/text injection, widget
rects, a log ring buffer, and `/gq` (grab every window, then quit). It exists
so a test or an agent can DRIVE a running app instead of reasoning about it
from source — the protocol is documented in AGENTS.md. Grabs are targeted per
window (`/g?w=N`), so a multi-window app is captured window by window rather
than whichever pass happens to present first, and `log!` mirrors into the ring
buffer without anyone owning the app's stdout.

platform/video grows a streaming half beside the file half. StreamEncoder /
StreamDecoder with Apple VideoToolbox and Windows Media Foundation backends,
Annex-B framing, and all-intra bound through pEncodingParameters on Windows —
the only control that MFT actually honors, as the readbacks claim success for
everything else. The file decoder can now be asked for a SPECIFIC frame rather
than only the next one, which is what frame-exact seek and bounce playback
need. Tests cover file seek and the stream round trip.

Draw shaders gain `Rgba16F` and `Rgba32F` color formats to pair with the
float render textures: blending off, whole-texel writes, meant for GPU
simulation state (particle position/velocity, fluid fields) rather than
pictures.

Windowing and dialogs:
  - `CxOsOp::SetChromelessWhenMaximized` drops the native maximized border
    strip on Windows, so a maximized window reads as a clean picture.
  - `Cx::open_select_folder_dialog` opens the native folder picker with a
    title and start location, answered by a `FileDialogAction` in the actions
    pass; cancelling is a first-class outcome, not an error.
  - Windows reports a user close the way macos.rs already did.
  - macOS swaps the titlebar container so WindowDragQuery alone decides window
    drags, and the delegates carry a panic shield.
  - `Windows::id_iter()` enumerates window slots generation-correctly.

Headless: the virtual GPU and its rasterizer are substantially rebuilt around
the shader runtime preamble, making `MAKEPAD=headless` render-to-PNG a real
test surface rather than a smoke check. `PerfMonitor::frames_painted()` lets a
scripted driver pace itself to PRESENTED frames instead of queueing passes
faster than the GPU retires them.
2026-08-23 00:43:20 +02:00
Kevin Boos
d223bf4697
Wayland: mark windows as created, fixing dpi override and pass dpi factor (#1188)
* Wayland: mark windows as created, fixing dpi override and pass dpi

Wayland was the only backend that never set `is_created = true` on the Cx
window; every other one does it in `CxOsOp::CreateWindow`.

That flag gates `Cx::dpi_override_scale()`, so every pointer event skipped
the native->layout remap and clicks missed their widgets by the UI zoom
factor. It also gates `get_delegated_dpi_factor()`, which was returning a
hardcoded 1.0 for every draw pass on Wayland, so pixel snapping and the
shader pixel size (SDF AA fringe) used the wrong scale on HiDPI screens.
`SetWindowVisuals` and `set_topmost` were dropped for the same reason.

Also seed the Cx window's geom at creation like the x11 backend does,
otherwise it sits at dpi_factor 0.0 until the first configure arrives.

* convert the seeded wayland geom to layout points and record os_dpi_factor

Seeding the raw native geom left window_geom in native units (and the
os_dpi_factor fallback unset) until the first configure arrived, which is
exactly the pre-configure window the dpi override needs to be correct in.
Do the same conversion the WindowGeomChange path already does.
2026-08-22 13:40:23 +02:00
Kevin Boos
94743e7687
Fingers: mark a second touch on an already-captured area as handled (#1187)
When an area that already captured a touch sees another touch start,
hits() returns a FingerDown for it (so the owner can handle multi-touch
gestures like pinch) but never marked the touch as handled. Widgets
behind the owner could then capture that second touch themselves: the
second finger of a pinch atop a fullscreen overlay could drag-scroll a
list behind it, or even press a button back there.

* mark such a touch as handled if it actually hit-tests within the
  area, mirroring the normal capture path below it
* only do so if nothing else has handled it yet, preserving the claim
  of a child widget that captured it earlier in the same dispatch
2026-08-21 08:49:10 +02:00
Kevin Boos
0d7816ea68
splash: a dead isolate's widgets must not call into the app VM (#1186)
Killing a Splash isolate while widgets it minted are still in the tree
could panic the whole process in the GC, somewhere else entirely:

    platform/script/src/gc.rs:300
    index out of bounds: the len is 12 but the index is 19

script_ref_vm_id() resolves a widget's owning VM from a ref the widget
holds. Isolate heaps live in heap_to_vm; anything else was assumed to be
the app VM. But an isolate's widgets outlive it by a frame or two — a
tile dropped mid-gesture, an app force-stopped while its buttons are
still on screen — and their refs were minted by a heap that is gone. So
those calls were routed INTO the app VM, which then stored a dead heap's
object ids in an args object of its own (make_call_args_object_with_
context). Nothing complained: the checked stores silently skip an index
they can't resolve. The next GC walked that object, indexed the app heap
with the other heap's index, and blew up with no trace of the cause.

Reclaimed heaps are now remembered, so a ref from one is told apart from
an app-VM ref and its call is dropped rather than redirected — the same
thing script_timer_dispatch_hook already does with a dead isolate's
timers. Three related tightenings while here:

- with_script_vm_id checks "is this VM already installed?" BEFORE the
  main-VM shortcut, and debug_asserts in the main branch. with_vm runs
  against whatever VM is parked on Cx, which during an isolate's own
  execution is that isolate's — so the old order could silently run
  app-VM work in an isolate's heap.
- gc_dead_splash_isolates purges the queues holding a dying isolate's
  values before dropping the heap those values live in, matching what
  gc_bridge above it already did.
- splash_host_respond checks it landed in the heap the request came
  from before minting the answer object there. One usize compare turns
  a future routing mistake into an undeliverable answer instead of a
  corrupted heap.

Found by a launcher that force-stops a mini-app for hammering the host
bridge: ~40 requests in flight, isolate torn down in the same event
pass, reopened moments later. That reproduced it every time; with this
it no longer does.
2026-08-21 08:48:59 +02:00
Admin
9daab06dcf DropDown2: fix type-mismatch errors applying instance() markers in the style block
The styled mod.widgets.DropDown2 block re-applied hover/active/up/enabled
as instance(...) markers onto the already-typed f32 fields of the draw
shaders, which fails at widget instantiation with "type mismatch:
expected f32, got object". The instance() declarations belong to the
script_shader type registrations; the style block just sets plain
float defaults.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 21:36:37 +02:00
Kevin Boos
04d5c5527a
PortalList: expose the total distance the user has scrolled (#1185)
Adds `PortalListRef::user_scroll_travel()`, the running sum of every
user-driven scroll delta: wheel/trackpad (including OS momentum), touch
drags, scroll-bar drags, and selection auto-scroll. Deltas absorbed at a
clamped edge still count, while the list's own movement (smooth scrolls,
fling coasting, bounce-back, tail-following) does not.

Sampling it at two points and subtracting tells a caller whether the
user scrolled in between and in which direction, which comparing
`first_id()` cannot do once items are inserted or removed.
2026-08-20 11:01:21 +02:00
Kevin Boos
5a251fba12
Shader: keep the value of an if body's last statement (#1182)
A bare `if cond { ... }` statement whose final body statement is a
non-void expression compiled to an empty `if(cond){ }`: the expression
never reached the generated shader, so its side effects were lost. In
robrix this silently removed every border drawn as

    if self.border_size > 0.0 {
        sdf.stroke(self.border_color, self.border_size)
    }

A call is not written to the output when it is compiled, it is pushed on
the stack as a string, and it only reaches the output via POP_TO_ME.
Since e0a5a23f2 the enclosing statement's POP_TO_ME is emitted as a
standalone opcode at the if's jump target instead of being fused onto the
body's last call, and the shader compiler closes an `IfBody` as soon as
`ip >= target_ip`, so the opcode sitting exactly at the target is never
seen while the body is open. The body's value was then dropped on the
floor by the `no outer phi` arm, whose comment assumed that could not
happen.

* Emit the leftover value as a statement inside the branch when nothing
  consumes it, mirroring the void path a few lines above.
* Add `gpu_stage_4m`, which asserts the call survives into the generated
  shader. It fails without the fix.

The parser side is deliberately untouched: `last_jump_target` is load
bearing for the widget-loss fix that `on_render_emission` guards.
2026-08-19 20:06:12 +02:00
Kevin Boos
6dd0b2c133
Splash: a host-services bridge for mini-apps, plus three VM/parser fixes (#1181)
* Splash: a host-services bridge so isolates can ask for brokered capabilities

Mini-apps are sandboxed hard (fs/run/res stripped, net gated), which also
means they can't do anything real. This adds the one doorway back: a
mod.host module in every isolate whose host.request(service, args, cb)
queues {app_tag, heap_key, req_id, service, args_json} on a thread-local
the EMBEDDING HOST drains and answers (splash_host_respond re-enters the
isolate under the normal budget and calls the callback with {ok, data,
error}). No policy lives in makepad: an undrained request never resolves,
tags are host-assigned (Splash::set_host_tag) so scripts can't spoof who
they are, and host.capabilities() just echoes whatever grant list the
host last pushed (set_host_caps). Callbacks are rooted ScriptFnRefs keyed
by heap, GC'd with the isolate alongside the storage-jail roots.

Also: call_script_fn_with_strings (string args must be minted in the
callee's own heap), 'let host = mod.host' in both Splash prefixes (line
offsets documented per prefix; the net prefix was already one line off),
and mod.cx.quit is now nil'd in isolates - a mini-app could quit the
whole host process with one call.

* script: stop validation from blessing scripts that failed to parse

The parser RECOVERS from errors (dangling else, missing expression), logs
them, sets had_error - which nothing ever read - and hands back a runnable
module. Nothing enters the trap queue, so a host validating with a
captured_errors sink + take_errors() got an empty list and reported
success; three freshly-written mini-apps shipped real parse errors straight
through host_launcher's validate this way, visible only as stray [E] log
lines.

report_error now also records the formatted message on the parser
(ScriptParser::errors), and both eval paths (eval_with_source and the
streaming eval_with_append_source) drain that into bx.captured_errors when
a sink is installed. No sink = logs only, exactly as before. Regression
tests in tests/parse_error_capture.rs, including the exact fn-final
if/else shape that slipped through.

* splash_host: review fixes — is_ok result field, silent surfaces, JSON hardening

Three classes of fixes from an adversarial review of the bridge:

- The result object's success field is now is_ok. 'ok' is the script
  dialect's ok-test KEYWORD, so r.ok never parsed as a field access — every
  callback that read it silently died. A pure-VM regression test
  (fn_ref_callback.rs) now exercises the exact store-callback-then-answer
  flow the bridge uses.

- SplashHostRequest carries may_prompt, set per isolate via
  Splash::set_host_prompts: background surfaces (home-screen widget tiles)
  are marked silent so a host can fail their permission-needing requests
  instead of popping consent dialogs nobody asked for. splash_host_respond
  also reports an outcome now (Delivered / NoCallback / IsolateGone) so
  hosts can log undeliverable answers, and Splash::isolate_heap_key lets a
  host relate a request to a specific widget (IPC fan-out skips the
  sender's own isolate with it).

- heap.to_json hardening: a cyclic object graph (script-buildable, host-
  serialized on every bridge request) recursed to a stack overflow — now a
  depth cap emits null leaves; backslashes were mis-escaped as a single
  backslash (invalid JSON downstream), tab and other control chars weren't
  escaped at all, and a handle serialized as unquoted junk.

* script: a closure's captured varargs must not shadow its own parameters

A call binds positional args by INDEXING the fn object's vec, which holds
declared parameters — but also, past that, any varargs the call received
(unnamed_fn_arg pushes them with a NIL key). A closure captures the scope
it was minted in, so those leftovers ride along ahead of the closure's own
parameters.

Concretely: script timers invoke their callback with one number (the time).
Hand start_timeout a zero-arg closure and that number lands in the scope as
a NIL-keyed vararg; any closure created in that body then binds its FIRST
parameter against the leftover — first failing the typecheck ("arg 0 (nil)
type mismatch: expected number, got object"), and once that was relaxed,
binding the value under the NIL key so the real parameter stayed nil. It
cost a full debug cycle in host_launcher, where every host-service callback
created inside a boot timer silently never ran.

Both binding paths now walk the DECLARED (named) entries in order, so
captured varargs can never be mistaken for a parameter. Regression test in
tests/extra_arg_typecheck.rs reproduces the timer shape exactly.

(Pre-existing and unrelated: widget_tree's test_observe_and_find_single_node
and test_property_patch_no_structural_rebuild fail on upstream dev too.)

* script: stop parse_json silently dropping negative numbers

The tokenizer emits a leading `-` as its own Operator token, and none of
the three JSON value positions (object value, array element, root) had a
case for it. The sign was swallowed — and inside an object the KEY went
with it, because the minus consumed the value slot and the parser resynced
on the next token.

So `{"lat":37.7,"lon":-122.4}` parsed to `{"lat":37.7}`. No error, no
warning, just a missing field. That is how it was found: a mini-app asked
the host where it was, got coordinates with no longitude, and quietly fell
back to a default city. Sub-zero temperatures and negative UTC offsets
(New York is -14400) were being dropped the same way.

A pending-sign flag is applied to the next number in all three positions.
Bare scalar roots stay unsupported ("42" never parsed either) — separate
pre-existing gap, not touched here. Tests in
platform/script/tests/json_negative_numbers.rs.

* splash_storage: let the host raise a single isolate's jail quota

The jail's 16MB whole-app cap is a constant, so "this app may keep more
than the standard amount" had nowhere to live. A per-heap quota map beside
SANDBOX_ROOTS gives the host one, set through Splash::set_storage_quota
and cleared with the isolate like every other per-isolate binding. Script
still can't see or raise its own cap.

Lowering a quota never deletes anything — it just stops further growth —
so revoking the grant is safe on an app that already wrote past the
default.

host_launcher uses this for a `storage-large` permission (64MB), which is
the point: a capability the user can revoke and have it actually mean
something.
2026-08-19 20:06:01 +02:00
Jason Yau
2c49150e3a
Don’t panic on a stale Area::Rect (#1184)
Co-authored-by: jasonqiu <jasonqiuchen@outlook.com>
2026-08-19 13:50:04 +02:00
Admin
6d708d9391 Clean the public tree for a checkout other people can use.
Drop the archived old/ tree, root AI notes, splashgame.md (Arcade lives
in the private sandbox), widgets-dll leftovers, and LAN/oracle helper
scripts. Move download_*.sh into tools/, and move sim/math out of
libs/game/ so the public repo no longer has a game/ directory.

Also quiet first-party compile noise and skip asset-ui tests that need
sandbox kits or uncommitted fixtures.
2026-08-18 15:23:19 +02:00
Admin
5421bc71c6 Remove widgets-dll, hotload_ui, and the weezl tokio test.
hotload_ui and xr pulled a dylib widgets wrapper that made
`cargo build --workspace` fail. XR now uses normal widgets.
The vendored weezl async test needed tokio, which we do not ship.
2026-08-18 14:45:09 +02:00
Admin
3b1ed5107b Drop unused examples from the public tree.
Remove godot, scratchpad, arracing, comfyui, exf, ddgo, and git.
Flux workflow JSON used by diffusion tests moves into libs/diffusion.
2026-08-18 14:41:52 +02:00
Admin
765f4785fc Let a clean makepad checkout load and compile.
Drop the private sandbox clone from required workspace members
and Studio runnables. Finish the platform_ops VecDeque merge
(push_back / Option remove), land the mip-repeat texture API
the renderer already calls, and unbreak the Q3 importer plus
the godot example template that .gitignore had hidden.
2026-08-18 14:36:05 +02:00
Admin
f0e1f83831 Close the rik2 land: workspace, splash, and sandbox cleanup.
Point the workspace at libs/asset and libs/render. Drop Arcade
frontend, gamemaker, rig, pack registry, physics, and the old
libs/game gameplay crates. Those live in the private sandbox repo.
2026-08-18 14:24:00 +02:00
Admin
bdc7c0a227 Land Asset UI and VJ from rik2.
Catalog/import/viewers and the live performance app. They use
libs/render preview play, not the Arcade sim crate as a dep.
2026-08-18 14:24:00 +02:00
Admin
7437b08f0e Land asset-ai and the pack/classic importer from rik2.
Fleet generate service and licensed/classic pack compiler.
2026-08-18 14:24:00 +02:00
Admin
8436f955d6 Land the Asset Server protocol stack from rik2.
data, client, store, and chat — the contract Asset UI and VJ speak.
2026-08-18 14:23:59 +02:00
Admin
3f1da40e89 Land native diffusion and TTS backends from rik2.
Flux2, SAM3, Music3, ACE, paint, DA3, RealESRGAN, Trellis, motion.
2026-08-18 14:23:59 +02:00
Admin
8fce54a7a8 Land the house Art-Net/DMX desk from rik2.
libs/show_control plus the automate example. Not Arcade gameplay.
2026-08-18 14:23:59 +02:00
Admin
55dde7b02a Land the 3D renderer and leftover sim/math from rik2.
libs/render (Renderer, SceneDraws, preview play) plus the Arcade
sim/math it still draws through. Gameplay crates stay out.
2026-08-18 14:23:59 +02:00
Admin
fbe877d746 Land glTF writers, remesh, PBR paint, and xatlas from rik2.
Animation/skin writers, COLOR_0, and the mesh tools the renderer uses.
2026-08-18 14:23:58 +02:00
Admin
27d0b1179d Land ggml, llama, and Metal/CUDA tensor runtimes from rik2.
GpuTensor, QMM, k-quant, and op-parity tests the gen backends sit on.
2026-08-18 14:23:58 +02:00
Admin
2a46d2a405 Land platform, studio, and widget infra from rik2.
HTTP progress, OS file drag, RunView controllers, DropDown2, video,
remote process helpers, and the Studio runbook.
2026-08-18 14:23:57 +02:00
4507 changed files with 1748049 additions and 523868 deletions

43
.gitignore vendored
View file

@ -2,6 +2,12 @@
# will have compiled files and executables
**/target/
# Private Arcade frontend. Clone into this path:
# git clone git@github.com:makepad/sandbox.git apps/sandbox
/apps/sandbox/
# git clone git@github.com:makepad/source-library.git apps/source-library
/apps/source-library/
# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
Cargo.lock
@ -38,6 +44,7 @@ GOOGLE_KEY
experiments/websocket-windows/nodejs-websocketserver
experiments/svgloader
CLAUDE.md
!examples/godot/resources/template/CLAUDE.md
/tools/__pycache__/
**/*.obsidian
@ -80,3 +87,39 @@ tesla_credentials.json
/route.md
/shiny.md
/status.md
# Baked AO atlases: regenerate with tools/ao_bake, never commit (41 MB).
**/ao_atlas.png
# LAN fleet configs — machine-local, never committed
fleet-hosts*.txt
# Local secrets and AI-session state — never commit
/GOOGLE_API_KEY
/aichat_history.json
/godot_chat_history.json
/codex2
/hello_world.py
/package-lock.json
/mapplan.md
/mix.md
/overlay.md
# Local data dumps, benches and scratch checkouts — never commit
/*.csv
/*.mbtiles
/kokoro_ref*
/silero_vad.onnx
/.rustup/
/build/
/box3d/
/sub/
/refdump/
/traces/
/xr/dump/
/examples/splash_preview/
/examples/automate/local/
/examples/cad/generated/
window_*_frame_*.png
remote-jobs/
apps/asset-ui/resources/

412
AGENTS.md
View file

@ -1,113 +1,297 @@
# Studio Remote Runbook
# Makepad Agent Runbook
> **Driving a running app: use `--remote`.** Every makepad app started with
> `--remote` serves a tiny localhost HTTP control surface: window list, PNG
> grabs, real mouse/key/text injection, widget rects, log tail, graceful quit.
> It replaces `screencapture -l`, `winid.swift`, CGEvent scripting and the
> studio websocket bridge for all agent work. Full spec: [App Remote Control](#app-remote-control---remote).
## Execution Policy
- Visual UI programs must be launched and controlled through the Makepad Studio remote protocol.
- Always use release builds for runtime validation, profiling, benchmarks, timing checks, or any performance-sensitive command. Use `--release` unless the user explicitly asks for a debug build.
- Do not use mount observation or runnable discovery from the bridge client. The bridge must not claim mount ownership from Studio desktop.
- Do not launch UI programs with raw `cargo run`, `cargo makepad`, or ad hoc cargo invocation when a runnable item exists.
- Do not use bridge `Cargo` requests to run applications. Only launch apps from runnable items via bridge `RunItem`.
- For UI runnable targets, do not prebuild or precheck the app from the shell before launching it in Studio. Let the Studio `RunItem` build be the single build path so Cargo fingerprints, env vars, target dirs, and flags stay identical.
- Before starting a new UI run for the same target, send `ClearBuild` for the previous build so Studio stops it and removes its run/log/profiler tabs.
- `cargo check` or `cargo build` never counts as UI verification. After changing UI/runtime code, you must clear the old build and start a fresh Studio run before trusting screenshots, widget dumps, or interaction results.
- Do not keep inspecting an older already-running app after code changes. Re-run the target and verify against the new `build_id`.
- Command-line-only tasks (builds, tests, linting, file ops, grep/ripgrep, etc.) can be run directly in the shell.
- Prefer studio remote control for any workflow that needs screenshots, widget queries, clicks, typing, or runtime UI inspection.
- Before using Studio protocol tools (`FindInFiles`, `ReadTextRange`, `WidgetTreeDump`, `WidgetQuery`, `Screenshot`, `Click`, `TypeText`, `Return`), always start one persistent Studio remote bridge process and reuse it for the entire interaction.
- When adding a new example crate, update both the Cargo workspace and `makepad.splash` so Studio exposes the new example as a runnable item.
## Assumptions
- Studio is started manually by the user.
- Studio remote target is `ip:port` only (no `http://`, no `ws://`), normally `127.0.0.1:8001`.
- Use `127.0.0.1:8002` only if Studio reports fallback because `8001` is occupied.
- Keep one persistent studio remote process for the whole interaction.
- **Designs stay local.** Design documents, plans, and reports are local files
(`local/agent_state/<topic>/DESIGN.md`) that lanes read from disk. Never
publish them to the web (no Artifacts, no hosted pages); summarize in the
terminal instead.
- Launch UI programs as standalone release binaries from this checkout. Do
not use the Studio remote bridge, `ObserveMount`, `RunItem`, or any
`cargo-makepad studio` websocket client.
- Launch with `--remote` whenever you intend to look at or drive the app,
and finish with `GET /gq`. **Nothing of yours may outlive your task**
never leave a test window on the user's screen.
- Always use release builds for runtime validation, profiling, benchmarks,
timing checks, or any performance-sensitive command. Use `--release`
unless the user explicitly asks for a debug build.
- Build with `cargo build --release -p <package>`, then launch the
resulting executable so its provenance is unambiguous. Do not use raw
`cargo run` / `cargo makepad` to start a UI you will keep inspecting.
- Stop or replace an older standalone instance of the same target before
launching a freshly built one.
- Keep an interactive standalone app running when the user asks to play
with it. Use a separate self-terminating capture run only when a
screenshot is also needed.
- `cargo check` or `cargo build` never counts as UI verification. After
changing UI/runtime code, rebuild and relaunch before trusting what you
see. Do not keep inspecting an older already-running binary.
- Command-line-only tasks (builds, tests, linting, file ops, grep, etc.)
can be run directly in the shell.
- A standalone app's built-in screenshot/capture hook is valid for visual
inspection.
- **System-level screenshots are FORBIDDEN.** Never run `screencapture`,
`CGWindowListCreateImage`/`CGDisplayCreateImage` scripts, `xcap`,
`import`, `scrot`, `grim`, `xwd`, PowerShell/Win32 screen grabs, or any
other OS screen capture — not of the display, not of a window, not
"just the caption". The user's screen is private. The only image of a
running app you may ever take is the app's own `--remote` grab (`/g`,
`/gq`, `/tweak/grab`), which renders the app's own drawable and nothing
else. If something only shows in the OS layer (native caption buttons,
other apps, the desktop), ask the user for a screenshot instead of
taking one.
- When adding a new example crate, update both the Cargo workspace and
`makepad.splash`.
- **Zero locking on the UI thread, one mechanism everywhere.** The UI
thread never takes a `Mutex`/`RwLock`/`Condvar` that another thread can
hold, and never blocks on a channel. UI → workers/audio is commands over
a channel (bounded, non-blocking send; a full queue is reported and
retried next frame). Workers/audio → UI is snapshots over atomics, a
triple buffer, or a channel read with `try_recv`. Large payloads (PCM,
stems, grids, images) travel as `Arc` through the channel; a replaced
payload is handed back so the UI thread does the drop, never a realtime
thread. A realtime callback (audio) owns its state, never takes a lock
the UI can hold, never allocates on the hot path. This is ONE code path
for native and wasm — no `cfg` fork where desktop keeps shared mutexes.
On wasm both the browser UI thread and the AudioWorklet thread abort on
`Atomics.wait`, and a spinning fallback against a busy audio callback is
a 100 % CPU feedback loop that kills audio and frame rate together (DJ
web, 2026-09-03). `lock_from_ui` is only acceptable on state provably
touched by the UI thread alone.
- **Standard operating flow — who does what.** The main session (Fable)
designs, briefs, manages and reviews; it does not write the code itself
except one-line fixes. **Codex writes the code**: every implementation lane
is a Codex lane with a precise brief (observations, files, rules, the
verification commands) launched through `local/tools/delegate` /
`local/agent_state/webdemos/tasks/queue.sh` and landed through
`local/tools/integrate`. **Grok does the tests and the token-heavy work**:
test suites, audits, surveys, log reading, conflict resolution passes,
reviews of large diffs (`delegate grok` / `research-grok` / `review-grok`).
A Fable subagent is the exception, only for a design-level change the
other two cannot carry (a new platform mechanism), and it stops as soon as
the API is fixed so Codex can do the conversions. Keep at most six lanes
per provider; land everything through the integrator; the user tries the
result — no routine captures.
- **No temporary threads — use the pool.** Never spawn a thread for one
job (`std::thread::spawn` is unsupported on wasm anyway; the platform
spawner works everywhere). Background work goes to the platform thread
pool (`cx.thread_spawner()` / the pool `TaskHandle` API) or to a
long-lived worker created once at start-up and fed over a channel. On
the web a Web Worker takes hundreds of milliseconds to come up, so a
per-job thread is a stall; on desktop it is still churn. One mechanism
on both targets.
## Start Studio Remote
- Command:
- `target/release/cargo-makepad studio --studio=127.0.0.1:8001`
- Send newline-delimited JSON requests on stdin.
- Read newline-delimited JSON responses on stdout.
- Protocol shape is raw `ClientToHub` requests on stdin and filtered `HubToClient` responses on stdout.
- Do not send `ObserveMount` from the bridge. It can take `primary` UI ownership for the mount and divert RunView/framebuffer traffic away from Studio desktop.
## Standalone Launch
1. `cargo build --release -p <package>` from this checkout.
2. Kill any older process of that same executable.
3. Run `target/release/<bin> --remote` from the repo root (so resource paths
resolve), parse the port from the startup line, drive it over HTTP.
4. After code changes, repeat 13 before drawing conclusions.
5. `GET /gq` when you are done. Always.
## Request Protocol (JSON Lines)
- `{"ListBuilds":[]}`
- `{"ClearBuild":{"build_id":[6]}}` stops a running build and immediately clears its Studio UI tabs; use this before rerunning the same app.
- `{"StopBuild":{"build_id":[6]}}` stops/kills a running build but does not clear Studio tabs.
- `{"RunItem":{"mount":"makepad","name":"makepad-example-todo"}}`
- `{"RunItem":{"mount":"makepad","name":"makepad-example-xr-quest"}}`
- `{"FindInFiles":{"mount":"makepad","pattern":"ClientToHub::","is_regex":false,"glob":null,"max_results":200}}`
- `{"FindInFiles":{"mount":"makepad","pattern":"ClientToHub::(FindInFiles|ReadTextRange)","is_regex":true,"glob":"**/*.rs","max_results":200}}`
- `{"ReadTextRange":{"path":"makepad/studio/backend/src/dispatch.rs","start_line":640,"end_line":720}}`
- `{"WidgetTreeDump":{"build_id":[6]}}`
- `{"WidgetQuery":{"build_id":[6],"query":"id:todo_input"}}`
- `{"Screenshot":{"build_id":[6],"kind_id":0}}` (`kind_id` optional; defaults to `0`)
- `{"Click":{"build_id":[6],"x":1274,"y":342}}`
- `{"TypeText":{"build_id":[6],"text":"hello"}}`
- `{"Return":{"build_id":[6],"auto_dump":false}}`
- `{"ForwardToApp":{"build_id":[6],"msg_bin":[...]}}` (advanced; binary payload)
## App Remote Control (`--remote`)
## `StudioToApp` API (Updated)
- The studio remote bridge supports raw app event passthrough via `UIToStudio::ForwardToApp`.
- Current `StudioToApp` variants include:
- `Screenshot`, `WidgetTreeDump`, `KeepAlive`, `LiveChange`, `Swapchain`, `WindowGeomChange`, `Tick`
- `MouseDown`, `MouseUp`, `MouseMove`, `Scroll`
- `KeyDown`, `KeyUp`, `TextInput`, `TextCopy`, `TextCut`
- `None`, `Kill`
- Use direct studio remote requests (`Screenshot`, `WidgetTreeDump`, `Click`, `TypeText`, `Return`) for normal automation.
- Use raw `StudioToApp` only for low-level event injection/debugging.
> **Focus law.** A `--remote` app opens its window VISIBLE BUT UNFOCUSED and
> stays that way: it never activates, never becomes key, and bridge clicks
> never raise it. The user keeps typing wherever they were. Everything the
> bridge does (grabs, `/m`, `/k`, `/t`, `/snap`) works without focus because
> input is injected through the app's event loop, not the OS. Do not work
> around this (`MAKEPAD_FOCUS=1` exists only for a run the user asks to see
> in front); `MAKEPAD_NO_FOCUS=1` gives a non-remote launch the same manners.
## Response Notes (Current)
- Bridge stdout is filtered to: `Hello`, `Error`, `TextFileRead`, `TextFileRange`, `FindFileResults`, `SearchFileResults`, `Builds`, `RunItems`, `BuildStarted`, `BuildStopped`, `BuildCleared`, `AppStarted`, `RunViewCreated`, `QueryLogResults`, `Screenshot`, `WidgetTreeDump`, `WidgetQuery`, `QueryCancelled`.
- `BuildCleared` is a Studio frontend cleanup signal routed to the primary UI for the build's mount; bridge clients should not wait for it before starting the next run.
- `RunViewFrame` and the terminal stream are not exposed by the bridge.
- `Screenshot` responses include file metadata (`path`, `width`, `height`) and not inline PNG bytes.
- `WidgetTreeDump` responses include text dump content keyed by `request_id`.
- `FindInFiles` responds as `SearchFileResults` with concise entries (`path`, `line`, `column`, `line_text`) and `done`.
- `FindInFiles` defaults to searching only `.rs`, `.md`, `.toml` files unless `glob` is provided.
- `ReadTextRange` responds as `TextFileRange` with `path`, requested `start_line`/`end_line`, `total_lines`, and `content`.
- Query-scoped responses are lane-filtered by `query_id.client_id`; only this bridge client's query results are emitted.
- Build ids and query ids are `QueryId` tuple structs, so JSON encodes them as one-element arrays like `[6]`.
- `FindInFiles`/`SearchFiles` execution is worker-pooled in backend (not main dispatch thread).
> **Who may open a visible window.** Subagent/lane verification runs HIDDEN:
> launch with `MAKEPAD_HIDE_WINDOWS=1 <bin> --remote` — the window never
> appears, grabs (`/g`), `/snap`, `/m`, `/k`, `/t` all still work offscreen.
> Only the integrating session opens the one visible, unfocused window the
> user watches; several look-alike windows on screen made the user "go
> insane" (2026-08-26).
## Recommended Control Flow
1. Start studio remote process once.
2. Determine the target runnable item name locally from the repo or from the user request.
3. Call `ListBuilds` and find any existing build for the same runnable item.
4. Send `ClearBuild` for that old `build_id`; do not wait for an acknowledgment before the next launch.
5. Start the new UI app through `RunItem`, and wait for `BuildStarted` and `AppStarted`.
6. After any code change that affects runtime/UI behavior, repeat steps 3-5 before doing screenshots, widget dumps, clicks, or visual conclusions.
7. For code search, use `FindInFiles` first, then `ReadTextRange` to window exact regions.
8. Use direct shell cargo commands for non-launch tasks such as `check`, `build`, `test`, or `bench`.
9. Use `WidgetQuery` / `WidgetTreeDump` to get click targets.
10. For text input, click field first, then send text, then return.
11. Keep control packets compact (`auto_dump:false` on click/type/return for low latency).
## `RunItem` Launch
- `RunItem` executes a Studio-defined runnable item by name.
- Use the runnable item name shown in Studio, not a Cargo package name.
- `RunItem` does not implicitly replace an older build tab; agents should clear the old build themselves first with `ClearBuild`.
Any makepad app launched with `--remote` runs a localhost HTTP server inside
the process and prints one line before the UI appears:
## One-Flow Input Burst
- Send this as one stdin write (multiple JSON lines, no sleeps):
- `Click` (input field center)
- `TypeText`
- `Return`
- Then request `WidgetTreeDump` or `Screenshot` to confirm.
```
[makepad-remote] listening on 127.0.0.1:53412 pid=9931 app=makepad-example-splash grabs=/var/folders/…/T/makepad-remote/makepad-example-splash-9931
```
## Coordinates
- Use coordinates from dump as-is.
- `W3` dump uses integer pixel coordinates in the same space expected by `Click`.
- Do not apply extra DPI math in the agent loop.
Port, pid, app name and the grab directory — everything needed to drive and
clean up the instance, with no discovery step. `--remote=PORT` pins the port;
`MAKEPAD_REMOTE=1` (or `=PORT`) does the same via the environment. No app code
is involved: it lives in `app_main!`, so every app gets it for free.
## Reliability Notes
- `Screenshot` can arrive before visible redraw after rapid input bursts.
- If screenshot looks stale, request a follow-up `WidgetTreeDump`/`Screenshot`.
- If input does nothing:
- Verify `build_id` with `ListBuilds`.
- Refresh dump and retry click on input before typing.
- If request errors with no active websocket:
- app is not connected yet; wait for startup completion and retry.
### Cheat sheet
Every route is a plain `GET`. Every answer is **one line of JSON** with short
keys and real numbers. Errors are `{"err":"..."}` with HTTP 404.
`GET /` returns this table as plain text, so an agent that finds the port
learns the whole API in one request.
| Route | Answer | Notes |
|---|---|---|
| `/` `/help` | plain-text cheat sheet | self-describing; read this first |
| `/s` `?w=ID` | `{"app":…,"pid":…,"w":[{"i":0,"t":"Title","sz":[w,h],"px":[w,h],"dpi":2,"pos":[x,y]}]}` | `sz` = layout points, `px` = physical pixels |
| `/g` `?w=&scale=&raw=` | `{"png":"/abs/path.png","w":0,"sz":[w,h]}` | writes a file and returns the **path** (agents read images as files). `raw=1` sends `image/png` bytes instead. `scale=0.5` halves it |
| `/gq` `?w=&scale=` | `{"png":[paths…],"quit":1}` | **grab every window, then quit.** The canonical last call of a session |
| `/m` `?k=&x=&y=&w=&b=&dx=&dy=&wait=` | `{"ok":1,"f":frame}` | `k=move\|down\|up\|click\|scroll`; `b=0` left, `1` right, `2` middle |
| `/click` `?x=&y=&w=&wait=` | `{"ok":1}` | alias for `/m?k=click` (move + down + up) |
| `/k` `?t=TEXT` or `?k=down\|up\|press&c=CODE` | `{"ok":1}` | `t=` goes through the IME text path; `c=` takes `KeyA`/`a`/`enter`/`Escape`/`ArrowLeft`/`F1`/`Key1`… plus `&shift=1&ctrl=1&alt=1&cmd=1` |
| `/t` `?t=TEXT` | `{"ok":1}` | same as `/k?t=` |
| `/snap` `?q=&w=&all=` | `{"s":[{"i":"id","ty":"Button","r":[x,y,w,h],"w":0,"t":"Click me"}]}` | **how you find things to click.** `q=` filters id/type/text; rects are window-local, ready to feed to `/click` |
| `/d` `/dump` | plain text widget tree | one indented line per widget, ending `x y w h` |
| `/log` `?n=50&since=N` | `{"n":lastseq,"l":["[E] …"]}` | ring buffer of the app's own log output — see errors without owning stdout |
| `/close` `?w=ID` | `{"ok":1}` | closes one window the normal way |
| `/quit` | `{"ok":1}` | graceful shutdown, no final grab |
Add `&wait=1` to any input route to have it answer only **after the next frame
is drawn**, so a following `/g` sees the result with no `sleep`.
Add `&w=ID` to target a window; omit it for the first one.
`POST` the same routes with a flat JSON body (`{"x":10,"y":20}`) when quoting a
query string is painful; the key names are the long ones (`window`, `kind`,
`button`, `text`, `code`).
### The standard pattern
```bash
cargo build --release -p makepad-example-splash
./target/release/makepad-example-splash --remote > /tmp/app.log 2>&1 &
sleep 4
P=$(grep -o 'listening on 127.0.0.1:[0-9]*' /tmp/app.log | grep -o '[0-9]*$')
curl -s "http://127.0.0.1:$P/s" # {"app":…,"w":[{"i":0,…}]}
curl -s "http://127.0.0.1:$P/snap?q=press_demo" # find the button's rect
curl -s "http://127.0.0.1:$P/click?x=352&y=472&wait=1"
curl -s "http://127.0.0.1:$P/snap?q=press_status" # assert the app reacted
curl -s "http://127.0.0.1:$P/log?n=20" # any errors?
curl -s "http://127.0.0.1:$P/gq?scale=0.5" # final PNGs + quit
```
Read the returned `png` path with your image tool. `tools/remote_smoke.sh` is
this pattern as an executable end-to-end test across three example apps.
### Rules
- **Close what you open.** When you are done with an instance you launched,
`GET /gq` (or `/close` each window, then `/quit`). Never leave test windows
on the user's screen, and never `pkill` when the protocol is available.
- **Never touch an instance the user is running.** Launch your own.
- **`/g` is the only camera.** No OS-level screen capture of any kind (see
Execution Policy) — the remote grab is what you get.
- **A vanished window or app with `[makepad-remote] user closed …` in the log
means the human dismissed it — it was in their way.** Do **not** treat that
as a crash and do **not** relaunch it. The app prints
`[makepad-remote] user closed window 1 ("Inspector Panel")` and, when that
was the last window, `[makepad-remote] app exit: user closed the last
window`. Both lines go to stdout with or without `--remote`, and into the
`/log` ring. While the app lives, `/s?w=1` on such a window answers
`{"err":"window 1 closed by user"}` rather than "no window 1".
- **`--remote` windows are tagged.** Their title gets a ` [remote]` suffix
(both the OS title bar and makepad's own caption bar) so a human who finds
one lingering knows it is an agent instance and can close it guilt-free.
`--remote-title-tag=NAME` changes the tag; `--remote-title-tag=off` removes
it.
### Semantics worth knowing
- **Coordinates** are layout points, window-local, y down — the same space
`MouseDownEvent.abs` uses, and the same space `/snap` reports rects in. No
dpi maths: a rect from `/snap` goes straight into `/click`.
- **Window ids** are stable `usize` slots (`/s` `"i"`). Every window-targeting
route takes `w=`; omitting it means the first created window. A request for
a window that never existed 404s with `{"err":"no window 3"}`.
- **Input takes the real path.** Events are injected through
`Cx::dispatch_studio_msg`, the same function the studio bridge uses, with
the same `fingers` bookkeeping — so hits, capture, tap counts and gestures
behave exactly as they do for a human. `/click` sends move + down + up so
hover-dependent widgets see what they expect.
- **Grabs are real frames**, read back from the window's own presented
drawable on the frame after the request (the studio screenshot pipeline,
extended with per-window targeting). The UI thread is never blocked; the
HTTP thread waits. Grabs are written to
`$TMPDIR/makepad-remote/<app>-<pid>/grab-w<window>-<seq>.png`, monotonically
numbered, with the last 32 per window retained.
- **Backends:** macOS/Metal is fully supported. Linux GL and Vulkan support
grabs too. Windows/D3D11 has no screenshot readback yet, so `/g` there times
out with `{"err":"grab timeout …"}` while every other route works. Android,
OHOS and wasm compile to a no-op.
- **Cost when idle is zero.** The event loop only upshifts its paint clock
while a remote request is in flight.
### The TWEAKER (`/tweak/*`) — design feedback and live styling
Every `--remote` app carries a design-feedback overlay (plan of record:
repo-root `tweaker.md`; implementation: `widgets/src/tweaker.rs`). Off it
costs nothing. On, the person (or you) points at the UI: pointer events over
the window body are swallowed before widget dispatch — **clicking a Button in
tweak mode outlines it and never fires it** — and the window grows a property
sidebar next to the (compressed) app UI. Shift+F10 toggles it in-app; every edit,
theirs or yours, lands in one shared diff log.
| Route | Answer | Notes |
|---|---|---|
| `/tweak` `?on=1\|0&annotate=1\|0` | `{"on":1,"annotate":0}` | toggle the overlay / the freehand draw mode (Alt-drag draws too) |
| `/tweak/state` | `{"on":1,"sel":{path,ty,r,band},"props":[{n,v,set}],"hover":…,"diff":[…],"ann":[…]}` | the STRUCTURE feedback: pinned selection, its real reflected properties (`set:1` = explicitly applied), the edit log, annotation strokes with the widget paths they touch |
| `/tweak/apply` (POST) | `{"ok":1,"path":…,"changed":[{path,prop,old,new}]}` | body `{"path":"a.b.c","splash":"{padding: Inset{left: 20}}"}` or the one-property shorthand `{"path":…,"prop":"draw_bg.border_radius","value":"8"}`. Evaluates the chunk onto that ONE instance through the ordinary apply machinery (`+:` merge rules intact) and triggers a full relayout. Answers after the next drawn frame |
| `/tweak/diff` | `{"diff":[{path,prop,old,new}…]}` | the raw edit log, in order |
| `/tweak/clear` | `{"ok":1}` | reset diff + annotations |
| `/tweak/final` | `{"final":[…coalesced…],"ann":[…],"drew":0\|1,"png":path?}` | **read this when tweaking is done**: per (path, prop) only the original and final value, churn collapsed. When the user drew, `png` is the composited screenshot — look at it, the strokes mean something |
| `/tweak/grab` | like `/g` | the overlay (outlines, strokes, sidebar) draws in the window's own pass, so any grab is already composited |
`local/tools/tweak` wraps all of this:
`tweak PORT on`, `tweak PORT state`, `tweak PORT apply PATH PROP VALUE`,
`tweak PORT splash PATH 'CHUNK'`, `tweak PORT final`, …
**How to listen.** Sidebar edits push to you: each one emits a marked
`TWEAK sidebar <path> <prop> <old> -> <new>` line into the app log — the
`/log` tail is your ear; you never poll `/tweak/state` for changes. Talk back
on `/tweak/apply` (values or whole shader chunks) to the same selected
instance.
**Write-back (you do this part — the overlay never writes source).** When the
session is done, take `/tweak/final` and edit the splash source:
1. Resolve each entry's widget path to its DSL site: the dotted path mirrors
the `script_mod!` tree (`/d` shows the same ids). `-` segments are
anonymous containers — skip them when searching the source.
2. Write each property at the **most specific existing site** — the widget's
own `name := Type{…}` block if it has one; create one only when none
exists.
3. Respect the merge law: a property inside a typed sub-struct goes through
`+:` (`draw_bg +: { border_radius: 8 }`), never a replacing
`draw_bg: {…}`. Plain walk/layout values (`padding`, `margin`, `width`)
are set directly (`padding: Inset{left: 20}`).
4. Values come back in source spelling (`#rrggbbaa` colors, plain numbers) —
paste them as-is. Mind the Rust-tokenizer hex-`e` trap in `script_mod!`:
`#1e1e2e` must be written `#x1e1e2e`.
5. Rebuild and relaunch; verify the value survived with `/tweak/state` or
`/snap` before calling it done.
Reflection truth: `props` come from the widget's live Rust fields plus the
type's DSL-declared shader inputs (`instance()`/`uniform()`), so the list is
what the widget actually exposes — there is no synthetic schema to drift.
### Studio remote bridge (the older path)
The studio (`studio/desktop` + `studio/hub`) drives a hosted app over a
websocket with the `StudioToApp` / `AppToStudio` protocol
(`platform/studio/src/studio.rs`): `MouseDown/Up/Move/Scroll`, `KeyDown/Up`,
`TextInput`, `TextCopy/Cut`, `GameInput`, `Screenshot`, `RunViewFrameRequest`,
`WidgetTreeDump`, `WidgetQuery`, `WidgetSnapshot`, `LiveChange`, `Custom`,
`Kill`, plus the shared-swapchain messages `Swapchain` / `WindowGeomChange` /
`Tick`. `libs/makepad_test` is the programmatic client for it
(`TestApp::try_click_center`, `try_type_text`, `try_screenshot`, …) and
`examples/*/tests/ui.rs` are its test suites.
`--remote` reuses that vocabulary — the same message types, the same injection
function, the same screenshot pipeline — but exposes it as HTTP on the app
itself, with no studio, no hub, no build ids, and with per-window targeting
that the studio path lacks. Use `--remote` for agent work; the studio bridge
remains for the studio and for `libs/makepad_test`.
## CLAUDE.md Body
The following is the current body of CLAUDE.md included verbatim for agent guidance parity.
@ -127,19 +311,35 @@ grep -r "texture_2d" widgets/src/
## Running UI Programs
Use the Studio bridge runnable-item flow instead of launching UI apps directly from the shell:
Launch UI apps as standalone release binaries from this checkout. Do not
use the Studio remote bridge.
1. Start the Studio remote bridge once.
2. Determine the runnable item name locally.
3. If an older instance is still running, clear it with `{"ClearBuild":{"build_id":[N]}}` and launch the replacement immediately without waiting for an acknowledgment.
4. Launch it with `{"RunItem":{"mount":"makepad","name":"<runnable-name>"}}`.
5. After editing UI/runtime code, do not inspect the previously running build. Always verify against the newly started build id from step 4.
```bash
cargo build --release -p makepad-app-asset-ui
# stop any older instance of the same binary, then:
./target/release/makepad-app-asset-ui
```
Do not use `ObserveMount` from the bridge. That call is for mount ownership/subscription and can steal RunView/framebuffer routing away from Studio desktop.
For one-shot visual smoke of a small example:
Use direct shell cargo commands only for non-UI tasks such as library checks, tests, and file/search operations. Do not run shell `cargo check`, `cargo build`, or `cargo run` for UI runnable targets that will be launched via Studio.
```bash
RUST_BACKTRACE=1 cargo run -p makepad-example-splash --release & PID=$!; sleep 15; kill $PID 2>/dev/null; echo "Process $PID killed"
```
When those non-UI tasks are used for runtime behavior or performance measurements, prefer their release variants (`cargo run --release`, `cargo test --release`, `cargo build --release`).
To look at or drive a running app, add `--remote`: the app serves a localhost
HTTP control surface (window list, PNG grabs, real mouse/key/text injection,
widget rects, log tail) and prints its port on startup. Finish every session
with `GET /gq`, which grabs each window and quits — never leave a test window
on screen. Full protocol: repo-root `AGENTS.md`.
```bash
./target/release/makepad-example-splash --remote > /tmp/app.log 2>&1 &
P=$(grep -o 'listening on 127.0.0.1:[0-9]*' /tmp/app.log | grep -o '[0-9]*$')
curl -s "http://127.0.0.1:$P/" # cheat sheet
curl -s "http://127.0.0.1:$P/gq" # final grab + quit
```
When measuring runtime or performance, prefer `--release`.
## Cargo.toml Setup

View file

@ -1,11 +1,24 @@
workspace.members = [
# === app ===
"apps/browser",
"apps/terminal",
"apps/wm",
"apps/aichat",
"apps/finance",
"apps/sheets",
"apps/photos",
"libs/wm_theme",
"libs/wm_api",
"libs/app_module",
"apps/task",
"apps/image",
"apps/video",
"apps/pdf",
"apps/files",
"apps/route",
# === arcade (game.md) ===
# === arcade (game.md) — networked AI game sandbox ===
"apps/arcade",
"libs/game/math",
"libs/game/render",
"libs/game/sim",
"libs/game/blocks",
"libs/game/gen",
"libs/game/audio",
@ -15,32 +28,67 @@ workspace.members = [
"libs/game/pkg",
"libs/game/session",
"libs/game/assets",
"apps/asset-ui",
"apps/asset-server",
"apps/flow-server",
"apps/flow-ui",
"libs/flowgraph",
"libs/media_view",
"apps/ai-hub",
"apps/vj",
"apps/mixer",
"libs/asset/importer",
# The tiled-RTS map generator, shared by the importer and the sandbox.
"libs/rtsmap",
"libs/texcomp",
"libs/asset/data",
"libs/asset/client",
"libs/asset/widgets",
"libs/asset/store",
"libs/asset/chat",
"libs/chat_ui",
"libs/ai/hub_ui",
"libs/ai/services",
"libs/asset/annotate",
"libs/render",
"libs/raytrace",
"libs/fab",
"libs/fab_tour",
"apps/fab/loaders/gltf",
"libs/sim",
"libs/sim/math",
"libs/show_control",
"libs/asset/creator",
"libs/bounded_http",
"libs/flow",
"libs/strict_json",
# === remesh (FaithC port) / xatlas (jpcy port) ===
"libs/remesh",
"libs/xatlas",
# === examples ===
"examples/hotload_ui",
"examples/teamtalk",
"examples/automate",
"examples/map",
"examples/arracing",
"examples/splash",
"examples/slides",
"examples/isolate",
"examples/uizoo",
"examples/video_player",
"examples/video_codec_test",
"examples/charts",
"examples/scratchpad",
"examples/ddgo",
"examples/datagrid",
"examples/todo",
"examples/git",
"examples/portallist_hit",
"examples/modal_footprint",
"examples/vector",
"examples/comfyui",
"examples/aichat",
"examples/godot",
"examples/gamemaker",
"examples/pdf",
"examples/exf",
"examples/shader",
"examples/raytrace",
"examples/text_input",
"examples/text_atlas_stress",
"examples/text_selection",
"examples/text_center",
"examples/counter",
"examples/browser",
"examples/camera",
@ -50,30 +98,97 @@ workspace.members = [
"examples/bugfix",
"examples/move_after_draw",
"examples/xr",
"examples/splat_bench",
"examples/cad",
"examples/box3d",
"examples/hello_world",
"examples/render_to_texture",
# === digital-fabrication product ===
"apps/fab",
"apps/fabric",
"libs/fabric/measure",
"libs/fabric/draft",
# === xr app ===
"xr",
# === studio ===
"studio/hub",
"studio/desktop",
# === own SQLite-format database engine (P0 reader, sqlq CLI) ===
"libs/sqlite_query",
# === headless music score engine ===
"libs/score",
"libs/score_layout",
"libs/score_play",
"libs/score_render",
"libs/score_view",
"libs/score_view/tests/embed_app",
"libs/score_ai",
"libs/score_import",
"libs/score_pdf",
"libs/score_ui",
"apps/score",
"libs/midi_file",
"libs/soundfont",
"libs/piano_model",
"libs/drumkit",
"libs/drumkit_phys",
"libs/musicxml",
# === own MP3 / Ogg Vorbis decoders ===
"libs/audio_decode",
# === own Ogg Vorbis encoder (stem side-channels) ===
"libs/audio_encode",
# === shared karaoke alignment core + lyrics schema ===
"libs/audio_lyrics",
# === stems + lyrics side-channel bake/publish (asset-ui + VJ) ===
"libs/audio_sidechannels",
"libs/vj_analysis",
"libs/teamtalk",
# === pictures of audio (spectrogram, wave strip, composite) ===
"libs/audio_picture",
# === mkfl motion payload + classical optical flow + all-intra re-encode
# (shared by the enhance backend and the VJ's flow-warp import) ===
"libs/video_flow",
# === realtime frame tweening: the classical GPU optical-flow pass
# chain, the RIFE producer behind the same warp, and the mode set
# (extracted from the VJ, which is still its reference user) ===
"libs/frametween",
# === archive.org search + download content input (VJ / asset-ui) ===
"libs/archive_org",
# === image tile wall: HEVC tape atlases + baker CLI + TileGrid widget
# (the engine extracted from the Source Library picture wall) ===
"libs/image_tiles",
"examples/image_tiles",
# === mp4 sample index for range-streaming playback ===
"libs/mp4_index",
# === necessary tools ===
"platform/video",
"tools/cargo_makepad",
"tools/makepad_loader",
# === OSM PBF -> tile archive + nav artifact bake passes (CLI + in-app) ===
"libs/map_build",
"tools/map_tiles",
"tools/map_bake",
"tools/remote",
"tools/arcade_eval",
"tools/dj_pack",
"libs/system_speech",
# === tests ===
"platform/script/test",
]
workspace.exclude = [
# Private Arcade frontend. Clone into apps/sandbox; path deps stay ../../
# like a subdirectory. Not a required member so a clean makepad tree
# still loads.
"apps/sandbox",
# Private Source Library picture wall. Same shape as sandbox: clone into
# apps/source-library; path deps stay ../../.
"apps/source-library",
"libs/terminal_core",
"libs/ggml",
"libs/voice",
"libs/voice2",
"libs/tts",
# standalone GPU generate service (own workspace, like diffusion)
"libs/ai/hub",
"libs/diffusion",
# the AI model workspace (loader + cuda/metal stores + model crates) — aiarch.md
"libs/ai",
"widgets/test",
"libs/stitch",
"libs/wasm_bridge/test",
@ -190,6 +305,9 @@ strip = true
inherits = "release"
debug = true
[profile.test.package.makepad-stitch]
opt-level = 1
#[profile.dev.package.makepad-live-tokenizer]
#opt-level = 3
#[profile.dev.package.makepad-live-compiler]

View file

@ -11,6 +11,41 @@ It also has a large set of AI backends integrated for embedding llms or generati
This repository contains the core engine, widgets, tools, and examples.
## Building the VJ example app
Rust stable is the toolchain everywhere: https://rustup.rs
**macOS** — install Rust and the Xcode command line tools, then:
```bash
cargo run -p makepad-vj --release
```
**Windows** — install Rust, Visual Studio 2022 (Desktop development with
C++), and the NVIDIA CUDA toolkit (any recent version; the build finds it
by itself). Then the same `cargo run -p makepad-vj --release`. Without a
CUDA toolkit the build still links — the GPU-AI lanes just stub out.
**Linux** — the VJ currently only compiles with CUDA present, and the
lane is not regularly tested; expect to fix small things. The errors are
shallow — point an AI coding assistant at them and it will get you
building.
**What CUDA is for**: the VJ uses the GPU-AI lane for audio source
separation (BS-RoFormer splits a track into vocals/drums/bass/other,
which drives the stem mutes and the karaoke word timing). The app runs
fine without it — those features just stay off.
**Model files install from inside the app.** The music decks use two
MIT-licensed model files: the BS-RoFormer stem splitter (527 MB) and the
Whisper large-v3-turbo transcriber (1.6 GB, karaoke word timing). On a
machine that lacks them the music page shows an **INSTALL MODELS** row
under the track explorer — accept the licenses and the app downloads both
(resumable, sha256-verified) into `local/` in the checkout. Until then the
VJ reports "stems: model not installed" and carries on. Existing copies
are found via `VJ_STEMS_CKPT` / `MAKEPAD_VOICE_MODEL` or the standard
`local/` paths.
## What Makepad Is
- A cross-platform UI runtime for native and web targets.
@ -90,8 +125,8 @@ cargo run -p makepad-example-map --release
For built-in maps and voice support, download the assets first:
```bash
./download_map.sh
./download_voice.sh
./tools/download_map.sh
./tools/download_voice.sh
```
## Run A WASM App

37
apps/ai-hub/Cargo.toml Normal file
View file

@ -0,0 +1,37 @@
[package]
name = "makepad-app-ai-hub"
version = "0.1.0"
edition = "2021"
publish = false
[[bin]]
name = "makepad-ai-hub"
path = "src/main.rs"
[dependencies]
makepad-ai-hub = { path = "../../libs/ai/hub", default-features = false }
# Feature passthrough: the headless node exposes exactly the library's
# feature set, so box launch scripts keep their --features flags working.
[features]
default = ["flux", "paint", "paint-cuda", "llm", "tts", "indextts", "video", "interpolate", "audio", "mesh", "matte-native", "depth-native", "segment-native", "body-native", "upscale-native", "motion-native", "rig-native", "splat-native", "stems-native"]
python-backends = ["makepad-ai-hub/python-backends"]
flux = ["makepad-ai-hub/flux"]
paint = ["makepad-ai-hub/paint"]
paint-cuda = ["makepad-ai-hub/paint-cuda"]
llm = ["makepad-ai-hub/llm"]
tts = ["makepad-ai-hub/tts"]
indextts = ["makepad-ai-hub/indextts"]
video = ["makepad-ai-hub/video"]
interpolate = ["makepad-ai-hub/interpolate"]
audio = ["makepad-ai-hub/audio"]
mesh = ["makepad-ai-hub/mesh"]
matte-native = ["makepad-ai-hub/matte-native"]
depth-native = ["makepad-ai-hub/depth-native"]
segment-native = ["makepad-ai-hub/segment-native"]
body-native = ["makepad-ai-hub/body-native"]
upscale-native = ["makepad-ai-hub/upscale-native"]
motion-native = ["makepad-ai-hub/motion-native"]
splat-native = ["makepad-ai-hub/splat-native"]
rig-native = ["makepad-ai-hub/rig-native"]
stems-native = ["makepad-ai-hub/stems-native"]

216
apps/ai-hub/src/main.rs Normal file
View file

@ -0,0 +1,216 @@
//! makepad-ai-hub service binary. Runs on each GPU box; wraps all AI
//! content generation behind a port.
//!
//! ```text
//! makepad-ai-hub [--port N] [--host ADDR] [--cache-dir PATH] [--registry PATH]
//!
//! --port listen port (env MAKEPAD_ASSET_AI_PORT, default 8765)
//! --host bind address (default 0.0.0.0)
//! --fleet partition name (env MAKEPAD_ASSET_AI_FLEET, default default)
//! --cache-dir model + artifact dir (env MAKEPAD_ASSET_AI_CACHE,
//! default <home>/.makepad/weights)
//! --registry registry json path (default: <cache-dir>/registry.json if it
//! exists, else the embedded registry)
//!
//! env HF_TOKEN bearer token for gated HF repos (flux1-dev)
//! env MAKEPAD_ASSET_AI_HF_BASE alternate HF endpoint / LAN mirror
//! ```
use makepad_ai_hub::download::Downloader;
use makepad_ai_hub::registry::Registry;
use makepad_ai_hub::server::{start_service, ServiceConfig};
use makepad_ai_hub::{AssetAiError, DEFAULT_PORT, SERVICE_NAME, SERVICE_VERSION};
use std::path::PathBuf;
fn main() {
if let Err(err) = run() {
eprintln!("{SERVICE_NAME}: {err}");
std::process::exit(1);
}
}
fn run() -> Result<(), AssetAiError> {
let mut port: Option<u16> = None;
let mut host = "0.0.0.0".to_string();
let mut fleet: Option<String> = None;
let mut cache_dir: Option<PathBuf> = None;
let mut registry_path: Option<PathBuf> = None;
let mut machine = false;
let mut activity_probe = None;
let mut args = std::env::args().skip(1);
while let Some(arg) = args.next() {
match arg.as_str() {
"--activity-probe" => {
activity_probe = Some(args.next().ok_or_else(|| AssetAiError::Io("--activity-probe needs seconds (1..3600)".into()))?.parse::<u64>().map_err(|_| AssetAiError::Io("invalid activity probe seconds".into()))?);
}
"--port" => {
let value = args
.next()
.ok_or_else(|| AssetAiError::Io("--port needs a value".into()))?;
port = Some(
value
.parse()
.map_err(|_| AssetAiError::Io(format!("bad --port {value:?}")))?,
);
}
"--host" => {
host = args
.next()
.ok_or_else(|| AssetAiError::Io("--host needs a value".into()))?;
}
"--fleet" => {
fleet = Some(args.next().ok_or_else(|| {
AssetAiError::Io("--fleet needs a value".into())
})?);
}
"--cache-dir" => {
cache_dir = Some(PathBuf::from(args.next().ok_or_else(|| {
AssetAiError::Io("--cache-dir needs a value".into())
})?));
}
"--registry" => {
registry_path = Some(PathBuf::from(args.next().ok_or_else(|| {
AssetAiError::Io("--registry needs a value".into())
})?));
}
// The machine node (aicore §3): loopback-only, registered in
// ~/.makepad/run for the apps on this machine, and gone on its
// own once nothing needs it — a cache, not a daemon.
"--machine" => {
machine = true;
}
"--help" | "-h" => {
println!(
"{SERVICE_NAME} {SERVICE_VERSION}\nusage: {SERVICE_NAME} [--port N] [--host ADDR] [--fleet NAME] [--cache-dir PATH] [--registry PATH] [--machine] [--activity-probe SECONDS]"
);
return Ok(());
}
other => {
return Err(AssetAiError::Io(format!("unknown argument {other:?}")));
}
}
}
if let Some(seconds) = activity_probe {
return makepad_ai_hub::activity::run_probe(seconds);
}
let port = match port {
Some(port) => port,
None => match std::env::var("MAKEPAD_ASSET_AI_PORT") {
Ok(value) => value
.parse()
.map_err(|_| AssetAiError::Io(format!("bad MAKEPAD_ASSET_AI_PORT {value:?}")))?,
Err(_) => DEFAULT_PORT,
},
};
let cache_dir = cache_dir.unwrap_or_else(default_cache_dir);
let fleet = makepad_ai_hub::discovery::normalize_fleet(
&fleet.unwrap_or_else(makepad_ai_hub::discovery::fleet_from_env),
);
// Registry: explicit path > registry.json dropped into the cache dir
// (per-box extension without a rebuild) > embedded seed registry.
let registry = if let Some(path) = registry_path {
Registry::load_file(&path)?
} else {
let cache_registry = cache_dir.join("registry.json");
if cache_registry.is_file() {
Registry::load_file(&cache_registry)?
} else {
Registry::embedded()?
}
};
// The machine node is machine-local by definition: loopback bind, no
// matter what --host said.
if machine {
host = "127.0.0.1".to_string();
}
let downloader = Downloader::from_env()?;
let handle = start_service(ServiceConfig {
host,
port,
cache_dir: cache_dir.clone(),
registry,
downloader,
// Peer-cache lane: everything resolves from env / cache-dir files
// (MAKEPAD_AI_PEER_SECRET or <cache>/peer-secret enables serving;
// MAKEPAD_AI_PEER_SOURCES injects download sources).
peer: Default::default(),
fleet: fleet.clone(),
})?;
println!("{SERVICE_NAME} {SERVICE_VERSION}");
println!(" listening on http://{}", handle.addr);
println!(" fleet {fleet}");
println!(" cache dir {}", cache_dir.display());
if std::env::var("HF_TOKEN").is_ok() {
println!(" HF_TOKEN present (gated repos enabled)");
}
println!(" lora dir {}", cache_dir.join("loras").display());
println!(
" endpoints: /health /models /jobs /loras POST:/generate /job/<id> POST:/job/<id>/cancel /artifact/<id> /v1/model_inventory /v1/model_blob/<sha256> POST:/realtime GET(ws):/realtime/<id>"
);
if machine {
return run_machine_node(handle);
}
// The http listener thread runs until the process is killed.
let _ = handle.http_thread.join();
Ok(())
}
/// The machine node's life: register in ~/.makepad/run so the apps on this
/// machine find it, then idle down and exit once nothing has needed it for
/// the TTL — it reads as a cache, not a daemon (aicore §3). Visible in any
/// process list as makepad-ai-hub.
fn run_machine_node(handle: makepad_ai_hub::server::ServiceHandle) -> Result<(), AssetAiError> {
use makepad_ai_hub::machine::{write_node_entry, NodeEntry};
use std::time::{Duration, Instant};
let ttl_min: u64 = std::env::var("MAKEPAD_AI_HUB_MACHINE_TTL_MIN")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(15);
let entry = NodeEntry {
pid: std::process::id() as u64,
port: handle.addr.port(),
pipes_hash: 0,
};
let entry_path = write_node_entry(&handle.shared.node_key, &entry)
.map_err(|e| AssetAiError::Io(format!("write node entry: {e}")))?;
println!(" machine node: registered {} (ttl {ttl_min}m idle)", entry_path.display());
let mut idle_since = Instant::now();
loop {
std::thread::sleep(Duration::from_secs(30));
// Busy = queued/running work, or a model somebody paid to load.
let pending = handle.shared.jobs.with(|store| store.pending_count()) > 0;
let resident = handle
.shared
.models
.lock()
.unwrap()
.values()
.any(|track| matches!(track, makepad_ai_hub::server::ModelTrack::Loaded));
if pending || resident {
idle_since = Instant::now();
} else if idle_since.elapsed() > Duration::from_secs(ttl_min * 60) {
println!("{SERVICE_NAME}: machine node idle for {ttl_min}m — exiting");
let _ = std::fs::remove_file(&entry_path);
return Ok(());
}
}
}
fn default_cache_dir() -> PathBuf {
if let Some(dir) = std::env::var_os("MAKEPAD_ASSET_AI_CACHE") {
return PathBuf::from(dir);
}
makepad_ai_hub::home::default_weights_dir_with_migration(&mut |message| {
eprintln!("{SERVICE_NAME}: {message}");
})
}

46
apps/aichat/Cargo.toml Normal file
View file

@ -0,0 +1,46 @@
# aichat — the assistant, as an app.
#
# One conversation that drives every application through the AI services
# layer. It is hosted three ways with one code base: under the window
# manager as a special child seated in the pane slot (this binary,
# `--stdin-loop`), standalone as its own window (this binary), and inside
# any app's Window as the F10 overlay or inside the mobile/web superbuild
# (this crate's lib, `makepad_aichat::script_mod` + `AiChatPanel{}`).
#
# The panel widget OWNS the engine: the service registry, the model, the
# transcript. Hosts only feed it links (in-process) or bus frames (the
# window manager's studio Custom frames) and give it a place to draw.
#
# Plan of record: repo-root aicontrol.md.
[package]
name = "makepad-aichat"
version = "0.1.0"
edition = "2021"
default-run = "aichat"
[lib]
name = "makepad_aichat"
path = "src/lib.rs"
[[bin]]
name = "aichat"
path = "src/main.rs"
[features]
default = ["engine"]
# The real models (the hub's local runtime, the cloud providers). Off for a
# build without a model runtime — the web page answers with NoModel and the
# tool console still works.
engine = ["makepad-ai-services/engine", "dep:makepad-asset-creator"]
[dependencies]
makepad-widgets = { path = "../../widgets" }
makepad-wm-theme = { path = "../../libs/wm_theme" }
makepad-wm-api = { path = "../../libs/wm_api" }
makepad-ai-services = { path = "../../libs/ai/services", default-features = false }
makepad-strict-json = { path = "../../libs/strict_json" }
# The generative pipelines behind the assistant's own `gen` service
# (src/gen.rs): the creator runner picks a fleet node and brings the
# picture back. Native only, with the engine.
makepad-asset-creator = { path = "../../libs/asset/creator", optional = true }

213
apps/aichat/src/bus.rs Normal file
View file

@ -0,0 +1,213 @@
//! The client half of the window manager's service bus.
//!
//! Under the WM the other apps are not in this process. The WM forwards
//! their up-frames to the aichat child as studio `Custom` frames, each
//! stamped with the endpoint the WM issued to the sender, and forwards
//! the aichat child's down-frames (which name their target endpoint) back
//! to the right client. This adapter turns those frames into ordinary
//! [`ServiceLink`]s in the panel's registry, so the engine never knows
//! whether a service is a channel away or a process away.
//!
//! One link per endpoint. A `Register` from an endpoint the registry does
//! not know creates the link and registers it under the WM's endpoint id
//! (`register_as`); a later `Register` from the same endpoint is just the
//! manifest going down the existing link, where the registry answers it.
//! The WM tells us about a dead client by sending `Unregister` on its
//! behalf. Everything the registry sends down a bus link is drained here
//! and put on the wire to the WM.
use makepad_ai_services::engine::ServiceRegistry;
use makepad_ai_services::port::{ServiceLink, ServiceLinkHost};
use makepad_ai_services::wire::*;
use makepad_widgets::makepad_platform::studio::AppToStudio;
use makepad_widgets::*;
use std::collections::HashMap;
#[derive(Default)]
pub struct ServiceBus {
hosts: HashMap<EndpointId, ServiceLinkHost>,
}
impl ServiceBus {
/// An up-frame from the WM. `None` for frames that are not the bus's.
pub fn on_custom(&mut self, registry: &ServiceRegistry, json: &str) -> bool {
let Some(frame) = HostedUp::parse(json) else { return false };
let Some(from) = frame.from.clone() else { return true };
match (&frame.msg, self.hosts.get(&from)) {
(ServiceUp::Register { manifest, .. }, None) => {
let (link, host) = ServiceLink::pair(manifest.clone());
if registry.register_as(link, from.clone(), "", None).is_ok() {
let _ = host.up.send(frame);
self.hosts.insert(from, host);
}
}
(_, Some(host)) => {
let _ = host.up.send(frame);
}
(_, None) => {}
}
true
}
/// Put every frame the registry sent down a bus link on the wire.
/// Links whose registry side is gone are dropped.
pub fn relay_down(&mut self, registry: &ServiceRegistry) {
for frame in self.drain_down(registry) {
Cx::send_studio_message(AppToStudio::Custom(frame.to_json()));
}
}
fn drain_down(&mut self, registry: &ServiceRegistry) -> Vec<HostedDown> {
let mut gone: Vec<EndpointId> = Vec::new();
let mut frames = Vec::new();
for (endpoint, host) in &self.hosts {
loop {
match host.down.try_recv() {
Ok(mut frame) => {
// `Registered` travels without a target; the WM
// routes it by the endpoint we register as.
if frame.to.is_none() {
frame.to = Some(endpoint.clone());
}
frames.push(frame);
}
Err(std::sync::mpsc::TryRecvError::Empty) => break,
Err(std::sync::mpsc::TryRecvError::Disconnected) => {
gone.push(endpoint.clone());
break;
}
}
}
}
for endpoint in gone {
self.hosts.remove(&endpoint);
registry.unregister(&endpoint);
}
frames
}
pub fn len(&self) -> usize {
self.hosts.len()
}
pub fn is_empty(&self) -> bool {
self.hosts.is_empty()
}
}
#[cfg(test)]
mod tests {
use super::*;
use makepad_ai_services::engine::{EngineCore, NoModelWithReason, RegistryUp};
#[test]
fn hosted_subscriptions_and_messages_cross_the_adapter_unchanged() {
let registry = ServiceRegistry::new();
let mut bus = ServiceBus::default();
let endpoint = EndpointId("w4".into());
let manifest = ServiceManifest::new("flow", "Flow", "A flow.")
.with_topic(TopicDef::new("run", "Run events."));
let register = HostedUp {
from: Some(endpoint.clone()),
msg: ServiceUp::Register { manifest, port_tag: 4 },
};
assert!(bus.on_custom(&registry, &register.to_json()));
registry.pump();
let host = bus.hosts.get(&endpoint).unwrap();
let _registered = host.down.try_recv().unwrap();
assert!(registry.send(
&endpoint,
ServiceDown::Subscribe {
sub_id: "s1".into(),
topic: "run".into(),
filter: None,
},
));
assert!(matches!(
host.down.try_recv().unwrap().msg,
ServiceDown::Subscribe { sub_id, topic, filter: None }
if sub_id == "s1" && topic == "run"
));
let message = HostedUp {
from: Some(endpoint.clone()),
msg: ServiceUp::Message {
sub_id: "s1".into(),
topic: "run".into(),
text: "finished".into(),
data: None,
final_: true,
},
};
assert!(bus.on_custom(&registry, &message.to_json()));
assert!(matches!(
registry.pump().as_slice(),
[RegistryUp::Message { endpoint: from, sub_id, message }]
if from == &endpoint && sub_id == "s1" && message.final_
));
}
#[test]
fn engine_shutdown_is_flushed_through_the_hosted_service_bus() {
let registry = ServiceRegistry::new();
let mut bus = ServiceBus::default();
let endpoint = EndpointId("w4".into());
let manifest = ServiceManifest::new("flow", "Flow", "A flow.")
.with_tool(ToolDef::new(
"watch",
"Watch a run.",
r#"{"type":"object","properties":{}}"#,
Risk::Read,
))
.with_topic(TopicDef::new("run", "Run events."));
assert!(bus.on_custom(
&registry,
&HostedUp {
from: Some(endpoint.clone()),
msg: ServiceUp::Register { manifest, port_tag: 4 },
}
.to_json(),
));
let mut engine = EngineCore::new(
registry.clone(),
Box::new(NoModelWithReason::new("not used by the tool console")),
None,
0x44,
);
engine.send("/flow.watch {}", 0.0);
let call_id = bus
.drain_down(&registry)
.into_iter()
.find_map(|frame| match frame.msg {
ServiceDown::Call(call) => Some(call.call_id),
_ => None,
})
.expect("the hosted service receives the call");
assert!(bus.on_custom(
&registry,
&HostedUp {
from: Some(endpoint.clone()),
msg: ServiceUp::Result(
ToolResult::ok(call_id, "watching", "")
.with_subscription(SubscriptionRequest::new("run")),
),
}
.to_json(),
));
engine.pump(0.1);
let sub_id = bus
.drain_down(&registry)
.into_iter()
.find_map(|frame| match frame.msg {
ServiceDown::Subscribe { sub_id, .. } => Some(sub_id),
_ => None,
})
.expect("the hosted service receives the subscription");
assert_eq!(sub_id, "l44-s1");
engine.shutdown();
assert!(matches!(
bus.drain_down(&registry).as_slice(),
[HostedDown { to: Some(to), msg: ServiceDown::Unsubscribe { sub_id: ended } }]
if to == &endpoint && ended == &sub_id
));
}
}

297
apps/aichat/src/gen.rs Normal file
View file

@ -0,0 +1,297 @@
//! The assistant's own generative service: `gen.image{prompt}`.
//!
//! The hub's pipelines are not an app, so no app registers them; the
//! panel registers this service in-process beside whatever apps join.
//! An `image` call runs the creator pipeline on a worker thread (the
//! runner is blocking: node pick over the LAN fleet, request, poll,
//! fetch), streams the node's progress into the card, writes the picture
//! under the makepad home's `gen` folder and answers with the path — the
//! model then hands that path to `photos.add`, which puts it on the wall.
//! Nothing goes through the asset store. Without the `engine` feature (the
//! web page) the service still exists and says it cannot.
use makepad_ai_services::engine::ServiceRegistry;
use makepad_ai_services::port::{AiServicePort, PortEvent};
use makepad_ai_services::wire::{Risk, ServiceCall, ServiceManifest, ToolDef, ToolResult};
use makepad_widgets::*;
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{Receiver, TryRecvError};
use std::sync::Arc;
/// The service id on the bus.
pub const SERVICE_ID: &str = "gen";
pub fn manifest() -> ServiceManifest {
ServiceManifest::new(
SERVICE_ID,
"Generate",
"The machine's generative pipelines, on the fleet's GPU nodes: a \
picture from a prompt. The picture is saved on this machine under \
the makepad home's gen folder and the answer carries its path. To \
SHOW it, call photos.add with that path (launch Photos first with \
os.launch if it is not running) the wall then glides onto it. A \
generation takes half a minute on a warm node, longer when a node \
must load the model.",
)
.with_tool(ToolDef::new(
"image",
"Generate one picture from a text prompt on a fleet image node; saves it under the makepad home's gen folder and returns the path.",
r#"{"type":"object","properties":{"prompt":{"type":"string","description":"what the picture shows, in plain words"},"width":{"type":"integer","description":"pixels, optional (default 1024)"},"height":{"type":"integer","description":"pixels, optional (default 1024)"}},"required":["prompt"]}"#,
Risk::Act,
))
}
/// What the worker reports back.
enum GenMsg {
Progress(String, u16),
Done(Result<GenDone, String>),
}
struct GenDone {
path: PathBuf,
node: String,
}
struct Job {
call_id: String,
cancel: Arc<AtomicBool>,
rx: Receiver<GenMsg>,
}
/// The in-process port plus the jobs in flight.
pub struct GenService {
port: AiServicePort,
jobs: Vec<Job>,
}
impl GenService {
/// Open the service and register it in the panel's registry. `None`
/// only when the manifest does not validate (a programming error).
pub fn open(registry: &ServiceRegistry) -> Option<GenService> {
let (port, link) = AiServicePort::in_process(manifest()).ok()?;
registry.register(link, "built in", None).ok()?;
Some(GenService { port, jobs: Vec::new() })
}
/// Drain the port and the workers; called on every panel event.
pub fn handle_event(&mut self, cx: &mut Cx, event: &Event) {
for ev in self.port.handle_event(cx, event) {
match ev {
PortEvent::Call(call) => self.start(call),
PortEvent::Cancel { call_id } => {
if let Some(job) = self.jobs.iter().find(|j| j.call_id == call_id) {
job.cancel.store(true, Ordering::Relaxed);
}
}
PortEvent::Registered(_)
| PortEvent::ChatOpen { .. }
| PortEvent::Subscribe { .. }
| PortEvent::Unsubscribe { .. } => {}
}
}
self.poll();
}
fn start(&mut self, call: ServiceCall) {
let id = call.call_id.clone();
if call.tool != "image" {
self.port.reply(ToolResult::refused(&id, format!("gen has no tool `{}`; it has image", call.tool)));
return;
}
let args = match parse_image_args(&call.args) {
Ok(a) => a,
Err(why) => {
self.port.reply(ToolResult::refused(&id, why));
return;
}
};
#[cfg(not(feature = "engine"))]
{
let _ = args;
self.port.reply(ToolResult::unavailable(&id, "this build has no pipeline runtime; pictures need the native app"));
}
#[cfg(feature = "engine")]
{
use makepad_widgets::makepad_platform::thread::SignalToUI;
let cancel = Arc::new(AtomicBool::new(false));
let (tx, rx) = std::sync::mpsc::channel();
let worker_cancel = cancel.clone();
let spawned = std::thread::Builder::new().name("gen-image".into()).spawn(move || {
let progress_tx = tx.clone();
let mut progress = |note: &str, permille: u16| {
let _ = progress_tx.send(GenMsg::Progress(note.to_string(), permille));
SignalToUI::set_ui_signal();
};
let result = run_image(&args, &worker_cancel, &mut progress);
let _ = tx.send(GenMsg::Done(result));
SignalToUI::set_ui_signal();
});
match spawned {
Ok(_) => self.jobs.push(Job { call_id: id, cancel, rx }),
Err(e) => self.port.reply(ToolResult::failed(&id, format!("could not start the generation: {e}"))),
}
}
}
fn poll(&mut self) {
let mut done: Vec<usize> = Vec::new();
for (i, job) in self.jobs.iter().enumerate() {
loop {
match job.rx.try_recv() {
Ok(GenMsg::Progress(note, permille)) => self.port.progress(&job.call_id, &note, permille),
Ok(GenMsg::Done(Ok(out))) => {
let path = out.path.to_string_lossy().to_string();
self.port.reply(
ToolResult::ok(
&job.call_id,
format!("saved {path} (made on {}). Show it on the wall with photos.add {{\"path\":\"{path}\"}}.", out.node),
"saved",
)
.with_data(format!("{{\"path\":{},\"node\":{}}}", json_string(&path), json_string(&out.node))),
);
done.push(i);
break;
}
Ok(GenMsg::Done(Err(e))) => {
let result = if e == "cancelled" { ToolResult::cancelled(&job.call_id) } else { ToolResult::failed(&job.call_id, e) };
self.port.reply(result);
done.push(i);
break;
}
Err(TryRecvError::Empty) => break,
Err(TryRecvError::Disconnected) => {
self.port.reply(ToolResult::failed(&job.call_id, "the generation worker died"));
done.push(i);
break;
}
}
}
}
for i in done.into_iter().rev() {
self.jobs.remove(i);
}
}
}
/// The arguments of one `image` call, checked.
#[derive(Clone, Debug, PartialEq)]
pub struct ImageArgs {
pub prompt: String,
pub width: u32,
pub height: u32,
}
/// Sizes are clamped to what the image nodes serve; a prompt is required.
pub fn parse_image_args(args: &str) -> Result<ImageArgs, String> {
use makepad_strict_json as json;
let fields = match json::parse(args.as_bytes()) {
Ok(json::Value::Obj(fields)) => fields,
_ => return Err("image needs a JSON object with a `prompt`".to_string()),
};
let get = |key: &str| fields.iter().find(|(k, _)| k == key).map(|(_, v)| v.clone());
let prompt = get("prompt").and_then(|v| v.as_str().map(|s| s.trim().to_string())).unwrap_or_default();
if prompt.is_empty() {
return Err("image needs a `prompt`".to_string());
}
if prompt.len() > 4000 {
return Err("the prompt is longer than 4000 bytes".to_string());
}
let dim = |key: &str| -> Result<u32, String> {
match get(key) {
None => Ok(1024),
Some(json::Value::Int(i)) if (256..=2048).contains(&i) => Ok(i as u32),
Some(_) => Err(format!("`{key}` must be an integer from 256 to 2048")),
}
};
Ok(ImageArgs { prompt, width: dim("width")?, height: dim("height")? })
}
/// A short file-name-safe slug of the prompt.
pub fn slug(prompt: &str) -> String {
let mut out = String::new();
for ch in prompt.chars().flat_map(|c| c.to_lowercase()) {
if ch.is_ascii_alphanumeric() {
out.push(ch);
} else if !out.ends_with('-') && !out.is_empty() {
out.push('-');
}
if out.len() >= 40 {
break;
}
}
let out = out.trim_matches('-').to_string();
if out.is_empty() { "picture".to_string() } else { out }
}
/// The file extension for a content type the nodes produce.
pub fn extension_for(content_type: &str) -> &'static str {
match content_type.split(';').next().unwrap_or("").trim() {
"image/jpeg" => "jpg",
"image/webp" => "webp",
"image/gif" => "gif",
_ => "png",
}
}
fn json_string(s: &str) -> String {
makepad_strict_json::s(s.to_string()).to_json()
}
#[cfg(feature = "engine")]
fn run_image(args: &ImageArgs, cancel: &Arc<AtomicBool>, progress: &mut dyn FnMut(&str, u16)) -> Result<GenDone, String> {
use makepad_asset_creator::makepad_ai_hub::home::makepad_home;
use makepad_asset_creator::makepad_strict_json as json;
use makepad_asset_creator::runner::generate_bytes;
let body = json::obj(vec![
("prompt", json::s(args.prompt.clone())),
("width", json::Value::Int(args.width as i64)),
("height", json::Value::Int(args.height as i64)),
]);
let seed = (Cx::time_now().max(0.0) * 1_000_000_000.0) as u64;
progress("finding an image node", 0);
let generated = generate_bytes("image.generate", &body, seed, cancel, progress)?;
let artifact = generated.artifact.ok_or("the node returned no picture")?;
let dir = makepad_home().join("gen");
std::fs::create_dir_all(&dir).map_err(|e| format!("cannot make {}: {e}", dir.display()))?;
let stamp = Cx::time_now().max(0.0) as u64;
let path = dir.join(format!("{stamp}-{}.{}", slug(&args.prompt), extension_for(&artifact.content_type)));
std::fs::write(&path, &artifact.bytes).map_err(|e| format!("cannot write {}: {e}", path.display()))?;
Ok(GenDone { path, node: generated.node })
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_manifest_validates_and_the_tool_acts() {
let m = manifest();
m.validate().expect("a manifest the wire accepts");
assert_eq!(m.id, "gen");
assert_eq!(m.tools.len(), 1);
assert_eq!(m.tools[0].name, "image");
assert_eq!(m.tools[0].risk, Risk::Act);
}
#[test]
fn image_args_need_a_prompt_and_sane_sizes() {
let a = parse_image_args(r#"{"prompt":"a red bicycle on the moon"}"#).unwrap();
assert_eq!((a.width, a.height), (1024, 1024));
let b = parse_image_args(r#"{"prompt":"x","width":512,"height":768}"#).unwrap();
assert_eq!((b.width, b.height), (512, 768));
assert!(parse_image_args(r#"{"width":512}"#).unwrap_err().contains("prompt"));
assert!(parse_image_args(r#"{"prompt":"x","width":16}"#).unwrap_err().contains("256"));
assert!(parse_image_args("nope").is_err());
}
#[test]
fn file_names_are_slugs_with_the_right_extension() {
assert_eq!(slug("A red bicycle, on the Moon!"), "a-red-bicycle-on-the-moon");
assert_eq!(slug(" "), "picture");
assert!(slug(&"word ".repeat(30)).len() <= 41);
assert_eq!(extension_for("image/png"), "png");
assert_eq!(extension_for("image/jpeg; charset=binary"), "jpg");
assert_eq!(extension_for("application/octet-stream"), "png");
}
}

89
apps/aichat/src/lib.rs Normal file
View file

@ -0,0 +1,89 @@
//! aichat as a library: the panel widget that owns the engine, the WM-bus
//! client half, the settings, and the module root. A host links this,
//! calls [`script_mod`] after the widgets' own, and puts `AiChatPanel{}`
//! where the chat goes — the standalone binary and the WM pane child do
//! that themselves; a plain `Window` does it for free through its F10
//! slot, which instantiates `mod.widgets.AiChatOverlay{}` by name; the
//! superbuild seats the same overlay in its pane.
//!
//! Linking this crate also gives the bridge its `/ai` routes: `script_mod`
//! installs `Cx::ai_callback`, the way the widgets crate installs the
//! tweaker's, so `/ai?on=1`, `/ai?say=…` and `/ai/transcript` drive and
//! read the chat in a hidden instance.
pub use makepad_widgets;
use makepad_widgets::ai_slot::AiSlotRequests;
use makepad_widgets::makepad_platform::ScriptVmCx;
use makepad_widgets::*;
pub mod bus;
pub mod gen;
pub mod overlay;
pub mod panel;
pub mod settings;
pub use bus::ServiceBus;
pub use overlay::{AiChatOverlay, AiTranscript};
pub use panel::{AiChatPanel, AiChatPanelAction};
pub use settings::AiSettings;
/// Register the panel and the overlay, and give the bridge its `/ai`
/// routes. Call once after `makepad_widgets::script_mod`.
pub fn script_mod(vm: &mut ScriptVm) {
crate::panel::script_mod(vm);
crate::overlay::script_mod(vm);
vm.cx_mut().ai_callback = Some(ai_callback);
}
fn arg<'a>(args: &'a [(String, String)], keys: &[&str]) -> Option<&'a str> {
for key in keys {
if let Some((_, value)) = args.iter().find(|(k, _)| k == key) {
return Some(value.as_str());
}
}
None
}
/// The bridge's `/ai` dispatcher. `toggle` (`on=1|0`, or flip) and `say`
/// (`say=TEXT`, opening the overlay if it is closed) are requests the
/// slot and the overlay take on their next event; `transcript` is what
/// the overlay last published. Never borrows a widget.
fn ai_callback(cx: &mut Cx, op: &str, args: &[(String, String)]) -> Result<String, String> {
match op {
"toggle" => {
let is_open = cx.global::<AiSlotRequests>().is_open;
let on = match arg(args, &["on"]) {
Some(value) => !matches!(value, "0" | "false" | "off" | "no"),
None => !is_open,
};
cx.global::<AiSlotRequests>().open = Some(on);
// The slot takes the request on its next event: make one.
cx.new_next_frame();
cx.redraw_all();
Ok(format!("{{\"on\":{}}}", on as u8))
}
"say" => {
let text = arg(args, &["say", "t"]).unwrap_or("").trim().to_string();
if text.is_empty() {
return Err("need say=TEXT".into());
}
let req = cx.global::<AiSlotRequests>();
if !req.is_open {
req.open = Some(true);
}
req.say.push(text);
cx.new_next_frame();
cx.redraw_all();
Ok("{\"ok\":1}".into())
}
"transcript" => {
let json = cx.global::<AiTranscript>().json.clone();
if json.is_empty() {
Ok("{\"status\":\"closed\",\"provider\":\"\",\"apps\":[],\"entries\":[],\"generation\":0}".into())
} else {
Ok(json)
}
}
other => Err(format!("no ai op `{other}`; there are toggle, say, transcript")),
}
}

79
apps/aichat/src/main.rs Normal file
View file

@ -0,0 +1,79 @@
//! aichat: the assistant as its own window, and as the window manager's
//! special child. Both are this binary: standalone it is a window with
//! the panel in it and whatever services are linked in-process (none,
//! until an app embeds it); under the WM (`--stdin-loop`) the WM seats it
//! in the pane slot and every other app's service reaches it over the bus
//! as studio `Custom` frames, which the panel turns into registry links.
pub use makepad_widgets;
use makepad_aichat::AiChatPanelAction;
use makepad_widgets::*;
app_main!(App);
script_mod! {
use mod.prelude.widgets.*
use mod.widgets.*
startup() do #(App::script_component(vm)){
ui: Root{
main_window := Window{
window.inner_size: vec2(460, 760)
window.title: "AI"
pass +: { clear_color: theme.color_bg_app }
body +: {
panel := AiChatPanel{
width: Fill
height: Fill
}
}
}
}
}
}
#[derive(Script, ScriptHook)]
pub struct App {
#[live]
ui: WidgetRef,
}
impl MatchEvent for App {
fn handle_startup(&mut self, cx: &mut Cx) {
makepad_wm_api::set_title(cx, "AI");
}
fn handle_actions(&mut self, cx: &mut Cx, actions: &Actions) {
for action in actions {
if let Some(widget_action) = action.as_widget_action() {
if let AiChatPanelAction::Close = widget_action.cast() {
// Under the WM the pane hides; standalone the window stays.
let _ = makepad_wm_api::send(cx, &makepad_wm_api::WmRequest::Close);
}
}
}
}
}
impl AppMain for App {
fn script_mod(vm: &mut ScriptVm) -> ScriptValue {
crate::makepad_widgets::script_mod(vm);
makepad_wm_theme::apply(vm);
makepad_aichat::script_mod(vm);
self::script_mod(vm)
}
fn handle_event(&mut self, cx: &mut Cx, event: &Event) {
// Bus frames reach the panel through its own handle_event; of the
// WM's own frames only the polite close is ours (the pane hides us
// by leaving us running; this is the desktop going down).
if let Event::Custom(json) = event {
if let Some(makepad_wm_api::WmEvent::CloseRequested) = makepad_wm_api::WmEvent::parse(json) {
cx.quit();
return;
}
}
self.match_event(cx, event);
self.ui.handle_event(cx, event, &mut Scope::empty());
}
}

261
apps/aichat/src/overlay.rs Normal file
View file

@ -0,0 +1,261 @@
//! The chat's module root: what a host seats in-process.
//!
//! `mod.widgets.AiChatOverlay{}` is what the Window's AI slot
//! (`widgets/src/ai_slot.rs`) instantiates by name on F10, and what the
//! superbuild seats in its pane. It is a `View` with the panel in it and
//! three duties around it: adopt the in-process service links the apps
//! parked on `Cx` ([`PendingServiceLinks`]) into the panel's registry,
//! send the lines the bridge asked to say ([`AiSlotRequests::say`]),
//! and publish the transcript as JSON ([`AiTranscript`]) after each draw
//! so `/ai/transcript` answers without touching a widget. The panel's
//! Escape (an idle, empty composer) asks the slot to close.
use crate::panel::{AiChatPanel, AiChatPanelAction};
use makepad_ai_services::port::PendingServiceLinks;
use makepad_ai_services::state::{Entry, EngineState, Status, ToolStatus};
use makepad_widgets::ai_slot::AiSlotRequests;
use makepad_widgets::makepad_micro_serde::*;
use makepad_widgets::*;
script_mod! {
use mod.prelude.widgets_internal.*
use mod.widgets.*
mod.widgets.AiChatOverlayBase = #(AiChatOverlay::register_widget(vm))
mod.widgets.AiChatOverlay = set_type_default() do mod.widgets.AiChatOverlayBase{
width: Fill
height: Fill
panel := AiChatPanel{
width: Fill
height: Fill
}
}
}
/// The transcript as the bridge reads it: a `Cx` global the overlay
/// rewrites after every draw of the panel.
#[derive(Default)]
pub struct AiTranscript {
pub json: String,
}
#[derive(SerJson)]
struct TranscriptRow {
kind: String,
text: String,
title: String,
status: String,
note: String,
}
#[derive(SerJson)]
struct Transcript {
status: String,
provider: String,
apps: Vec<String>,
entries: Vec<TranscriptRow>,
generation: u64,
}
/// The transcript JSON for a state: `status`, `provider`, the connected
/// apps, and one row per entry with what its card says.
pub fn transcript_json(state: &EngineState) -> String {
let status = match &state.status {
Status::Idle => "idle".to_string(),
Status::Loading { phase, fraction } => format!("loading {phase} {:.0}%", fraction * 100.0),
Status::Thinking => "thinking".to_string(),
Status::Streaming => "streaming".to_string(),
Status::WaitingForTool => "waiting_for_tool".to_string(),
Status::Error(e) => format!("error: {e}"),
};
let entries = state
.entries
.iter()
.map(|entry| match entry {
Entry::User { text } => TranscriptRow {
kind: "user".into(),
text: text.clone(),
title: String::new(),
status: String::new(),
note: String::new(),
},
Entry::Event(event) => TranscriptRow {
kind: "event".into(),
text: event.text.clone(),
title: format!("{} · {}", event.service_label, event.topic),
status: if event.final_ { "final".into() } else { "message".into() },
note: if event.dropped == 0 {
format!("sub_id: {}", event.sub_id)
} else {
format!("sub_id: {} · dropped: {}", event.sub_id, event.dropped)
},
},
Entry::Assistant { text, streaming } => TranscriptRow {
kind: "assistant".into(),
text: text.clone(),
title: String::new(),
status: if *streaming { "streaming".into() } else { "done".into() },
note: String::new(),
},
Entry::Tool(t) => {
let (status, note, text) = match &t.status {
ToolStatus::Confirm => ("confirm".to_string(), String::new(), String::new()),
ToolStatus::Running { note, permille } => ("running".to_string(), format!("{note} {permille}"), String::new()),
ToolStatus::Done { outcome, note, text } => (outcome.slug().to_string(), note.clone(), text.clone()),
};
TranscriptRow { kind: "tool".into(), text, title: t.title.clone(), status, note }
}
Entry::System { text } => TranscriptRow {
kind: "system".into(),
text: text.clone(),
title: String::new(),
status: String::new(),
note: String::new(),
},
})
.collect();
Transcript {
status,
provider: state.provider_label.clone(),
apps: state.services.iter().filter(|s| s.connected).map(|s| s.label.clone()).collect(),
entries,
generation: state.generation,
}
.serialize_json()
}
#[derive(Script, ScriptHook, Widget)]
pub struct AiChatOverlay {
#[source]
source: ScriptObjectRef,
#[deref]
view: View,
/// The state generation last published as JSON.
#[rust]
published: Option<u64>,
}
impl AiChatOverlay {
/// The transcript JSON, rewritten when the engine's state moved — on
/// every event as well as every draw, so a closed slot (no draws) still
/// answers `/ai/transcript` with the turn that finished behind it.
fn publish_transcript(&mut self, cx: &mut Cx) {
let json = self
.view
.widget(cx, ids!(panel))
.borrow::<AiChatPanel>()
.and_then(|panel| {
let state = panel.state()?;
if self.published == Some(state.generation) {
return None;
}
Some((state.generation, transcript_json(state)))
});
if let Some((generation, json)) = json {
self.published = Some(generation);
cx.global::<AiTranscript>().json = json;
}
}
/// The links the apps parked and the lines the bridge asked for, into
/// the panel.
fn adopt_requests(&mut self, cx: &mut Cx) {
let links = cx.global::<PendingServiceLinks>().take();
let says = std::mem::take(&mut cx.global::<AiSlotRequests>().say);
if links.is_empty() && says.is_empty() {
return;
}
let panel = self.view.widget(cx, ids!(panel));
let Some(mut panel) = panel.borrow_mut::<AiChatPanel>() else {
return;
};
for link in links {
let label = link.manifest.label.clone();
match panel.registry().register(link, "in this window", None) {
Ok(endpoint) => log!("aichat: {label} joined in-process as {}", endpoint.as_str()),
Err(e) => log!("aichat: {label} refused: {e}"),
}
}
for text in says {
panel.say(cx, text);
}
}
}
impl Widget for AiChatOverlay {
fn handle_event(&mut self, cx: &mut Cx, event: &Event, scope: &mut Scope) {
self.adopt_requests(cx);
self.view.handle_event(cx, event, scope);
self.publish_transcript(cx);
if let Event::Actions(actions) = event {
for action in actions {
if let Some(widget_action) = action.as_widget_action() {
if let AiChatPanelAction::Close = widget_action.cast() {
cx.global::<AiSlotRequests>().open = Some(false);
cx.redraw_all();
}
}
}
}
}
fn draw_walk(&mut self, cx: &mut Cx2d, scope: &mut Scope, walk: Walk) -> DrawStep {
let step = self.view.draw_walk(cx, scope, walk);
if step.is_done() {
self.publish_transcript(cx);
}
step
}
}
#[cfg(test)]
mod tests {
use super::*;
use makepad_ai_services::state::{EventEntry, ServiceInfo, ToolEntry};
use makepad_ai_services::wire::ToolOutcome;
#[test]
fn the_transcript_reads_as_json_rows() {
let mut state = EngineState::default();
state.provider_label = "No model".into();
state.services.push(ServiceInfo {
id: "sheets".into(),
endpoint: "e1".into(),
label: "Sheets".into(),
parent: None,
location: "in this window".into(),
connected: true,
launchable: false,
tool_count: 1,
});
state.entries.push(Entry::User { text: "/sheets.summary {}".into() });
state.entries.push(Entry::Event(EventEntry {
sub_id: "s1".into(),
service_label: "Sheets".into(),
topic: "changes".into(),
text: "A1 changed".into(),
data: None,
dropped: 2,
final_: false,
}));
state.entries.push(Entry::Tool(ToolEntry {
call_id: "c1".into(),
service: "sheets".into(),
service_label: "Sheets".into(),
tool: "summary".into(),
title: "Sheets · summary".into(),
args: "{}".into(),
status: ToolStatus::Done { outcome: ToolOutcome::Ok, note: "3 × 4".into(), text: "Sheet1, 3 rows".into() },
preview: false,
expanded: false,
}));
state.generation = 7;
let json = transcript_json(&state);
assert!(json.contains(r#""status":"idle""#), "{json}");
assert!(json.contains(r#""apps":["Sheets"]"#), "{json}");
assert!(json.contains(r#""kind":"user""#) && json.contains(r#""kind":"tool""#), "{json}");
assert!(json.contains(r#""kind":"event""#) && json.contains("sub_id: s1") && json.contains("dropped: 2"), "{json}");
assert!(json.contains(r#""title":"Sheets · summary""#) && json.contains(r#""status":"ok""#), "{json}");
assert!(json.contains(r#""generation":7"#), "{json}");
}
}

589
apps/aichat/src/panel.rs Normal file
View file

@ -0,0 +1,589 @@
//! The chat panel: the one widget every host shows. It owns the engine —
//! the service registry, the model, the transcript — and draws
//! `EngineState` as a transcript of user lines, assistant text, tool cards
//! (running, done, waiting for a confirm), and system lines, over a
//! composer.
//!
//! Hosts talk to it in two ways: they hand it service links
//! (`registry()`) or bus frames (`on_custom`), and they listen for
//! [`AiChatPanelAction`]s. The engine runs on the panel's own events: every
//! event pumps it, and while a turn is in flight the panel asks for the
//! next frame so streaming, deadlines and the cloud provider's polling all
//! advance without a host timer.
use crate::bus::ServiceBus;
use crate::gen::GenService;
use crate::settings::AiSettings;
#[cfg(feature = "engine")]
use makepad_ai_services::engine::models::{build_model, provider_rows};
use makepad_ai_services::engine::NoModelWithReason;
use makepad_ai_services::engine::{EngineCore, EngineEvent, ServiceRegistry};
use makepad_ai_services::state::*;
use makepad_ai_services::wire::ToolOutcome;
use makepad_widgets::*;
script_mod! {
use mod.prelude.widgets_internal.*
use mod.widgets.*
mod.widgets.AiChatPanelBase = #(AiChatPanel::register_widget(vm))
let Line = Label{
width: Fill
height: Fit
draw_text +: {
color: theme.color_text
text_style: theme.font_regular{font_size: 9.5}
}
}
let Row = View{
width: Fill
height: Fit
flow: Down
padding: Inset{left: 14 right: 14 top: 4 bottom: 4}
}
mod.widgets.AiChatPanel = set_type_default() do mod.widgets.AiChatPanelBase{
width: Fill
height: Fill
flow: Down
draw_bg +: { color: theme.color_bg_app }
header := SolidView{
width: Fill
height: 36
flow: Right
spacing: 10
padding: Inset{left: 14 right: 8}
align: Align{y: 0.5}
draw_bg +: { color: theme.color_bg_container }
title := Label{
text: "AI"
draw_text +: {
color: theme.color_text
text_style: theme.font_bold{font_size: 10.5}
}
}
provider := Label{
width: Fill
text: ""
draw_text +: {
color: theme.color_text_meta
text_style: theme.font_regular{font_size: 8.5}
}
}
clear_button := ButtonFlatter{ text: "Clear" }
}
apps_row := Label{
width: Fill
height: Fit
padding: Inset{left: 14 right: 14 top: 6 bottom: 2}
max_lines: 2
text: ""
draw_text +: {
color: theme.color_text_meta
text_style: theme.font_regular{font_size: 8.5}
}
}
transcript := PortalList{
width: Fill
height: Fill
auto_tail: true
UserRow := Row{
padding: Inset{left: 14 right: 14 top: 10 bottom: 4}
user_text := Line{
draw_text +: {
color: theme.color_text
text_style: theme.font_bold{font_size: 9.5}
}
}
}
EventRow := Row{
margin: Inset{left: 10 right: 10 top: 5 bottom: 5}
padding: Inset{left: 10 right: 10 top: 7 bottom: 7}
draw_bg +: { color: theme.color_bg_container }
event_title := Line{
draw_text +: {
color: theme.color_text_hl
text_style: theme.font_bold{font_size: 8.5}
}
}
event_text := Line{}
event_meta := Line{
draw_text +: {
color: theme.color_text_meta
text_style: theme.font_regular{font_size: 8.0}
}
}
}
AssistantRow := Row{
assistant_md := Markdown{
width: Fill
height: Fit
body: ""
}
}
StreamRow := Row{
stream_text := Line{}
}
ToolRow := Row{
padding: Inset{left: 22 right: 14 top: 3 bottom: 3}
tool_head := View{
width: Fill
height: Fit
cursor: MouseCursor.Hand
tool_title := Line{
draw_text +: {
color: theme.color_text_meta
text_style: theme.font_regular{font_size: 8.5}
}
}
}
tool_note := Line{
draw_text +: {
color: theme.color_text_meta
text_style: theme.font_regular{font_size: 8.5}
}
}
tool_bar := SolidView{
width: Fill
height: 2
margin: Inset{top: 3}
draw_bg +: { color: theme.color_text_hl }
}
tool_detail := Line{
visible: false
draw_text +: {
color: theme.color_text_meta
text_style: theme.font_regular{font_size: 8.0}
}
}
}
ConfirmRow := Row{
padding: Inset{left: 22 right: 14 top: 6 bottom: 6}
confirm_title := Line{
draw_text +: {
color: theme.color_text
text_style: theme.font_regular{font_size: 9.0}
}
}
confirm_buttons := View{
width: Fill
height: Fit
flow: Right
spacing: 8
margin: Inset{top: 4}
run_button := Button{ text: "Run" }
deny_button := ButtonFlat{ text: "Cancel" }
}
}
SystemRow := Row{
system_text := Line{
draw_text +: {
color: theme.color_text_hl
text_style: theme.font_regular{font_size: 8.5}
}
}
}
}
status := Label{
width: Fill
height: Fit
padding: Inset{left: 14 right: 14 top: 4 bottom: 2}
max_lines: 2
text: ""
draw_text +: {
color: theme.color_text_meta
text_style: theme.font_regular{font_size: 8.5}
}
}
composer := View{
width: Fill
height: Fit
flow: Right
spacing: 8
padding: Inset{left: 12 right: 12 top: 6 bottom: 10}
align: Align{y: 0.5}
input := TextInput{
width: Fill
height: Fit
empty_text: "Ask AI"
// The prompt is a hint, not text: a dark grey in every state,
// never the typed colour (the composer is always focused).
draw_text +: {
color_empty: #666666
color_empty_hover: #777777
color_empty_focus: #666666
}
}
send_button := Button{ text: "Send" }
}
}
}
/// What the panel tells its host.
#[derive(Clone, Debug, PartialEq, Default)]
pub enum AiChatPanelAction {
/// Esc with an empty composer and no turn in flight: the host may hide
/// the pane.
Close,
#[default]
None,
}
#[derive(Script, ScriptHook, Widget)]
pub struct AiChatPanel {
#[source]
source: ScriptObjectRef,
#[deref]
view: View,
#[rust]
engine: Option<EngineCore>,
#[rust]
registry: ServiceRegistry,
#[rust]
bus: ServiceBus,
/// The assistant's own `gen` service (pictures from the fleet), joined
/// to the registry with the engine.
#[rust]
gen: Option<GenService>,
#[rust]
settings: Option<AiSettings>,
#[rust]
drawn_generation: u64,
#[rust]
next_frame: NextFrame,
/// The composer took the keyboard once it existed on screen — a
/// focus set before the first draw lands on no area at all.
#[rust]
composer_focused: bool,
}
impl AiChatPanel {
/// The registry a host plugs in-process links into.
pub fn registry(&self) -> &ServiceRegistry {
&self.registry
}
/// A studio `Custom` frame that may be a bus frame from the WM.
pub fn on_custom(&mut self, json: &str) -> bool {
self.bus.on_custom(&self.registry, json)
}
pub fn state(&self) -> Option<&EngineState> {
self.engine.as_ref().map(|e| e.state())
}
fn settings(&mut self) -> &AiSettings {
if self.settings.is_none() {
self.settings = Some(AiSettings::load());
}
self.settings.as_ref().unwrap()
}
/// The engine comes up on first use — a local model is a long load
/// and no host pays it before the person opens the pane.
fn ensure_engine(&mut self) -> &mut EngineCore {
if self.engine.is_none() {
let lease_id = self.widget_uid().0;
let settings = self.settings().clone();
// The real models ride the `engine` feature; a build without a
// runtime (the web page) says so and keeps the tool console.
#[cfg(feature = "engine")]
let (model, rows): (Box<dyn makepad_ai_services::Model>, Vec<ProviderRow>) = (
match build_model(&settings.provider, settings.local_only) {
Ok(m) => m,
Err(reason) => Box::new(NoModelWithReason::new(reason)),
},
provider_rows(settings.local_only),
);
#[cfg(not(feature = "engine"))]
let (model, rows): (Box<dyn makepad_ai_services::Model>, Vec<ProviderRow>) =
(Box::new(NoModelWithReason::new("this build has no model runtime")), Vec::new());
let mut core = EngineCore::new(self.registry.clone(), model, None, lease_id);
// The state is the panel's window into the core; the core owns
// it, so provider facts go in through the core.
core.set_provider_facts(settings.provider.clone(), rows, settings.local_only);
self.engine = Some(core);
self.gen = GenService::open(&self.registry);
}
self.engine.as_mut().unwrap()
}
fn now(cx: &Cx) -> f64 {
cx.seconds_since_app_start()
}
/// A line as if typed and sent — the bridge's `/ai?say=`.
pub fn say(&mut self, cx: &mut Cx, text: String) {
self.send(cx, text);
}
fn send(&mut self, cx: &mut Cx, text: String) {
let now = Self::now(cx);
self.ensure_engine().send(&text, now);
let input = self.view.text_input(cx, ids!(input));
input.set_text(cx, "");
// The widget drops the keyboard on submit; a chat composer keeps
// it, so the next line can be typed straight away.
input.take_key_focus(cx);
self.view.redraw(cx);
}
fn apps_line(&self, state: &EngineState) -> String {
if state.services.is_empty() {
return "No apps connected.".into();
}
state
.services
.iter()
.map(|s| if s.connected { s.label.clone() } else { format!("{} (not running)", s.label) })
.collect::<Vec<_>>()
.join(" · ")
}
fn status_line(state: &EngineState) -> String {
let rate = state.rate.map(|r| format!(" · {r:.0} tok/s")).unwrap_or_default();
match &state.status {
Status::Idle => rate.trim_start_matches(" · ").to_string(),
Status::Loading { phase, fraction } => format!("loading {phase} {:.0}%", fraction * 100.0),
Status::Thinking => {
if state.thinking.is_empty() {
"thinking…".to_string()
} else {
let tail: String = state.thinking.chars().rev().take(120).collect::<Vec<_>>().into_iter().rev().collect();
format!("thinking… {tail}")
}
}
Status::Streaming => format!("writing{rate}"),
Status::WaitingForTool => "waiting for the app…".to_string(),
Status::Error(e) => e.clone(),
}
}
}
impl Drop for AiChatPanel {
fn drop(&mut self) {
// The engine only enqueues Unsubscribe frames. Flush them while the
// hosted bus and registry still exist; field destruction is too late.
if let Some(engine) = self.engine.as_mut() {
engine.shutdown();
}
self.bus.relay_down(&self.registry);
}
}
impl Widget for AiChatPanel {
fn handle_event(&mut self, cx: &mut Cx, event: &Event, scope: &mut Scope) {
if let Event::Custom(json) = event {
self.on_custom(json);
}
// Esc: stop a running turn; with nothing running and an empty
// composer, ask the host to hide the pane.
if let Event::KeyDown(ke) = event {
if ke.key_code == KeyCode::Escape {
let busy = self.engine.as_ref().map(|e| e.state().status.is_busy()).unwrap_or(false);
let empty = self.view.text_input(cx, ids!(input)).text().trim().is_empty();
if busy {
let now = Self::now(cx);
self.ensure_engine().cancel(now);
self.view.redraw(cx);
} else if empty {
cx.widget_action(self.widget_uid(), AiChatPanelAction::Close);
}
}
}
// The built-in services answer before the engine pumps, so a
// finished picture lands in this same event.
if let Some(gen) = self.gen.as_mut() {
gen.handle_event(cx, event);
}
// Drive the engine on every event; the bus relays what it sent.
let now = Self::now(cx);
let mut changed = false;
let mut busy = false;
if let Some(engine) = self.engine.as_mut() {
for ev in engine.pump(now) {
match ev {
EngineEvent::Changed => changed = true,
EngineEvent::Confirm { .. } => changed = true,
}
}
busy = engine.needs_pump()
|| engine.state().status.is_busy()
|| matches!(engine.state().status, Status::Loading { .. });
}
self.bus.relay_down(&self.registry);
if changed {
self.view.redraw(cx);
}
if busy {
self.next_frame = cx.new_next_frame();
}
self.view.handle_event(cx, event, scope);
self.widget_match_event(cx, event, scope);
}
fn draw_walk(&mut self, cx: &mut Cx2d, scope: &mut Scope, walk: Walk) -> DrawStep {
// The engine exists once the panel is on screen.
let _ = self.ensure_engine();
let state = self.engine.as_ref().unwrap().state().clone();
self.drawn_generation = state.generation;
self.view.label(cx, ids!(provider)).set_text(cx, &format!("{}{}", state.provider_label, if state.local_only { " · local only" } else { "" }));
self.view.label(cx, ids!(apps_row)).set_text(cx, &self.apps_line(&state));
self.view.label(cx, ids!(status)).set_text(cx, &Self::status_line(&state));
while let Some(item) = self.view.draw_walk(cx, scope, walk).step() {
let Some(mut list) = item.borrow_mut::<PortalList>() else { continue };
let total = state.entries.len();
list.set_item_range(cx, 0, total);
while let Some(index) = list.next_visible_item(cx) {
if index >= total {
continue;
}
let entry = &state.entries[index];
let row = match entry {
Entry::User { text } => {
let row = list.item(cx, index, id!(UserRow));
row.label(cx, ids!(user_text)).set_text(cx, text);
row
}
Entry::Event(event) => {
let row = list.item(cx, index, id!(EventRow));
row.label(cx, ids!(event_title))
.set_text(cx, &format!("{} · {}", event.service_label, event.topic));
row.label(cx, ids!(event_text)).set_text(cx, &event.text);
let mut meta = format!("sub_id: {}", event.sub_id);
if event.dropped != 0 {
meta.push_str(&format!(" · dropped: {}", event.dropped));
}
if event.final_ {
meta.push_str(" · final");
}
if let Some(data) = &event.data {
meta.push_str(&format!("\n{data}"));
}
row.label(cx, ids!(event_meta)).set_text(cx, &meta);
row
}
Entry::Assistant { text, streaming: false } => {
let row = list.item(cx, index, id!(AssistantRow));
// No text, no row: a blank block above a card is a gap.
row.set_visible(cx, !text.trim().is_empty());
if let Some(mut md) = row.widget(cx, ids!(assistant_md)).borrow_mut::<Markdown>() {
md.set_text(cx, text);
}
row
}
Entry::Assistant { text, streaming: true } => {
let row = list.item(cx, index, id!(StreamRow));
row.set_visible(cx, !text.trim().is_empty());
row.label(cx, ids!(stream_text)).set_text(cx, text);
row
}
Entry::Tool(t) if matches!(t.status, ToolStatus::Confirm) => {
let row = list.item(cx, index, id!(ConfirmRow));
row.label(cx, ids!(confirm_title)).set_text(cx, &format!("{} — this changes things outside the app. Run it?", t.title));
row
}
Entry::Tool(t) => {
let row = list.item(cx, index, id!(ToolRow));
let (glyph, note, permille, detail) = match &t.status {
ToolStatus::Running { note, permille } => ("", note.clone(), *permille, String::new()),
ToolStatus::Done { outcome, note, text } => {
let glyph = if outcome.is_ok() { "" } else { "" };
let note = if note.is_empty() { outcome.slug().to_string() } else { note.clone() };
(glyph, note, 1000, text.clone())
}
ToolStatus::Confirm => ("?", String::new(), 0, String::new()),
};
row.label(cx, ids!(tool_title)).set_text(cx, &format!("{glyph} {}", t.title));
row.label(cx, ids!(tool_note)).set_text(cx, &note);
let bar_visible = matches!(t.status, ToolStatus::Running { .. });
row.view(cx, ids!(tool_bar)).set_visible(cx, bar_visible);
if bar_visible {
// The bar is a fraction of the row's inner width
// (the row's rect is last frame's; the first
// frame of a card draws it at a token width).
let inner = (row.area().rect(cx).size.x - 36.0).max(24.0);
let px = inner * (permille as f64 / 1000.0).max(0.05);
let mut bar = row.view(cx, ids!(tool_bar));
script_apply_eval!(cx, bar, { width: #(px) });
}
let detail_label = row.label(cx, ids!(tool_detail));
detail_label.set_visible(cx, t.expanded && !detail.is_empty());
if t.expanded {
detail_label.set_text(cx, &format!("{}\n{}", t.args, detail));
}
let _ = matches!(t.status, ToolStatus::Done { outcome: ToolOutcome::Ok, .. });
row
}
Entry::System { text } => {
let row = list.item(cx, index, id!(SystemRow));
row.label(cx, ids!(system_text)).set_text(cx, text);
row
}
};
row.draw_all(cx, &mut Scope::empty());
}
}
if !self.composer_focused {
self.composer_focused = true;
self.view.text_input(cx, ids!(input)).take_key_focus(cx);
}
DrawStep::done()
}
}
impl WidgetMatchEvent for AiChatPanel {
fn handle_actions(&mut self, cx: &mut Cx, actions: &Actions, scope: &mut Scope) {
let input = self.view.text_input(cx, ids!(input));
let returned = input.returned(actions).map(|(text, _)| text);
if let Some(text) = returned {
self.send(cx, text);
} else if self.view.button(cx, ids!(send_button)).clicked(actions) {
let text = input.text();
self.send(cx, text);
}
if self.view.button(cx, ids!(clear_button)).clicked(actions) {
let now = Self::now(cx);
self.ensure_engine().clear(now);
self.view.redraw(cx);
}
// Cards: confirm buttons and click-to-expand.
let list = self.view.portal_list(cx, ids!(transcript));
let items = list.items_with_actions(actions);
if !items.is_empty() {
let now = Self::now(cx);
let call_ids: Vec<Option<String>> = {
let state = self.ensure_engine().state();
items
.iter()
.map(|(index, _)| match state.entries.get(*index) {
Some(Entry::Tool(t)) => Some(t.call_id.clone()),
_ => None,
})
.collect()
};
for ((_, item), call_id) in items.iter().zip(call_ids) {
let Some(call_id) = call_id else { continue };
if item.button(cx, ids!(run_button)).clicked(actions) {
self.ensure_engine().confirm(&call_id, true, now);
self.view.redraw(cx);
} else if item.button(cx, ids!(deny_button)).clicked(actions) {
self.ensure_engine().confirm(&call_id, false, now);
self.view.redraw(cx);
} else if item.view(cx, ids!(tool_head)).finger_up(actions).is_some() {
self.ensure_engine().toggle_tool(&call_id);
self.view.redraw(cx);
}
}
}
let _ = scope;
}
}

101
apps/aichat/src/settings.rs Normal file
View file

@ -0,0 +1,101 @@
//! What the person chose: which model answers, and whether cloud models
//! are locked out. Two lines in `~/.makepad/aichat/settings`, the lock a
//! promise kept in code (a cloud choice under the lock normalises to
//! local at load and is refused at set), never a menu filter.
use makepad_ai_services::state::ProviderChoice;
use std::path::PathBuf;
#[derive(Clone, Debug, PartialEq)]
pub struct AiSettings {
pub provider: ProviderChoice,
/// Default ON: the person must switch it off before any cloud model
/// can be picked.
pub local_only: bool,
}
impl Default for AiSettings {
fn default() -> Self {
AiSettings { provider: ProviderChoice::Local, local_only: true }
}
}
impl AiSettings {
/// The lock, applied: a cloud provider under the lock becomes local.
/// "none" (no model) is always allowed — it reaches nothing.
pub fn normalized(mut self) -> Self {
if self.local_only {
if let ProviderChoice::Cloud(slug) = &self.provider {
if slug != "none" {
self.provider = ProviderChoice::Local;
}
}
}
self
}
/// Can this choice be made under the current lock?
pub fn allows(&self, choice: &ProviderChoice) -> Result<(), String> {
match choice {
ProviderChoice::Cloud(slug) if self.local_only && slug != "none" => Err("Local AI only is on".into()),
_ => Ok(()),
}
}
pub fn parse(text: &str) -> AiSettings {
let mut s = AiSettings::default();
for line in text.lines() {
let Some((k, v)) = line.split_once('=') else { continue };
match k.trim() {
"provider" => s.provider = ProviderChoice::from_slug(v.trim()),
"local_only" => s.local_only = v.trim() != "false",
_ => {}
}
}
s.normalized()
}
pub fn render(&self) -> String {
format!("provider={}\nlocal_only={}\n", self.provider.slug(), self.local_only)
}
pub fn path() -> Option<PathBuf> {
if cfg!(target_arch = "wasm32") {
return None;
}
let home = std::env::var_os("HOME").or_else(|| std::env::var_os("USERPROFILE"))?;
Some(PathBuf::from(home).join(".makepad").join("aichat").join("settings"))
}
pub fn load() -> AiSettings {
match Self::path().and_then(|p| std::fs::read_to_string(p).ok()) {
Some(text) => Self::parse(&text),
None => AiSettings::default(),
}
}
pub fn save(&self) -> Result<(), String> {
let Some(path) = Self::path() else { return Ok(()) };
if let Some(dir) = path.parent() {
std::fs::create_dir_all(dir).map_err(|e| e.to_string())?;
}
std::fs::write(&path, self.render()).map_err(|e| e.to_string())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_lock_is_kept_at_load_and_at_set() {
let s = AiSettings::parse("provider=claude-api\nlocal_only=true\n");
assert_eq!(s.provider, ProviderChoice::Local, "a cloud pref under the lock cannot resurrect a cloud model");
assert!(s.allows(&ProviderChoice::Cloud("claude-api".into())).is_err());
assert!(s.allows(&ProviderChoice::Cloud("none".into())).is_ok());
let open = AiSettings::parse("provider=claude-api\nlocal_only=false\n");
assert_eq!(open.provider, ProviderChoice::Cloud("claude-api".into()));
assert_eq!(AiSettings::parse(&open.render()), open);
assert_eq!(AiSettings::parse("garbage"), AiSettings::default());
}
}

View file

@ -15,8 +15,8 @@ path = "src/main.rs"
[dependencies]
makepad-widgets = { path = "../../widgets", version = "2.0.0" }
makepad-game-math = { path = "../../libs/game/math" }
makepad-game-sim = { path = "../../libs/game/sim" }
makepad-game-math = { path = "../../libs/sim/math" }
makepad-game-sim = { path = "../../libs/sim" }
makepad-game-session = { path = "../../libs/game/session" }
makepad-game-blocks = { path = "../../libs/game/blocks" }
makepad-game-render = { path = "../../libs/game/render" }
@ -26,7 +26,9 @@ makepad-game-script = { path = "../../libs/game/script" }
makepad-game-net = { path = "../../libs/game/net" }
makepad-game-coedit = { path = "../../libs/game/coedit" }
makepad-game-pkg = { path = "../../libs/game/pkg" }
makepad-ai = { path = "../../libs/makepad_ai" }
# The improved AI layer replaced the old makepad-ai agent crate: chat and
# provider flows now come from makepad-ai-hub's headless ChatProvider model.
makepad-ai-hub = { path = "../../libs/ai/hub", default-features = false }
# default-features off drops speech synthesis (~10 MB of binary, ~327 MB
# resident). Re-enabled by `voice` below: a device that can capture speech is
# the one that has any use for producing it.

View file

@ -6,17 +6,15 @@
//! thing that writes code. Where there is no local compute, the text box IS
//! the gate and typing goes straight through.
use crate::capability::{Capabilities, Tier};
use crate::capability::Tier;
use crate::library::{classify_deterministic, Action, Librarian, Library, Manifest};
use crate::pairing::{has_key, load_key, Provider};
use makepad_ai::agent::{Agent, SessionConfig, StatelessBackendAdapter};
use makepad_ai::backend::BackendConfig;
use makepad_ai::backends::claude::ClaudeBackend;
use makepad_ai::backends::claude_code::ClaudeCodeAgent;
use makepad_ai::backends::gemini::GeminiBackend;
use makepad_ai::backends::openai::OpenAiBackend;
use makepad_converse::filter::{FilterDecision, PassthroughFilter, TranscriptFilter};
use makepad_converse::pipeline::ConversePipeline;
use makepad_ai_hub::chat_wire::{ChatMessage, ChatRole, ProviderAvailability, ProviderKind};
use makepad_ai_hub::providers::claude_api::ClaudeApiChatProvider;
use makepad_ai_hub::providers::provider::{ChatProvider, ProviderEvent, TurnInput};
use makepad_converse::filter::{FilterDecision, TranscriptFilter};
use std::sync::mpsc::{self, Receiver, Sender};
use std::time::Duration;
use makepad_widgets::*;
/// Tools the authoring session may use. Without this the CLI is launched with
/// `--tools ""` and the agent is chat-only: it answers, sounds correct, and
@ -31,17 +29,118 @@ pub const PERMISSION_POLICY: &str = r#"{"permissions":{
"allow":["Read","Glob","Grep","Edit(./**)","Write(./**)"],
"deny":["Bash","Edit(../**)","Write(../**)"]}}"#;
/// The session config for authoring one game in `dir`.
pub fn authoring_session(dir: Option<String>, api: &str, model: Option<String>) -> SessionConfig {
SessionConfig {
cwd: dir,
system_prompt: Some(system_prompt(api)),
model,
allowed_tools: AUTHORING_TOOLS.iter().map(|t| t.to_string()).collect(),
permission_mode: Some("dontAsk".to_string()),
settings_json: Some(PERMISSION_POLICY.to_string()),
..Default::default()
// --- AI worker ------------------------------------------------------------
// The improved makepad-ai-hub layer drives the agent headless on a worker
// thread (exactly the `example/cad` pattern): the UI sends a [`TurnInput]
// and drains [`AiWorkerEvent`]s back from `poll`. The CLI owns its own tool
// loop, so it is the natural single provider; direct-API providers remain
// available through the same seam via their `ProviderKind`.
enum AiWorkerCommand {
Send(TurnInput),
Cancel,
}
pub enum AiWorkerEvent {
Availability(ProviderAvailability),
Delta(String),
Done(String),
Error(String),
}
pub struct AiWorker {
command_tx: Sender<AiWorkerCommand>,
event_rx: Receiver<AiWorkerEvent>,
}
impl AiWorker {
pub fn new(cx: &mut Cx) -> Self {
let (command_tx, command_rx) = mpsc::channel();
let (event_tx, event_rx) = mpsc::channel();
cx.spawn_thread(move || ai_worker_loop(command_rx, event_tx));
Self {
command_tx,
event_rx,
}
}
pub fn send(&self, input: TurnInput) -> Result<(), String> {
self.command_tx
.send(AiWorkerCommand::Send(input))
.map_err(|_| "AI provider worker ended".to_string())
}
pub fn cancel(&self) {
let _ = self.command_tx.send(AiWorkerCommand::Cancel);
}
pub fn poll(&self) -> Vec<AiWorkerEvent> {
self.event_rx.try_iter().collect()
}
}
pub fn emit_ai_worker_event(event_tx: &Sender<AiWorkerEvent>, event: AiWorkerEvent) -> bool {
if event_tx.send(event).is_err() {
return false;
}
SignalToUI::set_ui_signal();
true
}
fn ai_worker_loop(command_rx: Receiver<AiWorkerCommand>, event_tx: Sender<AiWorkerEvent>) {
let mut provider = ClaudeApiChatProvider::from_env(ProviderKind::ClaudeCli, None);
if !emit_ai_worker_event(
&event_tx,
AiWorkerEvent::Availability(provider.availability()),
) {
return;
}
loop {
match command_rx.recv_timeout(Duration::from_millis(16)) {
Ok(AiWorkerCommand::Send(input)) => {
if let Err(error) = provider.begin_turn(&input) {
if !emit_ai_worker_event(&event_tx, AiWorkerEvent::Error(error)) {
return;
}
}
}
Ok(AiWorkerCommand::Cancel) => provider.cancel(),
Err(mpsc::RecvTimeoutError::Timeout) => {}
Err(mpsc::RecvTimeoutError::Disconnected) => {
provider.cancel();
return;
}
}
for event in provider.poll() {
let event = match event {
ProviderEvent::Delta(text) => Some(AiWorkerEvent::Delta(text)),
ProviderEvent::Done { text } => Some(AiWorkerEvent::Done(text)),
ProviderEvent::Error(error) => Some(AiWorkerEvent::Error(error)),
ProviderEvent::FunctionCall { .. } => Some(AiWorkerEvent::Error(
"AI provider requested an unsupported function".to_string(),
)),
ProviderEvent::Status { .. } | ProviderEvent::Serving(_) => None,
};
if let Some(event) = event {
if !emit_ai_worker_event(&event_tx, event) {
provider.cancel();
return;
}
}
}
}
}
/// Build the turn for one authoring request against one game. `api` is the
/// `game.*` verb table from [`makepad_game_script::api_text`]; the CLI runs in
/// the game directory so it can edit `game.splash` with its own tools.
pub fn build_turn(api: &str, request: &str) -> TurnInput {
TurnInput::new(
system_prompt(api),
vec![ChatMessage::new(ChatRole::User, request)],
)
}
/// A worked example is worth more than any amount of prose: this is the whole
@ -111,118 +210,6 @@ pub fn system_prompt(api: &str) -> String {
)
}
/// Which backend this device should use, and why — the reason is worth showing
/// in settings, because "no key" and "no CLI" need different fixes.
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum BackendChoice {
Ready(Provider),
/// A direct-API provider is selected but has no key yet: this device can
/// still play and join, and in a hosted room it can still ASK the host's
/// agent — it just cannot create on its own.
NeedsKey(Provider),
/// Nothing configured at all.
None,
}
/// Prefer a CLI agent when one exists (it runs its own tool loop and needs no
/// key); otherwise fall back to whichever direct API has a key.
pub fn choose_backend(preferred: Option<Provider>) -> BackendChoice {
if let Some(p) = preferred {
if p == Provider::ClaudeCode && !ClaudeCodeAgent::is_available() {
// Asked for the CLI but it is not installed — say so rather than
// silently using something else.
return BackendChoice::None;
}
return if has_key(p) {
BackendChoice::Ready(p)
} else {
BackendChoice::NeedsKey(p)
};
}
if ClaudeCodeAgent::is_available() {
return BackendChoice::Ready(Provider::ClaudeCode);
}
for p in [Provider::Anthropic, Provider::OpenAi, Provider::Gemini] {
if has_key(p) {
return BackendChoice::Ready(p);
}
}
BackendChoice::None
}
/// Default models per provider: small edits should not cost flagship prices
/// (game.md: "default small edits to cheaper models").
pub fn default_model(provider: Provider) -> &'static str {
match provider {
Provider::ClaudeCode => "claude-sonnet-5",
Provider::Anthropic => "claude-sonnet-5",
Provider::OpenAi => "gpt-5",
Provider::Gemini => "gemini-2.5-flash",
}
}
pub fn build_agent(provider: Provider, model: Option<String>) -> Option<Box<dyn Agent>> {
let model = model.unwrap_or_else(|| default_model(provider).to_string());
Some(match provider {
// The CLI runs its own tool loop, so it is an Agent directly.
Provider::ClaudeCode => Box::new(ClaudeCodeAgent::new()) as Box<dyn Agent>,
// The HTTP backends are stateless; the adapter owns the session and
// executes tool calls on our side (the Quest/mobile path).
Provider::Anthropic => Box::new(StatelessBackendAdapter::new(Box::new(
ClaudeBackend::new(BackendConfig::Claude {
api_key: Some(load_key(provider)?),
oauth_token: None,
model,
}),
))) as Box<dyn Agent>,
Provider::OpenAi => Box::new(StatelessBackendAdapter::new(Box::new(
OpenAiBackend::new(BackendConfig::OpenAI {
api_key: load_key(provider)?,
model,
base_url: None,
reasoning_effort: None,
}),
))) as Box<dyn Agent>,
Provider::Gemini => Box::new(StatelessBackendAdapter::new(Box::new(
GeminiBackend::new(BackendConfig::Gemini {
api_key: load_key(provider)?,
model,
}),
))) as Box<dyn Agent>,
})
}
/// Build the conversational pipeline for this device's tier.
///
/// The filter is constructed ON the worker thread (a local LLM session is not
/// Send), which is why this takes capabilities rather than a built filter.
pub fn build_pipeline(
agent: Box<dyn Agent>,
caps: &Capabilities,
voice: &str,
) -> ConversePipeline {
let tier = caps.tier();
#[cfg(feature = "local-llm")]
let qwen = caps.qwen_model.clone();
let make_filter = move || -> Box<dyn TranscriptFilter> {
#[cfg(feature = "local-llm")]
{
if matches!(tier, Tier::Voice) {
if let Some(path) = qwen {
if let Some(f) = makepad_converse::qwen_filter::QwenFilter::new(&path) {
return Box::new(f);
}
}
}
}
// No judge available: every transcript passes. Safe because in this
// tier the mic is push-to-talk, so the human is the gate.
let _ = tier;
Box::new(PassthroughFilter)
};
ConversePipeline::new(agent, make_filter, voice)
}
/// Route one utterance. Voice goes through the pipeline's filter; typed text
/// bypasses it (typing IS the gate) — but BOTH go through the librarian first,
/// so "play the racing game" never costs a cloud call.
@ -268,29 +255,11 @@ mod tests {
use super::*;
#[test]
fn a_missing_key_is_reported_not_silently_swapped() {
let _guard = crate::pairing::tests::KEY_STORE_LOCK
.lock()
.unwrap_or_else(|e| e.into_inner());
// Point the key store at an empty dir so nothing is configured.
let dir = std::env::temp_dir().join(format!("arcade-ai-test-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
std::env::set_var("ARCADE_CONFIG_DIR", &dir);
for var in ["ANTHROPIC_API_KEY", "OPENAI_API_KEY", "GOOGLE_API_KEY"] {
std::env::remove_var(var);
}
assert_eq!(
choose_backend(Some(Provider::Anthropic)),
BackendChoice::NeedsKey(Provider::Anthropic)
);
std::env::set_var("ANTHROPIC_API_KEY", "sk-test");
assert_eq!(
choose_backend(Some(Provider::Anthropic)),
BackendChoice::Ready(Provider::Anthropic)
);
std::env::remove_var("ANTHROPIC_API_KEY");
std::fs::remove_dir_all(&dir).ok();
std::env::remove_var("ARCADE_CONFIG_DIR");
fn build_turn_carries_the_request_and_verb_system() {
let turn = build_turn(&makepad_game_script::api_text(), "add a ramp over the lake");
assert!(turn.system.contains("game.car"), "prompt must list the blocks");
assert_eq!(turn.messages.len(), 1);
assert!(turn.messages[0].text.contains("ramp"));
}
#[test]

View file

@ -488,6 +488,7 @@ fn spawn(
restitution: 0.0,
// 0.0 reads as 1.0 (see Entity::push_mass): normal shove resistance.
push_mass: 0.0,
..Default::default()
});
id
}

View file

@ -21,9 +21,6 @@ pub mod xr_input;
use crate::authoring::{Applied, Authoring};
use crate::capability::{arcade_home, Capabilities};
use crate::chat::{ChatData, ChatRole};
// Named rather than glob-imported: `makepad_ai::*` and the widgets prelude
// both re-export a `makepad_widgets`, and the collision is a warning.
use makepad_ai::agent::{Agent, AgentEvent, PromptId, SessionId};
use makepad_widgets::*;
app_main!(App);
@ -144,11 +141,11 @@ pub struct App {
#[rust]
caps: Option<Capabilities>,
#[rust]
agent: Option<Box<dyn Agent>>,
ai_worker: Option<ai::AiWorker>,
#[rust]
session_id: Option<SessionId>,
backend_available: bool,
#[rust]
current_prompt: Option<PromptId>,
current_prompt: bool,
/// The intent log. Every edit — local agent or remote Claude — goes
/// through it; there is deliberately no faster path for the local one.
#[rust]
@ -188,7 +185,7 @@ impl App {
// it or a spoken sentence lands nowhere.
input.set_key_focus(cx);
let (Some(agent), Some(session_id)) = (&mut self.agent, self.session_id) else {
let Some(worker) = &self.ai_worker else {
ChatData::push(
ChatRole::System,
"No AI is connected. Open Settings to choose a provider and add a key.",
@ -196,18 +193,31 @@ impl App {
self.ui.redraw(cx);
return;
};
if self.current_prompt.is_some() {
if !self.backend_available {
ChatData::push(
ChatRole::System,
"The AI provider is not available. Open Settings to pair a key.",
);
self.ui.redraw(cx);
return;
}
if self.current_prompt {
// The newest instruction wins; the partial reply stays in the log.
if let Some(prompt) = self.current_prompt.take() {
agent.cancel_prompt(cx, prompt);
}
worker.cancel();
}
if let Some(authoring) = &mut self.authoring {
authoring.begin_turn();
}
let turn = ai::build_turn(&makepad_game_script::api_text(), &text.trim());
ChatData::begin_stream();
ChatData::set_activity("Thinking");
self.current_prompt = Some(agent.send_prompt(cx, session_id, &text));
if let Err(error) = worker.send(turn) {
ChatData::end_stream();
ChatData::push(ChatRole::System, format!("Error: {error}"));
self.ui.redraw(cx);
return;
}
self.current_prompt = true;
self.ui.widget(cx, ids!(stop_button)).set_visible(cx, true);
if let Some(speech) = self.speech.as_mut() {
@ -220,10 +230,13 @@ impl App {
if let Some(speech) = self.speech.as_mut() {
speech.stop();
}
let (Some(agent), Some(prompt)) = (&mut self.agent, self.current_prompt.take()) else {
if !self.current_prompt {
return;
};
agent.cancel_prompt(cx, prompt);
}
if let Some(worker) = &self.ai_worker {
worker.cancel();
}
self.current_prompt = false;
ChatData::end_stream();
self.ui.widget(cx, ids!(stop_button)).set_visible(cx, false);
self.set_status(cx, "Stopped.");
@ -287,18 +300,6 @@ impl App {
self.set_status(cx, &format!("{} {activity}", SPINNER[self.spinner_phase]));
cx.redraw_all();
}
/// Turn a tool call into something a child can understand.
fn describe_tool(tool_name: &str, subject: &str) -> Option<String> {
let file = subject.rsplit('/').next().unwrap_or(subject);
match tool_name {
"Edit" | "Write" => Some("Changing the game".to_string()),
"Read" => Some(format!("Looking at {file}")),
"Glob" | "Grep" => Some("Looking around".to_string()),
"Bash" => Some("Working on it".to_string()),
_ => None,
}
}
}
impl MatchEvent for App {
@ -396,38 +397,12 @@ impl MatchEvent for App {
}
}
// No explicit preference: prefer a CLI agent (no key needed), else
// whichever direct API has a key. Settings can override per device.
let backend = ai::choose_backend(None);
match &backend {
ai::BackendChoice::Ready(provider) => {
if let Some(mut agent) = ai::build_agent(*provider, None) {
let dir = self.game_path().parent().map(|p| p.to_path_buf());
if let Some(dir) = &dir {
let _ = std::fs::create_dir_all(dir);
}
let config = ai::authoring_session(
dir.map(|d| d.to_string_lossy().to_string()),
&makepad_game_script::api_text(),
None,
);
self.session_id = Some(agent.create_session(cx, config));
self.agent = Some(agent);
}
}
ai::BackendChoice::NeedsKey(p) => {
ChatData::push(
ChatRole::System,
format!("{p:?} needs an API key — open Settings to add one, or pair from another device."),
);
}
ai::BackendChoice::None => {
ChatData::push(
ChatRole::System,
"No AI backend found. Open Settings to choose a provider.",
);
}
}
// The AI worker probes the provider on its thread and reports
// availability back through `AiWorkerEvent::Availability`. The CLI
// needs no key; a direct-API provider needs one (Settings/pairing).
let worker = ai::AiWorker::new(cx);
self.ai_worker = Some(worker);
self.backend_available = false;
// The intent log starts from whatever game is on disk.
let path = self.game_path();
@ -518,57 +493,57 @@ impl AppMain for App {
self.next_frame = cx.new_next_frame();
}
let Some(agent) = &mut self.agent else { return };
for event in agent.handle_event(cx, event) {
let events = if let Some(worker) = &self.ai_worker {
worker.poll()
} else {
Vec::new()
};
for event in events {
match event {
AgentEvent::TextDelta { text, .. } => {
ai::AiWorkerEvent::Availability(
makepad_ai_hub::chat_wire::ProviderAvailability::Available { .. },
) => {
self.backend_available = true;
self.set_status(cx, "Ready.");
}
ai::AiWorkerEvent::Availability(
makepad_ai_hub::chat_wire::ProviderAvailability::Unavailable { reason },
) => {
self.backend_available = false;
self.current_prompt = false;
ChatData::push(
ChatRole::System,
format!("AI provider unavailable: {reason}"),
);
self.ui.widget(cx, ids!(stop_button)).set_visible(cx, false);
self.ui.redraw(cx);
}
ai::AiWorkerEvent::Delta(text) => {
ChatData::push_delta(&text);
if let Some(speech) = self.speech.as_mut() {
speech.feed(&text);
}
cx.redraw_all();
}
AgentEvent::ToolRequest {
tool_name,
tool_input,
..
} => {
if let Some(activity) = Self::describe_tool(&tool_name, &tool_input) {
ChatData::set_activity(&activity);
}
// An edit landed: propose it now so the world rebuilds
// while the agent keeps working.
if matches!(tool_name.as_str(), "Edit" | "Write") {
self.land_edit(cx, "edit");
}
}
AgentEvent::TurnComplete { .. } => {
ai::AiWorkerEvent::Done(_full_text) => {
ChatData::end_stream();
if let Some(speech) = self.speech.as_mut() {
speech.flush();
}
self.current_prompt = None;
self.current_prompt = false;
self.ui.widget(cx, ids!(stop_button)).set_visible(cx, false);
// Catch a final edit the tool stream did not announce.
self.land_edit(cx, "turn");
self.set_status(cx, "Ready.");
self.ui.text_input(cx, ids!(input)).set_key_focus(cx);
cx.redraw_all();
}
AgentEvent::SessionError { error, .. } => {
ChatData::push(ChatRole::System, format!("Session error: {error}"));
cx.redraw_all();
}
AgentEvent::PromptError { prompt_id, error } => {
if self.current_prompt == Some(prompt_id) {
ChatData::end_stream();
self.current_prompt = None;
self.ui.widget(cx, ids!(stop_button)).set_visible(cx, false);
}
ai::AiWorkerEvent::Error(error) => {
self.current_prompt = false;
ChatData::end_stream();
ChatData::push(ChatRole::System, format!("Error: {error}"));
cx.redraw_all();
self.ui.widget(cx, ids!(stop_button)).set_visible(cx, false);
self.ui.redraw(cx);
}
AgentEvent::SessionReady { .. } => {}
}
}
}

View file

@ -0,0 +1,37 @@
# The standalone Asset Server host.
#
# The asset store is inherently MULTIPLAYER: asset-ui, the VJ, the sandbox
# and headless workers are all clients of one catalog. So the catalog's
# lifetime must not be any client's lifetime — closing a window must not
# take the store down under everyone else. This binary is that process: the
# catalog/CAS service, the chat broker, the events hub, LAN discovery, plus
# the two background loops a fleet needs (the ai-content library publisher
# and the GPU-fleet job coordinator), composed in one supervisor.
#
# It owns no UI and derives no content that needs a GPU surface: those loops
# stay in the app that has the resources, as CLIENTS of this server.
[package]
name = "makepad-app-asset-server"
version = "0.1.0"
edition = "2021"
description = "Standalone Asset Server host: catalog service, chat broker, library publisher and fleet job coordinator in one process"
license = "MIT OR Apache-2.0"
[lib]
name = "makepad_app_asset_server"
path = "src/lib.rs"
[[bin]]
name = "makepad-asset-server"
path = "src/main.rs"
[dependencies]
# The service itself: catalog + CAS, chat broker, events hub, discovery.
makepad-asset-store = { path = "../../libs/asset/store" }
# The headless background loops, shared verbatim with the Asset Worker and
# with the Asset UI's embedded mode: `watch` (library publication) and
# `gen_service` (claim -> fleet dispatch -> publish, plus profile announce).
makepad-asset-importer = { path = "../../libs/asset/importer" }
# Endpoints + the publishing client the library watcher writes through.
makepad-asset-client = { path = "../../libs/asset/client" }

156
apps/asset-server/README.md Normal file
View file

@ -0,0 +1,156 @@
# `makepad-asset-server` — the standalone Asset Server host
The asset store is inherently **multiplayer**. asset-ui, the VJ, the sandbox
and headless workers are all clients of one catalog, and they open and close
independently of each other. So the catalog's lifetime must not be any one
client's lifetime.
Historically the Asset UI *embedded* the server: it took `<root>/server.lock`,
served the catalog, ran the chat broker and the events hub — and every time
that window was closed or rebuilt, every other connected client lost the
store mid-session (`503 state unavailable`). This binary is the fix: the same
service, in its own process, that survives every window.
```
cargo build --release -p makepad-app-asset-server
./target/release/makepad-asset-server
```
With no flags it serves the checkout's standard store root
(`local/asset-ui/asset-server`, or `$AI_CONTENT_ASSET_ROOT`) on ephemeral
ports, announces itself on the LAN beacon, publishes
`local/ai_content_library`, and coordinates fleet jobs. `--help` lists every
flag.
---
## What this process carries
| Part | Where it lives | Notes |
| --- | --- | --- |
| Catalog + CAS, control/data planes | `AssetServer::start` | search, blobs, ranges/ETags, retire |
| Auth, tokens, grants | ” | `--root/admin-token` bootstrapped at start |
| Job queue, worker/lease protocol | ” | plus `GET /v1/job-profiles` |
| Chat broker (+ client-executed tool parking for game sessions) | ” | `--chat-fleet` / LAN fleet discovery |
| Games publish path, operations, import routes | ” | |
| Committed events hub (`/v1/events`) | ” | every client's live view |
| LAN discovery beacon | ” | `--no-beacon` to stay silent |
| Lease janitor + bounded blob GC | ” | runs whether or not anyone polls |
| **ai-content library publisher** | `makepad_asset_importer::watch` | `--library` / `--no-library` |
| **GPU-fleet job coordinator + profile announce** | `makepad_asset_importer::gen_service` | `--no-jobs` / `--no-announce` / `--fleet` |
The last two are the loops the Asset UI used to run *only while it was
hosting*. They are headless — no `Cx`, no window, no GPU surface — so they
belong beside the server, and running them here is what makes a UI-less
deployment a complete fleet citizen. Without the coordinator, jobs any client
enqueues sit at "waiting for agent" forever.
## What deliberately stays client-side
These are **content-deriving** loops that need resources only a UI process
has. They reach the catalog as ordinary clients; moving them here would mean
giving a headless daemon a window.
- **Thumbnail / preview backfill** (`apps/asset-ui`: `thumbnail_renderer.rs`,
the splat + mesh offscreen renders). Needs a real `Cx`, render passes and
GPU readback.
- **Classic-game import** (`import_classic.rs`) and **pack import** wizards —
interactive, and their icon bakes are GPU renders. The one-shot headless
equivalents already exist as `makepad-asset-importer --import-pack /
--import-games / --import-music / --import-ai-library` when a batch run is
wanted.
- **Stems / lyrics analysis bake** (`analysis.rs`) — a large local model on
the operator's machine, driven from the surface that asks for it.
- **Generation pipelines** driven from the Create surface (`pipeline.rs`) —
these are *requests*; the queue and the dispatch live server-side.
## Recommended deployment
One daemon, every app attached:
```bash
# 1. the store (leave it running; it survives every UI restart)
./target/release/makepad-asset-server > /tmp/asset-server.log 2>&1 &
# 2. every client, told never to hold the root itself
export ASSET_UI_ASSET_EMBED=never
./target/release/makepad-app-asset-ui
./target/release/makepad-vj
```
`ASSET_UI_ASSET_EMBED=never` (aliases: `no`, `off`, `0`, `false`, `attach`,
`client`) makes the Asset UI a pure client: it never takes
`<root>/server.lock`, at startup or during a succession. Without it the app
still attaches when the daemon holds the root — but whichever process starts
*first* wins the lock, so a UI launched before the daemon would put the store
back inside a window.
The VJ and the sandbox are already pure clients; they need no flag.
### How a client finds the server
Three ways, in order of precedence:
1. `ASSET_UI_ASSET_SERVER=ip:controlport:dataport` — an explicit pin.
2. `<root>/listen`, `<root>/server-id`, `<root>/admin-token` — what the
daemon writes at startup. This is the attach path, and it is why the
default bind is `0.0.0.0:0`: the *file* is the address of record.
3. The LAN discovery beacon — how a client on another machine finds it with
no configuration at all.
## What survives what
- **Closing / rebuilding / crashing any app** — the server keeps serving.
Every other client stays connected; nothing is lost.
- **Restarting the daemon** — attached clients notice the silence within
~3.5 s, say so honestly in the status chip, and rejoin the new process as
soon as it rewrites `listen`. Cursors and leases are re-established; the
events journal reports an explicit `gap` rather than a silent hole.
- **A daemon crash with `EmbedPolicy::Auto`** — an attached Asset UI succeeds
it (takes the lock, starts the loops, reconnects to itself). This is the
transition-mode safety net, not the deployment.
- **A daemon crash with `ASSET_UI_ASSET_EMBED=never`** — no app fills the
vacancy; clients wait and rejoin whoever takes the root next. Restart the
daemon.
- **Two daemons on one root** — refused immediately and by name
(`server root: locked by another server process`). One process per root is
a law: the job routing metadata assumes a single enqueuer, and two writers
over one WAL catalog would be two sources of recovery truth.
## Bouncing the daemon safely
```bash
# find it (it holds the lock and wrote the listen file)
cat local/asset-ui/asset-server/listen # ip:control:data
pkill -TERM -f 'makepad-asset-server' # clean shutdown: joins every thread
./target/release/makepad-asset-server > /tmp/asset-server.log 2>&1 &
```
SIGTERM/SIGINT shut down in order: background loops first (so nothing is
still publishing into a closing catalog), then the planes, then the state
thread. Startup logs what recovery found — `recovered N cas temps / M leases`
— which is the honest report of what the previous life left behind.
## Isolated instances
Never point a scratch instance at the live root, and never let one beacon:
```bash
./target/release/makepad-asset-server \
--root /tmp/scratch-store --work /tmp/scratch-work \
--no-beacon --no-jobs --no-library
```
`--no-beacon` keeps peers hunting for the real store from finding the scratch
one; `--no-jobs` keeps it from claiming the real fleet's queued work.
## Tests
`cargo test -p makepad-app-asset-server --release` covers the composition:
the host serves a catalog over real sockets and shuts down cleanly, a second
host on one root is refused by name, a missing library directory never costs
the catalog its server, and the defaults are the deployment defaults. The
parts themselves are covered where they live — `libs/asset/store` (HTTP,
chat, events, jobs, security, operations suites) and `libs/asset/importer`
(watch, coordinator, gen-service). The attach/succession contract is in
`apps/asset-ui/src/asset_store_state.rs`.

View file

@ -0,0 +1,358 @@
//! FULLY-LOCAL MODE, shared: an app hosting its own Asset Server.
//!
//! The VJ and the sandbox are thin clients over a store somebody else runs —
//! normally asset-ui's embedded one, or the standalone host. That stays the
//! default and still wins whenever it is actually reachable. This module is
//! the shared answer to "and if it is not?": the app brings up [`Host`] in
//! its own process, on 127.0.0.1 only, rooted in the user's main library
//! when one exists on disk (a private seed root only when none does), with
//! the ai-content library publisher riding along.
//!
//! The thin-client law does not bend for this. Hosting changes WHERE the
//! store runs, never who owns the content: the app still browses, publishes
//! and fetches over HTTP, through the same `AssetClient`, with the same
//! catalog-event subscription. Nothing durable moves into the app.
//!
//! ## Loopback, deliberately
//!
//! The embedded host binds 127.0.0.1 for both planes (port 0 = OS-assigned,
//! or pinned by `<PREFIX>_ASSET_PORT`) and runs NO discovery beacon: an app
//! hosting for itself has no business accepting connections from the
//! network, and announcing the store would invite other apps onto a private
//! instance of the user's library. That is also what makes reference
//! imports safe to enable here (see `makepad_asset_store::blobrefs`).
//!
//! ## Choosing
//!
//! `<PREFIX>_ASSET_EMBED` decides, `auto` by default:
//!
//! - `never` — attach only; if nothing is reachable, behave as before.
//! - `auto` — attach if an external store ANSWERS, else host.
//! - `always` — host, regardless of what else is running.
//!
//! "Answers" is a real probe, not a guess: `GET /v1/health` against the
//! endpoints the caller advertises, failing that a short listen for a UDP
//! beacon, failing that the main store root's `server.lock` — a held lock
//! is proof the user's library is hosted RIGHT NOW whatever its ports, and
//! self-hosting then would silently split their library in two.
use crate::{Host, HostConfig};
use makepad_asset_client::ApiEndpoints;
use makepad_asset_store::BlobRefPolicy;
use std::io::{Read, Write};
use std::net::{SocketAddr, TcpStream};
use std::path::{Path, PathBuf};
use std::time::Duration;
/// How the app got its store this run.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum StoreMode {
/// Attached to a store some other process runs (the usual case).
Attached,
/// Hosting the store in this process, on loopback.
Hosting,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum EmbedPolicy {
Never,
Auto,
Always,
}
/// `<PREFIX>_ASSET_EMBED`, `auto` when unset or unrecognised.
pub fn embed_policy(prefix: &str) -> EmbedPolicy {
match std::env::var(format!("{prefix}_ASSET_EMBED"))
.unwrap_or_default()
.trim()
.to_ascii_lowercase()
.as_str()
{
"never" | "off" | "0" => EmbedPolicy::Never,
"always" | "force" | "1" => EmbedPolicy::Always,
_ => EmbedPolicy::Auto,
}
}
/// The checkout this binary was built in — every default root hangs off it.
fn checkout_root() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../..")
}
/// Where asset-ui keeps the user's main store.
pub fn main_store_root() -> PathBuf {
checkout_root().join("local/asset-ui/asset-server")
}
/// The ai-content library this checkout generates into — the same default
/// the standalone asset-server and asset-ui use.
fn library_root() -> PathBuf {
checkout_root().join("local/ai_content_library")
}
/// The root the app hosts over when it hosts.
///
/// `<PREFIX>_ASSET_ROOT` pins it. Otherwise the user's MAIN library —
/// asset-ui's store root — whenever it holds a catalog: hosting only happens
/// after the probe found nobody serving, and `AssetServer::start` takes the
/// same `server.lock` the daemon would, so losing the race resolves to
/// "attach instead", never to two servers over one WAL. The private seed
/// root is only for a machine with no main library at all — self-hosting a
/// fresh empty root next to a full library reads as "the app lost my
/// content".
pub fn default_store_root(prefix: &str, seed_dir: &str) -> PathBuf {
if let Ok(root) = std::env::var(format!("{prefix}_ASSET_ROOT")) {
return PathBuf::from(root);
}
let main = main_store_root();
if main.join("catalog.sqlite3").exists() {
return main;
}
checkout_root().join("local").join(seed_dir)
}
/// Does something on the other end of `addr` answer `GET /v1/health` like an
/// Asset Server?
///
/// Deliberately minimal and deliberately SHORT: this runs on the startup
/// path, and its only job is to tell "the user's server is up" from "that
/// port file is stale". A 400 ms budget is generous for loopback and
/// invisible to a human. Anything unexpected reads as "no" — the cost of a
/// false negative is one extra local store; the cost of a false positive is
/// an app that never connects.
fn health_answers(addr: SocketAddr) -> bool {
let Ok(mut stream) = TcpStream::connect_timeout(&addr, Duration::from_millis(400)) else {
return false;
};
let _ = stream.set_read_timeout(Some(Duration::from_millis(400)));
let _ = stream.set_write_timeout(Some(Duration::from_millis(400)));
let req = format!(
"GET /v1/health HTTP/1.1\r\nHost: {}\r\nConnection: close\r\n\r\n",
addr
);
if stream.write_all(req.as_bytes()).is_err() {
return false;
}
let mut buf = [0u8; 256];
let mut got = 0usize;
// One short read is enough: the status line is the first 15 bytes.
while got < 16 {
match stream.read(&mut buf[got..]) {
Ok(0) => break,
Ok(n) => got += n,
Err(_) => break,
}
}
buf[..got].starts_with(b"HTTP/1.1 200")
}
/// Listen briefly for an Asset Server beacon on the LAN discovery port.
///
/// This is the second chance for "a server is running": HTTP ports are
/// ephemeral and `listen` files go stale every launch, but a beacon is
/// live. One beacon period is 2 s, so we wait a little over one.
fn beacon_heard(wait_ms: u64) -> bool {
use makepad_asset_client::discovery::DiscoveryListener;
use makepad_asset_client::util::now_ms;
let Ok(listener) = DiscoveryListener::start(
makepad_asset_client::wire::DEFAULT_DISCOVERY_PORT,
10_000,
now_ms,
) else {
return false;
};
let deadline = std::time::Instant::now() + Duration::from_millis(wait_ms);
while std::time::Instant::now() < deadline {
if !listener.snapshot(now_ms()).is_empty() {
return true;
}
std::thread::sleep(Duration::from_millis(100));
}
false
}
/// Is an external store actually reachable right now?
pub fn external_store_reachable(hinted: Option<ApiEndpoints>) -> bool {
if let Some(endpoints) = hinted {
if health_answers(endpoints.control) {
return true;
}
}
// Ports move every launch; the beacon does not.
if beacon_heard(2_400) {
return true;
}
// THE USER'S MAIN STORE IS ALIVE BUT NOT ANSWERING YET (a succession
// handover, a stale `listen` file, a beacon missed by a hair): the
// lock holder is proof it exists. Self-hosting here would SILENTLY
// put this app on a private empty store — "no content in the grid" —
// so treat a held lock as reachable and let the attach path keep
// discovering; it retries on its own.
main_store_lock_held()
}
/// True when another process holds the main (asset-ui) store's server
/// lock — i.e. the user's library is hosted right now, whatever its ports.
fn main_store_lock_held() -> bool {
let root = main_store_root();
let Ok(file) = std::fs::OpenOptions::new()
.write(true)
.open(root.join("server.lock"))
else {
return false;
};
// The SAME advisory lock the store takes (File::try_lock): if we can
// take it, nobody serves that root; the file drop releases it.
match file.try_lock() {
Ok(()) => {
let _ = file.unlock();
false
}
Err(_) => true,
}
}
/// The in-process host (server + library publisher), held for as long as
/// the app runs.
///
/// Dropping it stops the publisher first and the server last, so the field
/// holding this must be declared AFTER anything that talks to the store —
/// the same drop-order discipline asset-ui's `AssetStore` uses.
pub struct LocalStore {
host: Host,
root: PathBuf,
}
impl LocalStore {
pub fn endpoints(&self) -> ApiEndpoints {
self.host.endpoints()
}
pub fn root(&self) -> &Path {
&self.root
}
pub fn control_addr(&self) -> SocketAddr {
self.host.endpoints().control
}
pub fn data_addr(&self) -> SocketAddr {
self.host.endpoints().data
}
pub fn server_id(&self) -> [u8; 16] {
self.host.server_id()
}
pub fn token(&self) -> &str {
self.host.token()
}
pub fn publisher_running(&self) -> bool {
self.host.publisher_running()
}
}
/// What [`resolve`] decided, and why — the `note` belongs in the app's
/// status line, because "which store am I on" is the first thing that
/// matters when content does not show up.
pub struct Resolved {
pub mode: StoreMode,
/// The embedded host, when hosting. The caller owns it for the life of
/// the app and points its client at [`LocalStore::endpoints`] with
/// [`LocalStore::server_id`] and [`LocalStore::token`].
pub local: Option<LocalStore>,
pub note: String,
}
/// Decide between attaching and hosting.
///
/// `pinned` is the caller saying "an explicit server was named" — naming a
/// server is not something you do by accident, so it always means attach.
/// `hinted` is where the caller last knew a server to live, for the health
/// probe.
pub fn resolve(
prefix: &str,
seed_dir: &str,
pinned: bool,
hinted: Option<ApiEndpoints>,
) -> Resolved {
let policy = embed_policy(prefix);
if pinned || policy == EmbedPolicy::Never {
let note = if pinned {
format!("asset server pinned by {prefix}_ASSET_SERVER")
} else {
format!("attach-only ({prefix}_ASSET_EMBED=never)")
};
return Resolved { mode: StoreMode::Attached, local: None, note };
}
if policy == EmbedPolicy::Auto && external_store_reachable(hinted) {
return Resolved {
mode: StoreMode::Attached,
local: None,
note: "attached to the running asset server".to_string(),
};
}
let root = default_store_root(prefix, seed_dir);
match host_at(&root, prefix) {
Ok(local) => {
let note = format!(
"local store on {} over {} · library publisher {}",
local.control_addr(),
root.file_name().and_then(|n| n.to_str()).unwrap_or("store"),
if local.publisher_running() { "on" } else { "off" }
);
Resolved { mode: StoreMode::Hosting, local: Some(local), note }
}
Err(error) => {
// Hosting failed — most often because another process already
// holds this root's lock. Say so and fall back to the attach
// path, which keeps retrying discovery on its own.
Resolved {
mode: StoreMode::Attached,
local: None,
note: format!("local store unavailable ({error}); attaching instead"),
}
}
}
}
/// Bring up the in-process host on loopback.
///
/// This is the standalone asset-server's own [`Host`] — catalog + CAS plus
/// the ai-content LIBRARY PUBLISHER, so a self-hosted app sees the same
/// `local/ai_content_library` rows asset-ui and the standalone server
/// publish, not a bare seed store. Two deployment defaults are overridden,
/// deliberately, and both stay: LOOPBACK ONLY, and NO discovery beacon.
fn host_at(root: &Path, prefix: &str) -> Result<LocalStore, String> {
std::fs::create_dir_all(root).map_err(|e| format!("create store root: {e}"))?;
let port: u16 = std::env::var(format!("{prefix}_ASSET_PORT"))
.ok()
.and_then(|p| p.trim().parse().ok())
.unwrap_or(0);
let mut cfg = HostConfig::new(root.to_path_buf());
cfg.control_addr = SocketAddr::from(([127, 0, 0, 1], port));
// The data plane always takes an OS-assigned port: pinning one would
// only create a second thing to collide.
cfg.data_addr = SocketAddr::from(([127, 0, 0, 1], 0));
cfg.beacon = false;
// The user's generation library, when this checkout has one: the
// publisher keeps turning it into catalog rows exactly as the
// standalone server would.
let library = library_root();
cfg.library = library.join("index.json").exists().then_some(library);
// Reference imports ON: the whole point of the local mode is pointing
// the store at content that stays where it is. Loopback-only and no
// prefix restriction, which is exactly the privilege this process
// already has over the user's own files.
cfg.blob_refs = BlobRefPolicy::local_host();
cfg.log = true;
let host = Host::start(&cfg).map_err(|e| format!("{e}"))?;
if host.token().is_empty() {
return Err("admin token file empty".to_string());
}
Ok(LocalStore { host, root: root.to_path_buf() })
}

View file

@ -0,0 +1,412 @@
//! The standalone Asset Server host.
//!
//! # Why this process exists
//!
//! The asset store is inherently multiplayer. asset-ui, the VJ, the sandbox
//! and headless workers are all clients of ONE catalog, and they come and go
//! independently. When the catalog lives inside one of those apps, that
//! app's lifetime becomes everybody's lifetime: closing the Asset UI window
//! takes the store and the events hub down under every other connected
//! client, which they see as `503 state unavailable` mid-session. This
//! binary breaks that coupling — the server outlives every window.
//!
//! # What it carries
//!
//! [`Host::start`] composes two things, both existing, tested code:
//!
//! 1. [`makepad_asset_store::AssetServer`] — catalog + CAS over the control
//! and data planes, the games publish path, the committed events hub,
//! game rooms, the blob GC janitor, and the LAN discovery beacon.
//! 2. The **library publisher** (`makepad_asset_importer::watch`) — whatever
//! the generation pipelines write into the ai-content library becomes
//! catalog rows. Headless, so it belongs beside the server rather than
//! inside a UI.
//!
//! # What deliberately stays client-side
//!
//! Everything that CREATES content (aicore: "the store stores, the client
//! creates"). Generation runs in the creating apps over their own ai-hub
//! fleet connections (`makepad-asset-creator`), chat sessions live in-app,
//! and loops that derive content with resources only a UI process has — the
//! offscreen thumbnail renders, the classic-game import wizards, the
//! stems/lyrics analysis bake — stay in the app and reach the catalog as
//! ordinary clients.
//!
//! # Single-owner laws
//!
//! - One process per server root (`<root>/server.lock`). Starting this
//! binary while the Asset UI still hosts the same root fails cleanly on
//! the lock instead of corrupting anything; stop the other host first, or
//! let it attach (see the README).
//! - An `AssetClient` cache root is single-owner too, so each loop gets its
//! own child of the work root.
pub mod embed;
use makepad_asset_client::{ApiEndpoints, AssetClient, ClientConfig, PublishRights};
use makepad_asset_store::{AssetServer, BlobRefPolicy, DiscoveryConfig, ServerConfig};
use std::net::{IpAddr, Ipv4Addr, SocketAddr};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::thread::JoinHandle;
/// Default namespace the library publisher publishes into. Same value the
/// creator apps publish with, so rows from either path sit side by side.
pub const DEFAULT_NAMESPACE: &str = "gen";
/// Everything the host needs. Every knob is a named field: nothing here is
/// read from the environment behind the operator's back (the binary maps
/// flags and two documented env fallbacks onto it).
#[derive(Clone, Debug)]
pub struct HostConfig {
/// Server root: catalog, CAS, `server.lock`, `server-id`, `listen`,
/// `admin-token`.
pub root: PathBuf,
/// Control plane bind address. The default is `0.0.0.0:0` — an
/// ephemeral port on every interface, published in `<root>/listen`,
/// which is exactly what the Asset UI's attach path reads.
pub control_addr: SocketAddr,
pub data_addr: SocketAddr,
/// Announce on the LAN discovery beacon so clients find this server
/// without configuration.
pub beacon: bool,
/// ai-content library to publish continuously. `None` = no publisher.
pub library: Option<PathBuf>,
/// Namespace for published rows.
pub namespace: String,
/// Parent for the loops' single-owner client caches. Defaults to the
/// server root's parent, which puts them exactly where the Asset UI's
/// own hosting mode puts them.
pub work_root: PathBuf,
/// Reference-import policy handed to the server. The deployment default
/// (owned blobs only); a loopback-only embedder may open this up.
pub blob_refs: BlobRefPolicy,
/// Log to stderr.
pub log: bool,
}
impl HostConfig {
/// Deployment defaults: ephemeral planes on every interface, beacon on.
pub fn new(root: PathBuf) -> Self {
let work_root = root
.parent()
.map(Path::to_path_buf)
.unwrap_or_else(|| root.clone());
Self {
root,
control_addr: SocketAddr::from(([0, 0, 0, 0], 0)),
data_addr: SocketAddr::from(([0, 0, 0, 0], 0)),
beacon: true,
library: None,
namespace: DEFAULT_NAMESPACE.to_string(),
work_root,
blob_refs: BlobRefPolicy::default(),
log: true,
}
}
}
/// A running host. Dropping it (or calling [`Host::shutdown`]) stops the
/// publisher first and the server last, so nothing is still publishing into
/// a catalog that is closing.
pub struct Host {
// Declaration order IS drop order: the loop is joined while the server
// it talks to is still answering.
publish: Option<BackgroundLoop>,
server: Option<AssetServer>,
endpoints: ApiEndpoints,
server_id: [u8; 16],
token: String,
library_error: Option<String>,
}
/// One owned background thread plus the flag that stops it.
struct BackgroundLoop {
stop: Arc<AtomicBool>,
join: Option<JoinHandle<()>>,
}
impl Drop for BackgroundLoop {
fn drop(&mut self) {
self.stop.store(true, Ordering::Release);
if let Some(join) = self.join.take() {
let _ = join.join();
}
}
}
impl Host {
/// Bring the server up, then the publisher.
///
/// The SERVER is the only thing whose failure is fatal — a publisher
/// that cannot start is reported ([`Host::library_error`]) and logged,
/// never a reason to deny every client its catalog.
pub fn start(config: &HostConfig) -> Result<Host, String> {
let mut cfg = ServerConfig::new(config.root.clone());
cfg.control_addr = config.control_addr;
cfg.data_addr = config.data_addr;
// The host mints (or re-mints) the root admin token: its own loops
// authenticate with it, and an attaching client reads it from the
// root the same way it does against an Asset-UI-hosted server.
cfg.bootstrap_admin = true;
cfg.discovery = config.beacon.then(DiscoveryConfig::lan_default);
cfg.blob_refs = config.blob_refs.clone();
cfg.log = config.log;
let server = AssetServer::start(cfg).map_err(|error| format!("asset server: {error}"))?;
let token = std::fs::read_to_string(config.root.join("admin-token"))
.map_err(|error| format!("admin token: {error}"))?
.trim()
.to_string();
if token.is_empty() {
return Err("admin token file is empty".to_string());
}
let endpoints = localized_endpoints(&server);
let server_id = server.server_id();
let (publish, library_error) = match &config.library {
None => (None, None),
Some(dir) => match start_publisher(config, dir, endpoints, server_id, &token) {
Ok(handle) => (Some(handle), None),
Err(error) => {
log(config.log, &format!("library publisher: {error}"));
(None, Some(error))
}
},
};
log(
config.log,
&format!(
"asset host up: control {} data {} · publisher {}",
endpoints.control,
endpoints.data,
publish.as_ref().map_or("off", |_| "on"),
),
);
Ok(Host {
publish,
server: Some(server),
endpoints,
server_id,
token,
library_error,
})
}
/// Loopback-reachable planes (a `0.0.0.0` bind resolved to `127.0.0.1`),
/// which is what `<root>/listen` advertises too.
pub fn endpoints(&self) -> ApiEndpoints {
self.endpoints
}
pub fn server_id(&self) -> [u8; 16] {
self.server_id
}
/// The root admin bearer token this host bootstrapped.
pub fn token(&self) -> &str {
&self.token
}
pub fn publisher_running(&self) -> bool {
self.publish.is_some()
}
/// Why the library publisher is not running, when it was asked for.
pub fn library_error(&self) -> Option<&str> {
self.library_error.as_deref()
}
/// Stop the publisher, then the server. Idempotent; also runs on drop.
pub fn shutdown(&mut self) {
drop(self.publish.take());
if let Some(mut server) = self.server.take() {
server.shutdown();
}
}
}
impl Drop for Host {
fn drop(&mut self) {
self.shutdown();
}
}
/// A server bound to `0.0.0.0` is reached over loopback from this process.
fn localized_endpoints(server: &AssetServer) -> ApiEndpoints {
let localize = |addr: SocketAddr| {
if addr.ip().is_unspecified() {
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), addr.port())
} else {
addr
}
};
ApiEndpoints {
control: localize(server.control_addr()),
data: localize(server.data_addr()),
}
}
/// Publish everything the pipelines write into the ai-content library.
///
/// The client connects HERE, on the caller's thread, so "the publisher could
/// not reach the server it is hosted beside" is an answer the operator (and
/// the tests) get immediately instead of a log line scrolling past.
fn start_publisher(
config: &HostConfig,
dir: &Path,
endpoints: ApiEndpoints,
server_id: [u8; 16],
token: &str,
) -> Result<BackgroundLoop, String> {
let mut client_config = ClientConfig::new(config.work_root.join("publish-cache"));
client_config.token = Some(token.to_string());
let mut client = AssetClient::connect(client_config, endpoints, Some(server_id))
.map_err(|error| format!("cannot connect to the local server: {error}"))?;
let stop = Arc::new(AtomicBool::new(false));
let thread_stop = stop.clone();
let dir = dir.to_path_buf();
let namespace = config.namespace.clone();
let log_enabled = config.log;
let join = std::thread::Builder::new()
.name("asset-host-publish".to_string())
.spawn(move || {
log(
log_enabled,
&format!("library publisher: watching {}", dir.display()),
);
makepad_asset_importer::watch::run(
&mut client,
&dir,
&namespace,
&PublishRights::generated_cc0(),
log_enabled,
&thread_stop,
);
log(log_enabled, "library publisher: stopped");
})
.map_err(|error| format!("cannot spawn the publisher thread: {error}"))?;
Ok(BackgroundLoop { stop, join: Some(join) })
}
fn log(enabled: bool, message: &str) {
if enabled {
eprintln!("[asset-host] {message}");
}
}
#[cfg(test)]
mod tests {
use super::*;
fn test_root(name: &str) -> PathBuf {
std::env::temp_dir().join(format!(
"mp_asset_host_{}_{}_{}",
std::process::id(),
name,
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0)
))
}
/// An isolated host: loopback-only ephemeral planes, NO beacon (a test
/// must never advertise itself to the operator's LAN).
fn isolated(name: &str) -> HostConfig {
let base = test_root(name);
let mut config = HostConfig::new(base.join("server"));
config.control_addr = "127.0.0.1:0".parse().unwrap();
config.data_addr = "127.0.0.1:0".parse().unwrap();
config.beacon = false;
config.log = false;
config.work_root = base.join("work");
config
}
fn cleanup(config: &HostConfig) {
if let Some(parent) = config.root.parent() {
let _ = std::fs::remove_dir_all(parent);
}
}
#[test]
fn the_host_serves_a_catalog_and_shuts_down_cleanly() {
let config = isolated("serves");
let mut host = Host::start(&config).expect("host starts");
assert!(!host.token().is_empty(), "the host bootstraps an admin token");
assert!(
config.root.join("listen").is_file(),
"the listen file is how every client attaches"
);
// A real client over real sockets, exactly as any other process
// would reach it.
let mut client_config = ClientConfig::new(config.work_root.join("probe-cache"));
client_config.token = Some(host.token().to_string());
let client =
AssetClient::connect(client_config, host.endpoints(), Some(host.server_id()))
.expect("client connects to the standalone host");
assert_eq!(client.server_id(), host.server_id());
drop(client);
host.shutdown();
host.shutdown();
cleanup(&config);
}
#[test]
fn a_second_host_on_one_root_is_refused_by_the_root_lock() {
let config = isolated("locked");
let host = Host::start(&config).expect("first host starts");
let error = match Host::start(&config) {
Ok(_) => panic!("the root lock is single-owner: a second host must be refused"),
Err(error) => error,
};
assert!(
error.contains("locked by another server process"),
"the refusal must name the lock, got {error}"
);
drop(host);
cleanup(&config);
}
/// The publisher is a CLIENT of the server beside it, and a library that
/// is not there yet must not cost anyone their catalog: the loop starts,
/// the server keeps serving, and shutdown still joins cleanly.
#[test]
fn a_missing_library_directory_never_costs_the_catalog_its_server() {
let mut config = isolated("library");
config.library = Some(config.work_root.join("nonexistent-library"));
let mut host = Host::start(&config).expect("host starts");
assert!(
host.publisher_running(),
"the publisher connected: {:?}",
host.library_error()
);
assert_eq!(host.library_error(), None);
let mut client_config = ClientConfig::new(config.work_root.join("probe-cache"));
client_config.token = Some(host.token().to_string());
AssetClient::connect(client_config, host.endpoints(), Some(host.server_id()))
.expect("the server serves while the publisher waits for its library");
host.shutdown();
cleanup(&config);
}
#[test]
fn the_work_root_defaults_beside_the_server_root() {
let config = HostConfig::new(PathBuf::from("/store/local/asset-ui/asset-server"));
assert_eq!(config.work_root, PathBuf::from("/store/local/asset-ui"));
assert!(config.beacon, "a fleet host is discoverable by default");
assert_eq!(config.namespace, DEFAULT_NAMESPACE);
assert_eq!(
config.control_addr.port(),
0,
"ephemeral by default: the listen file is the address of record"
);
}
}

View file

@ -0,0 +1,187 @@
//! `makepad-asset-server` — run the Asset Server as its own process.
//!
//! Parse flags into a [`HostConfig`], start the host, and wait for
//! SIGINT/SIGTERM to shut it down cleanly. With no flags at all it serves the
//! checkout's standard store root on ephemeral ports, announces itself on the
//! LAN, and publishes the ai-content library — i.e. everything the Asset
//! UI's embedded mode does, minus the window.
//!
//! See `README.md` beside this file for the deployment runbook.
use makepad_app_asset_server::{Host, HostConfig};
use std::net::SocketAddr;
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::Duration;
const USAGE: &str = "\
makepad-asset-server [options]
The standalone, multiplayer-first Asset Server: catalog + CAS, events hub,
game rooms, LAN beacon, plus the ai-content library publisher. Clients
(asset-ui, vj, sandbox) attach to it and may come and go without ever
taking the store down. Generation runs in the creating apps over their own
fleet connections the store stores.
Options:
--root <dir> Server root. Default: $AI_CONTENT_ASSET_ROOT, else
<checkout>/local/asset-ui/asset-server
--control <addr> Control plane bind (default 0.0.0.0:0 ephemeral;
the real address is written to <root>/listen)
--data <addr> Data plane bind (default 0.0.0.0:0)
--no-beacon Do not announce on the LAN discovery beacon
--library <dir> ai-content library to publish continuously.
Default: <checkout>/local/ai_content_library
--no-library Do not run the library publisher
--namespace <ns> Namespace for published rows (default gen)
--work <dir> Parent for the loops' client caches
(default: the server root's parent)
--quiet No stderr logging
--help This text
";
static STOP: AtomicBool = AtomicBool::new(false);
// Minimal async-signal-safe handling: the handler only flips an atomic and
// the main thread polls it.
extern "C" fn on_signal(_sig: i32) {
STOP.store(true, Ordering::SeqCst);
}
fn install_signal_handlers() {
#[cfg(unix)]
{
extern "C" {
fn signal(signum: i32, handler: usize) -> usize;
}
const SIGINT: i32 = 2;
const SIGTERM: i32 = 15;
unsafe {
signal(SIGINT, on_signal as *const () as usize);
signal(SIGTERM, on_signal as *const () as usize);
}
}
}
fn fail(message: &str) -> ! {
eprintln!("makepad-asset-server: {message}");
eprintln!("{USAGE}");
std::process::exit(2);
}
/// The checkout this binary was built from, so the default root and library
/// are the same paths the Asset UI uses. `MAKEPAD_ROOT` overrides, exactly
/// as it does there.
fn checkout_root() -> PathBuf {
if let Ok(root) = std::env::var("MAKEPAD_ROOT") {
return PathBuf::from(root);
}
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../..")
}
fn default_root() -> PathBuf {
if let Ok(root) = std::env::var("AI_CONTENT_ASSET_ROOT") {
return PathBuf::from(root);
}
checkout_root().join("local/asset-ui/asset-server")
}
fn parse_config() -> HostConfig {
let mut args = std::env::args().skip(1);
let mut root: Option<PathBuf> = None;
let mut control: Option<SocketAddr> = None;
let mut data: Option<SocketAddr> = None;
let mut beacon = true;
let mut library: Option<PathBuf> = None;
let mut no_library = false;
let mut namespace: Option<String> = None;
let mut work: Option<PathBuf> = None;
let mut log = true;
let value_of = |name: &str, args: &mut dyn Iterator<Item = String>| -> String {
match args.next() {
Some(value) => value,
None => fail(&format!("{name} needs a value")),
}
};
while let Some(arg) = args.next() {
match arg.as_str() {
"--root" => root = Some(PathBuf::from(value_of("--root", &mut args))),
"--control" => {
let value = value_of("--control", &mut args);
control = Some(value.parse().unwrap_or_else(|_| fail("malformed --control address")));
}
"--data" => {
let value = value_of("--data", &mut args);
data = Some(value.parse().unwrap_or_else(|_| fail("malformed --data address")));
}
"--no-beacon" => beacon = false,
"--library" => library = Some(PathBuf::from(value_of("--library", &mut args))),
"--no-library" => no_library = true,
"--namespace" => namespace = Some(value_of("--namespace", &mut args)),
"--work" => work = Some(PathBuf::from(value_of("--work", &mut args))),
"--quiet" => log = false,
"--help" | "-h" => {
println!("{USAGE}");
std::process::exit(0);
}
other => fail(&format!("unknown flag {other}")),
}
}
if no_library && library.is_some() {
fail("--library and --no-library contradict each other");
}
let mut config = HostConfig::new(root.unwrap_or_else(default_root));
if let Some(addr) = control {
config.control_addr = addr;
}
if let Some(addr) = data {
config.data_addr = addr;
}
config.beacon = beacon;
config.library = if no_library {
None
} else {
Some(library.unwrap_or_else(|| checkout_root().join("local/ai_content_library")))
};
if let Some(namespace) = namespace {
if namespace.is_empty() {
fail("--namespace needs a value");
}
config.namespace = namespace;
}
if let Some(work) = work {
config.work_root = work;
}
config.log = log;
config
}
fn main() {
let config = parse_config();
install_signal_handlers();
if config.log {
eprintln!(
"[asset-host] root {} · library {} · beacon {}",
config.root.display(),
config
.library
.as_ref()
.map(|dir| dir.display().to_string())
.unwrap_or_else(|| "off".to_string()),
if config.beacon { "on" } else { "off" },
);
}
let mut host = match Host::start(&config) {
Ok(host) => host,
Err(error) => {
eprintln!("makepad-asset-server: failed to start: {error}");
std::process::exit(1);
}
};
while !STOP.load(Ordering::SeqCst) {
std::thread::sleep(Duration::from_millis(200));
}
host.shutdown();
}

61
apps/asset-ui/Cargo.toml Normal file
View file

@ -0,0 +1,61 @@
[package]
name = "makepad-app-asset-ui"
version = "0.1.0"
edition = "2021"
description = "Asset UI: catalog, licensed pack import, viewers, and fleet generation over the Asset Server"
license = "MIT OR Apache-2.0"
[dependencies]
makepad-widgets = { path = "../../widgets" }
# Service wire types + fleet/affinity scheduler (transport here is
# cx.http_request; the lib's blocking client is for worker-thread consumers).
makepad-ai-hub = { path = "../../libs/ai/hub", default-features = false }
makepad-micro-serde = { path = "../../libs/micro_serde" }
makepad-base64 = { path = "../../libs/base64" }
# Mesh viewer: Renderer stage for a single GLB.
makepad-render = { path = "../../libs/render" }
# Animated-character viewer: base-color texture extraction from the GLB
# (skin.rs ignores materials by design — the caller binds the texture).
makepad-gltf = { path = "../../libs/gltf" }
# Splat viewer: ViewSplat + XrSceneView desktop host.
makepad-xr = { path = "../../xr" }
makepad-media-view = { path = "../../libs/media_view" }
# REAL Asset Server frontend: shared session lifecycle (discovery/auth/
# retry), catalog runtimes, committed-event subscriber, verified cache. The
# VJ worker owns these crates; this app consumes the public API only.
makepad-asset-client = { path = "../../libs/asset/client" }
makepad-asset-creator = { path = "../../libs/asset/creator" }
# Shared preview/viewer widgets: the pool every host draws catalog content
# with (VJ and the DJ surface adopt the same set).
# The renderer feature brings the 3D faces (mesh turntable, world walker).
makepad-asset-widgets = { path = "../../libs/asset/widgets", features = ["renderer"] }
makepad-asset-data = { path = "../../libs/asset/data" }
makepad-asset-chat = { path = "../../libs/asset/chat" }
# The chat pane itself — ONE component, shared with the game sandbox: the
# broker session + its worker channel, the transcript with its tool chips
# and rate meter, and the list widget. This app supplies the personality
# (namespace `gen`, client profile `gen`, its own client-executed tools).
makepad-chat-ui = { path = "../../libs/chat_ui" }
# Embedded catalog/chat broker for now: the app starts the real Asset
# Server in-process instead of waiting for a LAN beacon.
makepad-asset-store = { path = "../../libs/asset/store" }
# Local licensed-pack compiler (Kenney Import). Same fail-closed walk as
# `makepad-asset-importer --import-pack`; no network fetch.
makepad-asset-importer = { path = "../../libs/asset/importer" }
# The vision-annotation pass, for its VERSION only: this app draws the
# annotation bar and must agree with the store on which annotator version
# the catalog owes. The work itself is `annotate.asset`, a vision job the
# fleet coordinator claims — nothing here runs a model.
makepad-asset-annotate = { path = "../../libs/asset/annotate" }
# The library is 241 MP3s and 4 WAVs: the transport has to be able to play
# what the catalog actually holds, and the container has to be knowable from
# the bytes when a digest-named cache object has no name to read. Zero
# dependencies, and the same decoder the preview well draws with.
makepad-audio-decode = { path = "../../libs/audio_decode" }
# "Split audio layers": the analysis bake. Separation (BS-RoFormer on the
# ggml Metal runtime), the word-aligned lyrics bake, and the ONE shared
# encode/publish of both as typed side-channel files — the same crate the VJ
# calls, so a track analysed here is a track a deck never has to separate.
makepad-ai-stems = { path = "../../libs/ai/models/stems" }
makepad-audio-lyrics = { path = "../../libs/audio_lyrics" }
makepad-audio-sidechannels = { path = "../../libs/audio_sidechannels" }

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

1188
apps/asset-ui/src/audio.rs Normal file

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,866 @@
//! 3D camera-facing viewer for a stateful billboard.
//!
//! The sprite is an unlit, nearest-sampled, alpha-tested quad (Doom/Quake
//! pixels, not a PBR statue). Orbit/zoom the camera; the quad yaws to face
//! it. Named animation states are listed at the bottom — click one to play
//! it. Preview always loops so walk/attack/pain can be inspected.
use makepad_render::{
preview_scene_state, set_pass_camera, DrawSceneAlpha, DrawSceneCube,
DrawSceneScreen, DrawSceneShadow, DrawSceneSkinned, DrawSceneSky, DrawSceneTerrain, DrawSceneTexture,
PreviewLook, PreviewStage, SceneDraws, Renderer,
};
use makepad_asset_importer::stateful_billboard::{SpriteFrame, StatefulBillboard};
use makepad_widgets::*;
use std::path::Path;
script_mod! {
use mod.prelude.widgets_internal.*
use mod.widgets.*
mod.widgets.BillboardViewBase = #(BillboardView::register_widget(vm))
mod.widgets.BillboardView = set_type_default() do mod.widgets.BillboardViewBase{
width: Fill
height: Fill
draw_cube +: {
light_dir: vec3(0.35, 0.8, 0.45)
}
draw_alpha +: {
light_dir: vec3(0.35, 0.8, 0.45)
}
draw_terrain +: {
light_dir: vec3(0.35, 0.8, 0.45)
}
draw_models +: {
light_dir: vec3(0.35, 0.8, 0.45)
}
// Unlit + nearest + discard keyed texels. The stock screen shader
// forces alpha 1 and bilinear-samples, which is why sprites looked
// like a dark cardboard cutout.
draw_sprite +: {
alpha_blend: true
backface_culling: false
pixel: fn() {
let color = self.tex.sample_nearest(self.v_uv)
if color.w < 0.08 {
discard()
}
return vec4(color.x, color.y, color.z, color.w)
}
}
draw_hud +: {
text_style: theme.font_regular{font_size: 9}
color: #xffffffcc
}
draw_state +: {
text_style: theme.font_bold{font_size: 9}
color: #xc6cfd8
}
draw_state_on +: {
text_style: theme.font_bold{font_size: 9}
color: #x7db8f0
}
}
}
#[derive(Clone)]
struct BbFrame {
texture: Texture,
w: u32,
h: u32,
flip: bool,
}
struct BbState {
name: String,
fps: f32,
/// `by_rot[0]` = omnidirectional; `by_rot[1..=facings]` = view.
by_rot: Vec<Vec<BbFrame>>,
}
impl BbState {
fn frames_for(&self, facing: u8) -> &[BbFrame] {
let i = facing as usize;
if let Some(frames) = self.by_rot.get(i) {
if !frames.is_empty() {
return frames;
}
}
if let Some(frames) = self.by_rot.first() {
if !frames.is_empty() {
return frames;
}
}
self.by_rot.get(1).map(Vec::as_slice).unwrap_or(&[])
}
}
#[derive(Script, ScriptHook, WidgetRef, WidgetRegister)]
pub struct BillboardView {
#[uid]
uid: WidgetUid,
#[source]
source: ScriptObjectRef,
#[walk]
walk: Walk,
#[layout]
layout: Layout,
#[live]
draw_bg: DrawSceneTexture,
#[live]
draw_cube: DrawSceneCube,
#[live]
draw_alpha: DrawSceneAlpha,
#[live]
draw_sky: DrawSceneSky,
#[live]
draw_terrain: DrawSceneTerrain,
#[live]
draw_shadow: DrawSceneShadow,
#[live]
draw_models: DrawSceneSkinned,
#[live]
draw_sprite: DrawSceneScreen,
#[live]
draw_hud: DrawText,
#[live]
draw_state: DrawText,
#[live]
draw_state_on: DrawText,
#[live(vec4(0.03, 0.045, 0.075, 1.0))]
clear_color: Vec4f,
#[new]
pass: DrawPass,
#[new]
draw_list: DrawList,
#[new]
color_texture: Texture,
#[new]
depth_texture: Texture,
#[rust]
area: Area,
#[rust(false)]
initialized: bool,
#[rust(false)]
world_built: bool,
#[rust]
renderer: Renderer,
#[rust]
look: PreviewLook,
#[rust]
states: Vec<BbState>,
#[rust(0usize)]
state_i: usize,
#[rust(0usize)]
frame_i: usize,
#[rust]
frame_accum: f64,
#[rust]
last_time: Option<f64>,
#[rust]
status: String,
#[rust(0.0f32)]
orbit_yaw: f32,
#[rust(1u8)]
facings: u8,
#[rust(-0.12f32)]
orbit_pitch: f32,
#[rust]
orbit_last_abs: Option<DVec2>,
#[rust]
view_rect: Rect,
#[rust]
chip_rects: Vec<Rect>,
#[rust]
next_frame: NextFrame,
/// World metres per authored texel. Locked from the standing
/// (walk/idle) frame so a wide death splat does not grow to 1.55m tall.
#[rust(0.02f32)]
px_scale: f32,
}
const STAND_HEIGHT: f32 = 1.55;
fn viewer_state_name(name: &str) -> bool {
// pose_a / pose_b are just one letter from the sheet (walk frame A,
// walk frame B, …), not authored poses. Keep walk/attack/pain/death.
!name.starts_with("pose_")
}
fn pixel_scale_from_stand(stand_h_px: u32) -> f32 {
STAND_HEIGHT / stand_h_px.max(1) as f32
}
fn frame_world_size(w: u32, h: u32, px: f32) -> (f32, f32) {
((w as f32 * px).max(0.04), (h as f32 * px).max(0.04))
}
/// Frame pixels of a stateful billboard, decoded once per source file.
///
/// A packed-sheet manifest (`sheet <cols> <cell_w> <cell_h>` + `cell <n>`
/// per frame) names ONE PNG for every frame: it is decoded once and each
/// frame is cut out of its cell at the authored size. A legacy manifest
/// still reads one PNG per frame. Both hand back the same pixels, so every
/// viewer plays a sheet-backed actor exactly as it played loose frames.
pub struct BillboardFrames<'a> {
/// Where the frame files sit, for a manifest that came off disk with its
/// pictures beside it. `None` when the caller handed over the ONE sheet
/// in memory — which is how a catalog asset arrives: its manifest and
/// its sheet are two blobs of one asset, named by digest, with no
/// directory to be siblings in.
manifest: Option<&'a Path>,
bb: &'a StatefulBillboard,
decoded: Vec<(String, Option<ImageBuffer>)>,
/// The single decoded sheet, when there is one.
sheet: Option<ImageBuffer>,
}
impl<'a> BillboardFrames<'a> {
pub fn new(manifest: &'a Path, bb: &'a StatefulBillboard) -> Self {
Self {
manifest: Some(manifest),
bb,
decoded: Vec::new(),
sheet: None,
}
}
/// Every frame cut out of ONE sheet already in memory. The packed form
/// names one PNG for all frames and gives each a cell, so nothing needs
/// to be resolved against a directory.
pub fn from_sheet(bb: &'a StatefulBillboard, sheet_png: &[u8]) -> Self {
Self {
manifest: None,
bb,
decoded: Vec::new(),
sheet: ImageBuffer::from_png(sheet_png).ok(),
}
}
/// Decoded pixels of one frame, or `None` when its picture is missing.
pub fn image(&mut self, frame: &SpriteFrame) -> Option<ImageBuffer> {
let source = match (&self.sheet, self.manifest) {
(Some(sheet), _) => sheet,
(None, Some(manifest)) => {
if !self.decoded.iter().any(|(f, _)| *f == frame.file) {
let path = self.bb.resolve_frame(manifest, frame);
let image = std::fs::read(&path)
.ok()
.and_then(|bytes| ImageBuffer::from_png(&bytes).ok());
self.decoded.push((frame.file.clone(), image));
}
self.decoded
.iter()
.find(|(f, _)| *f == frame.file)
.and_then(|(_, i)| i.as_ref())?
}
(None, None) => return None,
};
match self.bb.frame_rect(frame) {
Some(rect) => cut_cell(source, rect),
// A loose-frame manifest handed only a sheet cannot say which
// pixels are this frame's: better nothing than the whole sheet
// drawn as if it were one sprite.
None if self.sheet.is_some() => None,
None => Some(source.clone()),
}
}
}
/// Copy `(x, y, w, h)` out of a decoded sheet. Refuses a rect that leaves
/// the sheet rather than smearing whatever follows it in memory.
fn cut_cell(sheet: &ImageBuffer, rect: (u32, u32, u32, u32)) -> Option<ImageBuffer> {
let (x, y, w, h) = (
rect.0 as usize,
rect.1 as usize,
rect.2 as usize,
rect.3 as usize,
);
if w == 0 || h == 0 || x + w > sheet.width || y + h > sheet.height {
return None;
}
if sheet.data.len() < sheet.width * sheet.height {
return None;
}
let mut rgba = Vec::with_capacity(w * h * 4);
for row in 0..h {
let start = (y + row) * sheet.width + x;
for &p in &sheet.data[start..start + w] {
rgba.push(((p >> 16) & 0xFF) as u8);
rgba.push(((p >> 8) & 0xFF) as u8);
rgba.push((p & 0xFF) as u8);
rgba.push(((p >> 24) & 0xFF) as u8);
}
}
ImageBuffer::new(&rgba, w, h).ok()
}
impl BillboardView {
pub fn load_manifest(&mut self, cx: &mut Cx, path: &Path) {
let text = std::fs::read_to_string(path).unwrap_or_default();
let Ok(bb) = StatefulBillboard::parse(&text) else {
self.no_billboard(cx, "not a stateful billboard");
return;
};
let pixels = BillboardFrames::new(path, &bb);
self.build(cx, &bb, pixels);
}
/// The same actor, from BYTES: a catalog asset's manifest text and the
/// one packed sheet its frames are cut from. The store names both by
/// digest — there is no directory for them to be siblings in — so the
/// well hands them over instead of a path to walk.
///
/// This is the SAME widget the Create surface uses, with the same state
/// list, the same 8 rotations and the same clock; only where the pixels
/// came from differs.
pub fn show_sheet(&mut self, cx: &mut Cx, manifest: &str, sheet_png: &[u8]) {
let Ok(bb) = StatefulBillboard::parse(manifest) else {
self.no_billboard(cx, "not a stateful billboard");
return;
};
let pixels = BillboardFrames::from_sheet(&bb, sheet_png);
self.build(cx, &bb, pixels);
}
fn no_billboard(&mut self, cx: &mut Cx, why: &str) {
self.states.clear();
self.status = why.to_string();
self.area.redraw(cx);
}
fn build(&mut self, cx: &mut Cx, bb: &StatefulBillboard, mut pixels: BillboardFrames<'_>) {
self.facings = bb.resolved_facings();
let mut states = Vec::new();
let names: Vec<String> = if bb.states.is_empty() {
vec![bb.preview.clone()]
} else {
bb.states.iter().map(|s| s.name.clone()).collect()
};
for name in names {
if !viewer_state_name(&name) {
continue;
}
let nrot = self.facings.max(1) as usize;
let mut by_rot = vec![Vec::new(); nrot + 1];
for rot in 0..=self.facings {
for faced in bb.frames_for_state_facing(&name, rot) {
if let Some(image) = pixels.image(faced.frame) {
let w = image.width.max(1) as u32;
let h = image.height.max(1) as u32;
by_rot[rot as usize].push(BbFrame {
texture: image.into_new_texture(cx),
w: faced.frame.w.max(w).max(1),
h: faced.frame.h.max(h).max(1),
flip: faced.flip,
});
}
}
}
if by_rot.iter().all(|f| f.is_empty()) {
continue;
}
states.push(BbState {
fps: bb.state_fps(&name) as f32,
name,
by_rot,
});
}
let start = states
.iter()
.position(|s| s.name == bb.preview)
.unwrap_or(0);
let stand_h = states
.iter()
.find(|s| matches!(s.name.as_str(), "walk" | "idle" | "ready" | "see"))
.or_else(|| states.first())
.and_then(|s| {
s.by_rot
.iter()
.flat_map(|f| f.iter())
.map(|f| f.h)
.max()
})
.unwrap_or(56);
self.px_scale = pixel_scale_from_stand(stand_h);
self.states = states;
self.state_i = start;
self.frame_i = 0;
self.frame_accum = 0.0;
self.last_time = None;
self.status = if self.states.is_empty() {
"billboard has no frames".into()
} else {
format!(
"{} · {} states · click a state · drag orbit",
bb.prefix.to_ascii_uppercase(),
self.states.len()
)
};
self.next_frame = cx.new_next_frame();
self.area.redraw(cx);
}
fn current_frame(&self) -> Option<&BbFrame> {
let state = self.states.get(self.state_i)?;
let facing = makepad_asset_importer::stateful_billboard::StatefulBillboard::facing_for_yaw(
self.orbit_yaw,
self.facings,
);
let frames = state.frames_for(facing);
if frames.is_empty() {
return None;
}
frames.get(self.frame_i % frames.len())
}
fn set_state(&mut self, cx: &mut Cx, index: usize) {
if index >= self.states.len() || index == self.state_i {
return;
}
self.state_i = index;
self.frame_i = 0;
self.frame_accum = 0.0;
self.area.redraw(cx);
}
fn ensure_initialized(&mut self, cx: &mut Cx) {
if self.initialized {
return;
}
self.initialized = true;
self.color_texture = Texture::new_with_format(
cx,
TextureFormat::RenderBGRAu8 {
size: TextureSize::Auto,
initial: true,
},
);
self.depth_texture = Texture::new_with_format(
cx,
TextureFormat::DepthD32 {
size: TextureSize::Auto,
initial: true,
},
);
self.pass.set_color_texture(
cx,
&self.color_texture,
DrawPassClearColor::ClearWith(self.clear_color),
);
self.pass
.set_depth_texture(cx, &self.depth_texture, DrawPassClearDepth::ClearWith(1.0));
cx.passes[self.pass.draw_pass_id()].keep_camera_matrix = true;
}
fn ensure_world(&mut self) {
if self.world_built {
return;
}
self.world_built = true;
self.look.target = vec3f(0.0, 0.7, 0.0);
self.look.distance = 2.6;
self.look.fov = 45.0;
}
fn pose_sprite(&mut self) {
let Some((texture, w, h, flip)) = self
.current_frame()
.map(|f| (f.texture.clone(), f.w, f.h, f.flip))
else {
self.draw_sprite.screen_size = vec4(0.0, 0.0, 0.0, 0.0);
return;
};
let (width, height) = frame_world_size(w, h, self.px_scale);
self.draw_sprite.draw_vars.set_texture(0, &texture);
// Constant texel scale: a short wide death frame stays short, not
// stretched back up to stand height. Feet on the slab.
// Negative width X-flips Duke/Doom mirrored side views.
let width = if flip { -width } else { width };
self.draw_sprite.screen_pos = vec4(0.0, height * 0.5, 0.0, self.orbit_yaw);
self.draw_sprite.screen_size = vec4(width, height, 0.0, 0.0);
self.draw_sprite.depth_clip = 1.0;
}
fn tick_anim(&mut self, now: f64) -> bool {
let Some(state) = self.states.get(self.state_i) else {
return false;
};
let facing = makepad_asset_importer::stateful_billboard::StatefulBillboard::facing_for_yaw(
self.orbit_yaw,
self.facings,
);
let n = state.frames_for(facing).len();
if n < 2 {
return false;
}
let last = self.last_time.replace(now).unwrap_or(now);
self.frame_accum += (now - last).min(0.25);
let step = 1.0 / state.fps.max(1.0) as f64;
let mut changed = false;
while self.frame_accum >= step {
self.frame_accum -= step;
// Preview always loops — attack/pain/death are one-shots in the
// manifest for the game, but the viewer has to keep playing.
self.frame_i = (self.frame_i + 1) % n;
changed = true;
}
changed
}
}
impl WidgetNode for BillboardView {
fn widget_uid(&self) -> WidgetUid {
self.uid
}
fn walk(&mut self, _cx: &mut Cx) -> Walk {
self.walk
}
fn area(&self) -> Area {
self.area
}
fn redraw(&mut self, cx: &mut Cx) {
self.area.redraw(cx);
}
}
impl Widget for BillboardView {
fn handle_event(&mut self, cx: &mut Cx, event: &Event, _scope: &mut Scope) {
if self.next_frame.is_event(event).is_some() {
if self.tick_anim(cx.seconds_since_app_start()) {
self.area.redraw(cx);
}
if !self.states.is_empty() {
self.next_frame = cx.new_next_frame();
}
}
// Raw mouse only while orbit-dragging. Clicks start through hits()
// so a dropdown over/near this pane keeps its FingerDown.
if self.orbit_last_abs.is_some() {
match event {
Event::MouseMove(me) => {
if let Some(last) = self.orbit_last_abs {
let delta = me.abs - last;
self.orbit_yaw -= delta.x as f32 * 0.01;
self.orbit_pitch =
(self.orbit_pitch + delta.y as f32 * 0.01).clamp(-1.2, 1.2);
self.orbit_last_abs = Some(me.abs);
self.area.redraw(cx);
}
}
Event::MouseUp(me) if me.button.is_primary() => {
self.orbit_last_abs = None;
}
_ => {}
}
}
match event.hits(cx, self.area) {
Hit::FingerDown(fe) if fe.is_primary_hit() => {
let mut hit_chip = false;
for (i, rect) in self.chip_rects.iter().enumerate() {
if rect.contains(fe.abs) {
self.set_state(cx, i);
hit_chip = true;
break;
}
}
if !hit_chip {
self.orbit_last_abs = Some(fe.abs);
cx.set_cursor(MouseCursor::Grabbing);
}
}
Hit::FingerScroll(se) => {
let axis = if se.scroll.y.abs() > f64::EPSILON {
se.scroll.y
} else {
se.scroll.x
};
if axis.abs() > f64::EPSILON {
let factor = if axis > 0.0 { 1.0 / 0.92 } else { 0.92 };
self.look.distance =
(self.look.distance * factor).clamp(1.2, 12.0);
self.area.redraw(cx);
}
}
Hit::FingerHoverIn(_) => {
cx.set_cursor(MouseCursor::Grab);
}
_ => {}
}
}
fn draw_walk(&mut self, cx: &mut Cx2d, _scope: &mut Scope, walk: Walk) -> DrawStep {
let rect = cx.walk_turtle_with_area(&mut self.area, walk);
if rect.size.x <= 1.0 || rect.size.y <= 1.0 {
return DrawStep::done();
}
self.ensure_initialized(cx.cx);
self.ensure_world();
self.pose_sprite();
self.view_rect = rect;
self.pass.set_size(cx, rect.size);
self.pass.set_color_texture(
cx,
&self.color_texture,
DrawPassClearColor::ClearWith(self.clear_color),
);
self.pass
.set_depth_texture(cx, &self.depth_texture, DrawPassClearDepth::ClearWith(1.0));
cx.make_child_pass(&self.pass);
cx.begin_pass(&self.pass, None);
self.look.yaw = self.orbit_yaw;
self.look.pitch = self.orbit_pitch;
let scene_state = preview_scene_state(self.look, rect, cx.time());
if let Some(scene_state) = scene_state {
set_pass_camera(cx.cx, &self.pass, &scene_state);
let cx3d = &mut Cx3d::new(cx.cx);
self.renderer.set_models(Vec::new());
let mut draws = SceneDraws {
cube: &mut self.draw_cube,
alpha: &mut self.draw_alpha,
sky: &mut self.draw_sky,
sky_analytic: None,
terrain: &mut self.draw_terrain,
shadow: Some(&mut self.draw_shadow),
shadow_sdf: None,
firework: None,
flare: None,
water: None,
screen: Some(&mut self.draw_sprite),
screen_instances: &[],
view_model: None,
};
let mut stage = PreviewStage::statue();
stage.ground_half = 6.0;
stage.ground_color = vec4(0.22, 0.24, 0.28, 1.0);
self.renderer.draw_preview(
cx3d,
&mut self.draw_list,
&mut draws,
self.look,
stage,
scene_state,
None,
Some(&mut self.draw_models),
);
}
cx.end_pass(&self.pass);
self.draw_bg.draw_vars.set_texture(0, &self.color_texture);
self.draw_bg.draw_abs(cx, rect);
self.area = self.draw_bg.area();
cx.set_pass_area(&self.pass, self.area);
self.chip_rects.clear();
let mut x = rect.pos.x + 10.0;
let y = rect.pos.y + rect.size.y - 22.0;
for (i, state) in self.states.iter().enumerate() {
let label = if i == self.state_i {
format!("[{}]", state.name)
} else {
state.name.clone()
};
let w = (label.len() as f64) * 6.4 + 8.0;
let chip = Rect {
pos: dvec2(x, y - 2.0),
size: dvec2(w, 16.0),
};
self.chip_rects.push(chip);
if i == self.state_i {
self.draw_state_on.draw_abs(cx, dvec2(x, y), &label);
} else {
self.draw_state.draw_abs(cx, dvec2(x, y), &label);
}
x += w + 8.0;
}
if self.states.is_empty() {
self.draw_hud.draw_abs(cx, dvec2(rect.pos.x + 10.0, y), &self.status);
}
DrawStep::done()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn viewer_hides_sheet_letter_poses() {
assert!(viewer_state_name("walk"));
assert!(viewer_state_name("attack"));
assert!(viewer_state_name("idle"));
assert!(!viewer_state_name("pose_a"));
assert!(!viewer_state_name("pose_b"));
}
#[test]
fn death_frame_keeps_stand_texel_scale() {
let px = pixel_scale_from_stand(56);
let (sw, sh) = frame_world_size(40, 56, px);
assert!((sh - STAND_HEIGHT).abs() < 0.01, "{sh}");
let (dw, dh) = frame_world_size(80, 20, px);
// Wide splat: wider, much shorter — not a 1.55m-tall billboard.
assert!(dw > sw);
assert!(dh < sh * 0.5);
assert!(dh < STAND_HEIGHT * 0.5);
}
fn sheet_scratch(name: &str) -> std::path::PathBuf {
let dir = std::env::temp_dir().join(format!("mp-bbview-{name}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
dir
}
/// 2 cells of 4x6, cell 0 red, cell 1 green, padding transparent.
fn write_two_cell_sheet(path: &std::path::Path) {
let (w, h) = (8usize, 6usize);
let mut rgba = vec![0u8; w * h * 4];
for y in 0..h {
for x in 0..w {
let i = (y * w + x) * 4;
let color = match (x < 4, x >= 4) {
(true, _) => [255u8, 0, 0, 255],
(_, true) => [0, 255, 0, 255],
_ => [0, 0, 0, 0],
};
rgba[i..i + 4].copy_from_slice(&color);
}
}
let png =
makepad_asset_importer::classic_import::encode_png_rgba(&rgba, w as u32, h as u32)
.unwrap();
std::fs::write(path, png).unwrap();
}
#[test]
fn sheet_frames_are_cut_out_of_their_cells() {
let dir = sheet_scratch("cells");
write_two_cell_sheet(&dir.join("troo.png"));
let manifest = dir.join("troo.billboard");
std::fs::write(
&manifest,
"stateful-billboard 1\n\
prefix troo\n\
role character\n\
preview walk\n\
sheet 2 4 6\n\
state walk 0 2 1 8\n\
frame 0 A 1 4 6 troo.png cell 0\n\
frame 1 B 1 3 5 troo.png cell 1\n",
)
.unwrap();
let text = std::fs::read_to_string(&manifest).unwrap();
let bb = StatefulBillboard::parse(&text).unwrap();
let mut pixels = BillboardFrames::new(&manifest, &bb);
let a = pixels.image(&bb.frames[0]).expect("cell 0");
assert_eq!((a.width, a.height), (4, 6));
assert_eq!(a.data[0] & 0x00ff_ffff, 0x00ff_0000, "cell 0 is red");
// A frame smaller than its cell keeps its authored size, top-left.
let b = pixels.image(&bb.frames[1]).expect("cell 1");
assert_eq!((b.width, b.height), (3, 5));
assert_eq!(b.data[0] & 0x00ff_ffff, 0x0000_ff00, "cell 1 is green");
let _ = std::fs::remove_dir_all(&dir);
}
/// The catalog hands over BYTES, not a directory: a stateful billboard
/// is one asset carrying two blobs, both named by digest, so nothing can
/// be resolved as a sibling file. The same cells come out either way —
/// the Library well and the Create viewer play the same actor.
#[test]
fn a_sheet_in_memory_cuts_the_same_cells_as_one_on_disk() {
let dir = sheet_scratch("bytes");
let sheet_path = dir.join("troo.png");
write_two_cell_sheet(&sheet_path);
let manifest = "stateful-billboard 1\n\
prefix troo\n\
role character\n\
preview walk\n\
sheet 2 4 6\n\
state walk 0 2 1 8\n\
frame 0 A 1 4 6 troo.png cell 0\n\
frame 1 B 1 3 5 troo.png cell 1\n";
let bb = StatefulBillboard::parse(manifest).unwrap();
let png = std::fs::read(&sheet_path).unwrap();
let mut pixels = BillboardFrames::from_sheet(&bb, &png);
let a = pixels.image(&bb.frames[0]).expect("cell 0");
assert_eq!((a.width, a.height), (4, 6));
assert_eq!(a.data[0] & 0x00ff_ffff, 0x00ff_0000, "cell 0 is red");
let b = pixels.image(&bb.frames[1]).expect("cell 1");
assert_eq!((b.width, b.height), (3, 5));
assert_eq!(b.data[0] & 0x00ff_ffff, 0x0000_ff00, "cell 1 is green");
let _ = std::fs::remove_dir_all(&dir);
}
/// A loose-frame manifest names one PNG per frame and gives no cells, so
/// a single sheet cannot say which pixels are whose. It draws nothing
/// rather than showing the whole sheet as if it were one sprite.
#[test]
fn loose_frames_cannot_be_cut_out_of_one_sheet() {
let dir = sheet_scratch("loose-bytes");
let sheet_path = dir.join("trooa1.png");
write_two_cell_sheet(&sheet_path);
let manifest = "stateful-billboard 1\n\
prefix troo\n\
role character\n\
preview walk\n\
state walk 0 1 1 8\n\
frame 0 A 1 8 6 trooa1.png\n";
let bb = StatefulBillboard::parse(manifest).unwrap();
let png = std::fs::read(&sheet_path).unwrap();
let mut pixels = BillboardFrames::from_sheet(&bb, &png);
assert!(pixels.image(&bb.frames[0]).is_none());
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn legacy_frames_still_decode_whole_files() {
let dir = sheet_scratch("legacy");
write_two_cell_sheet(&dir.join("trooa1.png"));
let manifest = dir.join("troo.billboard");
std::fs::write(
&manifest,
"stateful-billboard 1\n\
prefix troo\n\
role character\n\
preview walk\n\
state walk 0 1 1 8\n\
frame 0 A 1 8 6 trooa1.png\n",
)
.unwrap();
let text = std::fs::read_to_string(&manifest).unwrap();
let bb = StatefulBillboard::parse(&text).unwrap();
let mut pixels = BillboardFrames::new(&manifest, &bb);
let a = pixels.image(&bb.frames[0]).expect("whole png");
assert_eq!((a.width, a.height), (8, 6), "no sheet header, no cropping");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn a_cell_outside_the_sheet_is_refused() {
let dir = sheet_scratch("outside");
write_two_cell_sheet(&dir.join("troo.png"));
let manifest = dir.join("troo.billboard");
std::fs::write(
&manifest,
"stateful-billboard 1\n\
prefix troo\n\
role character\n\
preview walk\n\
sheet 2 4 6\n\
state walk 0 1 1 8\n\
frame 0 A 1 4 6 troo.png cell 9\n",
)
.unwrap();
let text = std::fs::read_to_string(&manifest).unwrap();
let bb = StatefulBillboard::parse(&text).unwrap();
let mut pixels = BillboardFrames::new(&manifest, &bb);
assert!(
pixels.image(&bb.frames[0]).is_none(),
"smeared pixels are worse than a missing frame"
);
let _ = std::fs::remove_dir_all(&dir);
}
}

1161
apps/asset-ui/src/chat.rs Normal file

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,167 @@
//! Post-process pass: rewrite path-like library labels into catalog titles
//! (Doom lump → Imp, Quake MDL → Shambler). No model, no invented lore.
//!
//! This file also held a stub "vision caption" layer that wrote
//! `<file>.vision.json` sidecars saying `skipped`: nothing ever called it
//! and nothing ever read one. The real vision pass is
//! [`crate::annotate_queue`], which writes descriptions into the STORE,
//! where search can index them — a sidecar beside a local GLB was never
//! reachable by a catalog query.
use crate::library::{Library, LibraryMeta};
use makepad_asset_importer::stateful_billboard::{mesh_title, sprite_title, world_title};
/// Rewrite path-like labels to catalog titles. Does not invent lore.
pub fn apply_catalog_names(library: &mut Library) -> usize {
let mut changed = 0usize;
for item in &mut library.index.items {
if let Some(next) = better_label(item) {
if next != item.label {
item.label = next;
changed += 1;
}
}
}
if changed > 0 {
let _ = library.persist();
}
changed
}
fn better_label(item: &LibraryMeta) -> Option<String> {
let domain = item.domain.as_str();
let stem = prompt_asset_stem(&item.prompt)
.or_else(|| label_asset_stem(&item.label))
.unwrap_or(item.label.as_str());
let title = match domain {
"billboard" => sprite_title(stem),
"world" => world_title(stem),
"character" | "weapon" | "prop" => mesh_title(stem),
_ => return None,
};
if title.is_empty() || title == item.label {
None
} else {
Some(title)
}
}
fn is_meta_segment(s: &str) -> bool {
let l = s.trim().to_ascii_lowercase();
if l.is_empty() {
return true;
}
if l.starts_with("http://") || l.starts_with("https://") {
return true;
}
if l.contains("cc0")
|| l.contains("cc-by")
|| l.contains("cc by")
|| l.contains("public domain")
|| l.contains("shareware")
|| l.contains("license")
{
return true;
}
matches!(
l.as_str(),
"character"
| "world"
| "billboard"
| "weapon"
| "prop"
| "audio"
| "music"
| "texture"
| "image"
| "video"
)
}
/// Classic prompts are `source · kind · stem · license · url`. Kenney is
/// `source · stem · license · url`. Never take the license as the title.
fn prompt_asset_stem(prompt: &str) -> Option<&str> {
let segs: Vec<&str> = prompt
.split('·')
.map(str::trim)
.filter(|s| !s.is_empty() && !is_meta_segment(s))
.collect();
segs.iter()
.rev()
.find(|s| !s.contains(' '))
.copied()
.or_else(|| {
segs.first().and_then(|s| {
s.rsplit([' ', '/'])
.map(str::trim)
.find(|w| !w.is_empty() && !is_meta_segment(w))
})
})
}
fn label_asset_stem(label: &str) -> Option<&str> {
let stem = label.rsplit('/').next().unwrap_or(label).trim();
if stem.is_empty() || is_meta_segment(stem) {
None
} else {
Some(stem)
}
}
#[cfg(test)]
mod tests {
use super::*;
fn item(domain: &str, label: &str, prompt: &str) -> LibraryMeta {
LibraryMeta {
file: "lib-1.glb".into(),
label: label.into(),
domain: domain.into(),
content_type: "model/gltf-binary".into(),
prompt: prompt.into(),
group_id: None,
group_label: None,
tags: None,
enhanced_tags: None,
product: None,
}
}
#[test]
fn kaykit_prompt_does_not_become_the_license() {
let broken = item(
"character",
"Cc0 1.0",
"KayKit knight · character · CC0-1.0 · https://kaylousberg.com/",
);
assert_eq!(better_label(&broken).as_deref(), Some("Knight"));
let pathy = item(
"character",
"kaykit/adventurers/barbarian",
"KayKit adventurers · character · barbarian · CC0-1.0 · https://kaylousberg.com/",
);
assert_eq!(better_label(&pathy).as_deref(), Some("Barbarian"));
}
#[test]
fn kenney_and_classic_stems_stay_the_asset() {
let kenney = item(
"character",
"kenney/characters/rogue",
"Kenney characters · rogue · CC-BY-4.0 · https://kenney.nl/",
);
assert_eq!(better_label(&kenney).as_deref(), Some("Rogue"));
let classic = item(
"character",
"imp",
"Freedoom freedoom2 · character · imp · CC0-1.0 · https://freedoom.github.io/",
);
assert_eq!(better_label(&classic).as_deref(), Some("Imp"));
let duke = item(
"billboard",
"tile-0123",
"Duke Nukem 3D (shareware) duke3d · billboard · tile-0123 · 3D-Realms-shareware · https://archive.org/",
);
assert_eq!(better_label(&duke).as_deref(), Some("TILE-0123"));
}
}

View file

@ -0,0 +1,288 @@
//! User-saved pipeline snapshots. Click to run, × to delete.
use crate::pipeline::{
format_music_duration, GenParams, EDIT_STRENGTHS, VIDEO_INTERPOLATE, PRESETS, IMAGE_SIZES, IMAGE_STEPS, MESH_FACE_COUNTS,
MESH_TEXTURE_SIZES, MUSIC_LENGTHS, VIDEO_LENGTHS, VIDEO_SIZES,
};
use makepad_micro_serde::*;
use std::fs;
use std::path::{Path, PathBuf};
pub const MAX_FAST_PRESETS: usize = 8;
#[derive(Clone, Debug, Default, SerJson, DeJson)]
pub struct SavedModelPin {
pub domain: String,
pub model: String,
}
#[derive(Clone, Debug, SerJson, DeJson)]
pub struct SavedFastPreset {
pub id: String,
pub name: String,
pub pipeline: String,
pub models: Vec<SavedModelPin>,
pub voice: Option<String>,
pub image_w: u32,
pub image_h: u32,
pub image_steps: Option<u32>,
pub mesh_texture: u32,
pub mesh_faces: Option<u32>,
/// Option so presets saved before the flag existed still load (None =
/// false).
pub mesh_trellis_texture: Option<bool>,
/// Motion-prompt override (None/empty = playable set).
pub motion_prompt: Option<String>,
pub video_w: u32,
pub video_h: u32,
pub video_frames: u32,
pub video_steps: u32,
/// Option so presets saved before the flag existed still load (None =
/// true — the H3 default is an audible clip).
pub video_audio: Option<bool>,
pub edit_strength: Option<f32>,
pub video_interpolate: Option<u32>,
pub image_lora: Option<String>,
pub image_lora_strength: Option<f32>,
pub music_seconds: u32,
/// Options so presets saved before the enhance stage existed still load.
pub enhance_upscale: Option<u32>,
pub enhance_interpolate: Option<u32>,
pub enhance_flow: Option<bool>,
}
#[derive(Clone, Debug, Default, SerJson, DeJson)]
struct SavedFastFile {
presets: Vec<SavedFastPreset>,
}
pub fn store_path() -> PathBuf {
PathBuf::from(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../local/ai_content_library/fast_presets.json"
))
}
pub fn load(path: &Path) -> Vec<SavedFastPreset> {
let Ok(text) = fs::read_to_string(path) else {
return Vec::new();
};
SavedFastFile::deserialize_json(&text)
.map(|file| file.presets)
.unwrap_or_default()
}
pub fn save(path: &Path, presets: &[SavedFastPreset]) -> Result<(), String> {
if let Some(parent) = path.parent() {
fs::create_dir_all(parent).map_err(|e| e.to_string())?;
}
let file = SavedFastFile {
presets: presets.to_vec(),
};
let tmp = path.with_extension("json.tmp");
fs::write(&tmp, file.serialize_json()).map_err(|e| e.to_string())?;
fs::rename(&tmp, path).map_err(|e| e.to_string())
}
/// Short label from the current panel. Keep it chip-sized.
pub fn auto_name(
pipeline: &str,
models: &[(String, String)],
gen: &GenParams,
) -> String {
let short_pipe = pipeline
.replace("expand → ", "exp→")
.replace("image → ", "img→")
.replace(" (playable)", "")
.replace(" (small)", "");
let mut bits = vec![short_pipe];
for (domain, model) in models {
bits.push(short_model(domain, model));
}
if pipeline.contains("music") {
bits.push(format_music_duration(gen.music_seconds));
} else if pipeline.contains("video") {
bits.push(format!("{}f", gen.video_frames));
} else if pipeline.contains("image") && !pipeline.contains("mesh") {
bits.push(format!("{}×{}", gen.image_size.0, gen.image_size.1));
}
let name = bits.join(" · ");
if name.chars().count() > 36 {
name.chars().take(34).collect::<String>() + ""
} else {
name
}
}
fn short_model(_domain: &str, model: &str) -> String {
model
.trim_start_matches("minimax-")
.trim_end_matches("-1.5-xl")
.replace("music3-python", "py")
.replace("music3", "m3")
.replace("ace-step", "ace")
.replace("flux1-", "fx1-")
.replace("sa3-sfx", "sa3")
.replace("moss-sfx", "moss")
.replace("woosh-sfx", "woosh")
.replace("indextts-2.5", "clone")
.replace("birefnet-hr", "matte")
.replace("da3-metric-large", "da3")
.replace("sam3-1-multiplex", "sam3")
.replace("hunyuan3d-paint-2.1", "hy")
.replace("pbr-testpattern", "pbr-test")
.replace("trellis-2", "trellis")
}
pub fn snapshot(
pipeline: &str,
models: Vec<(String, String)>,
voice: Option<String>,
gen: &GenParams,
name: String,
) -> SavedFastPreset {
SavedFastPreset {
id: format!(
"fp-{}",
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis())
.unwrap_or(0)
),
name,
pipeline: pipeline.to_string(),
models: models
.into_iter()
.map(|(domain, model)| SavedModelPin { domain, model })
.collect(),
voice,
image_w: gen.image_size.0,
image_h: gen.image_size.1,
image_steps: gen.image_steps,
mesh_texture: gen.mesh_texture_size,
mesh_faces: gen.mesh_faces,
mesh_trellis_texture: Some(gen.mesh_trellis_texture),
motion_prompt: Some(gen.motion_prompt.clone()),
video_w: gen.video_size.0,
video_h: gen.video_size.1,
video_frames: gen.video_frames,
video_steps: gen.video_steps,
video_audio: Some(gen.video_audio),
edit_strength: Some(gen.edit_strength),
video_interpolate: Some(gen.video_interpolate),
image_lora: gen.image_lora.as_ref().map(|(name, _)| name.clone()),
image_lora_strength: gen.image_lora.as_ref().map(|(_, strength)| *strength),
music_seconds: gen.music_seconds,
enhance_upscale: Some(gen.enhance_upscale),
enhance_interpolate: Some(gen.enhance_interpolate),
enhance_flow: Some(gen.enhance_flow),
}
}
pub fn apply_gen(saved: &SavedFastPreset) -> GenParams {
GenParams {
image_size: (saved.image_w, saved.image_h),
image_steps: saved.image_steps,
mesh_texture_size: saved.mesh_texture,
mesh_faces: saved.mesh_faces,
mesh_trellis_texture: saved.mesh_trellis_texture.unwrap_or(false),
motion_prompt: saved.motion_prompt.clone().unwrap_or_default(),
video_size: (saved.video_w, saved.video_h),
video_frames: saved.video_frames,
video_steps: saved.video_steps,
video_audio: saved.video_audio.unwrap_or(true),
edit_strength: saved.edit_strength.unwrap_or(1.0),
video_interpolate: saved.video_interpolate.unwrap_or(1),
image_lora: saved
.image_lora
.clone()
.filter(|name| !name.is_empty())
.map(|name| (name, saved.image_lora_strength.unwrap_or(1.0))),
music_seconds: saved.music_seconds,
enhance_upscale: saved.enhance_upscale.unwrap_or(2),
enhance_interpolate: saved.enhance_interpolate.unwrap_or(2),
enhance_flow: saved.enhance_flow.unwrap_or(true),
// Loop-ness derives from the preset row at dispatch, never a spec.
video_loop: false,
}
}
/// Dropdown index of `factor` in [`VIDEO_INTERPOLATE`] (off when unknown).
pub fn nearest_video_interpolate(factor: u32) -> usize {
VIDEO_INTERPOLATE.iter().position(|f| *f == factor).unwrap_or(0)
}
/// Dropdown index of the nearest [`EDIT_STRENGTHS`] entry.
pub fn nearest_edit_strength(strength: f32) -> usize {
EDIT_STRENGTHS
.iter()
.enumerate()
.min_by(|a, b| {
(a.1 - strength)
.abs()
.partial_cmp(&(b.1 - strength).abs())
.unwrap_or(std::cmp::Ordering::Equal)
})
.map(|(i, _)| i)
.unwrap_or(0)
}
pub fn nearest_image_size(w: u32, h: u32) -> usize {
IMAGE_SIZES
.iter()
.position(|size| *size == (w, h))
.unwrap_or(0)
}
pub fn nearest_image_steps(steps: Option<u32>) -> usize {
match steps {
None => 0,
Some(s) => IMAGE_STEPS
.iter()
.position(|v| *v == s)
.map(|i| i + 1)
.unwrap_or(0),
}
}
pub fn nearest_mesh_texture(size: u32) -> usize {
MESH_TEXTURE_SIZES
.iter()
.position(|v| *v == size)
.unwrap_or(0)
}
pub fn nearest_mesh_faces(faces: Option<u32>) -> usize {
match faces {
None => 0,
Some(n) => MESH_FACE_COUNTS
.iter()
.position(|v| *v == n)
.unwrap_or(0),
}
}
pub fn nearest_video_size(w: u32, h: u32) -> usize {
VIDEO_SIZES
.iter()
.position(|size| *size == (w, h))
.unwrap_or(0)
}
pub fn nearest_video_len(frames: u32, steps: u32) -> usize {
VIDEO_LENGTHS
.iter()
.position(|pair| *pair == (frames, steps))
.unwrap_or(0)
}
pub fn nearest_music_len(seconds: u32) -> usize {
MUSIC_LENGTHS
.iter()
.position(|v| *v == seconds)
.unwrap_or(0)
}
pub fn pipeline_index(name: &str) -> Option<usize> {
PRESETS.iter().position(|p| p.name == name)
}

View file

@ -0,0 +1,765 @@
//! Fleet discovery over `cx.http_request`: polls `GET /health` +
//! `GET /models` on every fleet endpoint and feeds the parsed JSON into
//! [`makepad_ai_hub::fleet::BoxSnapshot`]s — the scheduler
//! (`fleet::pick_box` / `pick_for_domain`) is pure over those snapshots.
//!
//! Endpoint lifecycle (the part that keeps the fleet free of duplicates):
//!
//! - Every row is LEASED from the LAN beacon set. Reconcile adds live
//! beacons that alias no existing row and drops rows whose beacon lease
//! expired.
//! - After health responses land, rows are COALESCED by durable identity:
//! `node_key` (persisted per service instance, stable across restarts)
//! first, live `node_id` (random per process start) as the fallback for
//! services predating `node_key`. One service reachable under several
//! addresses (127.0.0.1 + LAN ip) collapses to one row; a restart keeps
//! the row because the beacon url and `node_key` both survive it.
//! - Rows are NEVER merged merely for sharing a host: two service
//! instances on one box (e.g. 10.0.0.169:8123 and :8080) have distinct
//! node_keys, distinct model sets and both stay. [`host_groups`] offers a
//! display-only grouping so the UI can render one physical-node block
//! with its service endpoints; scheduling still routes per endpoint.
//!
//! One request pair in flight per endpoint, so a down box (OS connect
//! timeout can be a minute) never stacks requests. In-flight requests are
//! keyed by url, not row index — rows may be coalesced away mid-flight and
//! the late response is then dropped instead of updating the wrong box.
use makepad_ai_hub::discovery::DiscoveredNode;
use makepad_ai_hub::fleet::BoxSnapshot;
use makepad_ai_hub::protocol::{HealthJson, JobStatusJson, JobsJson, LorasJson, ModelsJson};
use makepad_micro_serde::DeJson;
use makepad_widgets::*;
use std::collections::HashMap;
enum Pending {
Health,
Models,
Jobs,
Loras,
}
struct InFlight {
pending: Pending,
base_url: String,
sent_at: std::time::Instant,
}
pub struct FleetPoll {
pub snapshots: Vec<BoxSnapshot>,
/// /health round-trip per endpoint, from the last completed poll.
pub latency_ms: Vec<Option<u64>>,
/// `GET /jobs` per endpoint (running first, then queued) — other
/// clients' work included, so the UI can show and cancel it. Empty for
/// services without the endpoint.
pub jobs: Vec<Vec<JobStatusJson>>,
/// `GET /loras` per endpoint: adapter names the box can apply to FLUX.1.
/// Empty for services without the endpoint.
pub loras: Vec<Vec<String>>,
/// Endpoints with any request in flight (index-parallel to snapshots).
busy: Vec<bool>,
/// Consecutive failed `/health` probes per endpoint. A box is only
/// declared offline after [`OFFLINE_AFTER`] of them: a node denoising a
/// whole GPU or streaming 17 GB of weights answers late, and late is not
/// the same as gone. The panel used to flip such a box to "down" on the
/// first slow answer and back on the next, once a second.
health_fails: Vec<u32>,
/// A failing streak has already been reported for this endpoint, per
/// probe kind (health, models); cleared by that probe's next success.
/// Without it a busy node writes one log line per poll — the log filled
/// with thousands of them while every box in fact answered 200 to curl.
/// The two are separate because a box that answers `/health` and drops
/// `/models` would otherwise clear the latch every round and complain
/// every round.
health_quiet: Vec<bool>,
models_quiet: Vec<bool>,
in_flight: HashMap<LiveId, InFlight>,
}
/// Consecutive `/health` failures before a box is shown as offline.
/// Three misses at the poll cadence is several seconds of real silence, well
/// past any answer a loaded box is merely slow to give.
const OFFLINE_AFTER: u32 = 3;
/// What a failed `/health` probe means for a row.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum ProbeVerdict {
/// Keep showing the box exactly as it was; `remaining` more consecutive
/// misses would declare it offline.
Hold { remaining: u32 },
/// Enough silence: the box is gone, not merely busy.
Offline,
}
fn verdict_after_misses(misses: u32) -> ProbeVerdict {
if misses >= OFFLINE_AFTER {
ProbeVerdict::Offline
} else {
ProbeVerdict::Hold { remaining: OFFLINE_AFTER - misses }
}
}
impl FleetPoll {
pub fn new() -> Self {
Self {
snapshots: Vec::new(),
latency_ms: Vec::new(),
jobs: Vec::new(),
loras: Vec::new(),
busy: Vec::new(),
health_fails: Vec::new(),
health_quiet: Vec::new(),
models_quiet: Vec::new(),
in_flight: HashMap::new(),
}
}
fn remove_row(&mut self, index: usize) {
self.snapshots.remove(index);
self.latency_ms.remove(index);
self.jobs.remove(index);
self.loras.remove(index);
self.busy.remove(index);
self.health_fails.remove(index);
self.health_quiet.remove(index);
self.models_quiet.remove(index);
}
/// One `/health` answer landed: the row is live and its miss streak
/// (and the log-once latch) resets. Returns the streak it ended.
fn note_health_ok(&mut self, index: usize) -> u32 {
let missed = self.health_fails[index];
self.health_fails[index] = 0;
self.health_quiet[index] = false;
missed
}
/// One `/health` probe missed. The row is NOT touched until the verdict
/// says offline — a box mid-denoise or mid-download is still there.
fn note_health_miss(&mut self, index: usize) -> ProbeVerdict {
self.health_fails[index] = self.health_fails[index].saturating_add(1);
verdict_after_misses(self.health_fails[index])
}
fn row_by_url(&self, url: &str) -> Option<usize> {
self.snapshots
.iter()
.position(|snapshot| snapshot.base_url == url)
}
fn live_node_id(&self, index: usize) -> Option<u64> {
self.snapshots[index].health.as_ref().and_then(|h| h.node_id)
}
fn get(&mut self, cx: &mut Cx, base_url: String, url: String, pending: Pending) {
let request_id = LiveId::unique();
let request = crate::http::get(url);
cx.http_request(request_id, request);
self.in_flight.insert(
request_id,
InFlight {
pending,
base_url,
sent_at: std::time::Instant::now(),
},
);
}
/// Reconcile against the FULL live beacon set: add live nodes that
/// alias no existing row, and drop rows whose lease expired. A row
/// survives by beacon url match or by live `node_id` match (its beacon
/// source address may change while the identity holds). Returns true
/// when a row was added or removed.
pub fn reconcile_discovered(&mut self, nodes: &[DiscoveredNode]) -> bool {
let mut changed = false;
for node in nodes {
let known = (0..self.snapshots.len()).any(|index| {
self.snapshots[index].base_url == node.base_url
|| self.live_node_id(index) == Some(node.node_id)
});
if !known {
self.snapshots.push(BoxSnapshot::new(&node.base_url));
self.latency_ms.push(None);
self.jobs.push(Vec::new());
self.loras.push(Vec::new());
self.busy.push(false);
self.health_fails.push(0);
self.health_quiet.push(false);
self.models_quiet.push(false);
changed = true;
}
}
let mut index = 0;
while index < self.snapshots.len() {
let keep = nodes.iter().any(|node| {
node.base_url == self.snapshots[index].base_url
|| self.live_node_id(index) == Some(node.node_id)
});
if keep {
index += 1;
} else {
self.remove_row(index);
changed = true;
}
}
if self.coalesce_aliases() {
changed = true;
}
changed
}
/// Collapse rows that PROVE to be the same service instance: identical
/// durable `node_key`, or identical live `node_id` (pre-node_key
/// services). Sharing a host is deliberately NOT identity — same-box
/// sibling instances (own cache dirs, own node_keys, own model sets)
/// must both stay; see [`host_groups`] for the display-side grouping.
/// The surviving row is the earlier index.
fn coalesce_aliases(&mut self) -> bool {
let mut changed = false;
let mut index = 0;
while index < self.snapshots.len() {
let key = self.snapshots[index]
.health
.as_ref()
.and_then(|h| h.node_key.clone());
let id = self.live_node_id(index);
let mut other = index + 1;
while other < self.snapshots.len() {
let other_key = self.snapshots[other]
.health
.as_ref()
.and_then(|h| h.node_key.clone());
let other_id = self.live_node_id(other);
let same_instance = (key.is_some() && key == other_key)
|| (id.is_some() && id == other_id);
if !same_instance {
other += 1;
continue;
}
log!(
"fleet: {} is the same service as {} — coalesced",
self.snapshots[other].base_url,
self.snapshots[index].base_url
);
self.remove_row(other);
changed = true;
}
if other >= self.snapshots.len() {
index += 1;
}
}
changed
}
/// Kick a health+models round for every endpoint without one in flight.
/// Call from an interval timer.
pub fn poll(&mut self, cx: &mut Cx) {
for index in 0..self.snapshots.len() {
if self.busy[index] {
continue;
}
self.busy[index] = true;
let base_url = self.snapshots[index].base_url.clone();
let url = format!("{base_url}/health");
self.get(cx, base_url, url, Pending::Health);
}
}
/// Routes one NetworkResponse; returns true when a snapshot changed
/// (caller redraws the fleet panel). A response for a row that was
/// coalesced or lease-expired while the request flew is dropped.
pub fn handle_response(&mut self, cx: &mut Cx, item: &NetworkResponse) -> bool {
let (request_id, response, transport_error) = match item {
NetworkResponse::HttpResponse {
request_id,
response,
} => (*request_id, Some(response), None),
NetworkResponse::HttpError { request_id, error } => {
(*request_id, None, Some(error.message.clone()))
}
_ => return false,
};
let Some(in_flight) = self.in_flight.remove(&request_id) else {
return false;
};
let Some(index) = self.row_by_url(&in_flight.base_url) else {
return false;
};
match in_flight.pending {
Pending::Health => {
let status = response.map(|r| r.status_code);
let health = response
.filter(|r| r.status_code == 200)
.and_then(|r| r.get_string_body())
.and_then(|body| HealthJson::deserialize_json_lenient(&body).ok());
let up = health.is_some();
if up {
let missed = self.note_health_ok(index);
if missed > 0 {
log!(
"fleet: {} answered again after {missed} missed probe(s)",
in_flight.base_url
);
}
self.latency_ms[index] = Some(in_flight.sent_at.elapsed().as_millis() as u64);
self.snapshots[index].health = health;
// A fresh health may prove this row aliases another
// (same node_key/node_id) — collapse before fetching
// models so the duplicate never renders.
self.coalesce_aliases();
if let Some(index) = self.row_by_url(&in_flight.base_url) {
let base_url = self.snapshots[index].base_url.clone();
let url = format!("{base_url}/models");
self.get(cx, base_url, url, Pending::Models);
}
return true;
}
// SLOW IS NOT GONE. A missed probe costs the row its
// latency reading and nothing else until the misses pile up.
let verdict = self.note_health_miss(index);
self.latency_ms[index] = None;
self.busy[index] = false;
let reason = transport_error.unwrap_or_else(|| match status {
Some(code) => format!("HTTP {code}"),
None => "unreadable /health".to_string(),
});
let was_up = self.snapshots[index].is_up();
match verdict {
ProbeVerdict::Offline => {
if was_up {
// ONE line, naming the reason, at the moment the
// box actually goes offline.
log!(
"fleet: {} offline after {OFFLINE_AFTER} failed /health probes: \
{reason}",
in_flight.base_url
);
}
self.snapshots[index].health = None;
self.snapshots[index].models.clear();
was_up
}
ProbeVerdict::Hold { remaining } => {
if !self.health_quiet[index] {
self.health_quiet[index] = true;
log!(
"fleet: {} missed a /health probe ({reason}) — holding it up \
for {remaining} more",
in_flight.base_url
);
}
// The row keeps its last known health and models: a
// busy box is still the same box.
false
}
}
}
Pending::Models => {
self.busy[index] = false;
// One line per failing STREAK. A saturated box drops a
// model probe now and then; that is worth knowing once, not
// thirty times a minute.
let complain = |quiet: &mut bool, message: String| {
if !*quiet {
*quiet = true;
log!("{}", message);
}
};
let quiet = &mut self.models_quiet[index];
let models = match response {
Some(response) if response.status_code == 200 => {
match response.get_string_body() {
Some(body) => match ModelsJson::deserialize_json_lenient(&body) {
Ok(models) => Some(models),
Err(error) => {
complain(
quiet,
format!(
"fleet: {} /models JSON rejected ({} bytes): {:?}",
in_flight.base_url,
body.len(),
error
),
);
None
}
},
None => {
complain(
quiet,
format!(
"fleet: {} /models returned no text body",
in_flight.base_url
),
);
None
}
}
}
Some(response) => {
complain(
quiet,
format!(
"fleet: {} /models returned HTTP {}",
in_flight.base_url, response.status_code
),
);
None
}
None => {
complain(
quiet,
format!(
"fleet: {} /models request failed ({}) — keeping its last model list",
in_flight.base_url,
transport_error.as_deref().unwrap_or("no response")
),
);
None
}
};
if let Some(models) = models {
*quiet = false;
self.snapshots[index].models = models.models;
}
// Live job list last (running + queued, other clients too).
let base_url = self.snapshots[index].base_url.clone();
let url = format!("{base_url}/jobs");
self.busy[index] = true;
self.get(cx, base_url, url, Pending::Jobs);
true
}
Pending::Jobs => {
self.busy[index] = false;
let jobs = response
.filter(|r| r.status_code == 200)
.and_then(|r| r.get_string_body())
.and_then(|body| JobsJson::deserialize_json_lenient(&body).ok())
.map(|j| j.jobs)
// Older services (no /jobs) → nothing to show, not an error.
.unwrap_or_default();
let changed = self.jobs[index].len() != jobs.len()
|| self.jobs[index]
.iter()
.zip(&jobs)
.any(|(a, b)| a.job_id != b.job_id || a.state != b.state || a.stage != b.stage);
self.jobs[index] = jobs;
// LoRA inventory after the job list (cheap directory listing).
let base_url = self.snapshots[index].base_url.clone();
let url = format!("{base_url}/loras");
self.busy[index] = true;
self.get(cx, base_url, url, Pending::Loras);
changed
}
Pending::Loras => {
self.busy[index] = false;
let loras: Vec<String> = response
.filter(|r| r.status_code == 200)
.and_then(|r| r.get_string_body())
.and_then(|body| LorasJson::deserialize_json_lenient(&body).ok())
.map(|l| l.loras.into_iter().map(|l| l.name).collect())
.unwrap_or_default();
let changed = self.loras[index] != loras;
self.loras[index] = loras;
changed
}
}
}
/// Every LoRA name any up endpoint offers (sorted, deduped).
pub fn all_loras(&self) -> Vec<String> {
let mut names: Vec<String> = self
.snapshots
.iter()
.zip(&self.loras)
.filter(|(snap, _)| snap.is_up())
.flat_map(|(_, loras)| loras.iter().cloned())
.collect();
names.sort();
names.dedup();
names
}
/// Fleet panel text: one block per physical host, its service
/// endpoints indented under it with queue depth, latency and per-model
/// loaded/ready badges.
pub fn panel_text(&self) -> String {
let mut out = String::new();
for (host, rows) in host_groups(&self.snapshots) {
let any_up = rows.iter().any(|&row| self.snapshots[row].is_up());
out.push_str(&format!("{} {}\n", if any_up { "" } else { "" }, host));
for row in rows {
let snap = &self.snapshots[row];
let port = port_of(&snap.base_url);
match &snap.health {
Some(health) => {
let latency = self.latency_ms[row]
.map(|ms| format!("{ms}ms"))
.unwrap_or_default();
out.push_str(&format!(
" :{port} {} queue {} {latency}\n",
health.gpu.as_deref().unwrap_or("no gpu info"),
snap.jobs_pending(),
));
if let (Some(free), Some(total)) =
(health.vram_free_mb, health.vram_total_mb)
{
out.push_str(&format!(" vram {free}/{total} MB\n"));
}
if snap.models.is_empty() {
out.push_str(" (no models reported)\n");
}
for model in &snap.models {
if !model.available {
continue;
}
let badge = match model.state.as_str() {
"loaded" => "◆ loaded",
"ready" => "◇ ready",
other => other,
};
out.push_str(&format!(
" {:<7} {} {}\n",
model.domain, model.id, badge
));
}
}
None => {
out.push_str(&format!(" :{port} offline\n"));
}
}
}
out.push('\n');
}
out
}
}
/// Host part of a fleet base url ("http://10.0.0.169:8123" → "10.0.0.169").
pub fn host_of(base_url: &str) -> &str {
let stripped = base_url
.trim_start_matches("http://")
.trim_start_matches("https://");
stripped
.rsplit_once(':')
.map_or(stripped, |(host, _port)| host)
}
pub fn port_of(base_url: &str) -> &str {
let stripped = base_url
.trim_start_matches("http://")
.trim_start_matches("https://");
stripped.rsplit_once(':').map_or("80", |(_host, port)| port)
}
/// DISPLAY-ONLY grouping of fleet rows by host, in first-appearance order.
/// This never merges data: each row keeps its own health, queue and model
/// set — the group exists so the UI can render one physical-node block with
/// several service endpoints instead of look-alike duplicate rows.
pub fn host_groups(snapshots: &[BoxSnapshot]) -> Vec<(String, Vec<usize>)> {
let mut groups: Vec<(String, Vec<usize>)> = Vec::new();
for (index, snapshot) in snapshots.iter().enumerate() {
let host = host_of(&snapshot.base_url).to_string();
match groups.iter_mut().find(|(name, _)| *name == host) {
Some((_, rows)) => rows.push(index),
None => groups.push((host, vec![index])),
}
}
groups
}
#[cfg(test)]
mod tests {
use super::*;
fn fleet(urls: &[&str]) -> FleetPoll {
let mut fleet = FleetPoll::new();
if !urls.is_empty() {
let nodes: Vec<DiscoveredNode> = urls
.iter()
.enumerate()
.map(|(index, url)| discovered(1000 + index as u64, url))
.collect();
fleet.reconcile_discovered(&nodes);
}
fleet
}
/// Health via the same lenient JSON path real responses take, so tests
/// keep passing when the (concurrently evolving) protocol gains fields.
fn health(node_id: u64, node_key: &str) -> HealthJson {
HealthJson::deserialize_json_lenient(&format!(
r#"{{"service":"makepad-asset-ai","version":"test","models_loaded":[],"node_id":{node_id},"node_key":"{node_key}"}}"#
))
.expect("test health json parses")
}
fn discovered(node_id: u64, base_url: &str) -> DiscoveredNode {
DiscoveredNode {
base_url: base_url.to_string(),
node_id,
fleet: makepad_ai_hub::discovery::DEFAULT_FLEET.to_string(),
}
}
/// SLOW IS NOT GONE. Under a full-GPU denoise or a 17 GB weight pull a
/// box answers late; the panel used to flip it to "down" on the first
/// miss and back on the next, once a second, while every box in fact
/// answered 200 to curl.
#[test]
fn a_busy_box_survives_missed_probes_and_only_then_goes_offline() {
let mut fleet = fleet(&["http://10.0.0.165:8123"]);
fleet.snapshots[0].health = Some(health(1000, "key-a"));
fleet.snapshots[0].models = ModelsJson::deserialize_json_lenient(
r#"{"models":[{"id":"minimax-h3","domain":"video","backend":"h3","available":true,"gated":false,"state":"ready"}]}"#,
)
.expect("test models json parses")
.models;
// Two misses: still up, still holding its model list.
assert_eq!(fleet.note_health_miss(0), ProbeVerdict::Hold { remaining: 2 });
assert!(fleet.snapshots[0].is_up());
assert_eq!(fleet.note_health_miss(0), ProbeVerdict::Hold { remaining: 1 });
assert!(fleet.snapshots[0].is_up());
assert_eq!(fleet.snapshots[0].models.len(), 1);
// One answer and the streak is gone — no flicker, no lost state.
assert_eq!(fleet.note_health_ok(0), 2);
assert_eq!(fleet.note_health_miss(0), ProbeVerdict::Hold { remaining: 2 });
// Sustained silence IS offline.
fleet.note_health_miss(0);
assert_eq!(fleet.note_health_miss(0), ProbeVerdict::Offline);
}
#[test]
fn the_offline_verdict_needs_a_full_streak() {
assert_eq!(verdict_after_misses(0), ProbeVerdict::Hold { remaining: 3 });
assert_eq!(verdict_after_misses(1), ProbeVerdict::Hold { remaining: 2 });
assert_eq!(verdict_after_misses(OFFLINE_AFTER - 1), ProbeVerdict::Hold { remaining: 1 });
assert_eq!(verdict_after_misses(OFFLINE_AFTER), ProbeVerdict::Offline);
assert_eq!(verdict_after_misses(99), ProbeVerdict::Offline);
}
/// The log-once latch: a failing streak says so once, not once per poll.
/// The old behaviour wrote 16-32 identical lines per box per minute.
#[test]
fn a_failing_streak_is_reported_once_not_once_per_poll() {
let mut fleet = fleet(&["http://10.0.0.165:8123"]);
fleet.snapshots[0].health = Some(health(1000, "key-a"));
assert!(!fleet.health_quiet[0], "a healthy row has nothing latched");
// The handler latches on the first complaint of a streak.
fleet.health_quiet[0] = true;
fleet.note_health_miss(0);
assert!(fleet.health_quiet[0], "still latched: no second line");
// A success unlatches it, so the NEXT streak is reported again.
fleet.note_health_ok(0);
assert!(!fleet.health_quiet[0]);
// A box that answers /health and drops /models keeps its OWN latch,
// so it does not complain once per poll forever.
fleet.models_quiet[0] = true;
fleet.note_health_ok(0);
assert!(fleet.models_quiet[0]);
}
#[test]
fn same_node_id_via_two_addresses_stays_one_row() {
// First beacon answered health with node_id 11; the same service
// also beacons from its LAN address. No second row.
let mut fleet = FleetPoll::new();
fleet.reconcile_discovered(&[discovered(11, "http://127.0.0.1:8768")]);
fleet.snapshots[0].health = Some(health(11, "aaaa"));
let changed =
fleet.reconcile_discovered(&[discovered(11, "http://10.0.0.4:8768")]);
assert!(!changed);
assert_eq!(fleet.snapshots.len(), 1);
assert_eq!(fleet.snapshots[0].base_url, "http://127.0.0.1:8768");
}
#[test]
fn same_node_key_via_two_addresses_coalesces() {
// Health had not yet arrived when a second address joined, so a
// duplicate row briefly exists — the node_key coalesce collapses it.
let mut fleet = FleetPoll::new();
fleet.reconcile_discovered(&[
discovered(1, "http://127.0.0.1:8768"),
discovered(2, "http://10.0.0.4:8768"),
]);
assert_eq!(fleet.snapshots.len(), 2);
fleet.snapshots[0].health = Some(health(11, "aaaa"));
fleet.snapshots[1].health = Some(health(11, "aaaa"));
assert!(fleet.coalesce_aliases());
assert_eq!(fleet.snapshots.len(), 1);
assert_eq!(fleet.snapshots[0].base_url, "http://127.0.0.1:8768");
}
#[test]
fn restart_with_new_node_id_keeps_one_row() {
// A discovered service restarts: node_id changes, url + node_key
// stay. The old row must be reused, not duplicated and not dropped.
let mut fleet = fleet(&[]);
fleet.reconcile_discovered(&[discovered(11, "http://10.0.0.9:8767")]);
fleet.snapshots[0].health = Some(health(11, "cafe"));
let changed =
fleet.reconcile_discovered(&[discovered(22, "http://10.0.0.9:8767")]);
assert!(!changed);
assert_eq!(fleet.snapshots.len(), 1);
// The next health refresh reports the new node_id; still one row.
fleet.snapshots[0].health = Some(health(22, "cafe"));
assert!(!fleet.coalesce_aliases());
assert_eq!(fleet.snapshots.len(), 1);
}
#[test]
fn lease_expiry_removes_discovered() {
let mut fleet = FleetPoll::new();
fleet.reconcile_discovered(&[discovered(33, "http://10.0.0.9:8767")]);
assert_eq!(fleet.snapshots.len(), 1);
let changed = fleet.reconcile_discovered(&[]);
assert!(changed);
assert!(fleet.snapshots.is_empty());
assert!(!fleet.reconcile_discovered(&[]));
}
#[test]
fn two_distinct_instances_on_one_host_never_merge() {
// Same box, two service instances with their own cache dirs: host
// grouping is display-only; the rows (and their future model sets)
// must both survive every reconcile + coalesce pass.
let nodes = [
discovered(41, "http://10.0.0.169:8123"),
discovered(42, "http://10.0.0.169:8080"),
];
let mut fleet = FleetPoll::new();
fleet.reconcile_discovered(&nodes);
fleet.snapshots[0].health = Some(health(41, "feed01"));
fleet.snapshots[1].health = Some(health(42, "feed02"));
assert!(!fleet.coalesce_aliases());
assert!(!fleet.reconcile_discovered(&nodes));
assert_eq!(fleet.snapshots.len(), 2);
let groups = host_groups(&fleet.snapshots);
assert_eq!(groups.len(), 1, "one physical host block for display");
assert_eq!(groups[0].0, "10.0.0.169");
assert_eq!(groups[0].1, vec![0, 1], "both service endpoints listed");
}
#[test]
fn pre_node_key_services_coalesce_by_live_node_id() {
// Services predating node_key: the live node_id is the only proof
// two urls are one process — documented fallback identity.
let mut fleet = fleet(&["http://127.0.0.1:8768", "http://10.0.0.4:8768"]);
fleet.snapshots[0].health =
Some(health(55, "")); // node_key "" parses to Some("") — use distinct json
fleet.snapshots[0].health = HealthJson::deserialize_json_lenient(
r#"{"service":"makepad-asset-ai","version":"t","models_loaded":[],"node_id":55}"#,
)
.ok();
fleet.snapshots[1].health = HealthJson::deserialize_json_lenient(
r#"{"service":"makepad-asset-ai","version":"t","models_loaded":[],"node_id":55}"#,
)
.ok();
assert!(fleet.coalesce_aliases());
assert_eq!(fleet.snapshots.len(), 1);
assert_eq!(fleet.snapshots[0].base_url, "http://127.0.0.1:8768");
}
}

28
apps/asset-ui/src/http.rs Normal file
View file

@ -0,0 +1,28 @@
//! Browser-like HTTP requests.
//!
//! Archive.org, TDM mirrors, and similar hosts treat unknown UAs as bots.
//! Every outbound request from Asset UI uses a stock Safari string.
use makepad_widgets::*;
pub const BROWSER_UA: &str = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) \
AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.3 Safari/605.1.15";
pub fn request(url: impl Into<String>, method: HttpMethod) -> HttpRequest {
let mut request = HttpRequest::new(url.into(), method);
request.set_header("User-Agent".into(), BROWSER_UA.into());
request.set_header("Accept".into(), "*/*".into());
request
}
pub fn get(url: impl Into<String>) -> HttpRequest {
request(url, HttpMethod::GET)
}
/// `end` is exclusive (zipsync byterange). HTTP Range is inclusive.
pub fn get_range(url: impl Into<String>, start: u64, end: u64) -> HttpRequest {
let mut request = get(url);
let last = end.saturating_sub(1);
request.set_header("Range".into(), format!("bytes={start}-{last}"));
request
}

5450
apps/asset-ui/src/import.rs Normal file

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

2571
apps/asset-ui/src/library.rs Normal file

File diff suppressed because it is too large Load diff

16103
apps/asset-ui/src/main.rs Normal file

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,498 @@
//! Mask painter for the FLUX.1 Fill-dev inpaint/outpaint lane: shows the
//! pinned input picture letterboxed in the viewer and lets the user paint a
//! repaint mask on it (left drag = paint, ⌥/right = erase). Exports the
//! canvas + mask as PNGs in the exact size the service expects (same size,
//! white/255 = repaint). Outpaint = grow the canvas around the picture (edge
//! replicate) and mask the new border, same request shape.
use makepad_widgets::*;
script_mod! {
use mod.prelude.widgets_internal.*
use mod.widgets.*
set_type_default() do #(DrawMaskPaint::script_shader(vm)){
..mod.draw.DrawQuad
image_tex: texture_2d(float)
mask_tex: texture_2d(float)
// Letterboxed image rect inside the widget, in 0..1 of rect_size.
fit_origin: vec2(0.0, 0.0)
fit_size: vec2(1.0, 1.0)
// Brush cursor (image uv + radius in uv units of the x axis).
cursor_uv: vec2(-1.0, -1.0)
cursor_r: 0.0
has_image: 0.0
pixel: fn() {
let uv = (self.pos - self.fit_origin) / self.fit_size
let inside = step(0.0, uv.x) * step(uv.x, 1.0) * step(0.0, uv.y) * step(uv.y, 1.0) * self.has_image
// Checkerboard ground like the image viewer.
let p = floor(self.pos * self.rect_size / 8.0)
let check = modf(p.x + p.y, 2.0)
let board = mix(#x26262b, #x3c3c42, check)
let color = self.image_tex.sample_as_bgra(uv)
let m = self.mask_tex.sample_as_bgra(uv).x
let painted = mix(color.xyz, vec3(1.0, 0.25, 0.2), m * 0.55)
let base = mix(board.xyz, painted, inside)
// Brush ring.
let d = length((uv - self.cursor_uv) * vec2(1.0, self.fit_size.y * self.rect_size.y / (self.fit_size.x * self.rect_size.x)))
let ring = smoothstep(self.cursor_r * 1.08, self.cursor_r, d) * (1.0 - smoothstep(self.cursor_r, self.cursor_r * 0.92, d))
let ringed = mix(base, vec3(1.0, 1.0, 1.0), ring * inside * step(0.0, self.cursor_r))
return vec4(ringed, 1.0)
}
}
mod.widgets.MaskPaintBase = #(MaskPaint::register_widget(vm))
mod.widgets.MaskPaint = set_type_default() do mod.widgets.MaskPaintBase{
width: Fill
height: Fill
}
}
#[derive(Script, ScriptHook)]
#[repr(C)]
pub struct DrawMaskPaint {
#[deref]
draw_super: DrawQuad,
#[live]
fit_origin: Vec2f,
#[live]
fit_size: Vec2f,
#[live]
cursor_uv: Vec2f,
#[live]
cursor_r: f32,
#[live]
has_image: f32,
}
/// Action emitted when the mask changed (so the host can relabel buttons).
#[derive(Clone, Debug, Default)]
pub enum MaskPaintAction {
MaskChanged,
#[default]
None,
}
#[derive(Script, ScriptHook, Widget)]
pub struct MaskPaint {
#[uid]
uid: WidgetUid,
#[source]
source: ScriptObjectRef,
#[visible]
#[live(true)]
visible: bool,
#[walk]
walk: Walk,
#[layout]
layout: Layout,
#[redraw]
#[live]
draw_bg: DrawMaskPaint,
#[rust]
area: Area,
/// Canvas pixels (BGRA u32, makepad order), `width * height`.
#[rust]
canvas: Vec<u32>,
#[rust]
width: usize,
#[rust]
height: usize,
/// Mask, 0 or 255 per canvas pixel.
#[rust]
mask: Vec<u8>,
#[rust]
image_texture: Option<Texture>,
#[rust]
mask_texture: Option<Texture>,
#[rust]
mask_dirty: bool,
/// Brush radius in canvas pixels.
#[rust(24.0)]
brush_radius: f32,
#[rust]
painting: Option<bool>,
#[rust]
last_pos: Option<(f32, f32)>,
#[rust]
hover_uv: Option<(f32, f32)>,
#[rust]
fit: (f64, f64, f64, f64),
/// Widget rect at the last draw (abs), for pointer → canvas mapping.
#[rust]
rect: Rect,
}
impl MaskPaint {
pub fn has_image(&self) -> bool {
self.width > 0 && self.height > 0
}
pub fn has_mask(&self) -> bool {
self.mask.iter().any(|&m| m > 0)
}
pub fn canvas_size(&self) -> (usize, usize) {
(self.width, self.height)
}
pub fn brush_radius(&self) -> f32 {
self.brush_radius
}
pub fn set_brush_radius(&mut self, radius: f32) {
self.brush_radius = radius.clamp(1.0, 512.0);
}
/// Paint (or erase) one disc — used by headless smoke hooks; the
/// interactive path goes through `stroke`.
pub fn paint_at(&mut self, x: f32, y: f32, radius: f32, paint: bool) {
paint_disc(&mut self.mask, self.width, self.height, x, y, radius, paint);
self.mask_dirty = true;
}
/// Load a decoded picture as the canvas; the mask resets.
pub fn set_image(&mut self, cx: &mut Cx, image: &ImageBuffer) {
self.width = image.width;
self.height = image.height;
self.canvas = image.data[..image.width * image.height].to_vec();
self.mask = vec![0u8; self.width * self.height];
self.image_texture = None;
self.mask_texture = None;
self.mask_dirty = true;
self.draw_bg.redraw(cx);
}
pub fn clear(&mut self, cx: &mut Cx) {
self.width = 0;
self.height = 0;
self.canvas.clear();
self.mask.clear();
self.image_texture = None;
self.mask_texture = None;
self.draw_bg.redraw(cx);
}
pub fn clear_mask(&mut self, cx: &mut Cx) {
for m in &mut self.mask {
*m = 0;
}
self.mask_dirty = true;
self.draw_bg.redraw(cx);
}
pub fn invert_mask(&mut self, cx: &mut Cx) {
for m in &mut self.mask {
*m = 255 - *m;
}
self.mask_dirty = true;
self.draw_bg.redraw(cx);
}
/// Outpaint: grow the canvas by `fraction` of each dimension on every
/// side (edge-replicated fill), rounded so the result is a multiple of
/// 16, and mask the new border. Existing mask strokes are kept.
pub fn outpaint(&mut self, cx: &mut Cx, fraction: f32) {
if !self.has_image() {
return;
}
let pad_x = ((self.width as f32 * fraction).round() as usize).max(16);
let pad_y = ((self.height as f32 * fraction).round() as usize).max(16);
let new_w = (self.width + 2 * pad_x).div_ceil(16) * 16;
let new_h = (self.height + 2 * pad_y).div_ceil(16) * 16;
let off_x = (new_w - self.width) / 2;
let off_y = (new_h - self.height) / 2;
let (canvas, mask) =
grow_canvas(&self.canvas, &self.mask, self.width, self.height, new_w, new_h, off_x, off_y);
self.canvas = canvas;
self.mask = mask;
self.width = new_w;
self.height = new_h;
self.image_texture = None;
self.mask_texture = None;
self.mask_dirty = true;
self.draw_bg.redraw(cx);
}
/// The canvas as RGBA8 PNG (what goes to the service as `image`).
pub fn canvas_png(&self) -> Option<Vec<u8>> {
if !self.has_image() {
return None;
}
let rgba = bgra_to_rgba8(&self.canvas);
makepad_ai_hub::testpattern::encode_png_rgba(&rgba, self.width, self.height).ok()
}
/// The mask as an opaque gray PNG (white = repaint).
pub fn mask_png(&self) -> Option<Vec<u8>> {
if !self.has_image() {
return None;
}
let mut rgba = Vec::with_capacity(self.mask.len() * 4);
for &m in &self.mask {
rgba.extend_from_slice(&[m, m, m, 255]);
}
makepad_ai_hub::testpattern::encode_png_rgba(&rgba, self.width, self.height).ok()
}
fn ensure_textures(&mut self, cx: &mut Cx) {
if !self.has_image() {
return;
}
if self.image_texture.is_none() {
self.image_texture = Some(Texture::new_with_format(
cx,
TextureFormat::VecBGRAu8_32 {
width: self.width,
height: self.height,
data: Some(self.canvas.clone()),
updated: TextureUpdated::Full,
},
));
}
if self.mask_texture.is_none() || self.mask_dirty {
let data: Vec<u32> = self
.mask
.iter()
.map(|&m| 0xff00_0000 | ((m as u32) << 16) | ((m as u32) << 8) | m as u32)
.collect();
match &self.mask_texture {
Some(texture) => {
texture.put_back_vec_u32(cx, data, None);
}
None => {
self.mask_texture = Some(Texture::new_with_format(
cx,
TextureFormat::VecBGRAu8_32 {
width: self.width,
height: self.height,
data: Some(data),
updated: TextureUpdated::Full,
},
));
}
}
self.mask_dirty = false;
}
}
/// Widget-space point → canvas pixel (None outside the picture).
fn canvas_point(&self, abs: Vec2d) -> Option<(f32, f32)> {
let rect = self.rect;
let (ox, oy, sw, sh) = self.fit;
if sw <= 0.0 || sh <= 0.0 {
return None;
}
let u = (abs.x - rect.pos.x - ox) / sw;
let v = (abs.y - rect.pos.y - oy) / sh;
if !(0.0..=1.0).contains(&u) || !(0.0..=1.0).contains(&v) {
return None;
}
Some(((u * self.width as f64) as f32, (v * self.height as f64) as f32))
}
fn stroke(&mut self, from: (f32, f32), to: (f32, f32), paint: bool) {
let steps = (((to.0 - from.0).powi(2) + (to.1 - from.1).powi(2)).sqrt() / (self.brush_radius * 0.35).max(1.0))
.ceil()
.max(1.0) as usize;
for i in 0..=steps {
let t = i as f32 / steps as f32;
let x = from.0 + (to.0 - from.0) * t;
let y = from.1 + (to.1 - from.1) * t;
paint_disc(&mut self.mask, self.width, self.height, x, y, self.brush_radius, paint);
}
self.mask_dirty = true;
}
}
/// Paint (or erase) a filled disc into `mask`.
pub fn paint_disc(mask: &mut [u8], width: usize, height: usize, cx: f32, cy: f32, radius: f32, paint: bool) {
let r2 = radius * radius;
let x0 = ((cx - radius).floor().max(0.0)) as usize;
let x1 = ((cx + radius).ceil().min(width as f32 - 1.0)).max(0.0) as usize;
let y0 = ((cy - radius).floor().max(0.0)) as usize;
let y1 = ((cy + radius).ceil().min(height as f32 - 1.0)).max(0.0) as usize;
if width == 0 || height == 0 || x0 > x1 || y0 > y1 {
return;
}
let value = if paint { 255 } else { 0 };
for y in y0..=y1 {
let dy = y as f32 + 0.5 - cy;
for x in x0..=x1 {
let dx = x as f32 + 0.5 - cx;
if dx * dx + dy * dy <= r2 {
mask[y * width + x] = value;
}
}
}
}
/// Edge-replicate `canvas` into a `new_w x new_h` frame at `(off_x, off_y)`;
/// the mask keeps its strokes and the whole new border becomes repaint.
pub fn grow_canvas(
canvas: &[u32],
mask: &[u8],
width: usize,
height: usize,
new_w: usize,
new_h: usize,
off_x: usize,
off_y: usize,
) -> (Vec<u32>, Vec<u8>) {
let mut out = vec![0u32; new_w * new_h];
let mut out_mask = vec![255u8; new_w * new_h];
for y in 0..new_h {
let sy = y.saturating_sub(off_y).min(height - 1);
for x in 0..new_w {
let sx = x.saturating_sub(off_x).min(width - 1);
out[y * new_w + x] = canvas[sy * width + sx];
let inside = x >= off_x && x < off_x + width && y >= off_y && y < off_y + height;
if inside {
out_mask[y * new_w + x] = mask[sy * width + sx];
}
}
}
(out, out_mask)
}
fn bgra_to_rgba8(bgra: &[u32]) -> Vec<u8> {
let mut out = Vec::with_capacity(bgra.len() * 4);
for px in bgra {
out.extend_from_slice(&[(px >> 16) as u8, (px >> 8) as u8, *px as u8, (px >> 24) as u8]);
}
out
}
impl Widget for MaskPaint {
fn handle_event(&mut self, cx: &mut Cx, event: &Event, _scope: &mut Scope) {
if !self.visible {
return;
}
let uid = self.widget_uid();
match event.hits(cx, self.area) {
Hit::FingerDown(fe) => {
if let Some(p) = self.canvas_point(fe.abs) {
let paint = !(fe.modifiers.alt || fe.mouse_button().is_some_and(|b| b.is_secondary()));
self.painting = Some(paint);
self.stroke(p, p, paint);
self.last_pos = Some(p);
self.draw_bg.redraw(cx);
}
}
Hit::FingerMove(fe) => {
let p = self.canvas_point(fe.abs);
if let (Some(paint), Some(p)) = (self.painting, p) {
let from = self.last_pos.unwrap_or(p);
self.stroke(from, p, paint);
self.last_pos = Some(p);
}
self.hover_uv = p.map(|(x, y)| (x / self.width.max(1) as f32, y / self.height.max(1) as f32));
self.draw_bg.redraw(cx);
}
Hit::FingerHoverIn(fe) | Hit::FingerHoverOver(fe) => {
self.hover_uv = self
.canvas_point(fe.abs)
.map(|(x, y)| (x / self.width.max(1) as f32, y / self.height.max(1) as f32));
self.draw_bg.redraw(cx);
}
Hit::FingerHoverOut(_) => {
self.hover_uv = None;
self.draw_bg.redraw(cx);
}
Hit::FingerUp(_) => {
if self.painting.take().is_some() {
cx.widget_action(uid, MaskPaintAction::MaskChanged);
}
self.last_pos = None;
self.draw_bg.redraw(cx);
}
_ => {}
}
}
fn draw_walk(&mut self, cx: &mut Cx2d, _scope: &mut Scope, walk: Walk) -> DrawStep {
if !self.visible {
return DrawStep::done();
}
self.ensure_textures(cx);
let rect = cx.peek_walk_turtle(walk);
self.rect = rect;
// Letterbox the picture ("Smallest" fit).
if self.has_image() && rect.size.x > 0.0 && rect.size.y > 0.0 {
let scale = (rect.size.x / self.width as f64).min(rect.size.y / self.height as f64);
let sw = self.width as f64 * scale;
let sh = self.height as f64 * scale;
let ox = (rect.size.x - sw) * 0.5;
let oy = (rect.size.y - sh) * 0.5;
self.fit = (ox, oy, sw, sh);
self.draw_bg.fit_origin = vec2((ox / rect.size.x) as f32, (oy / rect.size.y) as f32);
self.draw_bg.fit_size = vec2((sw / rect.size.x) as f32, (sh / rect.size.y) as f32);
self.draw_bg.has_image = 1.0;
self.draw_bg.cursor_r = self.brush_radius / self.width as f32;
} else {
self.fit = (0.0, 0.0, 0.0, 0.0);
self.draw_bg.fit_origin = vec2(0.0, 0.0);
self.draw_bg.fit_size = vec2(1.0, 1.0);
self.draw_bg.has_image = 0.0;
self.draw_bg.cursor_r = 0.0;
}
self.draw_bg.cursor_uv = match self.hover_uv {
Some((u, v)) => vec2(u, v),
None => vec2(-10.0, -10.0),
};
match (&self.image_texture, &self.mask_texture) {
(Some(image), Some(mask)) => {
self.draw_bg.draw_vars.set_texture(0, image);
self.draw_bg.draw_vars.set_texture(1, mask);
}
_ => {
self.draw_bg.draw_vars.empty_texture(0);
self.draw_bg.draw_vars.empty_texture(1);
}
}
self.draw_bg.draw_walk(cx, walk);
self.area = self.draw_bg.area();
DrawStep::done()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn disc_paint_and_erase_stay_in_bounds() {
let mut mask = vec![0u8; 8 * 8];
paint_disc(&mut mask, 8, 8, 4.0, 4.0, 2.0, true);
assert_eq!(mask[4 * 8 + 4], 255);
assert_eq!(mask[0], 0);
// Near the corner: no panic, partial disc.
paint_disc(&mut mask, 8, 8, 0.0, 0.0, 3.0, true);
assert_eq!(mask[0], 255);
paint_disc(&mut mask, 8, 8, 4.0, 4.0, 2.0, false);
assert_eq!(mask[4 * 8 + 4], 0);
// Degenerate sizes never index.
paint_disc(&mut [], 0, 0, 1.0, 1.0, 5.0, true);
}
#[test]
fn grow_canvas_replicates_edges_and_masks_the_border() {
// 2x1 canvas [A B] grown to 6x3 at offset (2,1).
let canvas = vec![0xff0000aa, 0xff0000bb];
let mask = vec![0u8, 255];
let (out, out_mask) = grow_canvas(&canvas, &mask, 2, 1, 6, 3, 2, 1);
assert_eq!(out.len(), 18);
// Left border replicates A, right border replicates B.
assert_eq!(out[1 * 6 + 0], 0xff0000aa);
assert_eq!(out[1 * 6 + 5], 0xff0000bb);
// Rows above/below replicate the single source row.
assert_eq!(out[0 * 6 + 2], 0xff0000aa);
assert_eq!(out[2 * 6 + 3], 0xff0000bb);
// Original mask kept inside, border all repaint.
assert_eq!(out_mask[1 * 6 + 2], 0);
assert_eq!(out_mask[1 * 6 + 3], 255);
assert_eq!(out_mask[0], 255);
assert_eq!(out_mask[17], 255);
}
}

View file

@ -0,0 +1,591 @@
//! The Import page's "Music" card: pick a folder with the platform's native
//! directory dialog, then publish every track under it into the catalog.
//!
//! Music belongs in the asset store like every other kind of content — once
//! it is there the DJ/VJ app lists it through the ordinary `kind=audio`
//! catalog query, with no second library and no second path. The walk,
//! metadata, tagging and publication all live in
//! `makepad_asset_importer::music_import`; this file is the UI half: a picked
//! path, a worker thread, and the same [`ImportPhase`] progress protocol the
//! pack imports already speak, so the shared queue row renders it unchanged.
use crate::import::{ImportPhase, ServerSession};
use makepad_asset_client::{AssetClient, ClientConfig};
use makepad_asset_importer::music_import::{self, MusicReport};
use makepad_widgets::makepad_platform::thread::{Lane, TaskPool};
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{self, Receiver, TryRecvError};
use std::sync::Arc;
/// The card's whole state: which folder, what the worker is doing, and the
/// one-line result of the last run.
pub struct MusicImportPage {
/// Folder the user picked. Empty until the dialog answers — the card
/// cannot start without one.
pub dir: String,
/// True between opening the dialog and its answer, so the button can say
/// so instead of looking dead.
pub picking: bool,
pub phase: ImportPhase,
/// Human summary of the finished run, kept after the phase goes idle.
pub summary: String,
cancel: Arc<AtomicBool>,
rx: Option<Receiver<MusicMsg>>,
/// The finished run, kept so the card can name skips and failures rather
/// than only a happy count.
pub last_report: Option<MusicReport>,
/// "split audio layers" as it stood when the run was QUEUED. Read at
/// queue time, not at finish time, so a toggle flipped while 200 tracks
/// import cannot retroactively change what the run promised.
pub split_layers: bool,
/// "+ lyrics", only meaningful with `split_layers`.
pub bake_lyrics: bool,
/// Aliases the finished run landed, waiting to be handed to the
/// analysis queue. Non-empty only when `split_layers` was on.
pending_analysis: Vec<String>,
pool: Option<TaskPool>,
}
/// What the worker sends back: live progress, then exactly one verdict.
pub enum MusicMsg {
Phase(ImportPhase),
Done(ImportPhase, Box<MusicReport>),
}
impl Default for MusicImportPage {
fn default() -> Self {
Self {
dir: String::new(),
picking: false,
phase: ImportPhase::Idle,
summary: String::new(),
cancel: Arc::new(AtomicBool::new(false)),
rx: None,
last_report: None,
split_layers: false,
bake_lyrics: false,
pending_analysis: Vec::new(),
pool: None,
}
}
}
/// Namespace imported music lands in. `music_import` collapses the repeated
/// class segment, so an alias reads `music/<artist>/<title>`.
pub const MUSIC_NAMESPACE: &str = "music";
impl MusicImportPage {
pub fn set_task_pool(&mut self, pool: TaskPool) {
self.pool = Some(pool);
}
/// The job this card would enqueue right now, or `None` while no folder
/// has been picked.
pub fn job(&self) -> Option<crate::import::ImportJob> {
(!self.dir.trim().is_empty()).then(|| crate::import::ImportJob::Music {
path: self.dir.trim().to_string(),
})
}
/// What the card shows beside the buttons: the picked folder, or the
/// invitation to pick one.
pub fn dir_label(&self) -> String {
if self.picking {
return "choosing folder…".into();
}
if self.dir.trim().is_empty() {
return "no folder chosen".into();
}
self.dir.clone()
}
pub fn set_dir(&mut self, dir: impl Into<String>) {
self.dir = dir.into();
self.picking = false;
}
pub fn compiling(&self) -> bool {
matches!(
self.phase,
ImportPhase::Compiling { .. } | ImportPhase::Publishing { .. }
)
}
pub fn request_stop(&mut self) {
self.cancel.store(true, Ordering::SeqCst);
}
/// One honest line for the queue row.
pub fn status_line(&self, server_connected: bool) -> String {
match &self.phase {
ImportPhase::Idle => {
if self.summary.is_empty() {
if server_connected {
"ready".into()
} else {
"no Asset Server — cannot publish".into()
}
} else {
self.summary.clone()
}
}
ImportPhase::Compiling {
done,
total,
current,
..
} => {
if *total == 0 {
"walking the folder…".into()
} else if current.is_empty() {
format!("reading tags · {done}/{total}")
} else {
format!("reading tags · {done}/{total} · {current}")
}
}
ImportPhase::Publishing {
assets, blob_done, ..
} => format!("publishing · {blob_done}/{assets}"),
ImportPhase::Published { assets, .. } => format!("published {assets} tracks"),
ImportPhase::Failed { message, .. } => message.clone(),
ImportPhase::Cancelled { message, .. } => message.clone(),
other => format!("{other:?}"),
}
}
/// Monotonic 0..1 for the queue row's bar, in two honest bands: the
/// metadata read pass owns 0.02..0.20, the publish pass 0.20..1.00. One
/// bar that filled twice would be a lie about how much work is left.
pub fn progress_fraction(&self) -> f32 {
match &self.phase {
ImportPhase::Compiling { done, total, .. } => {
if *total == 0 {
0.02
} else {
0.02 + 0.18 * (*done as f32 / *total as f32)
}
}
ImportPhase::Publishing {
assets, blob_done, ..
} => {
if *assets == 0 {
0.20
} else {
0.20 + 0.80 * (*blob_done as f32 / *assets as f32)
}
}
ImportPhase::Published { .. } => 1.0,
ImportPhase::Failed { .. } | ImportPhase::Cancelled { .. } => 1.0,
_ => 0.0,
}
}
/// Start the worker. `server` absent = refuse rather than pretend: this
/// import has no local-only half, it publishes or it does not run.
///
/// Every refusal is ALSO written to `summary`. The queue row for a job
/// that never started is removed on the next tick, so a refusal that only
/// returned `Err` would show the user nothing at all — "Load does
/// nothing" is the one failure mode this card must never have.
pub fn start_import(
&mut self,
path: String,
server: Option<ServerSession>,
) -> Result<(), String> {
let dir = PathBuf::from(path.trim());
if !dir.is_dir() {
return Err(self.refuse(format!("{} is not a folder", dir.display())));
}
let Some(server) = server else {
return Err(self.refuse(
"no Asset Server session yet — wait for the store to come up, then Load again"
.into(),
));
};
self.cancel = Arc::new(AtomicBool::new(false));
let cancel = self.cancel.clone();
let (tx, rx) = mpsc::channel();
self.rx = Some(rx);
self.summary.clear();
self.last_report = None;
self.phase = ImportPhase::Compiling {
pack: "Music".into(),
done: 0,
total: 0,
current: String::new(),
};
let Some(pool) = self.pool.clone() else {
return Err(self.refuse("runtime task pool is not configured".into()));
};
match pool.submit(Lane::Heavy, move || {
let msg = run_music_import(&dir, server, &tx, &cancel);
let _ = tx.send(msg);
}) {
Ok(handle) => handle.detach(),
Err(error) => {
return Err(self.refuse(format!("failed to submit music import job: {error}")));
}
}
Ok(())
}
/// Record a refusal where the card can show it, and hand the same text
/// back to the caller for the log.
fn refuse(&mut self, message: String) -> String {
self.phase = ImportPhase::Failed {
pack: "Music".into(),
message: message.clone(),
};
self.summary = message.clone();
self.rx = None;
message
}
/// Drain the worker channel. Returns true when anything changed, so the
/// caller only redraws on real news.
pub fn poll(&mut self) -> bool {
let mut changed = false;
loop {
let Some(rx) = &self.rx else { break };
match rx.try_recv() {
Ok(msg) => {
self.ingest(msg);
changed = true;
}
Err(TryRecvError::Empty) => break,
Err(TryRecvError::Disconnected) => {
self.rx = None;
if self.compiling() {
// The thread died without a verdict. Say so; never
// leave a bar spinning at a lie.
self.phase = ImportPhase::Failed {
pack: "Music".into(),
message: "music import thread exited without a result".into(),
};
self.summary = "music import thread exited without a result".into();
}
changed = true;
break;
}
}
}
changed
}
/// Tracks this run landed that the caller asked to have analysed, once.
/// Aliases, not ids — the importer reports aliases and the bake lane
/// resolves them; the UI thread never blocks on a lookup.
pub fn take_pending_analysis(&mut self) -> Vec<String> {
std::mem::take(&mut self.pending_analysis)
}
/// Every track the run left in the catalog under this folder: newly
/// published, re-published, and the ones already there byte-for-byte.
/// The bake is idempotent (a track that already carries stems is
/// skipped without touching the GPU), so including the unchanged ones is
/// what makes "import again with the switch on" analyse the library.
fn analysis_batch(report: &MusicReport) -> Vec<String> {
let mut aliases = Vec::with_capacity(report.landed());
aliases.extend(report.published.iter().cloned());
aliases.extend(report.updated.iter().cloned());
aliases.extend(report.unchanged.iter().cloned());
aliases
}
fn ingest(&mut self, msg: MusicMsg) {
let phase = match msg {
MusicMsg::Phase(phase) => phase,
MusicMsg::Done(phase, report) => {
// Only a run that was QUEUED with the switch on hands its
// tracks to the bake, and only when it actually landed some.
if self.split_layers {
self.pending_analysis = Self::analysis_batch(&report);
}
// The verdict line names every outcome, so a run that
// published nothing cannot read as a clean one.
self.summary = match &phase {
ImportPhase::Failed { message, .. }
| ImportPhase::Cancelled { message, .. } => message.clone(),
_ => summarise(&report),
};
self.last_report = Some(*report);
self.rx = None;
phase
}
};
if let ImportPhase::Failed { message, .. } = &phase {
self.summary = message.clone();
self.rx = None;
}
self.phase = phase;
}
}
/// Worker body: connect as an ordinary client and run the shared importer.
fn run_music_import(
dir: &std::path::Path,
server: ServerSession,
tx: &mpsc::Sender<MusicMsg>,
cancel: &Arc<AtomicBool>,
) -> MusicMsg {
let cache = crate::asset_store_state::asset_ui_home().join("music-import-cache");
let mut config = ClientConfig::new(cache);
config.token = Some(server.token.clone());
let mut client =
match AssetClient::connect(config, server.endpoints, Some(server.server_id)) {
Ok(client) => client,
Err(error) => {
return MusicMsg::Done(
ImportPhase::Failed {
pack: "Music".into(),
message: format!("asset client: {error}"),
},
Box::default(),
)
}
};
let rights = music_import::personal_library_rights(dir);
// One channel message per file would flood a 5000-track library; the UI
// polls at 5 Hz, so send the first, the last, and roughly one per
// percent in between.
let mut last_sent = usize::MAX;
let mut progress = |p: music_import::MusicProgress| {
let step = (p.total / 100).max(1);
let boundary = p.done == 0 || p.done + 1 >= p.total || p.done % step == 0;
if !boundary && last_sent != usize::MAX {
return;
}
last_sent = p.done;
let phase = match p.stage {
music_import::MusicStage::Reading => ImportPhase::Compiling {
pack: "Music".into(),
done: p.done,
total: p.total,
current: p.current.to_string(),
},
music_import::MusicStage::Publishing => ImportPhase::Publishing {
pack: "Music".into(),
assets: p.total,
blobs: p.total,
blob_done: p.done,
},
};
let _ = tx.send(MusicMsg::Phase(phase));
};
let stop = || cancel.load(Ordering::SeqCst);
match music_import::import_music(
&mut client,
dir,
MUSIC_NAMESPACE,
&rights,
false,
&mut progress,
&stop,
) {
Ok(report) if report.cancelled => MusicMsg::Done(
ImportPhase::Cancelled {
pack: "Music".into(),
message: format!("cancelled · {}", summarise(&report)),
},
Box::new(report),
),
Ok(report) => {
let phase = ImportPhase::Published {
pack: "Music".into(),
assets: report.landed(),
blobs: report.landed(),
created: !report.published.is_empty(),
annotated: report.landed(),
out: dir.to_path_buf(),
library: Vec::new(),
bake: Default::default(),
};
MusicMsg::Done(phase, Box::new(report))
}
Err(error) => MusicMsg::Done(
ImportPhase::Failed {
pack: "Music".into(),
message: error,
},
Box::default(),
),
}
}
/// One line naming every outcome that is not "published fine", so a run with
/// skips or failures cannot look like a clean one.
pub fn summarise(report: &MusicReport) -> String {
let mut parts = vec![format!("{} new", report.published.len())];
if !report.updated.is_empty() {
parts.push(format!("{} updated", report.updated.len()));
}
if !report.unchanged.is_empty() {
parts.push(format!("{} unchanged", report.unchanged.len()));
}
if !report.skipped.is_empty() {
parts.push(format!("{} skipped", report.skipped.len()));
}
if !report.failed.is_empty() {
parts.push(format!("{} failed", report.failed.len()));
}
parts.join(" · ")
}
#[cfg(test)]
mod tests {
use super::*;
use crate::import::ImportJob;
#[test]
fn no_folder_means_no_job() {
let mut page = MusicImportPage::default();
assert!(page.job().is_none());
assert_eq!(page.dir_label(), "no folder chosen");
page.picking = true;
assert_eq!(page.dir_label(), "choosing folder…");
page.set_dir("/Users/me/Music");
assert!(!page.picking);
assert_eq!(page.dir_label(), "/Users/me/Music");
assert_eq!(
page.job(),
Some(ImportJob::Music {
path: "/Users/me/Music".into()
})
);
}
#[test]
fn two_folders_are_two_jobs_but_one_folder_queues_once() {
let a = ImportJob::Music { path: "/a".into() };
let b = ImportJob::Music { path: "/b".into() };
assert!(a.conflicts(&a));
assert!(!a.conflicts(&b));
assert_eq!(a.title(), "Music · a");
}
#[test]
fn start_refuses_a_missing_folder_and_a_missing_server() {
let mut page = MusicImportPage::default();
let error = page
.start_import("/definitely/not/here".into(), None)
.expect_err("missing folder");
assert!(error.contains("not a folder"), "{error}");
let error = page
.start_import(std::env::temp_dir().to_string_lossy().into_owned(), None)
.expect_err("missing server");
assert!(error.contains("Asset Server"), "{error}");
// Refusing must not leave the card looking busy.
assert!(!page.compiling());
}
#[test]
fn progress_is_monotonic_and_ends_at_one() {
let mut page = MusicImportPage::default();
assert_eq!(page.progress_fraction(), 0.0);
page.phase = ImportPhase::Compiling {
pack: "Music".into(),
done: 0,
total: 0,
current: String::new(),
};
let scanning = page.progress_fraction();
page.phase = ImportPhase::Compiling {
pack: "Music".into(),
done: 5,
total: 10,
current: "Strange".into(),
};
let half = page.progress_fraction();
assert!(scanning < half && half < 1.0, "{scanning} {half}");
assert!(page.status_line(true).contains("5/10"));
page.phase = ImportPhase::Failed {
pack: "Music".into(),
message: "boom".into(),
};
assert_eq!(page.progress_fraction(), 1.0);
assert_eq!(page.status_line(true), "boom");
}
#[test]
fn a_dead_worker_becomes_a_visible_failure() {
let mut page = MusicImportPage::default();
let (tx, rx) = mpsc::channel();
page.rx = Some(rx);
page.phase = ImportPhase::Compiling {
pack: "Music".into(),
done: 0,
total: 3,
current: String::new(),
};
drop(tx);
assert!(page.poll());
assert!(matches!(page.phase, ImportPhase::Failed { .. }));
assert!(page.summary.contains("without a result"));
}
/// "split audio layers" hands the finished run's tracks to the bake —
/// and a run queued WITHOUT it hands over nothing, ever.
#[test]
fn only_a_run_queued_with_the_switch_on_feeds_the_bake() {
let report = MusicReport {
published: vec!["music/a/one".into()],
updated: vec!["music/a/two".into()],
unchanged: vec!["music/a/three".into()],
failed: vec![("music/a/four".into(), "boom".into())],
skipped: vec![("music/a/five".into(), "flac".into())],
cancelled: false,
};
let mut off = MusicImportPage::default();
off.ingest(MusicMsg::Done(
ImportPhase::Idle,
Box::new(report.clone()),
));
assert!(
off.take_pending_analysis().is_empty(),
"the switch is off: the import publishes and stops there"
);
let mut on = MusicImportPage::default();
on.split_layers = true;
on.bake_lyrics = true;
on.ingest(MusicMsg::Done(ImportPhase::Idle, Box::new(report)));
let batch = on.take_pending_analysis();
// Everything that LANDED, including the byte-identical re-imports:
// the bake skips a track that already carries stems without
// touching the GPU, so re-importing is how a library gets analysed.
assert_eq!(
batch,
vec![
"music/a/one".to_string(),
"music/a/two".to_string(),
"music/a/three".to_string()
]
);
// Failed and skipped files are not in the catalog; they cannot be.
assert!(!batch.iter().any(|alias| alias.ends_with("four")));
assert!(!batch.iter().any(|alias| alias.ends_with("five")));
// Handed over exactly once.
assert!(on.take_pending_analysis().is_empty());
}
#[test]
fn summary_names_every_non_clean_outcome() {
let report = MusicReport {
published: vec!["a".into()],
updated: vec!["b".into()],
unchanged: vec!["c".into()],
failed: vec![("d".into(), "boom".into())],
skipped: vec![("e".into(), "unsupported".into())],
cancelled: false,
};
let line = summarise(&report);
assert!(line.contains("1 new"), "{line}");
assert!(line.contains("1 updated"), "{line}");
assert!(line.contains("1 unchanged"), "{line}");
assert!(line.contains("1 skipped"), "{line}");
assert!(line.contains("1 failed"), "{line}");
assert_eq!(report.landed(), 3);
}
}

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,531 @@
//! The RUNS chip, the panel behind it, and the ONE card every spawned unit
//! of work is drawn as.
//!
//! All generation runs in THIS app now (aicore §9): the engine in
//! `pipeline.rs` talks to fleet boxes directly, and the store no longer has
//! a queue to poll. So this module owns [`RunCard`] — the card grammar of
//! F1 §5.7 (a title row, ONE aggregate bar, one compact stage strip, and a
//! fold holding the whole truth: sent prompts, params, box tags, errors) —
//! built from the local engine's runs and its waiting queue.
//!
//! The bar is never computed here: [`aggregate_permille`] is the client
//! crate's one implementation. What IS held here is the per-card high-water
//! mark — a stage retry legitimately re-starts one stage's bar, and a bar
//! that goes backwards reads as a bug even when it is honest.
use makepad_asset_client::{aggregate_permille, default_stage_weight};
use std::collections::HashMap;
use crate::pipeline::{format_clock, stage_display_name, Pipeline, StageState};
/// Longest prompt excerpt a card title carries.
const EXCERPT: usize = 60;
// ---------------------------------------------------------------------------
// The card grammar
// ---------------------------------------------------------------------------
/// Which spawned unit a card is; the panel's fold/cancel state keys on it.
#[derive(Clone, Debug, PartialEq, Eq, Hash)]
pub enum CardKey {
/// A run of this app's own engine, by run id.
Local(u64),
/// A run waiting in this app's queue, by queue position.
LocalQueued(usize),
}
impl CardKey {
pub fn as_text(&self) -> String {
match self {
Self::Local(id) => format!("local:{id}"),
Self::LocalQueued(index) => format!("queued:{index}"),
}
}
}
/// The five states a spawned unit reads as, in the order a list sorts them.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum CardState {
Running,
Queued,
Done,
Failed,
Cancelled,
}
impl CardState {
/// Running work first, then what is waiting, then what is over.
fn rank(self) -> u8 {
match self {
Self::Running => 0,
Self::Queued => 1,
Self::Done | Self::Failed | Self::Cancelled => 2,
}
}
pub fn is_terminal(self) -> bool {
matches!(self, Self::Done | Self::Failed | Self::Cancelled)
}
/// The state dot's colour. ONE accent (#3d9bf0) and it belongs to the
/// thing that is alive; everything else is grey, and only a failure
/// earns red. The state WORD is always on the card too (row 2), so the
/// dot is never the only signal.
pub fn dot(self) -> [f32; 4] {
match self {
Self::Running => [0.239, 0.608, 0.941, 1.0],
Self::Queued => [0.29, 0.32, 0.36, 1.0],
Self::Done => [0.35, 0.42, 0.48, 1.0],
Self::Failed => [0.851, 0.345, 0.310, 1.0],
Self::Cancelled => [0.29, 0.32, 0.36, 1.0],
}
}
/// The one bar's fill. Same hue family as the dot, for the same reason.
pub fn fill(self) -> [f32; 4] {
match self {
Self::Running => [0.239, 0.608, 0.941, 1.0],
Self::Queued => [0.20, 0.28, 0.36, 1.0],
Self::Done => [0.173, 0.373, 0.533, 1.0],
Self::Failed => [0.851, 0.345, 0.310, 1.0],
Self::Cancelled => [0.29, 0.32, 0.36, 1.0],
}
}
}
/// One chip of the stage strip. No bar — row 2 is the only bar on the card.
#[derive(Clone, Debug, PartialEq)]
pub struct StageChip {
/// `expand · 15s`, `music · 62%`, `publish`.
pub text: String,
pub tone: StageTone,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum StageTone {
Pending,
Running,
Done,
Failed,
/// An `on_fail: skip` expander that failed — the run went on with the
/// raw prompt, and that is a fact worth its own colour.
Skipped,
}
impl StageTone {
pub fn color(self) -> [f32; 4] {
match self {
Self::Pending => [0.35, 0.38, 0.42, 1.0],
Self::Running => [0.239, 0.608, 0.941, 1.0],
Self::Done => [0.60, 0.64, 0.69, 1.0],
Self::Failed => [0.851, 0.345, 0.310, 1.0],
Self::Skipped => [0.85, 0.65, 0.35, 1.0],
}
}
}
/// ONE spawned unit of work, however it was spawned. F1 §5.7 verbatim:
/// row 1 title, row 2 the only bar, row 3 the stage strip, row 4 the fold.
#[derive(Clone, Debug, PartialEq)]
pub struct RunCard {
pub key: CardKey,
pub state: CardState,
/// Where this ran: the store, or this app's own engine.
pub origin: &'static str,
/// The preset / pipeline label.
pub label: String,
/// The words the PERSON typed, quoted and truncated. Never the
/// expanded prompt — that lives in the fold.
pub excerpt: String,
/// `m:ss`, live while running, frozen when terminal. This REPLACES the
/// "Done in 538.3s" banner.
pub elapsed: String,
/// The aggregate, 0..=1000, already high-water clamped.
pub permille: u16,
/// The humanized current-stage word, or the failure reason.
pub status: String,
/// Empty for a single-stage unit — a heading with nothing under it does
/// not render, and neither does a strip of one.
pub stages: Vec<StageChip>,
/// The whole truth, only when the card is open.
pub fold: String,
/// Whether the × is offered (pending / running / publishing).
pub can_cancel: bool,
/// Whether this queued LOCAL run can still be moved up the queue.
pub can_promote: bool,
pub open: bool,
/// Newest first inside a state bucket.
created_ms: u64,
}
impl RunCard {
pub fn percent(&self) -> u32 {
(self.permille as u32 + 5) / 10
}
pub fn fraction(&self) -> f32 {
(self.permille as f32 / 1000.0).clamp(0.0, 1.0)
}
/// `RUNS · 3 running · 61%` needs to know which cards are still work.
pub fn is_active(&self) -> bool {
!self.state.is_terminal()
}
}
/// A run of this app's own engine, lent to the card builder.
pub struct LocalRun<'a> {
pub id: u64,
pub label: &'a str,
pub prompt: &'a str,
pub created_ms: u64,
pub pipeline: &'a Pipeline,
}
/// A run waiting in this app's queue — spawned, visible, cancellable, but
/// nothing has been sent yet.
pub struct LocalQueued<'a> {
pub index: usize,
pub label: &'a str,
pub prompt: &'a str,
}
// ---------------------------------------------------------------------------
// Text
// ---------------------------------------------------------------------------
fn excerpt(prompt: &str) -> String {
let one_line = prompt.replace('\n', " ");
let trimmed = one_line.trim();
if trimmed.is_empty() {
return String::new();
}
format!("\u{201c}{}\u{201d}", crate::store_views::truncate(trimmed, EXCERPT))
}
/// The job-kind spelling a local stage domain corresponds to, so the ONE
/// shared weight table (`default_stage_weight`) covers local runs too and a
/// local card's bar cannot mean something different from a store card's.
pub fn local_stage_kind(domain: &str) -> &'static str {
match domain {
"text" => "text.expand",
"image" => "image.generate",
"edit" => "image.edit",
"inpaint" => "image.inpaint",
"control" => "image.control",
"upscale" => "image.upscale",
"video" => "video.generate",
"enhance" => "video.enhance",
"music" | "speech" | "sfx" => "music.generate",
"mesh" | "splat" | "paint" | "rig" => "mesh.generate",
_ => "",
}
}
fn local_stage_weight(domain: &str) -> u16 {
default_stage_weight(local_stage_kind(domain))
}
// ---------------------------------------------------------------------------
// Cards from this app's own engine
// ---------------------------------------------------------------------------
/// The local engine's per-stage share of the bar, in the same 0..=1000 the
/// store speaks — so `aggregate_permille` can weigh them together.
///
/// A FAILED stage keeps the fraction it died at. The old rendering filled it
/// to 1.0 and drew "100% · FAILED" beside it, which is the exact lie this
/// design exists to remove.
fn local_stage_permille(state: &StageState, progress: f64) -> u16 {
match state {
StageState::Done | StageState::AwaitingChoice => 1000,
_ => (progress.clamp(0.0, 1.0) * 1000.0) as u16,
}
}
fn local_chip(pipeline: &Pipeline, index: usize) -> StageChip {
let stage = &pipeline.stages[index];
let name = stage.domain.clone();
let (tone, tail) = match &stage.state {
StageState::Waiting => (StageTone::Pending, String::new()),
StageState::Failed(_) => (StageTone::Failed, String::new()),
StageState::Done | StageState::AwaitingChoice => (
StageTone::Done,
match (stage.started, stage.finished) {
(Some(t0), Some(t1)) => format!(" \u{b7} {}", format_clock((t1 - t0).as_secs_f64())),
_ => String::new(),
},
),
_ => (
StageTone::Running,
format!(" \u{b7} {}%", (stage.progress.clamp(0.0, 1.0) * 100.0).round() as u32),
),
};
StageChip { text: format!("{name}{tail}"), tone }
}
fn local_status(pipeline: &Pipeline) -> String {
if let Some((index, error)) = pipeline.stages.iter().enumerate().find_map(|(i, s)| match &s.state
{
StageState::Failed(error) => Some((i, error.clone())),
_ => None,
}) {
return format!(
"failed at {} \u{2014} {}",
pipeline.stages[index].domain,
crate::store_views::truncate(error.trim(), 120)
);
}
if !pipeline.is_running() {
return "done".to_string();
}
let stage = &pipeline.stages[pipeline.current.min(pipeline.stages.len() - 1)];
match &stage.state {
StageState::Waiting if !stage.detail.is_empty() => stage.detail.clone(),
StageState::Waiting => "queued".to_string(),
StageState::FanOut | StageState::AwaitingChoice => stage.detail.clone(),
StageState::Submitting => "submitting".to_string(),
StageState::Polling if !stage.detail.is_empty() => stage.detail.clone(),
StageState::Polling if !stage.service_state.is_empty() => stage.service_state.clone(),
StageState::Polling => "rendering".to_string(),
StageState::Fetching => "fetching artifacts".to_string(),
StageState::Done => "done".to_string(),
StageState::Failed(error) => error.clone(),
}
}
/// The fold of a local run: every stage's inspect block, the SAME text the
/// RUNS surface's opened stage rows already show (`store_views::stage_detail`
/// is the one implementation), with the routing reasoning this engine knows
/// and nothing else does.
fn local_fold(run: &LocalRun) -> String {
let mut out = format!("PROMPT\n{}\n", run.prompt);
for (index, stage) in run.pipeline.stages.iter().enumerate() {
out.push_str(&format!(
"\n\u{2500}\u{2500} {} \u{b7} {} \u{b7} {}\n",
index + 1,
stage_display_name(&stage.domain),
match &stage.state {
StageState::Failed(_) => "failed",
StageState::Done => "succeeded",
StageState::Waiting => "pending",
_ => "running",
}
));
if !stage.box_url.is_empty() {
out.push_str(&format!(
"MODEL\n{} @ {}\n\n",
stage.model,
stage.box_url.trim_start_matches("http://")
));
}
if !stage.reason.is_empty() {
out.push_str(&format!("ROUTED\n{}\n\n", stage.reason));
}
out.push_str(&crate::store_views::stage_detail(stage));
out.push_str("\n\n");
if let StageState::Failed(error) = &stage.state {
out.push_str(&format!("ERROR\n{error}\n\n"));
}
}
while out.ends_with('\n') {
out.pop();
}
out
}
fn local_card(run: &LocalRun, open: bool) -> RunCard {
let failed = run
.pipeline
.stages
.iter()
.any(|s| matches!(s.state, StageState::Failed(_)));
let state = if failed {
CardState::Failed
} else if run.pipeline.is_running() {
if run
.pipeline
.stages
.iter()
.all(|s| s.state == StageState::Waiting)
{
CardState::Queued
} else {
CardState::Running
}
} else {
CardState::Done
};
let permille = aggregate_permille(run.pipeline.stages.iter().map(|stage| {
(
local_stage_weight(&stage.domain),
local_stage_permille(&stage.state, stage.progress),
)
}));
let elapsed: f64 = run
.pipeline
.stages
.iter()
.filter_map(|s| match (s.started, s.finished) {
(Some(t0), Some(t1)) => Some((t1 - t0).as_secs_f64()),
(Some(t0), None) => Some(t0.elapsed().as_secs_f64()),
_ => None,
})
.sum();
RunCard {
key: CardKey::Local(run.id),
state,
origin: "LOCAL",
label: run.label.to_string(),
excerpt: excerpt(run.prompt),
elapsed: format_clock(elapsed),
permille,
status: local_status(run.pipeline),
stages: if run.pipeline.stages.len() > 1 {
(0..run.pipeline.stages.len())
.map(|index| local_chip(run.pipeline, index))
.collect()
} else {
Vec::new()
},
fold: if open { local_fold(run) } else { String::new() },
can_cancel: run.pipeline.is_running(),
can_promote: false,
open,
created_ms: run.created_ms,
}
}
fn queued_card(queued: &LocalQueued, open: bool) -> RunCard {
RunCard {
key: CardKey::LocalQueued(queued.index),
state: CardState::Queued,
origin: "LOCAL",
label: queued.label.to_string(),
excerpt: excerpt(queued.prompt),
elapsed: String::new(),
permille: 0,
status: format!("waiting for a free slot \u{b7} #{}", queued.index + 1),
stages: Vec::new(),
fold: if open {
format!(
"not sent yet \u{2014} this run has not started\n\nPROMPT\n{}",
queued.prompt
)
} else {
String::new()
},
can_cancel: true,
can_promote: queued.index > 0,
open,
created_ms: 0,
}
}
// ---------------------------------------------------------------------------
// The chip line
// ---------------------------------------------------------------------------
/// The header chip beside SEARCHABLE. Always present: "nothing is running"
/// is itself an answer, and a chip that disappears cannot be clicked to see
/// what just finished.
pub fn chip_text(cards: &[RunCard]) -> String {
let running = cards
.iter()
.filter(|card| card.state == CardState::Running)
.count();
let queued = cards
.iter()
.filter(|card| card.state == CardState::Queued)
.count();
if running == 0 && queued == 0 {
return "RUNS \u{b7} idle".to_string();
}
// The percent covers everything still owed, queued runs included at 0 —
// "how far is the work I fired off", not "how far is the busiest box".
let permille = aggregate_permille(
cards
.iter()
.filter(|card| card.is_active())
.map(|card| (1u16, card.permille)),
);
if running == 0 {
return format!("RUNS \u{b7} {queued} queued");
}
let mut line = format!("RUNS \u{b7} {running} running");
if queued > 0 {
line.push_str(&format!(" \u{b7} {queued} queued"));
}
line.push_str(&format!(" \u{b7} {}%", (permille as u32 + 5) / 10));
line
}
// ---------------------------------------------------------------------------
// The chip state
// ---------------------------------------------------------------------------
/// Everything the app knows about work in flight — all of it local now.
#[derive(Default)]
pub struct RunsChip {
/// The cards the person has unfolded.
open: Vec<CardKey>,
/// Per-card high-water mark. A stage retry honestly restarts one
/// stage's bar; a bar that walks backwards reads as a broken app.
high_water: HashMap<CardKey, u16>,
pub panel_open: bool,
}
impl RunsChip {
pub fn set_panel_open(&mut self, open: bool) {
self.panel_open = open;
}
pub fn is_open(&self, key: &CardKey) -> bool {
self.open.contains(key)
}
/// Unfold / refold one card.
pub fn toggle_open(&mut self, key: &CardKey) {
match self.open.iter().position(|held| held == key) {
Some(at) => {
self.open.remove(at);
}
None => self.open.push(key.clone()),
}
}
/// Every spawned unit, one card each, in the one order: what is running,
/// what is waiting, what is over — newest first inside each.
pub fn cards(&mut self, local: &[LocalRun], queued: &[LocalQueued]) -> Vec<RunCard> {
let mut cards = Vec::new();
for run in local {
let key = CardKey::Local(run.id);
cards.push(local_card(run, self.is_open(&key)));
}
for entry in queued {
let key = CardKey::LocalQueued(entry.index);
cards.push(queued_card(entry, self.is_open(&key)));
}
// The high-water mark, held per card: a stage that retries restarts
// its own bar honestly, and the aggregate must still not walk back.
for card in &mut cards {
let seen = self.high_water.entry(card.key.clone()).or_insert(0);
*seen = (*seen).max(card.permille);
card.permille = *seen;
}
let live: Vec<CardKey> = cards.iter().map(|card| card.key.clone()).collect();
self.high_water.retain(|key, _| live.contains(key));
cards.sort_by(|a, b| {
a.state
.rank()
.cmp(&b.state.rank())
.then(b.created_ms.cmp(&a.created_ms))
});
cards
}
}

View file

@ -0,0 +1,449 @@
//! Bounded fleet-aware run scheduler — the pure planning core.
//!
//! SLOT IDENTITY: an endpoint is NOT a GPU slot. The deployment invariant
//! is exactly ONE makepad-asset-ai service process per physical PC, with
//! any additional GPUs advertised as slots BY that one service — never as
//! extra ports/processes. Host bucketing here is defense in depth: a live
//! probe once found a second service (10.0.0.169:8080) beside :8123 on the
//! same 96GB GPU, and scheduling the two independently would have
//! double-booked the card. If a duplicate/stale entry or a rogue second
//! process ever reappears, [`slot_key`] still collapses it: an advertised
//! stable gpu id when one exists, else the host IP. (`node_key` must never
//! be the key — it is per SERVICE INSTANCE, one per port.) A bucket admits
//! one run unless a service in it explicitly advertises more GPUs
//! (`capacity` > 1 — never assumed).
//!
//! Honesty rules:
//!
//! - capability comes from the same advertised-model affinity functions the
//! per-stage router uses — never from a hardcoded box list;
//! - per endpoint, busy = max(service-reported queue depth, our committed
//! stages) so a submission that has not yet appeared in a health poll is
//! neither missed nor double-counted; a bucket SUMS its endpoints (each
//! job queues on exactly one endpoint);
//! - a fresh run STARTS only onto a free compatible slot, WAITS in the app
//! queue while all compatible capacity is occupied, and fails visibly
//! (service gap) when no live endpoint has the capability at all.
//!
//! Repeated Generate clicks therefore fan out across distinct free GPU
//! slots, and a single-capacity GPU is never handed two of our runs — not
//! even through two different service ports.
/// Upper bound on concurrently active runs, above per-slot capacity. The
/// demo target is five simultaneous video jobs across five GPU slots; the
/// bound leaves headroom without ever being the effective limiter (slot
/// capacity is).
pub const MAX_ACTIVE_RUNS: usize = 8;
/// Physical-GPU slot identity for one endpoint. Preference order: a stable
/// advertised gpu id (none exists in `/health` today — the seam is here for
/// when one does), else the endpoint's host IP — two ports on one host are
/// assumed to share one GPU until the service says otherwise.
pub fn slot_key(base_url: &str, advertised_gpu_id: Option<&str>) -> String {
if let Some(gpu_id) = advertised_gpu_id {
return format!("gpu:{gpu_id}");
}
let stripped = base_url
.trim_start_matches("http://")
.trim_start_matches("https://");
let host = stripped
.split(['/', ':'])
.next()
.filter(|host| !host.is_empty())
.unwrap_or(stripped);
format!("host:{host}")
}
/// Expand occupied endpoint urls to every endpoint sharing their GPU slot.
/// `pairs` = (base_url, slot_key) for the whole fleet. The occupied urls
/// themselves are always included, even when no longer in the fleet list.
pub fn expand_occupied(pairs: &[(String, String)], occupied: &[String]) -> Vec<String> {
let occupied_keys: Vec<&str> = pairs
.iter()
.filter(|(url, _)| occupied.iter().any(|occ| occ == url))
.map(|(_, key)| key.as_str())
.collect();
let mut expanded: Vec<String> = occupied.to_vec();
for (url, key) in pairs {
if occupied_keys.iter().any(|occ| occ == key)
&& !expanded.iter().any(|existing| existing == url)
{
expanded.push(url.clone());
}
}
expanded
}
/// Honest load picture of one service endpoint for one prospective run.
#[derive(Clone, Debug)]
pub struct EndpointLoad {
pub base_url: String,
/// Physical-GPU bucket this endpoint belongs to (see [`slot_key`]).
pub slot_key: String,
/// `/health` answered on the last poll.
pub up: bool,
/// The run's FIRST stage can execute here (advertised models via the
/// affinity scorer, after the run's own box pin), and the GPU's TOTAL
/// memory can fit that model plus the service reserve.
pub capable: bool,
/// `capable`, but the latest health snapshot reports too little FREE
/// memory to admit the model now. This is transient pressure: the run
/// stays queued and is reconsidered on the next fleet poll.
pub vram_waiting: bool,
/// Service-reported queue depth (includes other clients' jobs and any
/// of ours the service has already seen).
pub reported_pending: u64,
/// Our runs whose CURRENT stage is committed to this endpoint —
/// authoritative between health polls.
pub ours_active: u32,
/// GPU slots this endpoint's service advertises for its bucket. 1
/// unless the service EXPLICITLY says it drives multiple GPUs.
pub capacity: u32,
}
/// This endpoint's own occupied-count: max() covers the window where our
/// committed job has not yet appeared in the endpoint's reported queue.
fn endpoint_busy(load: &EndpointLoad) -> u64 {
load.reported_pending.max(load.ours_active as u64)
}
/// Free capacity of the GPU slot (bucket) containing `member`. Busy sums
/// per-endpoint (each job queues on exactly one endpoint of the GPU);
/// capacity is the largest a member service advertises, floor 1, and a
/// bucket with no live member has none.
fn slot_free(loads: &[EndpointLoad], member: &EndpointLoad) -> u32 {
let bucket: Vec<&EndpointLoad> = loads
.iter()
.filter(|load| load.slot_key == member.slot_key)
.collect();
if !bucket.iter().any(|load| load.up) {
return 0;
}
let busy: u64 = bucket.iter().map(|load| endpoint_busy(load)).sum();
let capacity = bucket
.iter()
.map(|load| load.capacity.max(1))
.max()
.unwrap_or(1);
(capacity as u64).saturating_sub(busy) as u32
}
#[derive(Debug, PartialEq, Eq)]
pub enum DispatchPlan {
/// Start now. `avoid` = every endpoint of every full GPU slot; the
/// per-stage affinity router picks the best among the rest.
Start { avoid: Vec<String> },
/// Every compatible GPU slot is occupied — hold in the app-side queue.
Wait { reason: String },
/// No live endpoint advertises the capability: start anyway so the
/// stage fails visibly as a service gap instead of parking silently.
NoCapability,
}
pub fn plan_run(domain: &str, loads: &[EndpointLoad]) -> DispatchPlan {
let capable: Vec<&EndpointLoad> = loads
.iter()
.filter(|load| load.up && load.capable)
.collect();
if capable.is_empty() {
return DispatchPlan::NoCapability;
}
let admitted: Vec<&EndpointLoad> = capable
.iter()
.copied()
.filter(|load| !load.vram_waiting)
.collect();
if admitted.is_empty() {
return DispatchPlan::Wait {
reason: format!(
"all {} {domain}-capable GPU slot{} waiting for VRAM",
capable_slots(loads).len(),
if capable_slots(loads).len() == 1 { "" } else { "s" }
),
};
}
// Count SLOTS, not endpoints: two capable ports on one GPU are one unit
// of capacity in both the free check and the wait reason.
let mut capable_slots: Vec<&str> = Vec::new();
for load in &capable {
if !capable_slots.contains(&load.slot_key.as_str()) {
capable_slots.push(load.slot_key.as_str());
}
}
let free = admitted
.iter()
.filter(|load| slot_free(loads, load) > 0)
.count();
if free == 0 {
return DispatchPlan::Wait {
reason: format!(
"all {} {domain}-capable GPU slot{} busy",
capable_slots.len(),
if capable_slots.len() == 1 { "" } else { "s" }
),
};
}
// Avoid EVERY endpoint of every full slot — including a free-looking
// sibling port whose GPU is already taken. Free slots' endpoints stay
// unavoided even when not first-stage-capable: later chain stages may
// need exactly them.
let avoid = loads
.iter()
.filter(|load| slot_free(loads, load) == 0)
.map(|load| load.base_url.clone())
.collect();
DispatchPlan::Start { avoid }
}
fn capable_slots(loads: &[EndpointLoad]) -> Vec<&str> {
let mut slots = Vec::new();
for load in loads.iter().filter(|load| load.up && load.capable) {
if !slots.contains(&load.slot_key.as_str()) {
slots.push(load.slot_key.as_str());
}
}
slots
}
// ---------------------------------------------------------------------------
#[cfg(test)]
mod tests {
use super::*;
fn endpoint(url: &str, up: bool, capable: bool, reported: u64, ours: u32) -> EndpointLoad {
EndpointLoad {
base_url: url.to_string(),
slot_key: slot_key(url, None),
up,
capable,
vram_waiting: false,
reported_pending: reported,
ours_active: ours,
capacity: 1,
}
}
#[test]
fn slot_key_prefers_advertised_gpu_id_then_host_ip() {
assert_eq!(
slot_key("http://10.0.0.169:8123", Some("GPU-9f3a")),
"gpu:GPU-9f3a"
);
assert_eq!(slot_key("http://10.0.0.169:8123", None), "host:10.0.0.169");
assert_eq!(slot_key("http://10.0.0.169:8080", None), "host:10.0.0.169");
assert_eq!(slot_key("https://box.local/", None), "host:box.local");
assert_eq!(slot_key("10.0.0.5:9000", None), "host:10.0.0.5");
}
/// Host/node identity dedupe regression. The deployment invariant is
/// one service per PC (root removed the duplicate .169:8080 fleet
/// entry), but a stale entry or rogue second process must STILL never
/// double-book the card: committing a run through one port consumes the
/// whole slot and the second video run waits.
#[test]
fn two_ports_on_one_host_are_one_gpu_slot() {
let loads = vec![
endpoint("http://10.0.0.169:8123", true, true, 0, 1),
endpoint("http://10.0.0.169:8080", true, true, 0, 0),
];
assert_eq!(
plan_run("video", &loads),
DispatchPlan::Wait {
reason: "all 1 video-capable GPU slot busy".into()
}
);
// With a second HOST free, dispatch avoids BOTH .169 ports — the
// free-looking sibling included.
let loads = vec![
endpoint("http://10.0.0.169:8123", true, true, 0, 1),
endpoint("http://10.0.0.169:8080", true, true, 0, 0),
endpoint("http://10.0.0.217:8767", true, true, 0, 0),
];
match plan_run("video", &loads) {
DispatchPlan::Start { avoid } => {
assert!(avoid.contains(&"http://10.0.0.169:8123".to_string()));
assert!(avoid.contains(&"http://10.0.0.169:8080".to_string()));
assert!(!avoid.contains(&"http://10.0.0.217:8767".to_string()));
}
other => panic!("expected Start, got {other:?}"),
}
// Same story when the sibling port is the busy one via an EXTERNAL
// job and only the capable port looks idle.
let loads = vec![
endpoint("http://10.0.0.169:8123", true, true, 0, 0),
endpoint("http://10.0.0.169:8080", true, false, 1, 0),
];
assert!(matches!(plan_run("video", &loads), DispatchPlan::Wait { .. }));
}
/// The demo acceptance shape: five healthy slots on five DISTINCT hosts
/// take five clicks onto five distinct GPUs; the sixth waits.
#[test]
fn five_distinct_hosts_run_five_concurrent_videos_then_queue() {
let urls = [
"http://10.0.0.11:8767",
"http://10.0.0.12:8767",
"http://10.0.0.13:8767",
"http://10.0.0.14:8767",
"http://10.0.0.15:8767",
];
let mut ours = vec![0u32; urls.len()];
for click in 0..5 {
let loads: Vec<EndpointLoad> = urls
.iter()
.zip(&ours)
.map(|(url, ours)| endpoint(url, true, true, 0, *ours))
.collect();
match plan_run("video", &loads) {
DispatchPlan::Start { avoid } => {
// Exactly the committed hosts are avoided, so this run
// lands on a fresh GPU.
assert_eq!(avoid.len(), click, "click {click}");
let target = urls
.iter()
.position(|url| !avoid.iter().any(|a| a == url))
.expect("a free host remains");
ours[target] += 1;
}
other => panic!("click {click}: expected Start, got {other:?}"),
}
}
assert!(ours.iter().all(|count| *count == 1), "no double-booking");
let loads: Vec<EndpointLoad> = urls
.iter()
.zip(&ours)
.map(|(url, ours)| endpoint(url, true, true, 0, *ours))
.collect();
assert_eq!(
plan_run("video", &loads),
DispatchPlan::Wait {
reason: "all 5 video-capable GPU slots busy".into()
}
);
}
#[test]
fn advertised_multi_gpu_capacity_is_the_only_same_host_concurrency() {
// Conservative default: same host, no advertisement → one slot.
let default_cap = vec![
endpoint("http://10.0.0.169:8123", true, true, 0, 1),
endpoint("http://10.0.0.169:8080", true, true, 0, 0),
];
assert!(matches!(
plan_run("video", &default_cap),
DispatchPlan::Wait { .. }
));
// A service EXPLICITLY advertising two GPUs unlocks a second run…
let mut two_gpu = default_cap.clone();
two_gpu[0].capacity = 2;
match plan_run("video", &two_gpu) {
DispatchPlan::Start { avoid } => assert!(avoid.is_empty()),
other => panic!("expected Start, got {other:?}"),
}
// …and exactly two: the bucket then fills.
two_gpu[1].ours_active = 1;
assert!(matches!(
plan_run("video", &two_gpu),
DispatchPlan::Wait { .. }
));
}
#[test]
fn bucket_busy_sums_ports_but_never_double_counts_one_job() {
// Window A: our job committed via :8123 AND already visible in that
// port's reported queue — max() per endpoint counts it once.
let seen_both_ways = vec![
EndpointLoad {
capacity: 2,
..endpoint("http://10.0.0.169:8123", true, true, 1, 1)
},
endpoint("http://10.0.0.169:8080", true, true, 0, 0),
];
match plan_run("video", &seen_both_ways) {
DispatchPlan::Start { avoid } => assert!(avoid.is_empty()),
other => panic!("expected Start, got {other:?}"),
}
// Window B: an external job reported on :8123 plus our fresh commit
// on :8080 are DIFFERENT jobs — they sum to a full 2-GPU bucket.
let two_distinct_jobs = vec![
EndpointLoad {
capacity: 2,
..endpoint("http://10.0.0.169:8123", true, true, 1, 0)
},
endpoint("http://10.0.0.169:8080", true, true, 0, 1),
];
assert!(matches!(
plan_run("video", &two_distinct_jobs),
DispatchPlan::Wait { .. }
));
}
#[test]
fn capability_gaps_and_down_endpoints_are_honest() {
// Capability nowhere live: dispatch-to-visible-failure, never park.
let loads = vec![
endpoint("http://10.0.0.11:8767", true, false, 0, 0),
endpoint("http://10.0.0.12:8767", false, true, 0, 0),
];
assert_eq!(plan_run("world", &loads), DispatchPlan::NoCapability);
// A slot whose only member is down has no free capacity.
let down = endpoint("http://10.0.0.12:8767", false, true, 0, 0);
assert_eq!(slot_free(std::slice::from_ref(&down), &down), 0);
}
#[test]
fn full_h3_waits_on_96gb_pressure_and_excludes_24_32gb_nodes() {
// `capable` already incorporates TOTAL-VRAM fit from fleet.rs. The
// 24/32 GiB endpoints therefore cannot masquerade as H3 slots; .169
// is the sole compatible slot but its latest FREE-VRAM admission is
// temporarily blocked.
let mut loads = vec![
EndpointLoad {
vram_waiting: true,
..endpoint("http://10.0.0.169:8123", true, true, 0, 0)
},
endpoint("http://10.0.0.217:8767", true, false, 0, 0),
endpoint("http://10.0.0.100:8767", true, false, 0, 0),
];
assert_eq!(
plan_run("video", &loads),
DispatchPlan::Wait {
reason: "all 1 video-capable GPU slot waiting for VRAM".into()
}
);
// A later health poll reports enough free memory. The same queued
// run is now dispatchable on .169; undersized nodes remain excluded.
loads[0].vram_waiting = false;
assert_eq!(
plan_run("video", &loads),
DispatchPlan::Start { avoid: Vec::new() }
);
}
#[test]
fn expand_occupied_covers_slot_mates_and_unknown_urls() {
let pairs: Vec<(String, String)> = [
"http://10.0.0.169:8123",
"http://10.0.0.169:8080",
"http://10.0.0.217:8767",
]
.iter()
.map(|url| (url.to_string(), slot_key(url, None)))
.collect();
// Occupying one .169 port expands to its sibling, not to .217.
let expanded = expand_occupied(&pairs, &["http://10.0.0.169:8123".to_string()]);
assert!(expanded.contains(&"http://10.0.0.169:8123".to_string()));
assert!(expanded.contains(&"http://10.0.0.169:8080".to_string()));
assert!(!expanded.contains(&"http://10.0.0.217:8767".to_string()));
// An occupied url no longer in the fleet list is still avoided.
let gone = expand_occupied(&pairs, &["http://10.0.0.99:1".to_string()]);
assert_eq!(gone, vec!["http://10.0.0.99:1".to_string()]);
}
}

View file

@ -0,0 +1,365 @@
//! The asset UI is a thin client on top of the data in the asset store.
//!
//! Content is fetched from the server by digest and held only in RAM, under
//! a byte budget, evicted least-recently-used. Nothing here writes a file:
//! an app-owned copy is exactly what went stale under us — a fixed Doom map
//! sat in the catalog for hours while the viewer kept opening a local
//! `lib-13501.glb` from the night before.
//!
//! Staleness is impossible by construction: a blob is named by the hash of
//! its bytes, so a cached entry can never be the wrong content for its key,
//! and a new revision means a new digest, so a catalog event invalidates by
//! simply resolving the asset again.
//!
//! There is exactly ONE copy of a blob in memory, and it lives in the
//! client's budgeted, LRU-evicted RAM cache (`AssetClient::fetch_blob_bytes`
//! / `forget_blob` / `ram_cache_bytes`). This module deliberately keeps no
//! cache of its own — a second copy is a second thing to go wrong.
use makepad_asset_client::{dto::AssetDetailDto, AssetClient, ClientConfig};
use makepad_asset_data::{AssetId, BlobId, FileRole, MediaType};
/// What a viewer needs to draw one store asset: which blob it came from
/// (the cache key and the staleness proof) and its bytes.
#[derive(Debug)]
pub struct StorePayload {
pub blob: BlobId,
pub bytes: Vec<u8>,
/// The revision the bytes belong to, for the UI to show and to compare
/// against later catalog events.
pub revision: String,
}
/// Resolve an asset to its renderable payload and fetch it: detail → newest
/// PUBLISHED revision → that revision's manifest → the file role a viewer
/// can draw → blob bytes.
///
/// Fetching by digest is what makes this safe to cache: the server can hand
/// back the same bytes forever and they are still, provably, this revision's.
pub fn fetch_viewable(
client: &mut AssetClient,
asset: &AssetId,
prefer: &[FileRole],
) -> Result<StorePayload, String> {
let detail: AssetDetailDto = client
.asset_detail(asset)
.map_err(|e| format!("asset detail: {e}"))?;
let head = detail
.latest_published()
.ok_or("asset has no published revision")?;
// `fetch_asset_manifest` verifies the document against the revision id,
// so the file list cannot belong to another revision.
let manifest = client
.fetch_asset_manifest(&head.revision)
.map_err(|e| format!("revision manifest: {e}"))?;
let file = prefer
.iter()
.find_map(|role| manifest.files.iter().find(|f| f.role == *role))
.ok_or_else(|| {
format!(
"revision has no {:?} file (has {:?})",
prefer,
manifest.files.iter().map(|f| f.role).collect::<Vec<_>>()
)
})?;
let bytes = fetch_blob(client, &file.blob, file.byte_len)?;
Ok(StorePayload {
blob: file.blob,
bytes,
revision: head.revision.to_string(),
})
}
/// One blob, whole, by digest — verified against that digest by the client,
/// streamed into memory, never written to an app-owned file.
pub fn fetch_blob(client: &mut AssetClient, blob: &BlobId, byte_len: u64) -> Result<Vec<u8>, String> {
client
.fetch_blob_bytes(blob, Some(byte_len).filter(|n| *n > 0))
.map_err(|e| format!("blob {blob}: {e}"))
}
/// What the Create surface needs to WORK with a catalog asset: the payload
/// materialised as a file, plus what it is. The path is the client's
/// verified cache object — digest-named, re-hashed before it is handed out
/// — so the tools that take a file (AO bake, rig, drag-out, decoders) can
/// keep taking one without the app ever owning a copy that could drift.
#[derive(Clone, Debug)]
pub struct StoreFile {
pub blob: BlobId,
pub path: std::path::PathBuf,
pub role: FileRole,
/// The CONTAINER the manifest declares, which the role does not say: an
/// `Audio` file is WAV, Ogg or MP3 with equal right, and the path is
/// digest-named so there is no extension to fall back on. Handing a
/// decoder the wrong container is silent — it simply produces nothing.
pub media: MediaType,
pub revision: String,
/// The manifest's declared thumbnail views ([`ThumbnailMeta::views`]) —
/// what the picture IS, carried WITH the picture so no consumer ever
/// has to guess (or worse, measure). Empty for payload files and for
/// revisions baked before the views contract.
pub views: Vec<makepad_asset_data::ThumbnailView>,
}
/// Same resolution as [`fetch_viewable`] — detail → newest PUBLISHED
/// revision → manifest → preferred role — but materialised to a path
/// instead of read into memory.
pub fn materialize(
client: &mut AssetClient,
asset: &AssetId,
prefer: &[FileRole],
) -> Result<StoreFile, String> {
let (head, manifest) = head_manifest(client, asset)?;
let file = prefer
.iter()
.find_map(|role| manifest.files.iter().find(|f| f.role == *role))
.ok_or_else(|| {
format!(
"revision has no {:?} file (has {:?})",
prefer,
manifest.files.iter().map(|f| f.role).collect::<Vec<_>>()
)
})?;
let path = client
.blob_path(&file.blob, Some(file.byte_len).filter(|n| *n > 0))
.map_err(|e| format!("blob {}: {e}", file.blob))?;
Ok(StoreFile {
blob: file.blob,
path,
role: file.role,
media: file.media,
revision: head,
views: Vec::new(),
})
}
/// The asset's own thumbnail, materialised the same way. Small, and the
/// only thing a gallery row needs before anyone clicks it — a rail must not
/// pull whole payloads to draw a wall of cards.
pub fn materialize_thumbnail(
client: &mut AssetClient,
asset: &AssetId,
) -> Result<StoreFile, String> {
let (head, manifest) = head_manifest(client, asset)?;
let thumbnail = manifest
.thumbnail
.ok_or("revision has no thumbnail")?;
let path = client
.blob_path(&thumbnail.blob, Some(thumbnail.byte_len).filter(|n| *n > 0))
.map_err(|e| format!("thumbnail {}: {e}", thumbnail.blob))?;
Ok(StoreFile {
blob: thumbnail.blob,
path,
role: FileRole::Texture,
media: match thumbnail.media {
makepad_asset_data::ThumbnailMedia::Png => MediaType::Png,
makepad_asset_data::ThumbnailMedia::Jpeg => MediaType::Jpeg,
},
revision: head,
views: thumbnail.views,
})
}
/// Detail → newest published revision → its verified manifest.
fn head_manifest(
client: &mut AssetClient,
asset: &AssetId,
) -> Result<(String, makepad_asset_data::AssetManifest), String> {
let detail: AssetDetailDto = client
.asset_detail(asset)
.map_err(|e| format!("asset detail: {e}"))?;
let head = detail
.latest_published()
.ok_or("asset has no published revision")?;
let manifest = client
.fetch_asset_manifest(&head.revision)
.map_err(|e| format!("revision manifest: {e}"))?;
Ok((head.revision.to_string(), manifest))
}
/// Roles a viewer can draw, best first.
pub fn default_viewable_roles() -> Vec<FileRole> {
vec![
FileRole::RenderGlb,
FileRole::Splat,
FileRole::Texture,
FileRole::Albedo,
FileRole::Audio,
// The viewer plays catalog videos too — leaving this out made every
// store video open fail with "revision has no ... file (has
// [Video])".
FileRole::Video,
]
}
/// Client for the session a viewer request carries. Connecting is cheap
/// against the embedded server, but a cached one keeps the keep-alive pool
/// warm across opens.
pub fn connect(session: &crate::import::ServerSession, cache: &std::path::Path) -> Option<AssetClient> {
let mut config = ClientConfig::new(cache.to_path_buf());
config.token = Some(session.token.clone());
match AssetClient::connect(config, session.endpoints, Some(session.server_id)) {
Ok(client) => Some(client),
Err(error) => {
// The callers' one-line refusal ("cannot reach the asset
// server") is the right UI surface, but the log must carry the
// REASON — a held cache lock, a health mismatch and a dead
// endpoint all look identical without it.
makepad_widgets::log!(
"store connect to {} failed: {error}",
session.endpoints.control
);
None
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use makepad_asset_client::{ApiEndpoints, PublishFile, PublishRequest, PublishThumbnail};
use makepad_asset_data::{AssetKind, MediaType, ThumbnailMedia};
use std::str::FromStr;
fn test_root(name: &str) -> std::path::PathBuf {
let dir = std::env::temp_dir().join(format!(
"mp_store_content_{}_{name}_{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0)
));
std::fs::create_dir_all(&dir).unwrap();
dir
}
/// A REAL asset server in-process, with a real client publishing real
/// bytes — the same path the app uses, no mocks.
fn live_server() -> (makepad_asset_store::AssetServer, String, ApiEndpoints) {
let root = test_root("server");
let mut cfg = makepad_asset_store::ServerConfig::new(root.clone());
cfg.control_addr = "127.0.0.1:0".parse().unwrap();
cfg.data_addr = "127.0.0.1:0".parse().unwrap();
cfg.bootstrap_admin = true;
cfg.log = false;
cfg.gc_janitor_steps = 0;
cfg.gc_grace_ms = 0;
let server = makepad_asset_store::AssetServer::start(cfg).expect("server start");
let token = std::fs::read_to_string(root.join("admin-token"))
.expect("admin token")
.trim()
.to_string();
let endpoints = ApiEndpoints {
control: server.control_addr(),
data: server.data_addr(),
};
(server, token, endpoints)
}
fn client_for(token: &str, endpoints: ApiEndpoints, server_id: [u8; 16]) -> AssetClient {
let mut cfg = ClientConfig::new(test_root("client-cache"));
cfg.token = Some(token.to_string());
AssetClient::connect(cfg, endpoints, Some(server_id)).expect("client connect")
}
/// Publish a drawable asset: a PNG under the `Texture` role (a kind
/// with no mesh invariants — this test is about RESOLUTION, not about
/// what a valid world manifest looks like). `asset` re-publishes an
/// existing id as a NEW revision.
fn publish_drawable(
client: &mut AssetClient,
alias: &str,
body: &[u8],
asset: Option<AssetId>,
) -> AssetId {
let mut png = b"\x89PNG\r\n\x1a\n".to_vec();
png.extend_from_slice(body);
let mut request = PublishRequest::new(
"gen",
AssetKind::Texture,
"store_content drawable",
PublishFile {
bytes: png,
media: MediaType::Png,
role: FileRole::Texture,
media_millis: 0,
dims: Some((512, 512)),
},
PublishThumbnail {
bytes: vec![0x5A; 1_024],
media: ThumbnailMedia::Png,
width: 512,
height: 512,
views: Vec::new(),
},
);
request.alias = Some(makepad_asset_data::AssetAlias::from_str(alias).unwrap());
request.asset_id = asset;
match client.publish_artifact(&request) {
Ok(p) => p.asset_id,
Err(e) => panic!("publish {alias} (revision of {asset:?}): {e}"),
}
}
/// One live server, the whole contract: resolve → draw → re-publish →
/// the NEXT open shows the new bytes. (One server per test process: the
/// data root takes a process lock, so these scenarios share it.)
#[test]
fn the_viewer_reads_the_catalog_and_never_an_older_copy() {
let (server, token, endpoints) = live_server();
let mut client = client_for(&token, endpoints, server.server_id());
// 1. A published world resolves to its drawable file.
let asset = publish_drawable(&mut client, "gen/store-content-one", b"-first", None);
let first =
fetch_viewable(&mut client, &asset, &default_viewable_roles()).expect("viewable");
assert!(first.bytes.starts_with(b"\x89PNG"), "the drawable file came back");
assert!(first.bytes.ends_with(b"-first"));
assert_eq!(
first.blob,
makepad_asset_data::BlobId::hash_of(&first.bytes),
"bytes are exactly the blob they were named by"
);
assert!(!first.revision.is_empty());
// 2. Re-publishing the SAME asset makes the next open show the new
// mesh — the app holds nothing that can be older than the catalog.
let same = publish_drawable(&mut client, "gen/store-content-one", b"-new-and-fixed", Some(asset));
assert_eq!(same, asset, "re-publish is a new revision of one asset");
let second = fetch_viewable(&mut client, &asset, &default_viewable_roles()).unwrap();
assert!(
second.bytes.ends_with(b"-new-and-fixed"),
"re-opening after a publish shows the NEW content"
);
assert_ne!(second.blob, first.blob, "a new revision is a new digest");
assert_ne!(second.revision, first.revision);
// 3. Two assets never bleed into each other.
let other = publish_drawable(&mut client, "gen/store-content-two", b"-other", None);
let other_payload =
fetch_viewable(&mut client, &other, &default_viewable_roles()).unwrap();
assert!(other_payload.bytes.ends_with(b"-other"));
// 4. An asset with no file the viewer can draw says so, by name.
let err = fetch_viewable(&mut client, &asset, &[FileRole::ShadowSdf]).unwrap_err();
assert!(err.contains("ShadowSdf"), "{err}");
// 5. The single RAM copy stays inside its budget.
let (used, budget) = client.ram_cache_bytes();
assert!(used <= budget && budget > 0, "{used} / {budget}");
}
#[test]
fn viewable_roles_are_ordered_best_first() {
// The order the viewer asks for is the order it can DRAW: a mesh
// before a still, a still before audio. A world published with both
// a render mesh and a preview texture must open as the mesh.
let prefer = default_viewable_roles();
let index = |role: FileRole| prefer.iter().position(|r| *r == role);
assert!(index(FileRole::RenderGlb) < index(FileRole::Texture));
assert!(index(FileRole::Splat) < index(FileRole::Texture));
assert!(index(FileRole::Texture) < index(FileRole::Audio));
assert!(index(FileRole::RenderGlb).is_some());
}
}

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

293
apps/asset-ui/src/webcam.rs Normal file
View file

@ -0,0 +1,293 @@
//! Webcam input: the platform camera (NV12 / YUY2 frames off the capture
//! thread) becomes a live preview thumb and, on "snap", a PNG input asset —
//! the same kind of input as a dropped screenshot. `auto-run` keeps snapping
//! and generating with the selected img2X preset while idle.
//!
//! The capture callback runs on the platform's camera thread: it only
//! converts the newest frame into BGRA and parks it in [`WebcamFrames`];
//! the UI thread pulls from there on its own schedule (no allocation churn
//! per frame beyond one buffer swap, no UI work off-thread).
use makepad_widgets::makepad_platform::video::{
CameraFrameLayout, CameraFrameRef, VideoFormat, VideoInputDesc, VideoPixelFormat,
};
use std::sync::{Arc, Mutex};
/// Largest preview/snapshot edge we keep: snapshots go to image models as
/// PNG; a 1280-wide frame is plenty and keeps the per-frame conversion
/// cheap on the capture thread.
pub const MAX_CAPTURE_WIDTH: usize = 1280;
/// Newest converted frame: BGRA u32 pixels (makepad texture order) + size.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct WebcamFrame {
pub width: usize,
pub height: usize,
pub bgra: Vec<u32>,
/// Capture-thread frame counter; the UI uses it to skip re-uploading an
/// unchanged frame.
pub serial: u64,
}
/// Shared slot between the capture thread and the UI.
#[derive(Clone, Default)]
pub struct WebcamFrames {
inner: Arc<Mutex<WebcamFrame>>,
}
impl WebcamFrames {
pub fn new() -> Self {
Self::default()
}
/// Capture thread: convert `frame` (NV12/YUY2, any stride) and park it.
/// Unknown layouts are ignored (no panic on an exotic camera).
pub fn push(&self, frame: &CameraFrameRef<'_>) {
let Some((width, height, bgra)) = camera_frame_to_bgra(frame) else {
return;
};
if let Ok(mut slot) = self.inner.lock() {
slot.width = width;
slot.height = height;
slot.bgra = bgra;
slot.serial += 1;
}
}
/// UI thread: the newest frame if it changed since `seen_serial`.
pub fn take_newer(&self, seen_serial: u64) -> Option<WebcamFrame> {
let slot = self.inner.lock().ok()?;
(slot.serial > seen_serial && !slot.bgra.is_empty()).then(|| slot.clone())
}
pub fn latest(&self) -> Option<WebcamFrame> {
let slot = self.inner.lock().ok()?;
(!slot.bgra.is_empty()).then(|| slot.clone())
}
}
/// Pick the camera format to run: NV12 preferred (what the platform
/// delivers natively on Apple), else YUY2; the largest frame whose width
/// fits [`MAX_CAPTURE_WIDTH`], else the smallest available. `None` when the
/// device offers no raw YUV format (MJPEG-only cameras).
pub fn pick_format(desc: &VideoInputDesc) -> Option<&VideoFormat> {
let rank = |f: &VideoFormat| match f.pixel_format {
VideoPixelFormat::NV12 => 2,
VideoPixelFormat::YUY2 => 1,
_ => 0,
};
let candidates: Vec<&VideoFormat> = desc.formats.iter().filter(|f| rank(f) > 0).collect();
if candidates.is_empty() {
return None;
}
let best_rank = candidates.iter().map(|f| rank(f)).max().unwrap_or(0);
let same_rank: Vec<&VideoFormat> =
candidates.into_iter().filter(|f| rank(f) == best_rank).collect();
same_rank
.iter()
.filter(|f| f.width <= MAX_CAPTURE_WIDTH)
.max_by_key(|f| (f.width * f.height, f.frame_rate.map(|r| r as u64).unwrap_or(0)))
.or_else(|| same_rank.iter().min_by_key(|f| f.width * f.height))
.copied()
}
/// NV12 / YUY2 → BGRA u32 (BT.709 video range), honoring row strides.
pub fn camera_frame_to_bgra(frame: &CameraFrameRef<'_>) -> Option<(usize, usize, Vec<u32>)> {
let (w, h) = (frame.width, frame.height);
if w == 0 || h == 0 {
return None;
}
let mut out = vec![0u32; w * h];
match frame.layout {
CameraFrameLayout::NV12 => {
let y_plane = &frame.planes[0];
let uv_plane = &frame.planes[1];
if y_plane.bytes.len() < y_plane.row_stride * (h - 1) + w
|| uv_plane.bytes.len() < uv_plane.row_stride * (h / 2).max(1).saturating_sub(1) + w
{
return None;
}
for y in 0..h {
let yrow = &y_plane.bytes[y * y_plane.row_stride..];
let uvrow = &uv_plane.bytes[(y / 2) * uv_plane.row_stride..];
let orow = &mut out[y * w..(y + 1) * w];
for x in 0..w {
let yy = yrow[x];
let u = uvrow[(x / 2) * 2];
let v = uvrow[(x / 2) * 2 + 1];
orow[x] = yuv_to_bgra(yy, u, v);
}
}
}
CameraFrameLayout::YUY2 => {
let plane = &frame.planes[0];
if plane.bytes.len() < plane.row_stride * (h - 1) + w * 2 {
return None;
}
for y in 0..h {
let row = &plane.bytes[y * plane.row_stride..];
let orow = &mut out[y * w..(y + 1) * w];
for x in 0..w {
let pair = (x / 2) * 4;
let yy = row[pair + (x & 1) * 2];
let u = row[pair + 1];
let v = row[pair + 3];
orow[x] = yuv_to_bgra(yy, u, v);
}
}
}
_ => return None,
}
Some((w, h, out))
}
/// One BT.709 video-range pixel to 0xAARRGGBB.
#[inline]
pub fn yuv_to_bgra(y: u8, u: u8, v: u8) -> u32 {
let yf = (y as f32 - 16.0) * 1.164_383;
let cb = u as f32 - 128.0;
let cr = v as f32 - 128.0;
let r = yf + 1.792_741 * cr;
let g = yf - 0.213_249 * cb - 0.532_909 * cr;
let b = yf + 2.112_402 * cb;
let c = |v: f32| v.round().clamp(0.0, 255.0) as u32;
0xff00_0000 | (c(r) << 16) | (c(g) << 8) | c(b)
}
/// BGRA u32 → tightly packed RGBA8 (for the PNG encoder).
pub fn bgra_to_rgba8(bgra: &[u32]) -> Vec<u8> {
let mut out = Vec::with_capacity(bgra.len() * 4);
for px in bgra {
out.extend_from_slice(&[(px >> 16) as u8, (px >> 8) as u8, *px as u8, 0xff]);
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use makepad_widgets::makepad_platform::video::{
CameraColorMatrix, CameraFramePlaneRef, VideoFormatId, VideoInputId,
};
#[test]
fn yuv_extremes_map_to_black_white_and_saturated_red() {
assert_eq!(yuv_to_bgra(16, 128, 128) & 0xffffff, 0x000000);
assert_eq!(yuv_to_bgra(235, 128, 128) & 0xffffff, 0xffffff);
let red = yuv_to_bgra(81, 90, 240);
let (r, g, b) = ((red >> 16) & 0xff, (red >> 8) & 0xff, red & 0xff);
assert!(r > 200 && g < 60 && b < 60, "{r},{g},{b}");
assert_eq!(red >> 24, 0xff, "opaque");
}
#[test]
fn nv12_and_yuy2_decode_with_strides() {
// 4x2 NV12, Y plane stride 8 (padding), UV stride 8.
let y_bytes = [235u8, 16, 235, 16, 0, 0, 0, 0, 16, 235, 16, 235, 0, 0, 0, 0];
let uv_bytes = [128u8, 128, 128, 128, 0, 0, 0, 0];
let frame = CameraFrameRef {
timestamp_ns: 0,
width: 4,
height: 2,
layout: CameraFrameLayout::NV12,
matrix: CameraColorMatrix::BT709,
plane_count: 2,
planes: [
CameraFramePlaneRef { bytes: &y_bytes, row_stride: 8, pixel_stride: 1 },
CameraFramePlaneRef { bytes: &uv_bytes, row_stride: 8, pixel_stride: 2 },
CameraFramePlaneRef::empty(),
],
};
let (w, h, px) = camera_frame_to_bgra(&frame).unwrap();
assert_eq!((w, h), (4, 2));
assert_eq!(px[0] & 0xffffff, 0xffffff);
assert_eq!(px[1] & 0xffffff, 0x000000);
assert_eq!(px[4] & 0xffffff, 0x000000);
assert_eq!(px[5] & 0xffffff, 0xffffff);
// 2x1 YUY2: Y0 U Y1 V.
let packed = [235u8, 128, 16, 128];
let frame = CameraFrameRef {
timestamp_ns: 0,
width: 2,
height: 1,
layout: CameraFrameLayout::YUY2,
matrix: CameraColorMatrix::BT709,
plane_count: 1,
planes: [
CameraFramePlaneRef { bytes: &packed, row_stride: 4, pixel_stride: 2 },
CameraFramePlaneRef::empty(),
CameraFramePlaneRef::empty(),
],
};
let (_, _, px) = camera_frame_to_bgra(&frame).unwrap();
assert_eq!(px[0] & 0xffffff, 0xffffff);
assert_eq!(px[1] & 0xffffff, 0x000000);
// Short buffers are refused, never read past the end.
let short = CameraFrameRef {
planes: [
CameraFramePlaneRef { bytes: &packed[..2], row_stride: 4, pixel_stride: 2 },
CameraFramePlaneRef::empty(),
CameraFramePlaneRef::empty(),
],
..frame
};
assert!(camera_frame_to_bgra(&short).is_none());
}
#[test]
fn frames_slot_hands_out_only_newer_frames() {
let frames = WebcamFrames::new();
assert!(frames.take_newer(0).is_none());
let packed = [235u8, 128, 16, 128];
let frame = CameraFrameRef {
timestamp_ns: 0,
width: 2,
height: 1,
layout: CameraFrameLayout::YUY2,
matrix: CameraColorMatrix::BT709,
plane_count: 1,
planes: [
CameraFramePlaneRef { bytes: &packed, row_stride: 4, pixel_stride: 2 },
CameraFramePlaneRef::empty(),
CameraFramePlaneRef::empty(),
],
};
frames.push(&frame);
let first = frames.take_newer(0).unwrap();
assert_eq!(first.serial, 1);
assert!(frames.take_newer(first.serial).is_none());
frames.push(&frame);
assert_eq!(frames.take_newer(first.serial).unwrap().serial, 2);
assert_eq!(bgra_to_rgba8(&first.bgra)[..4], [0xff, 0xff, 0xff, 0xff]);
}
#[test]
fn format_pick_prefers_nv12_within_the_width_cap() {
let fmt = |id: u64, w: usize, h: usize, pf: VideoPixelFormat| VideoFormat {
format_id: VideoFormatId(makepad_widgets::LiveId(id)),
width: w,
height: h,
frame_rate: Some(30.0),
pixel_format: pf,
};
let desc = VideoInputDesc {
input_id: VideoInputId(makepad_widgets::LiveId(1)),
name: "cam".into(),
formats: vec![
fmt(1, 1920, 1080, VideoPixelFormat::NV12),
fmt(2, 1280, 720, VideoPixelFormat::YUY2),
fmt(3, 1280, 720, VideoPixelFormat::NV12),
fmt(4, 640, 480, VideoPixelFormat::NV12),
],
};
assert_eq!(pick_format(&desc).unwrap().format_id.0, makepad_widgets::LiveId(3));
let only_big = VideoInputDesc {
formats: vec![fmt(9, 3840, 2160, VideoPixelFormat::YUY2)],
..desc.clone()
};
assert_eq!(pick_format(&only_big).unwrap().format_id.0, makepad_widgets::LiveId(9));
let mjpeg = VideoInputDesc { formats: vec![fmt(5, 640, 480, VideoPixelFormat::MJPEG)], ..desc };
assert!(pick_format(&mjpeg).is_none());
}
}

29
apps/browser/Cargo.toml Normal file
View file

@ -0,0 +1,29 @@
# browser — a Chrome-like browser as a plain full-window Makepad app.
#
# Chromium (CEF, libs/cef) renders ONLY the page, GPU-accelerated straight
# into an IOSurface-backed Makepad texture. All browser chrome — the tab
# strip, the toolbar with back/forward/reload, the omnibox and the menu —
# is Makepad splash UI, styled from the wm theme when hosted by
# makepad-wm (`MAKEPAD_WM_THEME_SPLASH`) and from a Chrome-dark palette otherwise.
#
# Runs standalone, and unmodified inside makepad-wm / Studio tiles via the
# shared --stdin-loop client runtime every Makepad app has.
[package]
name = "makepad-browser"
version = "0.1.0"
edition = "2021"
default-run = "browser"
[[bin]]
name = "browser"
path = "src/main.rs"
[dependencies]
makepad-widgets = { path = "../../widgets", features = ["cef"] }
makepad-cef = { path = "../../libs/cef" }
makepad-ai-services = { path = "../../libs/ai/services" }
makepad-strict-json = { path = "../../libs/strict_json" }
makepad-wm-theme = { path = "../../libs/wm_theme" }
# The window manager's vocabulary: the bar's title, the polite close.
makepad-wm-api = { path = "../../libs/wm_api" }

View file

@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<svg width="24px" height="24px" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path d="M 15.60 4.40 L 17.00 5.80 L 10.80 12.00 L 17.00 18.20 L 15.60 19.60 L 8.00 12.00 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 231 B

View file

@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<svg width="24px" height="24px" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path d="M 6.00 7.40 L 7.40 6.00 L 12.00 10.60 L 16.60 6.00 L 18.00 7.40 L 13.40 12.00 L 18.00 16.60 L 16.60 18.00 L 12.00 13.40 L 7.40 18.00 L 6.00 16.60 L 10.60 12.00 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 310 B

View file

@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<svg width="24px" height="24px" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path d="M 8.40 4.40 L 7.00 5.80 L 13.20 12.00 L 7.00 18.20 L 8.40 19.60 L 16.00 12.00 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 228 B

View file

@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<svg width="24px" height="24px" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path d="M 20.00 12.00 L 19.73 14.07 L 18.93 16.00 L 17.66 17.66 L 16.00 18.93 L 14.07 19.73 L 12.00 20.00 L 9.93 19.73 L 8.00 18.93 L 6.34 17.66 L 5.07 16.00 L 4.27 14.07 L 4.00 12.00 L 4.27 9.93 L 5.07 8.00 L 6.34 6.34 L 8.00 5.07 L 9.93 4.27 L 12.00 4.00 L 14.07 4.27 L 16.00 5.07 L 17.66 6.34 L 18.93 8.00 L 19.73 9.93 L 20.00 12.00 L 18.40 12.00 L 18.18 10.34 L 17.54 8.80 L 16.53 7.47 L 15.20 6.46 L 13.66 5.82 L 12.00 5.60 L 10.34 5.82 L 8.80 6.46 L 7.47 7.47 L 6.46 8.80 L 5.82 10.34 L 5.60 12.00 L 5.82 13.66 L 6.46 15.20 L 7.47 16.53 L 8.80 17.54 L 10.34 18.18 L 12.00 18.40 L 13.66 18.18 L 15.20 17.54 L 16.53 16.53 L 17.54 15.20 L 18.18 13.66 L 18.40 12.00 Z"/>
<path d="M 11.20 4.40 L 12.80 4.40 L 12.80 19.60 L 11.20 19.60 Z"/>
<path d="M 4.40 11.20 L 19.60 11.20 L 19.60 12.80 L 4.40 12.80 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 950 B

View file

@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<svg width="24px" height="24px" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path d="M 10.80 4.60 L 13.20 4.60 L 13.20 7.00 L 10.80 7.00 Z"/>
<path d="M 10.80 10.80 L 13.20 10.80 L 13.20 13.20 L 10.80 13.20 Z"/>
<path d="M 10.80 17.00 L 13.20 17.00 L 13.20 19.40 L 10.80 19.40 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 346 B

View file

@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<svg width="24px" height="24px" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path d="M 11.00 5.00 L 13.00 5.00 L 13.00 11.00 L 19.00 11.00 L 19.00 13.00 L 13.00 13.00 L 13.00 19.00 L 11.00 19.00 L 11.00 13.00 L 5.00 13.00 L 5.00 11.00 L 11.00 11.00 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 314 B

View file

@ -0,0 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<svg width="24px" height="24px" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path d="M 15.75 5.50 L 17.07 6.47 L 18.14 7.70 L 18.93 9.13 L 19.39 10.70 L 19.49 12.33 L 19.24 13.94 L 18.65 15.46 L 17.75 16.82 L 16.57 17.95 L 15.17 18.80 L 13.62 19.32 L 12.00 19.50 L 10.38 19.32 L 8.83 18.80 L 7.43 17.95 L 6.25 16.82 L 5.35 15.46 L 4.76 13.94 L 4.51 12.33 L 4.61 10.70 L 5.07 9.13 L 5.86 7.70 L 6.93 6.47 L 8.25 5.50 L 9.25 7.24 L 8.28 7.94 L 7.49 8.85 L 6.92 9.90 L 6.58 11.04 L 6.51 12.24 L 6.69 13.42 L 7.12 14.54 L 7.79 15.54 L 8.65 16.36 L 9.68 16.98 L 10.81 17.37 L 12.00 17.50 L 13.19 17.37 L 14.32 16.98 L 15.35 16.36 L 16.21 15.54 L 16.88 14.54 L 17.31 13.42 L 17.49 12.24 L 17.42 11.04 L 17.08 9.90 L 16.51 8.85 L 15.72 7.94 L 14.75 7.24 Z"/>
<path d="M 18.45 5.17 L 14.25 2.77 L 14.65 8.77 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 867 B

View file

@ -0,0 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<svg width="24px" height="24px" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path d="M 16.00 10.00 L 15.80 11.55 L 15.20 13.00 L 14.24 14.24 L 13.00 15.20 L 11.55 15.80 L 10.00 16.00 L 8.45 15.80 L 7.00 15.20 L 5.76 14.24 L 4.80 13.00 L 4.20 11.55 L 4.00 10.00 L 4.20 8.45 L 4.80 7.00 L 5.76 5.76 L 7.00 4.80 L 8.45 4.20 L 10.00 4.00 L 11.55 4.20 L 13.00 4.80 L 14.24 5.76 L 15.20 7.00 L 15.80 8.45 L 16.00 10.00 L 14.00 10.00 L 13.86 8.96 L 13.46 8.00 L 12.83 7.17 L 12.00 6.54 L 11.04 6.14 L 10.00 6.00 L 8.96 6.14 L 8.00 6.54 L 7.17 7.17 L 6.54 8.00 L 6.14 8.96 L 6.00 10.00 L 6.14 11.04 L 6.54 12.00 L 7.17 12.83 L 8.00 13.46 L 8.96 13.86 L 10.00 14.00 L 11.04 13.86 L 12.00 13.46 L 12.83 12.83 L 13.46 12.00 L 13.86 11.04 L 14.00 10.00 Z"/>
<path d="M 14.20 13.00 L 15.60 14.40 L 20.60 19.40 L 19.20 20.80 L 14.20 15.80 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 892 B

View file

@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<svg width="24px" height="24px" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path d="M 12.00 4.60 L 14.00 9.85 L 19.61 10.13 L 15.23 13.65 L 16.70 19.07 L 12.00 16.00 L 7.30 19.07 L 8.77 13.65 L 4.39 10.13 L 10.00 9.85 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 284 B

220
apps/browser/src/ai.rs Normal file
View file

@ -0,0 +1,220 @@
//! The browser on the desktop's AI bus.
//!
//! The current CEF wrapper mirrors navigation metadata but does not bind its
//! frame text/source callbacks, so `page` reports the active title and URL.
use makepad_ai_services::wire::{Risk, ServiceCall, ServiceManifest, ToolDef, ToolResult};
use makepad_strict_json::{self as json, Value};
pub struct PageState {
pub title: String,
pub url: String,
}
pub struct TabState {
pub title: String,
pub url: String,
pub active: bool,
}
/// The bus-facing subset of the webview. Keeping it as a trait makes the
/// closed dispatcher testable without starting CEF or a window.
pub trait BrowserTarget {
fn page(&self) -> Option<PageState>;
fn tabs(&self) -> Vec<TabState>;
fn navigate(&mut self, url: &str) -> bool;
fn new_tab(&mut self, url: &str);
}
pub fn manifest() -> ServiceManifest {
ServiceManifest::new(
"browser",
"Browser",
"The live web browser. Its read tools report the active page and all tabs; its action tools steer the active tab or open a new one.",
)
.with_tool(ToolDef::new(
"page",
"Read the active tab's displayed title and URL. The current CEF binding does not expose visible page text or source, so this tool cannot return page text.",
r#"{"type":"object","properties":{},"additionalProperties":false}"#,
Risk::Read,
))
.with_tool(ToolDef::new(
"tabs",
"Read every open tab's displayed title and URL, with the active tab marked.",
r#"{"type":"object","properties":{},"additionalProperties":false}"#,
Risk::Read,
))
.with_tool(ToolDef::new(
"navigate",
"Navigate the active tab to an http:// or https:// URL, or to about:blank.",
r#"{"type":"object","properties":{"url":{"type":"string"}},"required":["url"],"additionalProperties":false}"#,
Risk::Act,
))
.with_tool(ToolDef::new(
"new_tab",
"Open and activate a new tab at an http:// or https:// URL, or at about:blank.",
r#"{"type":"object","properties":{"url":{"type":"string"}},"required":["url"],"additionalProperties":false}"#,
Risk::Act,
))
}
/// Answer one browser call through a closed match over the four advertised
/// names. URL actions use the webview's existing navigation methods.
pub fn answer(call: &ServiceCall, target: &mut impl BrowserTarget) -> ToolResult {
match call.tool.as_str() {
"page" => {
if let Err(error) = empty_args(&call.args) {
return ToolResult::refused(&call.call_id, error);
}
match target.page() {
Some(page) => ToolResult::ok(
&call.call_id,
format!("title: {}\nurl: {}", page.title, page.url),
format!("{}{}", page.title, page.url),
),
None => ToolResult::unavailable(&call.call_id, "there is no active browser tab"),
}
}
"tabs" => {
if let Err(error) = empty_args(&call.args) {
return ToolResult::refused(&call.call_id, error);
}
let tabs = target.tabs();
let mut text = String::new();
for (index, tab) in tabs.iter().enumerate() {
if index > 0 {
text.push('\n');
}
text.push_str(if tab.active { "[active] " } else { " " });
text.push_str(&tab.title);
text.push_str("");
text.push_str(&tab.url);
}
if text.is_empty() {
text.push_str("no tabs open");
}
ToolResult::ok(&call.call_id, text, format!("{} tabs", tabs.len()))
}
"navigate" => {
let url = match url_arg(&call.args, "navigate") {
Ok(url) => url,
Err(error) => return ToolResult::refused(&call.call_id, error),
};
if target.navigate(&url) {
ToolResult::ok(&call.call_id, format!("navigating to {url}"), "navigating")
} else {
ToolResult::unavailable(&call.call_id, "there is no active browser tab")
}
}
"new_tab" => {
let url = match url_arg(&call.args, "new_tab") {
Ok(url) => url,
Err(error) => return ToolResult::refused(&call.call_id, error),
};
target.new_tab(&url);
ToolResult::ok(&call.call_id, format!("opened a new tab at {url}"), "navigating")
}
other => ToolResult::refused(
&call.call_id,
format!("browser has no tool `{other}`; it has page, tabs, navigate, new_tab"),
),
}
}
fn empty_args(args: &str) -> Result<(), String> {
let fields = object_args(args)?;
if let Some((key, _)) = fields.first() {
return Err(format!("unknown argument `{key}`"));
}
Ok(())
}
fn url_arg(args: &str, tool: &str) -> Result<String, String> {
let fields = object_args(args)?;
if let Some((key, _)) = fields.iter().find(|(key, _)| key != "url") {
return Err(format!("unknown argument `{key}`"));
}
let Some(url) = fields
.iter()
.find(|(key, _)| key == "url")
.and_then(|(_, value)| value.as_str())
else {
return Err(format!("{tool}.url must be a string"));
};
if !allowed_url(url) {
return Err(format!(
"{tool}.url must start with http:// or https://, or be exactly about:blank"
));
}
Ok(url.to_string())
}
fn object_args(args: &str) -> Result<Vec<(String, Value)>, String> {
match json::parse(args.as_bytes()) {
Ok(Value::Obj(fields)) => Ok(fields),
Ok(_) => Err("tool arguments must be a JSON object".to_string()),
Err(error) => Err(format!("invalid tool arguments: {error}")),
}
}
fn allowed_url(url: &str) -> bool {
if url == "about:blank" {
return true;
}
if url.chars().any(char::is_whitespace) || url.chars().any(char::is_control) {
return false;
}
url.strip_prefix("http://")
.or_else(|| url.strip_prefix("https://"))
.is_some_and(|rest| !rest.is_empty())
}
#[cfg(test)]
mod tests {
use super::*;
use makepad_ai_services::wire::ToolOutcome;
#[derive(Default)]
struct FakeTarget {
navigated: Vec<String>,
}
impl BrowserTarget for FakeTarget {
fn page(&self) -> Option<PageState> {
None
}
fn tabs(&self) -> Vec<TabState> {
Vec::new()
}
fn navigate(&mut self, url: &str) -> bool {
self.navigated.push(url.to_string());
true
}
fn new_tab(&mut self, url: &str) {
self.navigated.push(url.to_string());
}
}
#[test]
fn browser_manifest_validates() {
manifest().validate().expect("a valid browser manifest");
}
#[test]
fn navigate_refuses_non_http_urls() {
let mut target = FakeTarget::default();
for url in ["file:///etc/passwd", "javascript:alert(1)", "data:text/plain,no"] {
let call = ServiceCall {
call_id: "c1".into(),
tool: "navigate".into(),
args: format!(r#"{{"url":"{url}"}}"#),
};
let result = answer(&call, &mut target);
assert_eq!(result.outcome, ToolOutcome::Refused);
}
assert!(target.navigated.is_empty());
}
}

512
apps/browser/src/chrome.rs Normal file
View file

@ -0,0 +1,512 @@
//! The browser chrome: a custom-drawn Chrome-style tab strip (favicon +
//! title tabs with a close x, a + for a new tab) and the toolbar
//! (back / forward / reload, the omnibox with search icon and bookmark star,
//! the menu button). Hard-square Omarchy look, colours from `mod.browser_theme`.
use crate::tabs::{TabId, TabSummary};
use makepad_widgets::image::DrawImage;
use makepad_widgets::*;
script_mod! {
use mod.prelude.widgets_internal.*
use mod.widgets.*
mod.widgets.TabStripBase = #(TabStrip::register_widget(vm))
mod.widgets.TabStrip = set_type_default() do mod.widgets.TabStripBase{
width: Fill
height: 36
draw_bg +: {
color: uniform(mod.browser_theme.darker_background)
pixel: fn() {
return self.color
}
}
draw_tab +: {
color: uniform(mod.browser_theme.darker_background)
pixel: fn() {
return self.color
}
}
draw_tab_hover +: {
color: uniform(mod.browser_theme.dark_background)
pixel: fn() {
return self.color
}
}
draw_tab_active +: {
color: uniform(mod.browser_theme.background)
pixel: fn() {
return self.color
}
}
draw_sep +: {
color: uniform(mod.browser_theme.muted)
pixel: fn() {
return self.color
}
}
draw_text +: {
color: mod.browser_theme.foreground
text_style: theme.font_regular{
font_size: 9.5
}
}
draw_text_dim +: {
color: mod.browser_theme.dark_foreground
text_style: theme.font_regular{
font_size: 9.5
}
}
draw_close +: {
svg: crate_resource("self:resources/icons/close.svg")
color: mod.browser_theme.dark_foreground
}
draw_close_active +: {
svg: crate_resource("self:resources/icons/close.svg")
color: mod.browser_theme.foreground
}
draw_plus +: {
svg: crate_resource("self:resources/icons/plus.svg")
color: mod.browser_theme.foreground
}
draw_globe +: {
svg: crate_resource("self:resources/icons/globe.svg")
color: mod.browser_theme.dark_foreground
}
}
// A square, flat icon button for the toolbar. A `let` so this block can
// instantiate it below (a `use mod.widgets.*` glob is a snapshot).
let MpToolButton = ButtonFlatterIcon{
width: 32
height: 32
padding: Inset{left: 0 right: 0 top: 0 bottom: 0}
margin: Inset{left: 0 right: 0 top: 0 bottom: 0}
align: Align{x: 0.5 y: 0.5}
icon_walk: Walk{width: 16 height: 16}
draw_icon +: {
color: mod.browser_theme.foreground
}
draw_bg +: {
border_radius: 0.0
border_size: 0.0
color: #00000000
color_hover: mod.browser_theme.lighter_background
color_down: mod.browser_theme.muted
color_focus: #00000000
}
}
mod.widgets.MpToolButton = MpToolButton
// A row of the ⋮ menu.
mod.widgets.MpMenuItem = ButtonFlatter{
width: Fill
height: 30
align: Align{x: 0.0 y: 0.5}
padding: Inset{left: 14 right: 14 top: 0 bottom: 0}
margin: Inset{left: 0 right: 0 top: 0 bottom: 0}
draw_text +: {
color: mod.browser_theme.foreground
color_hover: mod.browser_theme.bright_foreground
text_style: theme.font_regular{
font_size: 10
}
}
draw_bg +: {
border_radius: 0.0
border_size: 0.0
color: #00000000
color_hover: mod.browser_theme.lighter_background
color_down: mod.browser_theme.muted
}
}
// No `align y: 0.5` anywhere on the omnibox's ancestry: Makepad applies
// such alignment as a deferred shift that moves walked content (text)
// but not `draw_abs` quads — the TextInput's caret and selection would
// end up above the field and clipped. Heights and paddings centre
// everything explicitly instead.
mod.widgets.MpToolbar = SolidView{
width: Fill
height: 40
flow: Right
align: Align{x: 0.0 y: 0.0}
padding: Inset{left: 6 right: 6 top: 4 bottom: 4}
spacing: 2
draw_bg +: {
color: mod.browser_theme.background
}
back_btn := MpToolButton{
draw_icon.svg: crate_resource("self:resources/icons/back.svg")
}
forward_btn := MpToolButton{
draw_icon.svg: crate_resource("self:resources/icons/forward.svg")
}
reload_btn := MpToolButton{
draw_icon.svg: crate_resource("self:resources/icons/reload.svg")
}
View{width: 4 height: Fit}
// The omnibox: a darker square well with the search glyph, the
// text field and the bookmark star.
omnibox_frame := SolidView{
width: Fill
height: 32
flow: Right
align: Align{x: 0.0 y: 0.0}
padding: Inset{left: 10 right: 2 top: 0 bottom: 0}
spacing: 6
draw_bg +: {
color: mod.browser_theme.darker_background
}
Icon{
margin: Inset{top: 9 bottom: 0 left: 0 right: 0}
icon_walk: Walk{width: 14 height: 14}
draw_icon +: {
svg: crate_resource("self:resources/icons/search.svg")
color: mod.browser_theme.dark_foreground
}
}
omnibox := TextInputFlat{
width: Fill
height: 32
empty_text: "Search Google or type a URL"
// The line box is 16.8pt for this font size; text reads as
// centred by its x-height, not its full ink box, so 1pt
// less on top puts the x-height middle on the field's
// middle, level with the search glyph and the star.
padding: Inset{left: 4 right: 4 top: 6.6 bottom: 8.6}
margin: Inset{left: 0 right: 0 top: 0 bottom: 0}
draw_bg +: {
border_radius: 0.0
border_size: 0.0
color: #00000000
color_hover: #00000000
color_focus: #00000000
color_down: #00000000
color_empty: #00000000
border_color: #00000000
border_color_hover: #00000000
border_color_focus: #00000000
border_color_down: #00000000
border_color_empty: #00000000
}
draw_cursor +: {
color: mod.browser_theme.bright_foreground
}
draw_text +: {
color: mod.browser_theme.foreground
color_hover: mod.browser_theme.foreground
color_focus: mod.browser_theme.bright_foreground
color_empty: mod.browser_theme.dark_foreground
color_empty_hover: mod.browser_theme.dark_foreground
text_style: theme.font_regular{
font_size: 10.5
}
}
}
star_btn := MpToolButton{
width: 28
height: 28
margin: Inset{top: 2 bottom: 0 left: 0 right: 0}
icon_walk: Walk{width: 15 height: 15}
draw_icon +: {
svg: crate_resource("self:resources/icons/star.svg")
color: mod.browser_theme.dark_foreground
}
}
}
View{width: 4 height: Fit}
menu_btn := MpToolButton{
draw_icon.svg: crate_resource("self:resources/icons/menu.svg")
}
}
}
#[derive(Clone, Debug, Default, PartialEq)]
pub enum TabStripAction {
#[default]
None,
Activate(TabId),
Close(TabId),
New,
}
#[derive(Clone, Copy, Debug)]
struct TabHit {
id: TabId,
rect: Rect,
close: Rect,
}
#[derive(Script, ScriptHook, Widget)]
pub struct TabStrip {
#[uid]
uid: WidgetUid,
#[source]
source: ScriptObjectRef,
#[walk]
walk: Walk,
#[layout]
layout: Layout,
#[redraw]
#[live]
draw_bg: DrawQuad,
#[live]
draw_tab: DrawQuad,
#[live]
draw_tab_hover: DrawQuad,
#[live]
draw_tab_active: DrawQuad,
#[live]
draw_sep: DrawQuad,
#[live]
draw_text: DrawText,
#[live]
draw_text_dim: DrawText,
#[live]
draw_close: DrawSvg,
#[live]
draw_close_active: DrawSvg,
#[live]
draw_plus: DrawSvg,
#[live]
draw_globe: DrawSvg,
#[live]
draw_favicon: DrawImage,
#[rust]
tabs: Vec<TabSummary>,
#[rust]
hits: Vec<TabHit>,
#[rust]
plus_rect: Rect,
#[rust]
hover_tab: Option<TabId>,
#[rust]
hover_close: bool,
#[rust]
hover_plus: bool,
}
impl TabStrip {
const TAB_MAX_WIDTH: f64 = 240.0;
const TAB_MIN_WIDTH: f64 = 56.0;
const TOP_GAP: f64 = 6.0;
const LEFT_PAD: f64 = 8.0;
const PLUS_SIZE: f64 = 28.0;
pub fn set_tabs(&mut self, cx: &mut Cx, tabs: Vec<TabSummary>) {
self.tabs = tabs;
self.redraw(cx);
}
fn hit_at(&self, pos: Vec2d) -> (Option<TabId>, bool, bool) {
if self.plus_rect.contains(pos) {
return (None, false, true);
}
for hit in &self.hits {
if hit.rect.contains(pos) {
return (Some(hit.id), hit.close.contains(pos), false);
}
}
(None, false, false)
}
fn update_hover(&mut self, cx: &mut Cx, pos: Option<Vec2d>) {
let (tab, close, plus) = pos.map(|p| self.hit_at(p)).unwrap_or((None, false, false));
if tab != self.hover_tab || close != self.hover_close || plus != self.hover_plus {
self.hover_tab = tab;
self.hover_close = close;
self.hover_plus = plus;
self.redraw(cx);
}
}
}
impl Widget for TabStrip {
fn handle_event(&mut self, cx: &mut Cx, event: &Event, _scope: &mut Scope) {
match event.hits(cx, self.draw_bg.area()) {
Hit::FingerDown(fe) => {
let (tab, close, plus) = self.hit_at(fe.abs);
let middle = fe.mouse_button().map(|b| b.is_middle()).unwrap_or(false);
if plus {
cx.widget_action(self.uid, TabStripAction::New);
} else if let Some(id) = tab {
if close || middle {
cx.widget_action(self.uid, TabStripAction::Close(id));
} else {
cx.widget_action(self.uid, TabStripAction::Activate(id));
}
}
}
Hit::FingerHoverIn(fe) | Hit::FingerHoverOver(fe) => {
self.update_hover(cx, Some(fe.abs));
}
Hit::FingerMove(fe) => {
self.update_hover(cx, Some(fe.abs));
}
Hit::FingerHoverOut(_) => {
self.update_hover(cx, None);
}
_ => {}
}
}
fn draw_walk(&mut self, cx: &mut Cx2d, _scope: &mut Scope, walk: Walk) -> DrawStep {
self.draw_bg.begin(cx, walk, self.layout);
let strip = cx.turtle().rect();
self.hits.clear();
let count = self.tabs.len().max(1) as f64;
let available = (strip.size.x - Self::LEFT_PAD - Self::PLUS_SIZE - 12.0).max(0.0);
let tab_width = (available / count)
.min(Self::TAB_MAX_WIDTH)
.max(Self::TAB_MIN_WIDTH.min(available.max(1.0)));
let tab_height = strip.size.y - Self::TOP_GAP;
let mut x = strip.pos.x + Self::LEFT_PAD;
let y = strip.pos.y + Self::TOP_GAP;
let tabs = self.tabs.clone();
for (i, tab) in tabs.iter().enumerate() {
let rect = Rect {
pos: dvec2(x, y),
size: dvec2(tab_width, tab_height),
};
let hovered = self.hover_tab == Some(tab.id);
if tab.active {
self.draw_tab_active.draw_abs(cx, rect);
} else if hovered {
self.draw_tab_hover.draw_abs(cx, rect);
} else {
self.draw_tab.draw_abs(cx, rect);
// Separator between neighbouring inactive tabs.
let next_active = tabs.get(i + 1).map(|t| t.active).unwrap_or(true);
let next_hovered = tabs
.get(i + 1)
.map(|t| self.hover_tab == Some(t.id))
.unwrap_or(false);
if !next_active && !next_hovered {
self.draw_sep.draw_abs(
cx,
Rect {
pos: dvec2(x + tab_width - 0.5, y + 8.0),
size: dvec2(1.0, tab_height - 16.0),
},
);
}
}
let wide = tab_width >= 96.0;
let show_close = tab.active || hovered || wide;
let icon_size = 16.0;
let icon_y = y + (tab_height - icon_size) * 0.5;
let mut text_x = x + 10.0;
if tab_width >= 72.0 {
let icon_rect = Rect {
pos: dvec2(x + 10.0, icon_y),
size: dvec2(icon_size, icon_size),
};
match &tab.favicon {
Some(favicon) => {
self.draw_favicon.draw_vars.set_texture(0, favicon);
self.draw_favicon.draw_abs(cx, icon_rect);
}
None => {
self.draw_globe.draw_abs(cx, icon_rect);
}
}
text_x += icon_size + 8.0;
}
let close_size = 16.0;
let close_rect = Rect {
pos: dvec2(
x + tab_width - close_size - 8.0,
y + (tab_height - close_size) * 0.5,
),
size: dvec2(close_size, close_size),
};
let text_right = if show_close {
close_rect.pos.x - 6.0
} else {
x + tab_width - 8.0
};
let text_width = (text_right - text_x).max(0.0);
if text_width > 4.0 {
let title = if tab.loading && tab.title.is_empty() {
"Loading…".to_string()
} else {
tab.title.clone()
};
cx.begin_turtle(
Walk {
abs_pos: Some(dvec2(text_x, y)),
width: Size::Fixed(text_width),
height: Size::Fixed(tab_height),
..Walk::default()
},
Layout {
clip_x: true,
clip_y: true,
align: Align { x: 0.0, y: 0.5 },
..Layout::default()
},
);
if tab.active {
self.draw_text
.draw_walk(cx, Walk::fit(), Align::default(), &title);
} else {
self.draw_text_dim
.draw_walk(cx, Walk::fit(), Align::default(), &title);
}
cx.end_turtle();
}
if show_close {
let glyph = Rect {
pos: close_rect.pos + dvec2(3.0, 3.0),
size: dvec2(close_size - 6.0, close_size - 6.0),
};
if tab.active || (hovered && self.hover_close) {
self.draw_close_active.draw_abs(cx, glyph);
} else {
self.draw_close.draw_abs(cx, glyph);
}
}
self.hits.push(TabHit {
id: tab.id,
rect,
close: if show_close { close_rect } else { Rect::default() },
});
x += tab_width;
}
// The new-tab "+" square.
let plus_rect = Rect {
pos: dvec2(x + 4.0, y + (tab_height - Self::PLUS_SIZE) * 0.5),
size: dvec2(Self::PLUS_SIZE, Self::PLUS_SIZE),
};
if self.hover_plus {
self.draw_tab_hover.draw_abs(cx, plus_rect);
}
self.draw_plus.draw_abs(
cx,
Rect {
pos: plus_rect.pos + dvec2(7.0, 7.0),
size: dvec2(Self::PLUS_SIZE - 14.0, Self::PLUS_SIZE - 14.0),
},
);
self.plus_rect = plus_rect;
self.draw_bg.end(cx);
DrawStep::done()
}
}

749
apps/browser/src/main.rs Normal file
View file

@ -0,0 +1,749 @@
//! browser: a Chrome-like browser as a plain full-window Makepad app.
//! CEF renders the page (GPU-accelerated into a shared IOSurface texture);
//! every bit of chrome is Makepad. Runs standalone or inside makepad-wm /
//! Studio tiles via the shared --stdin-loop client runtime.
pub use makepad_widgets;
use makepad_ai_services::port::{AiServicePort, PortEvent};
use makepad_cef::BootstrapResult;
use makepad_widgets::*;
mod ai;
mod chrome;
mod tabs;
mod theme;
mod webview;
use chrome::{TabStrip, TabStripAction};
use tabs::TabId;
use std::cell::RefCell;
use std::rc::Rc;
use theme::Palette;
use webview::{WebView, WebViewAction};
script_mod! {
use mod.prelude.widgets.*
use mod.widgets.*
load_all_resources() do #(App::script_component(vm)){
ui: Root{
main_window := Window{
window.inner_size: vec2(1280, 860)
window.title: "browser"
body +: {
flow: Overlay
View{
width: Fill
height: Fill
flow: Down
tab_strip := TabStrip{}
toolbar := MpToolbar{}
webview := WebView{}
}
// The ⋮ menu: a square panel under the button.
menu_layer := View{
width: Fill
height: Fill
flow: Right
align: Align{x: 1.0 y: 0.0}
padding: Inset{top: 78 right: 6 left: 0 bottom: 0}
menu := SolidView{
visible: false
width: 240
height: Fit
flow: Down
padding: Inset{top: 4 bottom: 4 left: 0 right: 0}
draw_bg +: {
color: mod.browser_theme.background
}
menu_new_tab := MpMenuItem{text: "New tab"}
menu_close_tab := MpMenuItem{text: "Close tab"}
menu_reload := MpMenuItem{text: "Reload"}
Hr{}
menu_gpu := MpMenuItem{text: "chrome://gpu"}
menu_about := MpMenuItem{text: "About browser"}
}
}
}
}
}
}
}
thread_local! {
static PALETTE: RefCell<Option<Palette>> = const { RefCell::new(None) };
}
static START: std::sync::OnceLock<std::time::Instant> = std::sync::OnceLock::new();
/// Milliseconds since `main` started (startup measurements).
pub fn uptime_ms() -> u128 {
START.get_or_init(std::time::Instant::now).elapsed().as_millis()
}
/// Microseconds since `main` started (resize tracing).
pub fn uptime_us() -> u128 {
START.get_or_init(std::time::Instant::now).elapsed().as_micros()
}
fn palette() -> Palette {
PALETTE.with(|p| {
p.borrow_mut()
.get_or_insert_with(Palette::current)
.clone()
})
}
/// Cmd shortcuts the browser chrome owns (never forwarded to the page).
pub fn is_app_shortcut(key_event: &KeyEvent) -> bool {
if !key_event.modifiers.logo {
return false;
}
matches!(
key_event.key_code,
KeyCode::KeyT
| KeyCode::KeyW
| KeyCode::KeyL
| KeyCode::KeyR
| KeyCode::KeyN
| KeyCode::LBracket
| KeyCode::RBracket
| KeyCode::Key1
| KeyCode::Key2
| KeyCode::Key3
| KeyCode::Key4
| KeyCode::Key5
| KeyCode::Key6
| KeyCode::Key7
| KeyCode::Key8
| KeyCode::Key9
)
}
/// URLs given on the command line (everything that is not a flag).
fn initial_urls() -> Vec<String> {
let mut urls = Vec::new();
let mut args = std::env::args().skip(1);
while let Some(arg) = args.next() {
if arg == "--cwd" || arg == "--message-format" {
let _ = args.next();
continue;
}
if arg.starts_with("--") {
continue;
}
urls.push(tabs::resolve_omnibox(&arg));
}
urls
}
#[derive(Script, ScriptHook)]
pub struct App {
#[live]
ui: WidgetRef,
#[rust]
omnibox_focused: bool,
#[rust]
shown_url: String,
#[rust]
shown_tab: Option<TabId>,
#[rust]
menu_open: bool,
#[rust]
reported_mode: String,
#[rust]
focus_omnibox_pending: bool,
#[rust]
focus_frame: NextFrame,
#[rust]
focus_retries: u32,
#[rust]
ai_port: Option<AiServicePort>,
#[rust]
ai_context: String,
}
struct AiBrowserTarget<'a> {
cx: &'a mut Cx,
webview: &'a mut WebView,
}
impl ai::BrowserTarget for AiBrowserTarget<'_> {
fn page(&self) -> Option<ai::PageState> {
let info = self.webview.active_info();
info.id.map(|_| ai::PageState {
title: info.title,
url: info.url,
})
}
fn tabs(&self) -> Vec<ai::TabState> {
self.webview
.ai_tabs()
.into_iter()
.map(|(title, url, active)| ai::TabState { title, url, active })
.collect()
}
fn navigate(&mut self, url: &str) -> bool {
if self.webview.active_id().is_none() {
return false;
}
self.webview.navigate(self.cx, url);
true
}
fn new_tab(&mut self, url: &str) {
self.webview.new_tab(self.cx, url, true);
}
}
impl App {
fn with_webview<R>(&self, cx: &mut Cx, f: impl FnOnce(&mut Cx, &mut WebView) -> R) -> Option<R> {
let webview = self.ui.widget(cx, ids!(webview));
let mut inner = webview.borrow_mut::<WebView>()?;
Some(f(cx, &mut inner))
}
fn new_tab(&mut self, cx: &mut Cx, url: Option<String>) {
let url = url.unwrap_or_else(|| palette().new_tab_url());
let is_ntp = theme::is_new_tab_url(&url);
self.with_webview(cx, |cx, wv| {
wv.new_tab(cx, &url, true);
});
self.refresh_chrome(cx);
if is_ntp {
self.focus_omnibox(cx);
}
}
fn close_active_tab(&mut self, cx: &mut Cx) {
let closed_last = self
.with_webview(cx, |cx, wv| {
if let Some(id) = wv.active_id() {
wv.close_tab(cx, id);
}
wv.tab_count() == 0
})
.unwrap_or(false);
if closed_last {
self.new_tab(cx, None);
} else {
self.refresh_chrome(cx);
}
}
fn focus_omnibox(&mut self, cx: &mut Cx) {
let omnibox = self.ui.text_input(cx, ids!(omnibox));
omnibox.set_key_focus(cx);
omnibox.borrow_mut().map(|mut inner| inner.select_all(cx));
// A focus set while a mouse click is still being dispatched (the +
// button, a tab close) does not survive the rest of that click; the
// key-path (Cmd+L) proves the focus itself works. Re-assert it on the
// next frame.
self.focus_omnibox_pending = true;
self.focus_frame = cx.new_next_frame();
}
fn set_menu_open(&mut self, cx: &mut Cx, open: bool) {
if self.menu_open == open {
return;
}
self.menu_open = open;
self.ui.view(cx, ids!(menu)).set_visible(cx, open);
self.ui.redraw(cx);
}
/// Push the model into the chrome: tab strip, omnibox text (unless the
/// user is typing in it), nav button states, window title.
fn refresh_chrome(&mut self, cx: &mut Cx) {
let (summaries, info) = self
.with_webview(cx, |_cx, wv| (wv.summaries(), wv.active_info()))
.unwrap_or_default();
self.refresh_ai_context(&info);
if let Some(mut strip) = self.ui.widget(cx, ids!(tab_strip)).borrow_mut::<TabStrip>() {
strip.set_tabs(cx, summaries);
}
let shown = if theme::is_new_tab_url(&info.url) {
String::new()
} else {
info.url.clone()
};
// The omnibox follows the page unless the user is typing in it — but
// switching tabs always replaces what it shows.
let tab_changed = info.id != self.shown_tab;
self.shown_tab = info.id;
if shown != self.shown_url || tab_changed {
self.shown_url = shown.clone();
if !self.omnibox_focused || tab_changed {
self.ui.text_input(cx, ids!(omnibox)).set_text(cx, &shown);
}
}
let palette = palette();
let on = theme::parse_hex(&palette.foreground).unwrap_or_default();
let off = theme::parse_hex(&palette.muted).unwrap_or_default();
let mut back = self.ui.button(cx, ids!(back_btn));
let back_color = if info.can_go_back { on } else { off };
script_apply_eval!(cx, back, {
draw_icon +: {
color: #(back_color)
}
});
let mut forward = self.ui.button(cx, ids!(forward_btn));
let forward_color = if info.can_go_forward { on } else { off };
script_apply_eval!(cx, forward, {
draw_icon +: {
color: #(forward_color)
}
});
let title = if info.title.is_empty() {
"browser".to_string()
} else {
format!("{} — browser", info.title)
};
self.ui.window(cx, ids!(main_window)).set_title(cx, &title);
// The bar shows the page title too when the window manager hosts us.
makepad_wm_api::set_title(cx, &title);
if info.render_mode != self.reported_mode && info.render_mode != "None" {
self.reported_mode = info.render_mode.clone();
log!(
"browser: page rendering is {} (accelerated frames so far: {}, last blit {}us)",
info.render_mode,
info.accelerated_frames,
info.last_blit_micros
);
}
self.ui.redraw(cx);
}
fn refresh_ai_context(&mut self, info: &webview::ActiveInfo) {
let context = if info.id.is_some() {
format!("active tab: {}{}", info.title, info.url)
} else {
"no active tab".to_string()
};
if context == self.ai_context {
return;
}
self.ai_context = context.clone();
if let Some(port) = self.ai_port.as_ref() {
port.set_context(&context);
}
}
fn refresh_ai_context_from_webview(&mut self, cx: &mut Cx) {
let info = self
.with_webview(cx, |_cx, webview| webview.active_info())
.unwrap_or_default();
self.refresh_ai_context(&info);
}
fn drain_ai_port(&mut self, cx: &mut Cx, event: &Event) {
let events = match self.ai_port.as_mut() {
Some(port) => port.handle_event(cx, event),
None => return,
};
for event in events {
match event {
PortEvent::Registered(endpoint) => {
log!("browser: AI service registered as {}", endpoint.as_str());
self.ai_context.clear();
self.refresh_ai_context_from_webview(cx);
}
PortEvent::Call(call) => {
let result = self
.with_webview(cx, |cx, webview| {
let mut target = AiBrowserTarget { cx, webview };
ai::answer(&call, &mut target)
})
.unwrap_or_else(|| {
makepad_ai_services::wire::ToolResult::unavailable(
&call.call_id,
"the browser view is not ready",
)
});
if let Some(port) = self.ai_port.as_ref() {
port.reply(result);
}
self.refresh_chrome(cx);
}
PortEvent::Cancel { .. } | PortEvent::ChatOpen { .. } => {}
PortEvent::Subscribe { .. } | PortEvent::Unsubscribe { .. } => {}
}
}
}
fn navigate_from_omnibox(&mut self, cx: &mut Cx, text: &str) {
let url = tabs::resolve_omnibox(text);
if url.is_empty() {
return;
}
self.with_webview(cx, |cx, wv| {
if wv.tab_count() == 0 {
wv.new_tab(cx, &url, true);
} else {
wv.navigate(cx, &url);
}
});
self.refresh_chrome(cx);
}
fn handle_shortcut(&mut self, cx: &mut Cx, ke: &KeyEvent) -> bool {
// Ctrl+Tab / Ctrl+Shift+Tab cycle tabs (Chrome), as do Cmd+Shift+] / [.
if ke.modifiers.control && ke.key_code == KeyCode::Tab {
let delta = if ke.modifiers.shift { -1 } else { 1 };
self.with_webview(cx, |cx, wv| wv.activate_offset(cx, delta));
self.refresh_chrome(cx);
return true;
}
if !ke.modifiers.logo {
return false;
}
if ke.modifiers.shift
&& matches!(ke.key_code, KeyCode::LBracket | KeyCode::RBracket)
{
let delta = if ke.key_code == KeyCode::LBracket { -1 } else { 1 };
self.with_webview(cx, |cx, wv| wv.activate_offset(cx, delta));
self.refresh_chrome(cx);
return true;
}
match ke.key_code {
KeyCode::KeyT | KeyCode::KeyN => self.new_tab(cx, None),
KeyCode::KeyW => self.close_active_tab(cx),
KeyCode::KeyL => self.focus_omnibox(cx),
KeyCode::KeyR => {
self.with_webview(cx, |cx, wv| wv.reload(cx));
}
KeyCode::LBracket => {
self.with_webview(cx, |cx, wv| wv.go_back(cx));
}
KeyCode::RBracket => {
self.with_webview(cx, |cx, wv| wv.go_forward(cx));
}
KeyCode::Key1
| KeyCode::Key2
| KeyCode::Key3
| KeyCode::Key4
| KeyCode::Key5
| KeyCode::Key6
| KeyCode::Key7
| KeyCode::Key8 => {
let index = match ke.key_code {
KeyCode::Key1 => 0,
KeyCode::Key2 => 1,
KeyCode::Key3 => 2,
KeyCode::Key4 => 3,
KeyCode::Key5 => 4,
KeyCode::Key6 => 5,
KeyCode::Key7 => 6,
_ => 7,
};
self.with_webview(cx, |cx, wv| wv.activate_index(cx, index));
self.refresh_chrome(cx);
}
KeyCode::Key9 => {
self.with_webview(cx, |cx, wv| {
let last = wv.tab_count().saturating_sub(1);
wv.activate_index(cx, last);
});
self.refresh_chrome(cx);
}
_ => return false,
}
true
}
}
impl MatchEvent for App {
fn handle_startup(&mut self, cx: &mut Cx) {
// A warm-pool standby is not a running Browser: its service opens on
// `WmEvent::Adopted`, never while dormant — the assistant must not
// steer a page nobody can see.
if !makepad_wm_api::warm_start() {
self.ai_port = AiServicePort::open(cx, ai::manifest());
}
match makepad_cef::startup_phases() {
Some((bundle_ms, exec_gap_ms)) => log!(
"browser: window up at {} ms after main (app bundle prepared in {} ms, exec-to-main gap {} ms)",
uptime_ms(),
bundle_ms,
exec_gap_ms
),
None => log!("browser: window up at {} ms after main", uptime_ms()),
}
let urls = initial_urls();
if urls.is_empty() {
// Never boot empty: the first tab opens the web (Cmd+T tabs
// still get the themed New Tab page).
self.new_tab(cx, Some("https://www.google.com/".to_string()));
} else {
for url in urls {
self.new_tab(cx, Some(url));
}
}
}
fn handle_actions(&mut self, cx: &mut Cx, actions: &Actions) {
// Tab strip.
for action in actions {
let Some(action) = action.as_widget_action() else {
continue;
};
match action.cast::<TabStripAction>() {
TabStripAction::Activate(id) => {
self.with_webview(cx, |cx, wv| wv.activate(cx, id));
self.refresh_chrome(cx);
}
TabStripAction::Close(id) => {
let closed_last = self
.with_webview(cx, |cx, wv| {
wv.close_tab(cx, id);
wv.tab_count() == 0
})
.unwrap_or(false);
if closed_last {
self.new_tab(cx, None);
} else {
self.refresh_chrome(cx);
}
}
TabStripAction::New => self.new_tab(cx, None),
TabStripAction::None => {}
}
if let WebViewAction::TabsChanged = action.cast::<WebViewAction>() {
self.refresh_chrome(cx);
}
}
// Toolbar.
if self.ui.button(cx, ids!(back_btn)).clicked(actions) {
self.with_webview(cx, |cx, wv| wv.go_back(cx));
}
if self.ui.button(cx, ids!(forward_btn)).clicked(actions) {
self.with_webview(cx, |cx, wv| wv.go_forward(cx));
}
if self.ui.button(cx, ids!(reload_btn)).clicked(actions) {
// Chrome semantics: the button stops a page that is still loading.
self.with_webview(cx, |cx, wv| {
if wv.active_info().loading {
wv.stop(cx);
} else {
wv.reload(cx);
}
});
}
if self.ui.button(cx, ids!(star_btn)).clicked(actions) {
// Bookmarks are not wired yet; the star just re-focuses the page.
self.with_webview(cx, |cx, wv| wv.focus(cx));
}
if self.ui.button(cx, ids!(menu_btn)).clicked(actions) {
let open = !self.menu_open;
self.set_menu_open(cx, open);
}
// Menu.
if self.ui.button(cx, ids!(menu_new_tab)).clicked(actions) {
self.set_menu_open(cx, false);
self.new_tab(cx, None);
}
if self.ui.button(cx, ids!(menu_close_tab)).clicked(actions) {
self.set_menu_open(cx, false);
self.close_active_tab(cx);
}
if self.ui.button(cx, ids!(menu_reload)).clicked(actions) {
self.set_menu_open(cx, false);
self.with_webview(cx, |cx, wv| wv.reload(cx));
}
if self.ui.button(cx, ids!(menu_gpu)).clicked(actions) {
self.set_menu_open(cx, false);
self.new_tab(cx, Some("chrome://gpu".to_string()));
}
if self.ui.button(cx, ids!(menu_about)).clicked(actions) {
self.set_menu_open(cx, false);
let info = self
.with_webview(cx, |_cx, wv| wv.active_info())
.unwrap_or_default();
let html = format!(
"<!doctype html><title>About browser</title><body style=\"background:{bg};color:{fg};font:14px -apple-system,Helvetica,sans-serif;padding:32px\">\
<h2 style=\"font-weight:500\">browser</h2>\
<p>Makepad chrome, Chromium Embedded Framework {cef} page rendering.</p>\
<p>Page rendering path: <b>{mode}</b> (accelerated frames: {frames}, last GPU blit: {blit}µs)</p>\
<p>ANGLE backend: {angle}</p></body>",
bg = palette().darker_background,
fg = palette().foreground,
cef = makepad_cef::CEF_VERSION,
mode = info.render_mode,
frames = info.accelerated_frames,
blit = info.last_blit_micros,
angle = std::env::var("MAKEPAD_CEF_USE_ANGLE").unwrap_or_else(|_| "default".into()),
);
let url = format!("data:text/html;charset=utf-8,{}", theme::percent_encode(&html));
self.new_tab(cx, Some(url));
}
// Omnibox.
let omnibox = self.ui.text_input(cx, ids!(omnibox));
if let Some((text, _modifiers)) = omnibox.returned(actions) {
self.navigate_from_omnibox(cx, &text);
}
if omnibox.escaped(actions) {
let shown = self.shown_url.clone();
omnibox.set_text(cx, &shown);
self.with_webview(cx, |cx, wv| wv.focus(cx));
}
for action in actions {
let Some(widget_action) = action.as_widget_action() else {
continue;
};
if widget_action.widget_uid != omnibox.widget_uid() {
continue;
}
match widget_action.cast::<TextInputAction>() {
TextInputAction::KeyFocus => {
self.omnibox_focused = true;
if let Some(mut inner) = omnibox.borrow_mut() {
inner.select_all(cx);
}
}
TextInputAction::KeyFocusLost => {
if std::env::var_os("MAKEPAD_CEF_DEBUG").is_some() {
let now = cx.keyboard.key_focus();
let webview = self.ui.widget(cx, ids!(webview)).area();
log!(
"omnibox lost key focus to {}",
if now == webview { "the webview" } else if now == Area::Empty { "nothing (Area::Empty)" } else { "another widget" }
);
}
self.omnibox_focused = false;
let shown = self.shown_url.clone();
omnibox.set_text(cx, &shown);
}
_ => {}
}
}
}
}
impl AppMain for App {
fn script_mod(vm: &mut ScriptVm) -> ScriptValue {
crate::makepad_widgets::script_mod(vm);
// The family theme bridge retints the stock widgets from the WM
// theme; the chrome roles go into mod.browser_theme.
makepad_wm_theme::apply(vm);
palette().apply(vm);
chrome::script_mod(vm);
webview::script_mod(vm);
self::script_mod(vm)
}
fn handle_event(&mut self, cx: &mut Cx, event: &Event) {
// The window manager asked politely (SUPER+W): go now, ahead of the
// kill that follows its grace. A warm-pool browser needs no waking —
// CEF paints nothing until a tile presents it.
if let Event::Custom(json) = event {
match makepad_wm_api::WmEvent::parse(json) {
Some(makepad_wm_api::WmEvent::Adopted) if self.ai_port.is_none() => {
self.ai_port = AiServicePort::open(cx, ai::manifest());
}
_ => {}
}
if let Some(makepad_wm_api::WmEvent::CloseRequested) = makepad_wm_api::WmEvent::parse(json) {
cx.quit();
return;
}
}
self.drain_ai_port(cx, event);
if let Event::KeyDown(ke) = event {
if self.handle_shortcut(cx, ke) {
return;
}
}
if self.focus_omnibox_pending && self.focus_frame.is_event(event).is_some() {
let omnibox = self.ui.text_input(cx, ids!(omnibox));
if omnibox.area().is_valid(cx) || self.focus_retries >= 60 {
self.focus_omnibox_pending = false;
self.focus_retries = 0;
omnibox.set_key_focus(cx);
omnibox.borrow_mut().map(|mut inner| inner.select_all(cx));
} else {
// At startup the first tab arrives before the chrome has been
// drawn: no area to focus yet, so try again next frame.
self.focus_retries += 1;
self.focus_frame = cx.new_next_frame();
}
}
if let Event::MouseDown(_) = event {
if self.menu_open {
// Any click outside the menu closes it; the menu's own
// buttons still get the event through the ui below.
let menu = self.ui.view(cx, ids!(menu)).area();
if let Event::MouseDown(md) = event {
if !(menu.is_valid(cx) && menu.rect(cx).contains(md.abs)) {
self.set_menu_open(cx, false);
}
}
}
}
self.match_event(cx, event);
self.ui.handle_event(cx, event, &mut Scope::empty());
}
}
fn main() {
app_main();
}
/// The CEF-aware entry point: helper-process bootstrap (`cef_execute_process`),
/// the app-bundle re-exec macOS needs for Chromium's subprocesses, then the
/// ordinary Makepad event loop with the `--remote` control surface.
#[cfg(not(any(target_arch = "wasm32", target_os = "android", target_env = "ohos")))]
pub fn app_main() {
let _ = uptime_ms();
if let Err(err) = makepad_cef::reexec_into_app_bundle_if_needed() {
panic!("CEF bundle re-exec failed: {err}");
}
match makepad_cef::bootstrap() {
Ok(BootstrapResult::Continue) => {}
Ok(BootstrapResult::Exit(code)) => std::process::exit(code),
Err(err) => panic!("CEF bootstrap failed: {err}"),
}
Cx::init_log();
if Cx::pre_start() {
return;
}
// Chromium composites the page on this colour. Left at CEF's default the
// first frames of every page are BLACK — a dark dip then a bright jump on
// open, and a black margin wherever a resize outruns the reflow.
makepad_cef::set_background_color(palette().page_background_argb());
// Only the cheap NSApp/pump preparation here: the window goes up first,
// the WebView runs `cef_initialize` on the frame after it is drawn.
if let Err(err) = makepad_cef::prepare() {
panic!("CEF prepare failed: {err}");
}
let cx = Rc::new(RefCell::new(Cx::new(
makepad_widgets::_app_main_event_closure!(App),
)));
let studio_http = makepad_widgets::resolve_studio_http();
cx.borrow_mut().init_websockets(&studio_http);
if makepad_widgets::should_run_stdin_loop_from_env() {
cx.borrow_mut().in_makepad_studio = true;
}
cx.borrow_mut().init_cx_os();
makepad_widgets::makepad_platform::remote::start_if_requested();
Cx::event_loop(cx.clone());
drop(cx);
makepad_cef::shutdown();
}
#[cfg(any(target_arch = "wasm32", target_os = "android", target_env = "ohos"))]
pub fn app_main() {
panic!("browser is desktop-only");
}

321
apps/browser/src/tabs.rs Normal file
View file

@ -0,0 +1,321 @@
//! The tab model. Each tab owns one CEF browser (created lazily when the
//! view first knows its size), the texture that browser paints into, and the
//! navigation state mirrored from CEF's display/load handlers.
use makepad_widgets::*;
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
pub struct TabId(pub u64);
pub struct Tab {
pub id: TabId,
pub browser: Option<makepad_cef::Browser>,
/// The URL to load once the browser exists.
pub initial_url: String,
pub title: String,
pub url: String,
pub loading: bool,
pub can_go_back: bool,
pub can_go_forward: bool,
/// What the quad samples: an IOSurface-backed texture on the GPU path,
/// a BGRA upload texture on the software path. This is the LAST GOOD
/// frame — it is never dropped for a resize, only replaced once its
/// successor holds a page frame.
pub texture: Option<Texture>,
/// CEF has put at least one frame in `texture`. The GPU path hands the
/// browser an IOSurface up front, so the texture EXISTS long before the
/// page is on it; drawing it meanwhile paints an opaque black hole over
/// the themed ground — which, inside the WM, reads as the window
/// flashing black on the way in. Nothing samples the texture until this
/// is true.
pub painted: bool,
/// Pixel size `texture` is allocated at. The GPU surface is rounded up to
/// a coarse grid so a drag walks many sizes inside one surface.
pub texture_alloc: Option<(usize, usize)>,
/// The sub-rect of `texture` that actually holds page pixels (the last
/// blit's copy region). The quad samples exactly this and stretches it
/// over the current rect, so a frame from a slightly older size shows
/// scaled instead of cropped-with-a-blank-margin.
pub texture_valid: Option<(usize, usize)>,
/// A larger/smaller surface already handed to CEF that has NOT been
/// painted yet. It is not drawn until it holds a frame; `texture` keeps
/// covering the page area meanwhile.
pub pending_texture: Option<Texture>,
pub pending_alloc: Option<(usize, usize)>,
/// Allocation size of the surface CEF is painting into right now.
pub accel_target_size: Option<(usize, usize)>,
/// Page size last handed to `browser.resize`, and when — `was_resized` is
/// rate-limited, because a resize faster than CEF can paint starves the
/// paint callback completely (measured: 0 frames at ~2 kHz).
pub resized_to: Option<(usize, usize)>,
pub resized_at: Option<std::time::Instant>,
pub deferred_resize: Option<(usize, usize, f32)>,
/// Page size the layout last asked for, and when it last changed — the
/// settle detector behind shrinking a surface back down.
pub wanted_size: Option<(usize, usize)>,
pub wanted_at: Option<std::time::Instant>,
pub accel_frame_counter: u64,
pub nav_generation: u64,
pub favicon: Option<Texture>,
pub init_error: Option<String>,
pub render_mode: makepad_cef::RenderMode,
}
impl Tab {
fn new(id: TabId, url: &str) -> Self {
Self {
id,
browser: None,
initial_url: url.to_string(),
title: String::new(),
url: url.to_string(),
loading: true,
can_go_back: false,
can_go_forward: false,
texture: None,
painted: false,
texture_alloc: None,
texture_valid: None,
pending_texture: None,
pending_alloc: None,
accel_target_size: None,
resized_to: None,
resized_at: None,
deferred_resize: None,
wanted_size: None,
wanted_at: None,
accel_frame_counter: 0,
nav_generation: 0,
favicon: None,
init_error: None,
render_mode: makepad_cef::RenderMode::None,
}
}
/// Title for the strip: the page title, else the host, else "New Tab".
pub fn display_title(&self) -> String {
if !self.title.trim().is_empty() {
return self.title.clone();
}
if crate::theme::is_new_tab_url(&self.url) || self.url.is_empty() {
return "New Tab".to_string();
}
host_of(&self.url).unwrap_or_else(|| self.url.clone())
}
}
/// What the tab strip needs to draw one tab.
#[derive(Clone, Debug)]
pub struct TabSummary {
pub id: TabId,
pub title: String,
pub loading: bool,
pub active: bool,
pub favicon: Option<Texture>,
}
#[derive(Default)]
pub struct TabModel {
pub tabs: Vec<Tab>,
pub active: usize,
next_id: u64,
}
impl TabModel {
pub fn len(&self) -> usize {
self.tabs.len()
}
pub fn index_of(&self, id: TabId) -> Option<usize> {
self.tabs.iter().position(|t| t.id == id)
}
pub fn active(&self) -> Option<&Tab> {
self.tabs.get(self.active)
}
pub fn active_mut(&mut self) -> Option<&mut Tab> {
self.tabs.get_mut(self.active)
}
pub fn active_id(&self) -> Option<TabId> {
self.active().map(|t| t.id)
}
/// Insert a tab right after the active one (Chrome's placement), or at
/// the end when there is none.
pub fn insert(&mut self, url: &str, activate: bool) -> TabId {
self.next_id += 1;
let id = TabId(self.next_id);
let at = if self.tabs.is_empty() {
0
} else {
(self.active + 1).min(self.tabs.len())
};
self.tabs.insert(at, Tab::new(id, url));
if activate || self.tabs.len() == 1 {
self.active = at;
} else if at <= self.active {
self.active += 1;
}
id
}
/// Remove a tab. Returns the removed tab (dropping it closes its
/// browser) and whether it was the active one.
pub fn remove(&mut self, id: TabId) -> Option<(Tab, bool)> {
let index = self.index_of(id)?;
let was_active = index == self.active;
let tab = self.tabs.remove(index);
if self.tabs.is_empty() {
self.active = 0;
} else if index < self.active {
self.active -= 1;
} else if was_active {
// Chrome activates the tab to the right, else the new last one.
self.active = index.min(self.tabs.len() - 1);
}
Some((tab, was_active))
}
pub fn activate(&mut self, id: TabId) -> bool {
if let Some(index) = self.index_of(id) {
self.active = index;
true
} else {
false
}
}
pub fn activate_offset(&mut self, delta: isize) {
if self.tabs.is_empty() {
return;
}
let len = self.tabs.len() as isize;
let next = ((self.active as isize + delta) % len + len) % len;
self.active = next as usize;
}
pub fn activate_index(&mut self, index: usize) {
if index < self.tabs.len() {
self.active = index;
}
}
pub fn summaries(&self) -> Vec<TabSummary> {
self.tabs
.iter()
.enumerate()
.map(|(i, t)| TabSummary {
id: t.id,
title: t.display_title(),
loading: t.loading,
active: i == self.active,
favicon: t.favicon.clone(),
})
.collect()
}
}
pub fn host_of(url: &str) -> Option<String> {
let rest = url.split_once("://")?.1;
let host = rest.split(['/', '?', '#']).next()?;
if host.is_empty() {
return None;
}
Some(host.trim_start_matches("www.").to_string())
}
/// Turn omnibox input into a URL: keep explicit schemes, prefix `https://`
/// for things that look like hosts, otherwise search.
pub fn resolve_omnibox(input: &str) -> String {
let input = input.trim();
if input.is_empty() {
return String::new();
}
let lower = input.to_ascii_lowercase();
if lower.contains("://")
|| lower.starts_with("about:")
|| lower.starts_with("data:")
|| lower.starts_with("chrome:")
|| lower.starts_with("file:")
|| lower.starts_with("javascript:")
|| lower.starts_with("view-source:")
{
return input.to_string();
}
let has_space = input.contains(char::is_whitespace);
let first = input.split(['/', '?', '#']).next().unwrap_or("");
let (host, port) = match first.rsplit_once(':') {
Some((h, p)) if !p.is_empty() && p.chars().all(|c| c.is_ascii_digit()) => (h, Some(p)),
_ => (first, None),
};
let looks_like_host = !has_space
&& !host.is_empty()
&& (host == "localhost"
|| host.parse::<std::net::IpAddr>().is_ok()
|| (host.contains('.')
&& !host.starts_with('.')
&& !host.ends_with('.')
&& host
.chars()
.all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '.')
&& host
.rsplit('.')
.next()
.map(|tld| tld.len() >= 2 && tld.chars().all(|c| c.is_ascii_alphabetic()))
.unwrap_or(false)));
let _ = port;
if looks_like_host {
format!("https://{input}")
} else {
format!(
"https://www.google.com/search?q={}",
crate::theme::percent_encode(input)
)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn omnibox_resolution() {
assert_eq!(resolve_omnibox("makepad.nl"), "https://makepad.nl");
assert_eq!(resolve_omnibox("http://x.org/a b"), "http://x.org/a b");
assert_eq!(resolve_omnibox("localhost:8080/x"), "https://localhost:8080/x");
assert_eq!(
resolve_omnibox("rust async traits"),
"https://www.google.com/search?q=rust%20async%20traits"
);
assert_eq!(
resolve_omnibox("hello"),
"https://www.google.com/search?q=hello"
);
assert_eq!(resolve_omnibox("about:blank"), "about:blank");
}
#[test]
fn model_insert_remove() {
let mut m = TabModel::default();
let a = m.insert("a", true);
let b = m.insert("b", true);
assert_eq!(m.active_id(), Some(b));
let c = m.insert("c", false);
assert_eq!(m.active_id(), Some(b));
assert_eq!(m.index_of(c), Some(2));
m.remove(b);
assert_eq!(m.active_id(), Some(c));
m.remove(c);
assert_eq!(m.active_id(), Some(a));
m.activate_offset(1);
assert_eq!(m.active_id(), Some(a));
}
#[test]
fn hosts() {
assert_eq!(host_of("https://www.makepad.nl/x"), Some("makepad.nl".into()));
assert_eq!(host_of("nope"), None);
}
}

221
apps/browser/src/theme.rs Normal file
View file

@ -0,0 +1,221 @@
//! The browser-chrome palette. Theming lives in splash: the chrome reads
//! `mod.browser_theme.*` (tab strip, toolbar, omnibox, icon roles), which this
//! module evaluates into the VM before the UI modules.
//!
//! Under makepad-wm the roles come from the WM's theme.splash
//! (`MAKEPAD_WM_THEME_SPLASH`, line-scanned by `makepad_wm_theme`, the family bridge);
//! standalone runs get Chrome's own dark palette.
use makepad_widgets::*;
/// Chrome-dark roles, keyed like the wm theme so one mapping serves both.
#[derive(Clone, Debug)]
pub struct Palette {
/// Tab strip background (the "frame").
pub darker_background: String,
/// Active tab + toolbar.
pub background: String,
/// Hovered inactive tab.
pub dark_background: String,
/// Button hover squares, omnibox focus fill.
pub lighter_background: String,
pub foreground: String,
pub dark_foreground: String,
pub bright_foreground: String,
pub muted: String,
pub selection: String,
pub accent: String,
}
impl Palette {
pub fn chrome_dark() -> Self {
Self {
darker_background: "#202124".into(),
background: "#35363a".into(),
dark_background: "#2b2c2f".into(),
lighter_background: "#3c4043".into(),
foreground: "#e8eaed".into(),
dark_foreground: "#9aa0a6".into(),
bright_foreground: "#ffffff".into(),
muted: "#5f6368".into(),
selection: "#264f78".into(),
accent: "#8ab4f8".into(),
}
}
/// The WM palette when wm exported one, else Chrome dark.
pub fn current() -> Self {
let fallback = Self::chrome_dark();
let Some(p) = makepad_wm_theme::current() else {
return fallback;
};
Self {
darker_background: p.hex("darker_background", &fallback.darker_background),
background: p.hex("background", &fallback.background),
dark_background: p.hex("dark_background", &fallback.dark_background),
lighter_background: p.hex("lighter_background", &fallback.lighter_background),
foreground: p.hex("foreground", &fallback.foreground),
dark_foreground: p.hex("dark_foreground", &fallback.dark_foreground),
bright_foreground: p.hex("bright_foreground", &fallback.bright_foreground),
muted: p.hex("muted", &fallback.muted),
selection: p.hex("selection", &fallback.selection),
accent: p.hex("accent", &fallback.accent),
}
}
/// The `mod.browser_theme = {...}` splash source. Runtime-evaluated, so plain
/// `#hex` (the `#x` escape is a proc-macro-only hazard).
pub fn splash_source(&self) -> String {
format!(
"mod.browser_theme = {{\n\
\x20 darker_background: {}\n\
\x20 background: {}\n\
\x20 dark_background: {}\n\
\x20 lighter_background: {}\n\
\x20 foreground: {}\n\
\x20 dark_foreground: {}\n\
\x20 bright_foreground: {}\n\
\x20 muted: {}\n\
\x20 selection: {}\n\
\x20 accent: {}\n\
}}\n\
true\n",
self.darker_background,
self.background,
self.dark_background,
self.lighter_background,
self.foreground,
self.dark_foreground,
self.bright_foreground,
self.muted,
self.selection,
self.accent,
)
}
/// Evaluate `mod.browser_theme` into the VM. Call after
/// `makepad_widgets::script_mod(vm)` and before the chrome modules.
pub fn apply(&self, vm: &mut ScriptVm) {
let script_mod_id = ScriptMod {
cargo_manifest_path: env!("CARGO_MANIFEST_DIR").to_string(),
module_path: "browser_theme".to_string(),
file: "browser_theme.splash".to_string(),
line: 0,
column: 0,
code: self.splash_source(),
values: vec![],
};
vm.eval(script_mod_id);
for e in vm.take_errors() {
log!("browser theme: {}", e);
}
}
/// What CEF fills the page with before the site has composited anything,
/// and behind area a reflow has not reached yet. The same colour as the
/// ground the page quad sits on, so neither a cold tab nor a mid-resize
/// page shows a black hole.
pub fn page_background_argb(&self) -> u32 {
parse_argb(&self.darker_background).unwrap_or(0xff20_2124)
}
/// The new-tab page: a data URL in the theme's colours, so a fresh tab
/// never flashes white.
pub fn new_tab_url(&self) -> String {
let html = format!(
"<!doctype html><html><head><meta charset=utf-8><title>New Tab</title>\
<style>html,body{{margin:0;height:100%;background:{bg};color:{fg};\
font:14px -apple-system,Helvetica,Arial,sans-serif}}\
.c{{display:flex;height:100%;align-items:center;justify-content:center;\
flex-direction:column;gap:10px}}.n{{font-size:28px;letter-spacing:1px;color:{fgb}}}\
.s{{color:{fgd}}}</style></head><body><div class=c>\
<div class=n>browser</div><div class=s>Type a URL or search in the box above</div>\
</div></body></html>",
bg = self.darker_background,
fg = self.foreground,
fgb = self.bright_foreground,
fgd = self.dark_foreground,
);
format!("data:text/html;charset=utf-8,{}", percent_encode(&html))
}
}
pub fn parse_hex(s: &str) -> Option<Vec4f> {
let s = s.trim().trim_start_matches('#');
if s.len() != 6 {
return None;
}
let r = u8::from_str_radix(&s[0..2], 16).ok()?;
let g = u8::from_str_radix(&s[2..4], 16).ok()?;
let b = u8::from_str_radix(&s[4..6], 16).ok()?;
Some(vec4(
r as f32 / 255.0,
g as f32 / 255.0,
b as f32 / 255.0,
1.0,
))
}
/// A `#rrggbb` role as opaque ARGB (`0xFFRRGGBB`) — the form CEF wants for
/// `cef_browser_settings_t::background_color`.
pub fn parse_argb(s: &str) -> Option<u32> {
let s = s.trim().trim_start_matches('#');
if s.len() != 6 {
return None;
}
let rgb = u32::from_str_radix(s, 16).ok()?;
Some(0xff00_0000 | rgb)
}
/// Minimal percent-encoding for a `data:` URL payload.
pub fn percent_encode(s: &str) -> String {
let mut out = String::with_capacity(s.len() * 3);
for b in s.bytes() {
match b {
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' | b'.' | b'~' => {
out.push(b as char)
}
_ => out.push_str(&format!("%{:02X}", b)),
}
}
out
}
/// Is this new-tab-page URL (never shown in the omnibox)?
pub fn is_new_tab_url(url: &str) -> bool {
url.starts_with("data:text/html;charset=utf-8,%3C%21doctype%20html%3E%3Chtml%3E%3Chead%3E%3Cmeta%20charset%3Dutf-8%3E%3Ctitle%3ENew%20Tab")
|| url == "about:blank"
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn new_tab_url_is_recognised() {
let p = Palette::chrome_dark();
assert!(is_new_tab_url(&p.new_tab_url()));
assert!(!is_new_tab_url("https://makepad.nl"));
}
#[test]
fn hex_parses() {
let c = parse_hex("#8ab4f8").unwrap();
assert!((c.x - 0x8a as f32 / 255.0).abs() < 1e-6);
assert!(parse_hex("#12345").is_none());
}
#[test]
fn argb_is_opaque() {
// CEF paints the page on this before the site composites; a
// transparent (or zero) colour is what makes a page open black.
assert_eq!(parse_argb("#202124"), Some(0xff20_2124));
assert_eq!(parse_argb("202124"), Some(0xff20_2124));
assert_eq!(parse_argb("#nope"), None);
assert_eq!(
Palette::chrome_dark().page_background_argb(),
0xff20_2124,
"the CEF fill must match the themed ground the page sits on"
);
}
}

978
apps/browser/src/webview.rs Normal file
View file

@ -0,0 +1,978 @@
//! The page area. One widget hosts every tab's CEF browser; the active
//! tab's texture is what gets drawn, input goes to the active browser,
//! background tabs are told they are hidden so they stop painting.
//!
//! Rendering: with `shared_texture_enabled` CEF paints on the GPU into pooled
//! IOSurfaces and `libs/cef` blits each one into a Makepad-owned IOSurface
//! texture (`Cx::create_iosurface_render_texture`) that the quad samples —
//! no CPU readback anywhere. The classic `on_paint` BGRA upload stays as the
//! fallback (`MAKEPAD_CEF_SOFTWARE=1`, or a CEF build without GPU paint).
use crate::tabs::{TabId, TabModel, TabSummary};
// The surface policy is the stock Browser widget's — one source of truth for
// how a CEF page survives a resize.
#[cfg(target_os = "macos")]
use makepad_widgets::browser::{needs_new_surface, surface_alloc, SETTLE};
use makepad_widgets::browser::{Browser as BrowserKeys, RESIZE_INTERVAL};
use makepad_widgets::image::DrawImage;
use makepad_widgets::*;
script_mod! {
use mod.prelude.widgets_internal.*
use mod.widgets.*
mod.widgets.WebViewBase = #(WebView::register_widget(vm))
mod.widgets.WebView = set_type_default() do mod.widgets.WebViewBase{
width: Fill
height: Fill
draw_empty +: {
color: uniform(mod.browser_theme.darker_background)
pixel: fn() {
return self.color
}
}
draw_status +: {
color: mod.browser_theme.dark_foreground
text_style: theme.font_regular{
font_size: 11
}
}
}
}
#[derive(Clone, Debug, Default, PartialEq)]
pub enum WebViewAction {
#[default]
None,
/// Titles / URLs / loading state / favicons / the tab list changed.
TabsChanged,
}
/// What the chrome shows for the active tab.
#[derive(Clone, Debug, Default)]
pub struct ActiveInfo {
pub id: Option<TabId>,
pub url: String,
pub title: String,
pub loading: bool,
pub can_go_back: bool,
pub can_go_forward: bool,
pub render_mode: String,
pub accelerated_frames: u64,
pub last_blit_micros: u64,
}
#[derive(Script, ScriptHook, Widget)]
pub struct WebView {
#[uid]
uid: WidgetUid,
#[source]
source: ScriptObjectRef,
#[walk]
walk: Walk,
#[redraw]
#[live]
draw_bg: DrawImage,
#[live]
draw_empty: DrawQuad,
#[live]
draw_status: DrawText,
#[rust]
tabs: TabModel,
/// `cef_initialize` runs on the frame after the chrome first drew, so
/// the window is up before Chromium's processes spawn.
#[rust]
cef_ready: bool,
#[rust]
cef_init_frame: NextFrame,
#[rust]
cef_init_requested: bool,
#[rust]
cef_init_error: Option<String>,
#[rust]
first_frame_logged: bool,
#[rust]
pump_timer: Timer,
#[rust]
pressed_buttons: MouseButton,
#[rust]
suppress_next_paste_shortcut: bool,
#[rust]
pump_started: bool,
}
/// Env-gated resize tracing (`MAKEPAD_BROWSER_TRACE=1`): timestamps + sizes on every
/// draw, resize and target swap. Debug rig — not for committing.
pub fn trace_on() -> bool {
static ON: std::sync::OnceLock<bool> = std::sync::OnceLock::new();
*ON.get_or_init(|| std::env::var_os("MAKEPAD_BROWSER_TRACE").is_some())
}
macro_rules! trace {
($($arg:tt)*) => {
if crate::webview::trace_on() {
eprintln!("[mpb-trace {:>7}us] {}", crate::uptime_us(), format!($($arg)*));
}
};
}
impl WebView {
const PUMP_INTERVAL: f64 = 1.0 / 120.0;
// ------------------------------------------------------------------
// Tab operations (called by the app from chrome actions / shortcuts)
// ------------------------------------------------------------------
pub fn new_tab(&mut self, cx: &mut Cx, url: &str, activate: bool) -> TabId {
let id = self.tabs.insert(url, activate);
if activate {
self.apply_active_visibility();
self.focus(cx);
}
self.notify(cx);
id
}
pub fn close_tab(&mut self, cx: &mut Cx, id: TabId) {
if let Some((tab, _was_active)) = self.tabs.remove(id) {
// Dropping the tab closes its browser.
drop(tab);
}
self.apply_active_visibility();
self.notify(cx);
}
pub fn activate(&mut self, cx: &mut Cx, id: TabId) {
if self.tabs.activate(id) {
self.apply_active_visibility();
self.focus(cx);
self.notify(cx);
}
}
pub fn activate_offset(&mut self, cx: &mut Cx, delta: isize) {
self.tabs.activate_offset(delta);
self.apply_active_visibility();
self.focus(cx);
self.notify(cx);
}
pub fn activate_index(&mut self, cx: &mut Cx, index: usize) {
self.tabs.activate_index(index);
self.apply_active_visibility();
self.focus(cx);
self.notify(cx);
}
pub fn active_id(&self) -> Option<TabId> {
self.tabs.active_id()
}
pub fn tab_count(&self) -> usize {
self.tabs.len()
}
pub fn summaries(&self) -> Vec<TabSummary> {
self.tabs.summaries()
}
/// Title, URL and active marker for the browser's AI `tabs` tool.
pub fn ai_tabs(&self) -> Vec<(String, String, bool)> {
self.tabs
.tabs
.iter()
.enumerate()
.map(|(index, tab)| {
(tab.display_title(), tab.url.clone(), index == self.tabs.active)
})
.collect()
}
pub fn active_info(&self) -> ActiveInfo {
let Some(tab) = self.tabs.active() else {
return ActiveInfo::default();
};
let (accelerated_frames, last_blit_micros) = tab
.browser
.as_ref()
.map(|b| {
let s = b.accelerated_stats();
(s.frames, s.last_blit_micros)
})
.unwrap_or((0, 0));
ActiveInfo {
id: Some(tab.id),
url: tab.url.clone(),
title: tab.display_title(),
loading: tab.loading,
can_go_back: tab.can_go_back,
can_go_forward: tab.can_go_forward,
render_mode: format!("{:?}", tab.render_mode),
accelerated_frames,
last_blit_micros,
}
}
pub fn navigate(&mut self, cx: &mut Cx, url: &str) {
if let Some(tab) = self.tabs.active_mut() {
tab.url = url.to_string();
tab.title.clear();
tab.favicon = None;
tab.loading = true;
match &mut tab.browser {
Some(browser) => {
if let Err(err) = browser.set_url(url) {
log!("navigate failed: {err}");
}
}
None => tab.initial_url = url.to_string(),
}
}
self.focus(cx);
self.notify(cx);
}
pub fn go_back(&mut self, cx: &mut Cx) {
if let Some(browser) = self.active_browser() {
let _ = browser.go_back();
}
self.focus(cx);
}
pub fn go_forward(&mut self, cx: &mut Cx) {
if let Some(browser) = self.active_browser() {
let _ = browser.go_forward();
}
self.focus(cx);
}
pub fn reload(&mut self, cx: &mut Cx) {
if let Some(browser) = self.active_browser() {
let _ = browser.reload();
}
self.focus(cx);
}
pub fn stop(&mut self, cx: &mut Cx) {
if let Some(browser) = self.active_browser() {
let _ = browser.stop_load();
}
self.focus(cx);
}
pub fn focus(&mut self, cx: &mut Cx) {
let area = self.draw_bg.area();
if area.is_valid(cx) {
cx.set_key_focus(area);
}
if let Some(browser) = self.active_browser() {
let _ = browser.set_focus(true);
}
}
// ------------------------------------------------------------------
fn notify(&mut self, cx: &mut Cx) {
cx.widget_action(self.uid, WebViewAction::TabsChanged);
self.redraw(cx);
}
fn active_browser(&mut self) -> Option<&mut makepad_cef::Browser> {
self.tabs.active_mut().and_then(|t| t.browser.as_mut())
}
/// Background tabs stop painting; the active one is shown.
fn apply_active_visibility(&mut self) {
let active = self.tabs.active;
for (i, tab) in self.tabs.tabs.iter_mut().enumerate() {
if let Some(browser) = &mut tab.browser {
let _ = browser.set_hidden(i != active);
}
}
}
fn ensure_active_browser(&mut self, cx: &mut Cx, width: usize, height: usize, dpi: f32) {
if !self.cef_ready {
return;
}
let active = self.tabs.active;
let Some(tab) = self.tabs.tabs.get_mut(active) else {
return;
};
if tab.browser.is_none() && tab.init_error.is_none() {
match makepad_cef::Browser::new(&tab.initial_url, width, height, dpi) {
Ok(browser) => {
tab.browser = Some(browser);
}
Err(err) => {
let message = err.to_string();
log!("CEF browser creation failed: {message}");
tab.init_error = Some(message);
}
}
}
if tab.browser.is_none() {
return;
}
if let Some(browser) = &mut tab.browser {
let _ = browser.set_hidden(false);
}
let now = std::time::Instant::now();
if tab.wanted_size != Some((width, height)) {
tab.wanted_size = Some((width, height));
tab.wanted_at = Some(now);
}
Self::sync_browser_size(tab, width, height, dpi, now);
Self::sync_accel_surface(cx, tab, width, height, now);
}
/// Tell CEF the page size, at most once per `RESIZE_INTERVAL`; anything
/// faster is remembered and applied by the next pump, so the final size of
/// a drag always lands.
fn sync_browser_size(
tab: &mut crate::tabs::Tab,
width: usize,
height: usize,
dpi: f32,
now: std::time::Instant,
) {
if tab.resized_to == Some((width, height)) {
// `Browser::resize` no-ops on an unchanged size; this still lets a
// dpi change through.
if let Some(browser) = &mut tab.browser {
if let Err(err) = browser.resize(width, height, dpi) {
log!("CEF resize failed: {err}");
}
}
tab.deferred_resize = None;
return;
}
let due = tab
.resized_at
.is_none_or(|last| now.duration_since(last) >= RESIZE_INTERVAL);
if !due {
tab.deferred_resize = Some((width, height, dpi));
return;
}
if let Some(browser) = &mut tab.browser {
if let Err(err) = browser.resize(width, height, dpi) {
log!("CEF resize failed: {err}");
}
}
trace!("was_resized {}x{}", width, height);
tab.resized_to = Some((width, height));
tab.resized_at = Some(now);
tab.deferred_resize = None;
}
/// GPU path: hand the browser a Makepad-owned IOSurface to copy into.
///
/// The surface is over-allocated (grid-rounded) and grows only when the
/// page outgrows it; it shrinks only once the size has settled. A new
/// surface is BLANK, so it goes to `pending_texture` and the last good one
/// keeps being drawn until CEF has put a frame in the new one — that is
/// what keeps a drag-resize from flashing the themed ground.
fn sync_accel_surface(
cx: &mut Cx,
tab: &mut crate::tabs::Tab,
width: usize,
height: usize,
now: std::time::Instant,
) {
#[cfg(target_os = "macos")]
{
if !tab.browser.as_ref().is_some_and(|b| b.is_accelerated()) {
return;
}
let want = surface_alloc(width, height);
let settled = tab
.wanted_at
.is_some_and(|since| now.duration_since(since) >= SETTLE);
if !needs_new_surface(tab.accel_target_size, tab.pending_alloc, want, settled) {
return;
}
trace!(
"surface {:?} -> {}x{} for page {}x{}",
tab.accel_target_size,
want.0,
want.1,
width,
height
);
let (texture, iosurface, _id) = cx.create_iosurface_render_texture(want.0, want.1);
let Some(browser) = &mut tab.browser else {
return;
};
match browser.set_accelerated_target(iosurface, want.0, want.1) {
Ok(()) => {
tab.accel_target_size = Some(want);
if tab.painted && tab.texture.is_some() {
// Keep showing the last good frame until this one has
// one of its own.
tab.pending_texture = Some(texture);
tab.pending_alloc = Some(want);
} else {
tab.texture = Some(texture);
tab.texture_alloc = Some(want);
tab.texture_valid = None;
tab.pending_texture = None;
tab.pending_alloc = None;
}
}
Err(err) => {
log!("accelerated target failed, software frames only: {err}");
tab.accel_target_size = None;
tab.pending_texture = None;
tab.pending_alloc = None;
}
}
}
#[cfg(not(target_os = "macos"))]
{
let _ = (cx, tab, width, height, now);
}
}
fn apply_software_frame(cx: &mut Cx, tab: &mut crate::tabs::Tab, frame: makepad_cef::Frame) {
let size = (frame.width, frame.height);
match &tab.texture {
Some(texture)
if tab.accel_target_size.is_none()
&& texture.get_format(cx).vec_width_height() == Some(size) =>
{
texture.set_data_u32(cx, frame.width, frame.height, frame.pixels);
}
_ => {
tab.accel_target_size = None;
tab.pending_texture = None;
tab.pending_alloc = None;
tab.texture = Some(Texture::new_with_format(
cx,
TextureFormat::VecBGRAu8_32 {
data: Some(frame.pixels),
width: frame.width,
height: frame.height,
updated: TextureUpdated::Full,
},
));
}
}
// An upload texture is exactly the page: the whole thing is valid.
tab.texture_alloc = Some(size);
tab.texture_valid = Some(size);
tab.painted = true;
}
fn favicon_texture(cx: &mut Cx, frame: makepad_cef::Frame) -> Texture {
Texture::new_with_format(
cx,
TextureFormat::VecBGRAu8_32 {
data: Some(frame.pixels),
width: frame.width,
height: frame.height,
updated: TextureUpdated::Full,
},
)
}
/// One pump: run CEF's message loop, collect frames and navigation
/// changes from every tab, open queued popups as tabs.
fn pump(&mut self, cx: &mut Cx) {
makepad_cef::do_message_loop_work();
let active = self.tabs.active;
let mut changed = false;
let mut redraw = false;
let mut popups = Vec::new();
for (i, tab) in self.tabs.tabs.iter_mut().enumerate() {
let Some(browser) = &mut tab.browser else {
continue;
};
let mut latest = None;
while let Some(frame) = browser.take_frame() {
latest = Some(frame);
}
let counter = browser.accelerated_frame_counter();
if counter != tab.accel_frame_counter {
let stats = browser.accelerated_stats();
let valid = (stats.last_copy_width, stats.last_copy_height);
if stats.target_frames > 0 && valid.0 > 0 && valid.1 > 0 {
// The surface CEF is painting into now holds a real frame:
// if it was the pending one, this is the moment to swap —
// never before, so the page area never goes blank.
if let (Some(texture), Some(alloc)) =
(tab.pending_texture.take(), tab.pending_alloc.take())
{
trace!(
"promote pending {}x{} (valid {}x{})",
alloc.0,
alloc.1,
valid.0,
valid.1
);
tab.texture = Some(texture);
tab.texture_alloc = Some(alloc);
}
tab.texture_valid = Some(valid);
tab.painted = true;
}
if i == active {
trace!(
"paint landed {}x{} (+{} frames, surface {:?})",
stats.last_width,
stats.last_height,
counter - tab.accel_frame_counter,
tab.accel_target_size
);
}
tab.accel_frame_counter = counter;
if i == active {
redraw = true;
}
if !self.first_frame_logged {
self.first_frame_logged = true;
log!("browser: first page frame at {} ms", crate::uptime_ms());
}
}
let generation = browser.nav_generation();
if generation != tab.nav_generation {
tab.nav_generation = generation;
tab.title = browser.title();
let url = browser.url();
if !url.is_empty() {
tab.url = url;
}
tab.loading = browser.is_loading();
tab.can_go_back = browser.can_go_back();
tab.can_go_forward = browser.can_go_forward();
if let Some(favicon) = browser.take_favicon() {
tab.favicon = Some(Self::favicon_texture(cx, favicon));
}
popups.extend(browser.take_popup_requests());
changed = true;
}
let mode = browser.render_mode();
if mode != tab.render_mode {
tab.render_mode = mode;
log!("tab {:?} render mode: {:?}", tab.id, mode);
changed = true;
}
if let Some(frame) = latest {
Self::apply_software_frame(cx, tab, frame);
if i == active {
redraw = true;
}
}
// A resize that arrived faster than CEF can take them: apply the
// last one now, so the end of a drag always reaches the page.
if let Some((w, h, dpi)) = tab.deferred_resize {
Self::sync_browser_size(tab, w, h, dpi, std::time::Instant::now());
if i == active {
redraw = true;
}
}
}
for url in popups {
self.tabs.insert(&url, true);
self.apply_active_visibility();
changed = true;
redraw = true;
}
if changed {
cx.widget_action(self.uid, WebViewAction::TabsChanged);
}
if redraw {
self.redraw(cx);
}
}
// ------------------------------------------------------------------
// Input routing (same mapping as the stock Browser widget)
// ------------------------------------------------------------------
fn browser_rect(&self, cx: &mut Cx) -> Option<Rect> {
let area = self.draw_bg.area();
if area.is_valid(cx) {
Some(area.rect(cx))
} else {
None
}
}
/// CEF takes mouse coordinates in view points (it applies the device
/// scale factor itself).
fn cef_position(&self, cx: &mut Cx, abs: Vec2d) -> Option<(i32, i32)> {
let rect = self.browser_rect(cx)?;
let local = abs - rect.pos;
Some((local.x.round() as i32, local.y.round() as i32))
}
fn send_mouse_move(&mut self, cx: &mut Cx, abs: Vec2d, modifiers: KeyModifiers, leave: bool) {
let Some((x, y)) = self.cef_position(cx, abs) else {
return;
};
let m = BrowserKeys::cef_modifiers(modifiers, self.pressed_buttons);
if let Some(browser) = self.active_browser() {
let _ = browser.send_mouse_move(x, y, m, leave);
}
}
fn send_mouse_click(
&mut self,
cx: &mut Cx,
abs: Vec2d,
modifiers: KeyModifiers,
button: Option<MouseButton>,
mouse_up: bool,
click_count: i32,
) {
let Some((x, y)) = self.cef_position(cx, abs) else {
return;
};
let m = BrowserKeys::cef_modifiers(modifiers, self.pressed_buttons);
let b = BrowserKeys::cef_mouse_button(button);
if let Some(browser) = self.active_browser() {
let _ = browser.send_mouse_click(x, y, m, b, mouse_up, click_count.max(1));
}
}
fn send_mouse_wheel(&mut self, cx: &mut Cx, abs: Vec2d, modifiers: KeyModifiers, delta: Vec2d) {
let Some((x, y)) = self.cef_position(cx, abs) else {
return;
};
let m = BrowserKeys::cef_modifiers(modifiers, self.pressed_buttons)
| makepad_cef::EVENTFLAG_PRECISION_SCROLLING_DELTA;
if let Some(browser) = self.active_browser() {
let _ = browser.send_mouse_wheel(x, y, m, delta.x.round() as i32, delta.y.round() as i32);
}
}
fn send_key(&mut self, key_event: &KeyEvent, event_type: i32) {
let modifiers = BrowserKeys::key_event_modifiers(key_event);
let windows_key_code = BrowserKeys::windows_key_code(key_event.key_code);
let character = if key_event.modifiers.control
|| key_event.modifiers.alt
|| key_event.modifiers.logo
{
0
} else {
BrowserKeys::key_char(key_event.key_code, key_event.modifiers.shift)
.map(|ch| ch as u16)
.unwrap_or(0)
};
let send_char = event_type == makepad_cef::KEY_EVENT_KEYDOWN
&& character != 0
&& !key_event.modifiers.control
&& !key_event.modifiers.alt
&& !key_event.modifiers.logo
&& BrowserKeys::sends_char_on_keydown(key_event.key_code);
if let Some(browser) = self.active_browser() {
let _ = browser.send_key_event(
event_type,
modifiers,
windows_key_code,
windows_key_code,
character,
character,
false,
);
if send_char {
let _ = browser.send_key_event(
makepad_cef::KEY_EVENT_CHAR,
modifiers,
windows_key_code,
windows_key_code,
character,
character,
false,
);
}
}
}
fn update_ime_spot(&self, cx: &mut Cx, pos: Vec2d) {
let area = self.draw_bg.area();
if area.is_valid(cx) {
cx.show_text_ime(area, pos);
}
}
}
impl Widget for WebView {
fn handle_event(&mut self, cx: &mut Cx, event: &Event, _scope: &mut Scope) {
if let Event::Startup = event {
if !self.pump_started {
self.pump_started = true;
self.pump_timer = cx.start_interval(Self::PUMP_INTERVAL);
}
}
if let Event::Shutdown = event {
self.tabs.tabs.clear();
return;
}
if !self.cef_ready && self.cef_init_frame.is_event(event).is_some() {
// The chrome has been drawn: bring Chromium up now.
match makepad_cef::initialize() {
Ok(()) => {
self.cef_ready = true;
log!(
"browser: CEF {} initialized at {} ms",
makepad_cef::CEF_VERSION,
crate::uptime_ms()
);
}
Err(err) => {
let message = err.to_string();
log!("CEF initialize failed: {message}");
self.cef_init_error = Some(message);
}
}
self.redraw(cx);
}
if self.pump_timer.is_event(event).is_some() && self.cef_ready {
self.pump(cx);
}
match event.hits_with_capture_overload(cx, self.draw_bg.area(), true) {
Hit::KeyFocus(_) => {
if let Some(browser) = self.active_browser() {
let _ = browser.set_focus(true);
}
if let Some(rect) = self.browser_rect(cx) {
self.update_ime_spot(cx, rect.pos);
}
}
Hit::KeyFocusLost(_) => {
if let Some(browser) = self.active_browser() {
let _ = browser.set_focus(false);
}
cx.hide_text_ime();
self.suppress_next_paste_shortcut = false;
}
Hit::FingerDown(fe) => {
let button = fe.mouse_button().unwrap_or(MouseButton::PRIMARY);
self.pressed_buttons.insert(button);
cx.set_key_focus(self.draw_bg.area());
if let Some(browser) = self.active_browser() {
let _ = browser.set_focus(true);
}
self.update_ime_spot(cx, fe.abs);
self.send_mouse_move(cx, fe.abs, fe.modifiers, false);
self.send_mouse_click(
cx,
fe.abs,
fe.modifiers,
Some(button),
false,
fe.tap_count as i32,
);
}
Hit::FingerMove(fe) => {
self.send_mouse_move(cx, fe.abs, fe.modifiers, false);
}
Hit::FingerUp(fe) => {
let button = fe.mouse_button().unwrap_or(MouseButton::PRIMARY);
self.send_mouse_move(cx, fe.abs, fe.modifiers, false);
self.send_mouse_click(
cx,
fe.abs,
fe.modifiers,
Some(button),
true,
fe.tap_count as i32,
);
self.pressed_buttons.remove(button);
}
Hit::FingerHoverIn(fe) | Hit::FingerHoverOver(fe) => {
self.send_mouse_move(cx, fe.abs, fe.modifiers, false);
}
Hit::FingerHoverOut(fe) => {
self.send_mouse_move(cx, fe.abs, fe.modifiers, true);
}
Hit::FingerScroll(fe) => {
self.send_mouse_wheel(cx, fe.abs, fe.modifiers, fe.scroll);
}
Hit::KeyDown(key_event) => {
// Browser-level shortcuts (Cmd+T/W/L/R/[ ]) are the app's;
// they never reach the page.
if key_event.modifiers.logo && crate::is_app_shortcut(&key_event) {
return;
}
if self.suppress_next_paste_shortcut
&& key_event.key_code == KeyCode::KeyV
&& key_event.modifiers.is_primary()
{
self.suppress_next_paste_shortcut = false;
} else {
self.send_key(&key_event, makepad_cef::KEY_EVENT_KEYDOWN);
}
}
Hit::KeyUp(key_event) => {
if key_event.modifiers.logo && crate::is_app_shortcut(&key_event) {
return;
}
self.send_key(&key_event, makepad_cef::KEY_EVENT_KEYUP);
}
Hit::TextInput(text_event) => {
let ime_pos = self
.browser_rect(cx)
.map(|rect| rect.pos)
.unwrap_or_default();
self.update_ime_spot(cx, ime_pos);
if text_event.was_paste {
self.suppress_next_paste_shortcut = true;
}
let modifiers = BrowserKeys::cef_modifiers(cx.keyboard.modifiers(), MouseButton::empty());
let char_data = BrowserKeys::char_event_data(&text_event.input);
if let Some(browser) = self.active_browser() {
if text_event.was_paste || text_event.replace_last || char_data.is_none() {
let _ = browser.ime_commit_text(&text_event.input);
} else if let Some((windows_key_code, character)) = char_data {
let _ = browser.send_key_event(
makepad_cef::KEY_EVENT_CHAR,
modifiers,
windows_key_code,
windows_key_code,
character,
character,
false,
);
}
}
}
_ => {}
}
}
fn draw_walk(&mut self, cx: &mut Cx2d, _scope: &mut Scope, walk: Walk) -> DrawStep {
let rect = cx.peek_walk_turtle(walk);
let dpi = cx.current_dpi_factor() as f32;
let width = (rect.size.x.max(1.0) * dpi as f64).round().max(1.0) as usize;
let height = (rect.size.y.max(1.0) * dpi as f64).round().max(1.0) as usize;
self.ensure_active_browser(cx, width, height, dpi);
let shown = self.tabs.active().filter(|t| t.painted).and_then(|t| {
t.texture
.clone()
.map(|texture| (texture, t.texture_alloc, t.texture_valid))
});
// Themed ground under the page, so a tab that has not painted yet
// (or a transparent page) never shows black.
self.draw_empty.draw_abs(cx, rect);
if !self.cef_ready {
let status = match &self.cef_init_error {
Some(err) => format!("browser engine failed to start: {err}"),
None => "starting browser engine…".to_string(),
};
self.draw_status
.draw_abs(cx, rect.pos + dvec2(16.0, 14.0), &status);
if !self.cef_init_requested && self.cef_init_error.is_none() {
self.cef_init_requested = true;
self.cef_init_frame = cx.new_next_frame();
}
}
match shown {
Some((texture, alloc, valid)) => {
// Sample only the part of the surface that holds page pixels
// and stretch it over the current rect: mid-drag that is the
// last good frame at a slightly older size (a fraction of a
// percent of scale), never a blank margin.
let scale = match (alloc, valid) {
(Some((aw, ah)), Some((vw, vh))) if aw > 0 && ah > 0 && vw > 0 && vh > 0 => {
vec2(
(vw as f32 / aw as f32).min(1.0),
(vh as f32 / ah as f32).min(1.0),
)
}
_ => vec2(1.0, 1.0),
};
self.draw_bg.image_scale = scale;
self.draw_bg.draw_vars.set_texture(0, &texture);
self.draw_bg.opacity = 1.0;
}
None => {
self.draw_bg.image_scale = vec2(1.0, 1.0);
self.draw_bg.draw_vars.empty_texture(0);
self.draw_bg.opacity = 0.0;
}
}
if let Some(tab) = self.tabs.active() {
trace!(
"draw rect {}x{} px, surface {:?}, valid {:?}, pending {:?}, painted={}",
width,
height,
tab.accel_target_size,
tab.texture_valid,
tab.pending_alloc,
tab.painted
);
}
self.draw_bg.draw_walk(cx, walk);
cx.add_nav_stop(self.draw_bg.area(), NavRole::TextInput, Inset::default());
DrawStep::done()
}
}
#[cfg(test)]
mod tests {
use super::*;
use makepad_widgets::browser::SURFACE_GRID;
#[test]
fn surface_is_grid_rounded() {
assert_eq!(surface_alloc(1280, 860), (1280, 1024));
assert_eq!(surface_alloc(1281, 1025), (1536, 1280));
assert_eq!(surface_alloc(0, 0), (SURFACE_GRID, SURFACE_GRID));
}
/// The whole point of the grid: a drag walks many page sizes inside one
/// surface instead of handing CEF a blank one at every step (which is
/// what made the page area flash the themed ground for a whole drag).
#[test]
fn a_drag_walks_many_sizes_per_surface() {
let mut current = Some(surface_alloc(1600, 900));
let mut steps = 0;
let mut swaps = 0;
for w in (1600..2600).step_by(8) {
let want = surface_alloc(w, 900);
steps += 1;
if needs_new_surface(current, None, want, false) {
swaps += 1;
current = Some(want);
}
}
assert_eq!(steps, 125);
// 1600 -> 2592 crosses the 1792 / 2048 / 2304 / 2560 lines: four
// surfaces for 125 drag steps, instead of 125.
assert_eq!(swaps, 4);
}
#[test]
fn surface_grows_at_once_and_shrinks_only_when_settled() {
let current = Some((1536, 1024));
// Outgrowing the surface would clip the blit: no waiting.
assert!(needs_new_surface(current, None, (1792, 1024), false));
// Smaller page, still moving: keep the surface (and the frame in it).
assert!(!needs_new_surface(current, None, (1024, 1024), false));
// Settled: hand back the slack.
assert!(needs_new_surface(current, None, (1024, 1024), true));
// A bigger surface is already on its way; do not queue another.
assert!(!needs_new_surface(
current,
Some((1792, 1024)),
(1792, 1024),
true
));
assert!(needs_new_surface(
current,
Some((1792, 1024)),
(2048, 1024),
false
));
// Nothing yet: allocate.
assert!(needs_new_surface(None, None, (1280, 1024), false));
}
}

38
apps/fab/Cargo.toml Normal file
View file

@ -0,0 +1,38 @@
[package]
name = "makepad-fab"
version = "0.1.0"
authors = ["Makepad <info@makepad.nl>"]
edition = "2021"
description = "Fab digital-fabrication and architecture application"
license = "MIT OR Apache-2.0"
# src/bin/frames_to_mp4.rs makes this a two-binary package; without this,
# `cargo run -p makepad-fab` (the headless test harness) refuses to
# guess and demands --bin.
default-run = "makepad-fab"
[features]
default = ["gltf"]
gltf = ["dep:makepad-fab-loader-gltf"]
[dependencies]
makepad-widgets = { path = "../../widgets", version = "2.0.0" }
makepad-fab-shell = { path = "../../libs/fab", version = "0.1.0" }
makepad-fab-loader-gltf = { path = "loaders/gltf", version = "0.1.0", optional = true }
makepad-video = { package = "makepad-platform-video", path = "../../platform/video", version = "1.0.0" }
makepad-zune-png = { path = "../../libs/zune/zune-png", version = "0.5.2" }
[dev-dependencies]
makepad-test = { path = "../../libs/makepad_test", version = "0.1.0" }
makepad-micro-serde = { path = "../../libs/micro_serde", version = "1.0.0" }
makepad-raytrace = { path = "../../libs/raytrace", version = "0.1.0" }
# The headless harness (`MAKEPAD=headless`) execs one `target/release/<bin>`
# chosen from this manifest: a lone `[[bin]]` is treated as the whole package,
# so the UI binary has to be named here or `frames_to_mp4` is launched instead.
[[bin]]
name = "makepad-fab"
path = "src/main.rs"
[[bin]]
name = "frames_to_mp4"
path = "src/bin/frames_to_mp4.rs"

4
apps/fab/README.md Normal file
View file

@ -0,0 +1,4 @@
# Fab
The Fab product app composes the reusable shell with its in-tree glTF loader.
Run `makepad-fab --open model.glb` after building the app.

View file

@ -0,0 +1,11 @@
[package]
name = "makepad-fab-loader-gltf"
version = "0.1.0"
edition = "2021"
description = "glTF and GLB loader for makepad-fab-shell"
license = "MIT OR Apache-2.0"
[dependencies]
makepad-fab-shell = { path = "../../../../libs/fab", version = "0.1.0" }
makepad-raytrace = { path = "../../../../libs/raytrace", version = "0.1.0" }
makepad-gltf = { path = "../../../../libs/gltf", version = "1.0.0" }

View file

@ -0,0 +1,45 @@
# Fab glTF loader
The in-tree reference loader builds an editable Fab `Document` from glTF or GLB.
Architectural metadata uses this minimal open vocabulary:
```json
{
"node.extras": {
"arch": {
"kind": "wall|slab|roof|window|door|stair|column|beam|site|...",
"level": 0,
"level_name": "Ground Floor",
"id": "stable source id",
"material_class": "glass|wood|metal|concrete|plaster|..."
},
"props": {}
},
"scene.extras": {
"arch": {
"units": "m",
"levels": [{ "index": 0, "name": "Ground Floor", "elevation": 0.0 }]
}
}
}
```
`arch.kind` is an open lower-case set. Scene metadata may also carry
`arch.north_deg` and `arch.site`; readers accept the older `elevation_m` level
field for compatibility, but writers use `elevation`.
Node extras use `{ "arch": { "kind", "priority", "level", "level_name", "id",
"material_class", "area_m2", "volume_m3" }, "props": { ... } }`; scene extras
carry `arch.units`, `north_deg`, `site`, and indexed levels with elevations. The
loader maps `arch.kind` to the element class and retains numeric `arch.priority`
as the render-order hint used only to resolve measured coplanar overlaps (higher
values win); geometry, picking, and measurement remain unchanged.
For architectural depth ties, node `arch.coplanar_priority` is a stable integer and the higher value wins only when faces are coplanar. Material extras may carry `arch.material_class` (including `glass`), alongside standard `KHR_materials_transmission` and `KHR_materials_ior`; `site` additionally accepts `city`, `timezone_min`, `summer_time`, `date_local` (`YYYY-MM-DDTHH:MM` when a full local calendar date exists), or the source-exact `day_of_year` and `minute_of_day` when no year was authored.
The optional `arch.site` object uses `lat`, `lon`, and `elevation_m`, plus `date` (`YYYY-MM-DD`),
`time` (`HH:MM` local civil time), `utc_offset_hours` (standard hours east of UTC),
and `dst` (adds one hour); the converter may also write `city`, `timezone_min`,
`summer_time`, `date_local`, `day_of_year` and `minute_of_day`, and the loader keeps
every present key as `arch.site.*` Fab document metadata while project north remains `arch.north_deg`.

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,413 @@
//! Encode a directory of `frame_%06d.png` files into an H.264 mp4 via the
//! platform hardware encoder (`makepad-platform-video` / VideoToolbox on macOS).
//!
//! ```text
//! frames_to_mp4 <dir> <out.mp4> [--fps 24] [--start N] [--end M] [--bitrate BPS] [--crf N]
//! ```
use makepad_video::{VideoFileCodec, VideoFileEncoder, VideoFileEncoderOptions};
use makepad_zune_png::makepad_zune_core::bytestream::ZCursor;
use makepad_zune_png::makepad_zune_core::options::DecoderOptions;
use makepad_zune_png::PngDecoder;
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use std::process::ExitCode;
const USAGE: &str = "usage: frames_to_mp4 <dir> <out.mp4> [--fps 24] [--start N] [--end M] [--bitrate BPS] [--crf N]";
const DEFAULT_FPS: u32 = 24;
const DEFAULT_BITRATE: u32 = 8_000_000;
struct Args {
dir: PathBuf,
out: PathBuf,
fps: u32,
start: Option<u64>,
end: Option<u64>,
bitrate: u32,
}
fn parse_u32(flag: &str, value: &str) -> Result<u32, String> {
value
.parse::<u32>()
.map_err(|_| format!("{flag} expects an integer, got {value:?}"))
}
fn parse_u64(flag: &str, value: &str) -> Result<u64, String> {
value
.parse::<u64>()
.map_err(|_| format!("{flag} expects an integer, got {value:?}"))
}
/// AVFoundation exposes average bitrate, not CRF. Map a ffmpeg-like CRF
/// (0 = lossless-ish, 51 = worst) onto a bitrate so `--crf` is accepted.
fn bitrate_from_crf(crf: u32) -> u32 {
let crf = crf.min(51);
let scale = (51 - crf) as u64;
(500_000 + scale * scale * 7_500).min(40_000_000) as u32
}
fn take_value<'a>(flag: &str, rest: &mut std::iter::Peekable<std::slice::Iter<'a, String>>) -> Result<&'a str, String> {
rest.next()
.map(String::as_str)
.ok_or_else(|| format!("{flag} requires a value"))
}
fn parse_args(argv: &[String]) -> Result<Args, String> {
let mut pos = Vec::new();
let mut fps = DEFAULT_FPS;
let mut start = None;
let mut end = None;
let mut bitrate = DEFAULT_BITRATE;
let mut iter = argv.iter().peekable();
while let Some(arg) = iter.next() {
match arg.as_str() {
"-h" | "--help" => return Err(USAGE.to_string()),
"--fps" => {
let v = take_value("--fps", &mut iter)?;
fps = parse_u32("--fps", v)?;
if fps == 0 {
return Err("--fps must be > 0".into());
}
}
"--start" => {
let v = take_value("--start", &mut iter)?;
start = Some(parse_u64("--start", v)?);
}
"--end" => {
let v = take_value("--end", &mut iter)?;
end = Some(parse_u64("--end", v)?);
}
"--bitrate" => {
let v = take_value("--bitrate", &mut iter)?;
bitrate = parse_u32("--bitrate", v)?;
if bitrate == 0 {
return Err("--bitrate must be > 0".into());
}
}
"--crf" => {
let v = take_value("--crf", &mut iter)?;
bitrate = bitrate_from_crf(parse_u32("--crf", v)?);
}
flag if flag.starts_with('-') => return Err(format!("unknown flag {flag}")),
other => pos.push(other.to_string()),
}
}
if pos.len() != 2 {
return Err(USAGE.to_string());
}
if let (Some(a), Some(b)) = (start, end) {
if a > b {
return Err(format!("--start {a} is after --end {b}"));
}
}
Ok(Args {
dir: PathBuf::from(&pos[0]),
out: PathBuf::from(&pos[1]),
fps,
start,
end,
bitrate,
})
}
fn encoder_options(width: u32, height: u32, args: &Args) -> VideoFileEncoderOptions {
VideoFileEncoderOptions {
codec: VideoFileCodec::H264,
width,
height,
fps_num: args.fps,
fps_den: 1,
video_bitrate_bps: args.bitrate,
audio: None,
keyframe_only: false,
}
}
fn parse_frame_name(name: &str) -> Option<u64> {
let rest = name.strip_prefix("frame_")?.strip_suffix(".png")?;
if rest.is_empty() || !rest.bytes().all(|b| b.is_ascii_digit()) {
return None;
}
rest.parse().ok()
}
fn list_frame_files(dir: &Path) -> Result<BTreeMap<u64, PathBuf>, String> {
let entries = std::fs::read_dir(dir).map_err(|e| format!("read {}: {e}", dir.display()))?;
let mut map = BTreeMap::new();
for entry in entries {
let entry = entry.map_err(|e| format!("read {}: {e}", dir.display()))?;
let name = entry.file_name();
let Some(idx) = parse_frame_name(&name.to_string_lossy()) else {
continue;
};
map.insert(idx, entry.path());
}
Ok(map)
}
fn select_frames(
map: &BTreeMap<u64, PathBuf>,
start: Option<u64>,
end: Option<u64>,
) -> (Vec<(u64, PathBuf)>, Vec<u64>) {
if map.is_empty() {
return (Vec::new(), Vec::new());
}
let lo = start.unwrap_or_else(|| *map.keys().next().unwrap());
let hi = end.unwrap_or_else(|| *map.keys().next_back().unwrap());
let mut present = Vec::new();
let mut missing = Vec::new();
for i in lo..=hi {
match map.get(&i) {
Some(path) => present.push((i, path.clone())),
None => missing.push(i),
}
}
(present, missing)
}
fn decode_png_rgb(path: &Path) -> Result<(Vec<u8>, u32, u32), String> {
let bytes = std::fs::read(path).map_err(|e| format!("read {}: {e}", path.display()))?;
let options = DecoderOptions::default().png_set_strip_to_8bit(true);
let mut decoder = PngDecoder::new_with_options(ZCursor::new(&bytes), options);
let pixels = decoder
.decode_raw()
.map_err(|e| format!("decode {}: {e:?}", path.display()))?;
let (width, height) = decoder
.dimensions()
.ok_or_else(|| format!("{}: no dimensions", path.display()))?;
let components = decoder
.colorspace()
.ok_or_else(|| format!("{}: no colorspace", path.display()))?
.num_components();
if components == 0 {
return Err(format!("{}: empty colorspace", path.display()));
}
let n = width * height;
let need = n.saturating_mul(components);
if pixels.len() < need {
return Err(format!(
"{}: expected at least {need} bytes, got {}",
path.display(),
pixels.len()
));
}
let mut rgb = vec![0u8; n * 3];
if components == 3 {
rgb.copy_from_slice(&pixels[..n * 3]);
} else {
for i in 0..n {
let src = i * components;
if components == 1 {
let g = pixels[src];
rgb[i * 3] = g;
rgb[i * 3 + 1] = g;
rgb[i * 3 + 2] = g;
} else {
rgb[i * 3] = pixels[src];
rgb[i * 3 + 1] = pixels[src + 1];
rgb[i * 3 + 2] = pixels[src + 2.min(components - 1)];
}
}
}
Ok((rgb, width as u32, height as u32))
}
fn crop_rgb_even(rgb: &[u8], width: u32, height: u32, even_w: u32, even_h: u32) -> Vec<u8> {
if width == even_w && height == even_h {
return rgb.to_vec();
}
let mut out = vec![0u8; even_w as usize * even_h as usize * 3];
let src_stride = width as usize * 3;
let dst_stride = even_w as usize * 3;
for y in 0..even_h as usize {
let src = y * src_stride;
let dst = y * dst_stride;
out[dst..dst + dst_stride].copy_from_slice(&rgb[src..src + dst_stride]);
}
out
}
fn encode_dir(args: &Args) -> Result<(u64, f64, u32, u32), String> {
let map = list_frame_files(&args.dir)?;
if map.is_empty() {
return Err(format!("no frame_*.png files in {}", args.dir.display()));
}
let (frames, missing) = select_frames(&map, args.start, args.end);
for idx in &missing {
eprintln!("warning: missing frame_{idx:06}.png");
}
if frames.is_empty() {
return Err("no frames to encode in the requested range".into());
}
let mut encoder: Option<VideoFileEncoder> = None;
let mut width = 0u32;
let mut height = 0u32;
let mut encoded = 0u64;
for (_idx, path) in &frames {
let (rgb, w, h) = match decode_png_rgb(path) {
Ok(v) => v,
Err(e) => {
eprintln!("warning: skip {}: {e}", path.display());
continue;
}
};
let even_w = w & !1;
let even_h = h & !1;
if even_w == 0 || even_h == 0 {
eprintln!("warning: skip {}: size {w}x{h} is too small", path.display());
continue;
}
let rgb = if w != even_w || h != even_h {
crop_rgb_even(&rgb, w, h, even_w, even_h)
} else {
rgb
};
if encoder.is_none() {
width = even_w;
height = even_h;
let options = encoder_options(width, height, args);
let path_text = args.out.to_string_lossy();
encoder = Some(
VideoFileEncoder::new(&path_text, options)
.map_err(|e| format!("encoder open {}: {e}", args.out.display()))?,
);
} else if even_w != width || even_h != height {
eprintln!(
"warning: skip {}: size {w}x{h} != {width}x{height}",
path.display()
);
continue;
}
encoder
.as_mut()
.unwrap()
.push_frame_rgb8(&rgb, None)
.map_err(|e| format!("push {}: {e}", path.display()))?;
encoded += 1;
}
let Some(enc) = encoder else {
return Err("no frames encoded".into());
};
enc.finish().map_err(|e| format!("encoder finish: {e}"))?;
let duration = encoded as f64 / args.fps as f64;
Ok((encoded, duration, width, height))
}
fn main() -> ExitCode {
let argv: Vec<String> = std::env::args().skip(1).collect();
let args = match parse_args(&argv) {
Ok(a) => a,
Err(e) => {
eprintln!("{e}");
return ExitCode::from(2);
}
};
match encode_dir(&args) {
Ok((encoded, duration, width, height)) => {
println!(
"encoded {encoded} frames, duration {duration:.3}s ({width}x{height} @ {} fps) -> {}",
args.fps,
args.out.display()
);
ExitCode::SUCCESS
}
Err(e) => {
eprintln!("{e}");
ExitCode::FAILURE
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn argv(s: &str) -> Vec<String> {
s.split_whitespace().map(String::from).collect()
}
#[test]
fn parse_defaults() {
let a = parse_args(&argv("frames out.mp4")).unwrap();
assert_eq!(a.dir, PathBuf::from("frames"));
assert_eq!(a.out, PathBuf::from("out.mp4"));
assert_eq!(a.fps, 24);
assert_eq!(a.start, None);
assert_eq!(a.end, None);
assert_eq!(a.bitrate, DEFAULT_BITRATE);
}
#[test]
fn parse_flags_anywhere() {
let a = parse_args(&argv("--fps 30 d --start 10 o.mp4 --end 20 --bitrate 4000000")).unwrap();
assert_eq!(a.dir, PathBuf::from("d"));
assert_eq!(a.out, PathBuf::from("o.mp4"));
assert_eq!(a.fps, 30);
assert_eq!(a.start, Some(10));
assert_eq!(a.end, Some(20));
assert_eq!(a.bitrate, 4_000_000);
}
#[test]
fn parse_crf_maps_to_bitrate() {
let a = parse_args(&argv("d o.mp4 --crf 23")).unwrap();
assert_eq!(a.bitrate, bitrate_from_crf(23));
assert!(a.bitrate > 0);
}
#[test]
fn parse_rejects_bad_usage() {
assert!(parse_args(&[]).is_err());
assert!(parse_args(&argv("--fps 24")).is_err());
assert!(parse_args(&argv("only_one")).is_err());
assert!(parse_args(&argv("d o.mp4 --unknown 1")).is_err());
assert!(parse_args(&argv("d o.mp4 --fps 0")).is_err());
assert!(parse_args(&argv("d o.mp4 --start 5 --end 1")).is_err());
}
#[test]
fn constructs_h264_encoder_options() {
let a = parse_args(&argv("d o.mp4 --fps 24 --bitrate 16000000")).unwrap();
let o = encoder_options(1920, 1080, &a);
assert_eq!(o.codec, VideoFileCodec::H264);
assert_eq!(o.width, 1920);
assert_eq!(o.height, 1080);
assert_eq!(o.fps_num, 24);
assert_eq!(o.fps_den, 1);
assert_eq!(o.video_bitrate_bps, 16_000_000);
assert!(o.audio.is_none());
assert!(!o.keyframe_only);
}
#[test]
fn frame_names_sort_numerically() {
assert_eq!(parse_frame_name("frame_000001.png"), Some(1));
assert_eq!(parse_frame_name("frame_12.png"), Some(12));
assert_eq!(parse_frame_name("frame_000000.png"), Some(0));
assert_eq!(parse_frame_name("other.png"), None);
assert_eq!(parse_frame_name("frame_00a.png"), None);
}
#[test]
fn select_skips_missing_indices() {
let mut map = BTreeMap::new();
map.insert(0, PathBuf::from("frame_000000.png"));
map.insert(2, PathBuf::from("frame_000002.png"));
let (present, missing) = select_frames(&map, Some(0), Some(2));
assert_eq!(present.len(), 2);
assert_eq!(present[0].0, 0);
assert_eq!(present[1].0, 2);
assert_eq!(missing, vec![1]);
}
#[test]
fn bitrate_from_crf_is_sane() {
assert!(bitrate_from_crf(0) > bitrate_from_crf(23));
assert!(bitrate_from_crf(23) > bitrate_from_crf(51));
assert!(bitrate_from_crf(23) > 0);
}
}

477
apps/fab/src/main.rs Normal file
View file

@ -0,0 +1,477 @@
//! Fab product frontend over the reusable shell and feature-selected loaders.
use makepad_widgets::*;
use fab::{
api::*, loader::{self, LoadCoordinator}, nav, render, sheets, tools, tour, ui, viewport,
};
app_main!(App);
script_mod! {
use mod.prelude.fab.*
use mod.widgets.*
startup() do #(App::script_component(vm)){
ui: Root{
main_window := Window{
window.title: "Fab"
window.inner_size: vec2(1600, 1000)
pass +: {
clear_color: fab.color_area
}
body +: {
flow: Down
spacing: 0
margin: 0
padding: 0
shell := FabShell{}
}
}
}
}
}
#[derive(Script, ScriptHook)]
pub struct App {
#[live]
ui: WidgetRef,
#[rust]
state: AppState,
#[rust]
loader: LoadCoordinator,
#[rust]
loaders_registered: bool,
#[rust]
started: bool,
/// True while any `TextInput` owns key focus (outliner search, palette,
/// drag-number entry). Tracked from `TextInputAction::KeyFocus[Lost]` in
/// `handle_actions`; the bare viewport hotkeys (N/T/Z/Home) bail on it so
/// typing "window" in a search field does not toggle panels (review M3).
#[rust]
text_input_focused: bool,
/// Global movement keys remember the viewport that received their down
/// event so the matching up is delivered even after focus/modal changes.
#[rust]
nav_keys_down: Vec<(KeyCode, usize)>,
}
impl App {
/// Route one action through every lane hook, then the core state.
fn dispatch(&mut self, cx: &mut Cx, action: &ShellAction) -> bool {
let opens_modal = matches!(
action,
ShellAction::ShowFileBrowser(true) | ShellAction::ShowKeymapHelp(true)
) || matches!(action, ShellAction::ToggleCommandPalette)
&& !self.state.ui.command_palette_open;
if opens_modal || matches!(action, ShellAction::SetWorkspace(_)) {
self.release_nav_controls(cx);
}
let mut changed = false;
changed |= loader::apply(cx, &mut self.loader, &mut self.state, action);
changed |= self.state.apply_core(action);
changed |= viewport::apply(cx, &mut self.state, action);
changed |= nav::apply(cx, &mut self.state, action);
changed |= tools::apply(cx, &mut self.state, action);
changed |= sheets::apply(cx, &mut self.state, action);
changed |= render::apply(cx, &mut self.state, action);
changed |= tour::apply(cx, &mut self.state, action);
changed |= ui::apply(cx, &mut self.state, action);
match action {
ShellAction::Quit => cx.quit(),
ShellAction::Command(name) => {
self.run_command(cx, name);
}
_ => {}
}
changed
}
/// The F3 palette / menus speak in command names.
fn run_command(&mut self, cx: &mut Cx, name: &str) {
let v = self.state.active_view;
let n = name.trim().to_lowercase();
let action = match n.as_str() {
"frame all" | "view.frame_all" => Some(ShellAction::FrameAll(v)),
"frame selected" | "view.frame_selected" => Some(ShellAction::FrameSelected(v)),
"front" | "front view" => Some(ShellAction::PresetView(v, PresetView::Front)),
"top" | "top view" => Some(ShellAction::PresetView(v, PresetView::Top)),
"right" | "right view" => Some(ShellAction::PresetView(v, PresetView::Right)),
"iso" | "isometric" => Some(ShellAction::PresetView(v, PresetView::Isometric)),
"ortho" | "perspective" | "toggle ortho" => Some(ShellAction::ToggleOrtho(v)),
"hide selected" => Some(ShellAction::HideSelected),
"unhide all" => Some(ShellAction::UnhideAll),
"isolate" | "isolate selected" => Some(ShellAction::IsolateSelected),
"render image" | "render" => {
// F12 and the palette: the render happens in the Render
// workspace's `FabRenderView`, so go there first.
cx.action(ShellAction::SetWorkspace(Workspace::Render));
Some(ShellAction::RenderStart)
}
"save render" | "save png" | "export png" => {
self.save_render(cx);
None
}
"open" => Some(ShellAction::ShowFileBrowser(true)),
"open demo" | "demo" => Some(ShellAction::OpenDemo),
"wireframe" => Some(ShellAction::SetShading(v, Shading::Wireframe)),
"solid" => Some(ShellAction::SetShading(v, Shading::Solid)),
"material" => Some(ShellAction::SetShading(v, Shading::Material)),
"realtime" => Some(ShellAction::SetShading(v, Shading::Realtime)),
"rendered" => Some(ShellAction::SetShading(v, Shading::Rendered)),
"hidden line" | "ink" => Some(ShellAction::SetShading(v, Shading::HiddenLine)),
"quad view" => Some(ShellAction::ToggleQuadView),
"keymap" | "help" => Some(ShellAction::ShowKeymapHelp(true)),
"quit" => Some(ShellAction::Quit),
_ => None,
};
match action {
Some(a) => {
cx.action(a);
}
None if matches!(n.as_str(), "save render" | "save png" | "export png") => {}
None => {
self.state.ui.status_message = format!("Unknown command: {name}");
}
}
}
/// Where saved renders go: `local/fab/renders/<model>-<unix seconds>.png`
/// under the repo root (the same root the samples resolve against).
fn render_output_path(&self) -> std::path::PathBuf {
let repo_root = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
.ancestors()
.nth(2)
.map(|p| p.to_path_buf())
.unwrap_or_default();
let dir = repo_root.join("local/fab/renders");
let _ = std::fs::create_dir_all(&dir);
let stem: String = self
.state
.scene
.name
.chars()
.map(|c| if c.is_ascii_alphanumeric() { c.to_ascii_lowercase() } else { '-' })
.collect();
let stem = stem.trim_matches('-');
let stem = if stem.is_empty() { "render" } else { stem };
let secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
dir.join(format!("{stem}-{secs}.png"))
}
/// Cmd+S / "save render": the `FabRenderView` (Render workspace) owns
/// the tracer and answers `ExportPng` with a capture of the current
/// accumulation. When that view is not on screen yet, the first Cmd+S
/// starts the render there and says so; the next one saves.
fn save_render(&mut self, cx: &mut Cx) {
// A viewport in Rendered shading (the right pane by default) answers
// `ExportPng` with a capture of what it shows right now; the F12
// `FabRenderView` does the same when it is on screen.
let rendered_pane = self.state.views.iter().any(|v| v.shading == Shading::Rendered);
let render_view = self.state.ui.workspace == Workspace::Render && self.state.render.running;
if !rendered_pane && !render_view {
cx.action(ShellAction::SetWorkspace(Workspace::Render));
cx.action(ShellAction::RenderStart);
self.state.ui.status_message =
"Rendering in the Render workspace — Cmd+S again to save the PNG".into();
return;
}
let path = self.render_output_path();
self.state.ui.status_message = format!("Saving {}", path.display());
cx.action(ShellAction::ExportPng(path));
}
/// Global keys, before the widget tree sees the event. Anything with a
/// text field focused is left alone except the function keys.
fn global_key(&mut self, cx: &mut Cx, ke: &KeyEvent) -> bool {
let m = ke.modifiers;
let v = self.state.active_view;
let walking = matches!(self.state.view().nav_mode, NavMode::Walk | NavMode::Fly);
// Escape is the pointer-safety path before every modal rule. Release
// the OS lock and movement keys even if an overlay currently owns
// focus, then let the navigator leave first-person mode.
if ke.key_code == KeyCode::Escape {
self.release_nav_controls(cx);
if walking {
cx.action(ShellAction::NavKey {
view: v,
key: KeyCode::Escape,
down: true,
mods: m,
repeat: ke.is_repeat,
});
}
if self.state.ui.file_browser_open {
cx.action(ShellAction::ShowFileBrowser(false));
}
if self.state.ui.keymap_help_open {
cx.action(ShellAction::ShowKeymapHelp(false));
}
if self.state.ui.command_palette_open {
cx.action(ShellAction::ToggleCommandPalette);
}
return walking
|| self.state.ui.file_browser_open
|| self.state.ui.keymap_help_open
|| self.state.ui.command_palette_open;
}
if !ke.is_repeat {
let action = match ke.key_code {
KeyCode::F1 => Some(ShellAction::ShowKeymapHelp(!self.state.ui.keymap_help_open)),
KeyCode::F3 if m.shift => Some(ShellAction::TogglePerf),
KeyCode::F3 => Some(ShellAction::ToggleCommandPalette),
KeyCode::F12 => {
cx.action(ShellAction::SetWorkspace(Workspace::Render));
Some(ShellAction::RenderStart)
}
KeyCode::KeyS if m.is_primary() => {
self.save_render(cx);
return true;
}
KeyCode::KeyO if m.is_primary() && m.shift => Some(ShellAction::OpenDemo),
KeyCode::KeyO if m.is_primary() => Some(ShellAction::ShowFileBrowser(true)),
KeyCode::KeyQ if m.is_primary() => Some(ShellAction::Quit),
KeyCode::Space if m.control => Some(ShellAction::ToggleMaximizeArea),
_ => None,
};
if let Some(a) = action {
cx.action(a);
return true;
}
}
// Viewport-context keys that must work without viewport focus — but
// never while a text field is focused: these are bare letters and
// would swallow typing (review M3).
let modal_open = self.state.ui.file_browser_open
|| self.state.ui.keymap_help_open
|| self.state.ui.command_palette_open;
if modal_open || self.text_input_focused || cx.keyboard.modifiers().logo {
return false;
}
// W is both the entry command and forward movement. Post the mode
// action first so the viewport synchronises its Navigator before the
// same physical key-down reaches it.
if ke.key_code == KeyCode::KeyW
&& !m.control
&& !m.alt
&& !m.shift
&& !m.logo
&& !walking
{
cx.action(ShellAction::SetTool(Tool::Walk));
self.route_nav_key_down(cx, v, ke);
return true;
}
if walking && is_walk_control_key(ke.key_code) && !m.logo {
self.route_nav_key_down(cx, v, ke);
return true;
}
if ke.is_repeat {
return false;
}
let action = match ke.key_code {
KeyCode::KeyN => Some(ShellAction::ToggleSidebar),
KeyCode::KeyT => Some(ShellAction::ToggleToolbar),
KeyCode::KeyP
if !m.control
&& !m.alt
&& !m.shift
&& !m.logo
&& self.state.view_at(v).shading == Shading::Rendered =>
{
Some(ShellAction::SetRenderedPaused(
v,
!self.state.view_at(v).rendered_paused,
))
}
KeyCode::KeyZ if m.alt => Some(ShellAction::ToggleXray(v)),
KeyCode::Home => Some(ShellAction::FrameAll(v)),
_ => None,
};
if let Some(a) = action {
cx.action(a);
return true;
}
false
}
fn route_nav_key_down(&mut self, cx: &mut Cx, view: usize, ke: &KeyEvent) {
let routed_view = if let Some((_, routed_view)) = self
.nav_keys_down
.iter()
.find(|(key, _)| *key == ke.key_code)
{
*routed_view
} else {
self.nav_keys_down.push((ke.key_code, view));
view
};
cx.action(ShellAction::NavKey {
view: routed_view,
key: ke.key_code,
down: true,
mods: ke.modifiers,
repeat: ke.is_repeat,
});
}
fn global_key_up(&mut self, cx: &mut Cx, ke: &KeyEvent) -> bool {
let Some(i) = self
.nav_keys_down
.iter()
.position(|(key, _)| *key == ke.key_code)
else {
return false;
};
let (_, view) = self.nav_keys_down.swap_remove(i);
cx.action(ShellAction::NavKey {
view,
key: ke.key_code,
down: false,
mods: ke.modifiers,
repeat: false,
});
true
}
/// Clear every globally routed key and release every viewport capture.
/// This is safe to call repeatedly and is used by modal/focus transitions.
fn release_nav_controls(&mut self, cx: &mut Cx) {
for (key, view) in self.nav_keys_down.drain(..) {
cx.action(ShellAction::NavKey {
view,
key,
down: false,
mods: KeyModifiers::default(),
repeat: false,
});
}
nav::walk::request_capture_release();
cx.action(ShellAction::NavReleaseCapture);
}
fn open_from_args(&mut self, cx: &mut Cx) {
let args: Vec<String> = std::env::args().skip(1).collect();
let mut path: Option<String> = None;
let mut i = 0;
while i < args.len() {
let a = &args[i];
if a == "--open" {
path = args.get(i + 1).cloned();
i += 1;
} else if let Some(p) = a.strip_prefix("--open=") {
path = Some(p.to_string());
} else if !a.starts_with("--") {
path = Some(a.clone());
}
i += 1;
}
match path {
Some(p) => self.loader.open(cx, std::path::PathBuf::from(p)),
None => self.loader.open_demo(cx),
}
}
fn register_loaders(&mut self) {
if self.loaders_registered {
return;
}
#[cfg(feature = "gltf")]
self.loader.register(makepad_fab_loader_gltf::GltfLoader);
self.loaders_registered = true;
}
}
fn is_walk_control_key(key: KeyCode) -> bool {
matches!(
key,
KeyCode::KeyW
| KeyCode::KeyA
| KeyCode::KeyS
| KeyCode::KeyD
| KeyCode::KeyQ
| KeyCode::KeyE
| KeyCode::KeyF
| KeyCode::ArrowUp
| KeyCode::ArrowDown
| KeyCode::ArrowLeft
| KeyCode::ArrowRight
| KeyCode::Shift
| KeyCode::Control
| KeyCode::Space
)
}
impl MatchEvent for App {
fn handle_actions(&mut self, cx: &mut Cx, actions: &Actions) {
let mut changed = false;
let (mut focus_gained, mut focus_lost) = (false, false);
for action in actions {
if let Some(a) = action.downcast_ref::<ShellAction>() {
changed |= self.dispatch(cx, a);
}
if let Some(wa) = action.downcast_ref::<WidgetAction>() {
match wa.action.downcast_ref::<TextInputAction>() {
Some(TextInputAction::KeyFocus) => focus_gained = true,
Some(TextInputAction::KeyFocusLost) => focus_lost = true,
_ => (),
}
}
}
// A focus handoff between two inputs delivers Lost + Focus in one
// batch; the gain wins so the flag stays true across the handoff.
if focus_gained {
self.release_nav_controls(cx);
self.text_input_focused = true;
} else if focus_lost {
self.text_input_focused = false;
}
if changed {
self.ui.redraw(cx);
}
}
}
impl AppMain for App {
fn script_mod(vm: &mut ScriptVm) -> ScriptValue {
makepad_widgets::script_mod(vm);
fab::script_mod(vm);
self::script_mod(vm)
}
fn handle_event(&mut self, cx: &mut Cx, event: &Event) {
if !self.started {
if let Event::Startup | Event::Draw(_) = event {
self.started = true;
self.register_loaders();
cx.perf_monitor.set_enabled(self.state.ui.show_perf);
self.open_from_args(cx);
}
}
if let Event::KeyDown(ke) = event {
if self.global_key(cx, ke) {
return;
}
}
if let Event::KeyUp(ke) = event {
if self.global_key_up(cx, ke) {
return;
}
}
if matches!(
event,
Event::WindowLostFocus(_) | Event::Pause | Event::Background
) {
self.release_nav_controls(cx);
}
self.match_event(cx, event);
self.ui
.handle_event(cx, event, &mut Scope::with_data(&mut self.state));
}
}

228
apps/fab/tests/chrome.rs Normal file
View file

@ -0,0 +1,228 @@
//! Ink-centre of boxed chrome labels (workspace tabs, header menus).
//!
//! Same contract as `examples/text_center/tests/ui.rs`: the cap-height band of
//! a one-line label sits within 0.5 logical px of the widget box centre.
//! Specimens are live product controls (`Layout` workspace tab, `View` header
//! menu), measured from a headless grab — dpi 1, no GPU.
use makepad_test::{makepad_test, Selector, TestApp, WidgetSnapshot};
use std::path::Path;
const TOLERANCE_LPX: f64 = 0.5;
struct Frame {
width: usize,
height: usize,
lum: Vec<f64>,
}
impl Frame {
fn at(&self, x: usize, y: usize) -> f64 {
self.lum[y * self.width + x]
}
fn decode(path: &Path) -> Frame {
use makepad_zune_png::makepad_zune_core::bytestream::ZCursor;
use makepad_zune_png::PngDecoder;
let bytes = std::fs::read(path).unwrap_or_else(|e| panic!("read {path:?}: {e}"));
let mut dec = PngDecoder::new(ZCursor::new(&bytes));
dec.decode_headers().expect("png headers");
let (width, height) = dec.dimensions().expect("png dimensions");
let components = dec.colorspace().expect("png colorspace").num_components();
let decoded = dec.decode().expect("png decode");
let src = decoded.u8().expect("png 8-bit");
let lum = (0..width * height)
.map(|i| src[i * components] as f64)
.collect();
Frame {
width,
height,
lum,
}
}
}
fn band_edges(
frame: &Frame,
x: usize,
y0: usize,
y1: usize,
inside: f64,
outside: f64,
) -> (f64, f64) {
let span = inside - outside;
assert!(span.abs() > 8.0, "no contrast between band and surround");
let cov = |y: usize| ((frame.at(x, y) - outside) / span).clamp(0.0, 1.0);
let mid = (y0..=y1)
.max_by(|a, b| cov(*a).partial_cmp(&cov(*b)).unwrap())
.expect("non-empty window");
assert!(cov(mid) > 0.5, "no band on column {x} between {y0} and {y1}");
let mut top = mid as f64;
for y in y0..mid {
top -= cov(y);
}
let mut bottom = mid as f64;
for y in mid..=y1 {
bottom += cov(y);
}
(top, bottom)
}
fn fill_value(frame: &Frame, r: &DevRect) -> f64 {
let mut hist = [0usize; 256];
let y0 = (r.y0 + 2).min(r.y1);
let y1 = r.y1.saturating_sub(2).max(y0);
let x0 = (r.x0 + 2).min(r.x1);
let x1 = r.x1.saturating_sub(2).max(x0);
for y in y0..y1 {
for x in x0..x1 {
hist[frame.at(x, y) as usize] += 1;
}
}
hist.iter()
.enumerate()
.max_by_key(|(_, n)| **n)
.map(|(v, _)| v as f64)
.unwrap_or(0.0)
}
#[derive(Clone, Copy, Debug)]
struct DevRect {
x0: usize,
y0: usize,
x1: usize,
y1: usize,
}
struct Measurement {
box_center: f64,
ink_center: f64,
ink_height: f64,
box_source: &'static str,
}
impl Measurement {
fn delta_lpx(&self, dpi: f64) -> f64 {
(self.ink_center - self.box_center) / dpi
}
}
fn measure(frame: &Frame, snap: &WidgetSnapshot, dpi: f64) -> Measurement {
let rect = DevRect {
x0: (snap.x as f64 * dpi).round() as usize,
y0: (snap.y as f64 * dpi).round() as usize,
x1: ((snap.x + snap.width) as f64 * dpi).round() as usize,
y1: ((snap.y + snap.height) as f64 * dpi).round() as usize,
};
assert!(
rect.x1 <= frame.width && rect.y1 <= frame.height && rect.x1 > rect.x0 + 8,
"widget {} rect {rect:?} is not on the frame",
snap.id
);
let fill = fill_value(frame, &rect);
let rim = (2.0 * dpi).ceil() as usize;
let corner = (2.0 * dpi).ceil() as usize;
let columns = (rect.x0 + corner)..(rect.x1.saturating_sub(corner).max(rect.x0 + corner + 1));
let span = columns.len().max(1);
let lit = |y: usize| {
columns
.clone()
.filter(|x| (frame.at(*x, y) - fill).abs() > 40.0)
.count()
};
let text_rows: Vec<usize> = ((rect.y0 + rim)..(rect.y1.saturating_sub(rim).max(rect.y0 + rim + 1)))
.filter(|y| {
let n = lit(*y);
n > 0 && n * 2 < span
})
.collect();
assert!(
!text_rows.is_empty(),
"widget {} has no legible ink over fill {fill}",
snap.id
);
let window = (text_rows[0].saturating_sub(2))..=(text_rows[text_rows.len() - 1] + 2).min(frame.height - 1);
let mass = |x: usize| {
window
.clone()
.map(|y| (frame.at(x, y) - fill).abs())
.sum::<f64>()
};
let stem = columns
.clone()
.max_by(|a, b| mass(*a).partial_cmp(&mass(*b)).unwrap())
.expect("non-empty rect");
let peak = window.clone().map(|y| frame.at(stem, y)).fold(fill, |acc, v| {
if (v - fill).abs() > (acc - fill).abs() {
v
} else {
acc
}
});
assert!(
(peak - fill).abs() > 40.0,
"widget {} has no legible ink (peak {peak}, fill {fill})",
snap.id
);
let (ink_top, ink_bottom) =
band_edges(frame, stem, *window.start(), *window.end(), peak, fill);
let box_center = (snap.y as f64 + snap.height as f64 / 2.0) * dpi;
Measurement {
box_center,
ink_center: (ink_top + ink_bottom) / 2.0,
ink_height: ink_bottom - ink_top,
box_source: "widget rect",
}
}
fn snapshot_of<'a>(widgets: &'a [WidgetSnapshot], id: &str) -> &'a WidgetSnapshot {
widgets
.iter()
.find(|w| w.id == id && w.visible)
.unwrap_or_else(|| panic!("no visible widget with id `{id}`"))
}
#[makepad_test]
fn chrome_labels_are_ink_centered(app: TestApp) {
app.locator(Selector::id("ws_quad")).wait_visible();
let widgets = app.widget_snapshot();
let shot = app.screenshot();
let frame = Frame::decode(&shot);
let widest = widgets.iter().map(|w| w.width).max().unwrap_or(1);
let dpi = (frame.width as f64 / widest as f64).round().max(1.0);
println!(
"[fab-chrome] frame {:?} {}x{} dpi {dpi}",
shot, frame.width, frame.height
);
let mut worst: Option<(String, f64)> = None;
let mut check = |name: &str, snap: &WidgetSnapshot| {
let m = measure(&frame, snap, dpi);
let delta = m.delta_lpx(dpi);
println!(
"[fab-chrome] {name:16} box_c {:8.3} ({}) ink_c {:8.3} ink_h {:6.3} -> {delta:+.3} lpx",
m.box_center, m.box_source, m.ink_center, m.ink_height
);
let worse = match &worst {
None => true,
Some((_, w)) => delta.abs() > w.abs(),
};
if worse {
worst = Some((name.to_string(), delta));
}
};
check("ws_quad", snapshot_of(&widgets, "ws_quad"));
check("ws_render", snapshot_of(&widgets, "ws_render"));
if let Some(snap) = widgets.iter().find(|w| w.id == "menu_view" && w.visible) {
check("menu_view", snap);
}
let (name, delta) = worst.expect("measured nothing");
assert!(
delta.abs() <= TOLERANCE_LPX,
"[fab-chrome] `{name}`: label ink sits {delta:+.3} logical px off the box center \
(tolerance {TOLERANCE_LPX}); see {shot:?}"
);
}

View file

@ -0,0 +1,361 @@
//! Working-tree measurement rig for the Rendered-pane roof noise: loads a
//! GLB through the real loader + scene build (overlap analysis included),
//! rebuilds the tracer's `SceneInput` exactly as the seam does, and then
//! histograms per-sample primary hits and sun-shadow visibility on roof
//! pixels with the bit-exact CPU twin.
//!
//! Run explicitly (skipped without the env):
//! ```sh
//! FAB_PROBE_GLB=/abs/path/woodside.glb cargo test -p makepad-fab --release --test roof_probe -- --nocapture
//! ```
use fab::model::{LoadCancel, Loader, Scene, SceneSnapshot};
use fab::model::makepad_math::{vec3f, Vec3f};
use makepad_raytrace::{cpu_ref, pack::PackedScene, Camera, SceneInput, Sun};
fn v3(a: [f32; 3]) -> Vec3f {
vec3f(a[0], a[1], a[2])
}
fn scene_input_all_visible(snap: &SceneSnapshot) -> SceneInput {
let mut s = SceneInput { up: vec3f(0.0, 0.0, 1.0), ..Default::default() };
s.positions = snap.positions.clone();
s.normals = snap.normals.clone();
s.uvs = snap.uvs.clone();
let n_tris = snap.indices.len() / 3;
for t in 0..n_tris {
s.indices.extend_from_slice(&snap.indices[t * 3..t * 3 + 3]);
s.tri_material.push(snap.triangle_material.get(t).copied().unwrap_or(0));
s.tri_priority.push(snap.triangle_priority.get(t).copied().unwrap_or(0));
s.tri_coplanar_group.push(snap.triangle_coplanar_group.get(t).copied().unwrap_or(0));
}
s.materials = snap
.materials
.iter()
.map(|m| makepad_raytrace::Material {
albedo: [m.albedo[0], m.albedo[1], m.albedo[2]],
roughness: m.roughness,
metal: m.metallic,
emission: m.emission,
ior: if m.ior > 1.0 { m.ior } else { 1.5 },
transmission: m.transmission,
texture: if m.texture == u32::MAX { None } else { Some(m.texture as usize) },
two_sided: m.double_sided,
})
.collect();
if s.materials.is_empty() {
s.materials.push(makepad_raytrace::Material::default());
}
s.images = snap
.textures
.iter()
.map(|t| {
let data = t
.rgba
.chunks_exact(4)
.map(|p| {
((p[3] as u32) << 24) | ((p[0] as u32) << 16) | ((p[1] as u32) << 8) | p[2] as u32
})
.collect();
makepad_raytrace::Image { width: t.width as usize, height: t.height as usize, data }
})
.collect();
s
}
#[test]
fn roof_hit_distribution() {
let Ok(path) = std::env::var("FAB_PROBE_GLB") else {
eprintln!("roof_probe: FAB_PROBE_GLB not set; skipping");
return;
};
let loader = makepad_fab_loader_gltf::GltfLoader;
let cancel_fn = || false;
let cancel: LoadCancel = &cancel_fn;
let document = loader
.load_cancellable(std::path::Path::new(&path), &mut |_| {}, cancel)
.expect("load glb");
let scene = Scene::from_document_with(document, &mut |_, _| {});
let snap = SceneSnapshot::from_scene(&scene);
eprintln!(
"probe: {} tris, {} materials, priorities non-zero {} groups non-zero {}",
snap.indices.len() / 3,
snap.materials.len(),
snap.triangle_priority.iter().filter(|p| **p != 0).count(),
snap.triangle_coplanar_group.iter().filter(|g| **g != 0).count(),
);
// FAB_PROBE_MAT=1: per-material triangle counts, texture presence and
// uv spread — which surfaces the tracer shades flat that the raster
// textures (the lawn parity question).
if std::env::var("FAB_PROBE_MAT").is_ok() {
let n_tris = snap.indices.len() / 3;
let nm = snap.materials.len();
let mut tri_count = vec![0usize; nm];
let mut uv_min = vec![[f32::MAX; 2]; nm];
let mut uv_max = vec![[f32::MIN; 2]; nm];
for t in 0..n_tris {
let m = snap.triangle_material.get(t).copied().unwrap_or(0) as usize;
if m >= nm {
continue;
}
tri_count[m] += 1;
for k in 0..3 {
let vi = snap.indices[t * 3 + k] as usize;
if let Some(uv) = snap.uvs.get(vi) {
for a in 0..2 {
uv_min[m][a] = uv_min[m][a].min(uv[a]);
uv_max[m][a] = uv_max[m][a].max(uv[a]);
}
}
}
}
for (m, mat) in snap.materials.iter().enumerate() {
if tri_count[m] == 0 {
continue;
}
let tex = if mat.texture == u32::MAX {
"none".to_string()
} else {
let t = &snap.textures[mat.texture as usize];
format!("#{} {}x{}", mat.texture, t.width, t.height)
};
eprintln!(
"mat {m:2}: tris {:6} albedo {:?} rough {:.2} tex {} uv [{:.2},{:.2}]..[{:.2},{:.2}]",
tri_count[m], mat.albedo, mat.roughness, tex,
uv_min[m][0], uv_min[m][1], uv_max[m][0], uv_max[m][1]
);
}
}
let mut input = scene_input_all_visible(&snap);
// The interactive camera + sun of the live repro session (seam log).
input.camera = Camera {
pos: v3([-5.297323, 17.044632, 23.215685]),
target: v3([-31.696442, 64.76611, 3.9240212]),
up: v3([0.0, 0.0, 1.0]),
fov_y: 40.0f32.to_radians(),
..Default::default()
};
input.sun = Sun {
dir: v3([-0.4174682, 0.33721808, 0.84380347]),
turbidity: 3.0,
sky_strength: 1.0,
sun_strength: 4.0,
};
let packed = PackedScene::pack(&input);
if input.materials.len() > 18 {
let base = 18 * 16;
eprintln!(
"probe: images {} atlas {:?}; mat18 tex flag {} rect {:?}",
input.images.len(),
packed.atlas.as_ref().map(|a| (a.width, a.height)),
packed.mat.data[base + 10],
&packed.mat.data[base + 12..base + 16],
);
// The grass image's own contrast: if the source texels are nearly
// uniform, a flat traced lawn is faithful and the raster's texture
// comes from somewhere else.
if let Some(img) = input.images.get(8) {
let mut lum: Vec<f32> = img
.data
.iter()
.map(|p| {
let r = ((p >> 16) & 255) as f32;
let g = ((p >> 8) & 255) as f32;
let b = (p & 255) as f32;
(0.2126 * r + 0.7152 * g + 0.0722 * b) / 255.0
})
.collect();
lum.sort_by(|a, b| a.partial_cmp(b).unwrap());
let mean = lum.iter().sum::<f32>() / lum.len() as f32;
eprintln!(
"probe: image8 {}x{} lum mean {:.3} p5 {:.3} p95 {:.3}",
img.width, img.height, mean,
lum[lum.len() / 20], lum[lum.len() * 19 / 20]
);
}
}
let mut tracer = cpu_ref::cpu_tracer(&input, &packed);
if std::env::var("FAB_PROBE_NO_SKIN").is_ok() {
tracer.shadow_skin = 0.0;
}
eprintln!("probe: shadow_skin {:.4} m", tracer.shadow_skin);
let (w, h) = (1234u32, 1488u32); // the live pane's native size
let sun_dir = input.sun.dir;
// Pass 1: primary hits at pixel centre; find roof pixels (steep upward
// normal) in a native-resolution band across the roof area.
let mut roof_pixels: Vec<(u32, u32)> = Vec::new();
for py in (h * 32 / 100..h * 42 / 100).step_by(3) {
for px in (0..w).step_by(3) {
let hit = tracer.primary_hit(px, py, w, h, (0.0, 0.0));
if hit.x < 0.0 {
continue;
}
let ti = hit.x as usize;
let i0 = input.indices[ti * 3] as usize;
let i1 = input.indices[ti * 3 + 1] as usize;
let i2 = input.indices[ti * 3 + 2] as usize;
let p0 = v3(input.positions[i0]);
let p1 = v3(input.positions[i1]);
let p2 = v3(input.positions[i2]);
let ng = Vec3f::cross(p1 - p0, p2 - p0).normalize();
// Roof slope: normal well off vertical walls, well off flat ground.
if ng.z.abs() > 0.3 && ng.z.abs() < 0.95 {
roof_pixels.push((px, py));
}
}
}
eprintln!("probe: {} roof-slope pixels of {}", roof_pixels.len(), w * h);
// Pass 2: per-sample primary hit + sun shadow on those pixels.
let n_samples = 32u32;
let mut alternating_layer = 0usize;
let mut cross_group_alternating = 0usize;
let mut cross_group_examples: Vec<(u32, u32, f32, Vec<(i32, u32, u16)>)> = Vec::new();
let mut alternating_shadow = 0usize;
let mut dark_total = 0u64;
let mut lit_total = 0u64;
let mut sampled = 0usize;
let mut worst: Vec<(u32, u32, usize, f32, f32)> = Vec::new();
for &(px, py) in &roof_pixels {
let mut tris = std::collections::HashMap::<i32, u32>::new();
let mut t_min = f32::MAX;
let mut t_max = 0.0f32;
let mut shadow_lit = 0u32;
let mut shadow_dark = 0u32;
let mut facing_away = 0u32;
for sidx in 0..n_samples {
let (tri, t, sh) = tracer.probe_primary_and_sun(px, py, w, h, 1, sidx, sun_dir);
if tri < 0 {
continue;
}
*tris.entry(tri).or_insert(0) += 1;
t_min = t_min.min(t);
t_max = t_max.max(t);
if sh < -1.0 {
facing_away += 1;
} else if sh > 0.5 {
shadow_lit += 1;
} else {
shadow_dark += 1;
}
}
let _ = facing_away;
if tris.is_empty() {
continue;
}
dark_total += shadow_dark as u64;
lit_total += shadow_lit as u64;
sampled += 1;
// Distinct triangles whose depth band is tight = stacked layers, not
// a silhouette edge (silhouettes have big depth spread).
let spread = t_max - t_min;
if tris.len() > 1 && spread < 0.05 {
alternating_layer += 1;
// Same coplanar group = the deterministic priority tie-break
// already owns the decision; cross-group / zero-group pairs are
// the ones a per-sample flip can still slip through.
let groups: std::collections::HashSet<u32> = tris
.keys()
.map(|t| input.tri_coplanar_group.get(*t as usize).copied().unwrap_or(0))
.collect();
if groups.len() > 1 || groups.contains(&0) {
if cross_group_examples.len() < 8 {
let detail: Vec<(i32, u32, u16)> = tris
.keys()
.map(|t| {
(
*t,
input.tri_coplanar_group.get(*t as usize).copied().unwrap_or(0),
input.tri_priority.get(*t as usize).copied().unwrap_or(0),
)
})
.collect();
cross_group_examples.push((px, py, spread, detail));
}
cross_group_alternating += 1;
}
}
if shadow_lit > 0 && shadow_dark > 0 {
alternating_shadow += 1;
let dark = shadow_dark as f32 / (shadow_lit + shadow_dark) as f32;
if (0.2..=0.8).contains(&dark) && worst.len() < 12 {
worst.push((px, py, tris.len(), spread, dark));
}
}
}
eprintln!(
"probe: of {} sampled roof pixels — {} alternate between stacked layers (<5 cm band), {} of those cross groups, {} alternate sun shadow; dark samples {} / lit {} ({:.1} % dark)",
sampled, alternating_layer, cross_group_alternating, alternating_shadow, dark_total, lit_total,
100.0 * dark_total as f64 / (dark_total + lit_total).max(1) as f64
);
for (px, py, spread, detail) in &cross_group_examples {
eprintln!(" cross-group px({px},{py}) spread {spread:.4}: {detail:?} (tri, group, priority)");
}
// Blocker-distance histogram over the dark samples: how far away is the
// thing that blocks a roof pixel's sun ray?
let mut blockers: Vec<f32> = Vec::new();
for &(px, py) in &roof_pixels {
for sidx in 0..8 {
let t = tracer.probe_blocker_distance(px, py, w, h, 1, sidx, sun_dir);
if t.is_finite() && t >= 0.0 {
blockers.push(t);
}
}
}
blockers.sort_by(|a, b| a.partial_cmp(b).unwrap());
if !blockers.is_empty() {
let q = |f: f64| blockers[((blockers.len() - 1) as f64 * f) as usize];
eprintln!(
"probe: {} blocked sun samples; blocker t p10 {:.4} p50 {:.4} p90 {:.4} p95 {:.4} p99 {:.4} max {:.3}",
blockers.len(), q(0.10), q(0.50), q(0.90), q(0.95), q(0.99), blockers.last().unwrap()
);
for lim in [0.005f32, 0.01, 0.02, 0.05, 0.1, 0.5] {
let n = blockers.iter().filter(|t| **t < lim).count();
eprintln!(" < {:>5.3} m: {:5.1} %", lim, 100.0 * n as f32 / blockers.len() as f32);
}
}
for (px, py, n, spread, dark) in &worst {
eprintln!(" worst: px({px},{py}) tris {n} depth spread {spread:.4} m dark fraction {dark:.2}");
}
// Optional: write small converged CPU renders (skin on) for visual
// before/after evidence — FAB_PROBE_RENDER=<dir>.
if let Ok(dir) = std::env::var("FAB_PROBE_RENDER") {
let _ = std::fs::create_dir_all(&dir);
let (rw, rh, spp) = (308u32, 372u32, 24u32);
let t0 = std::time::Instant::now();
let img = tracer.render(rw, rh, spp, 1);
eprintln!("probe: {}x{} at {} spp rendered in {:.1}s (skin {:.4})", rw, rh, spp, t0.elapsed().as_secs_f64(), tracer.shadow_skin);
let mut bgra = vec![0u8; (rw * rh * 4) as usize];
for (i, px) in img.iter().enumerate() {
let m = makepad_raytrace::gpu::tonemap_rgb(*px, 1.0);
bgra[i * 4] = (m[2] * 255.0 + 0.5) as u8;
bgra[i * 4 + 1] = (m[1] * 255.0 + 0.5) as u8;
bgra[i * 4 + 2] = (m[0] * 255.0 + 0.5) as u8;
bgra[i * 4 + 3] = 255;
}
let name = if tracer.shadow_skin > 0.0 { "roof_skin_on.png" } else { "roof_skin_off.png" };
let path = std::path::Path::new(&dir).join(name);
makepad_raytrace::png::write_bgra8(&path, rw as usize, rh as usize, &bgra).expect("png");
eprintln!("probe: wrote {}", path.display());
}
// Per-sample detail on the four worst pixels: which tri/material/depth
// each sample lands on and whether its sun ray is lit.
for &(px, py, ..) in worst.iter().take(4) {
eprintln!(" detail px({px},{py}):");
for sidx in 0..12 {
let (tri, t, sh) = tracer.probe_primary_and_sun(px, py, w, h, 1, sidx, sun_dir);
if tri < 0 {
eprintln!(" s{sidx}: miss");
continue;
}
let mat = input.tri_material[tri as usize];
let pr = input.tri_priority[tri as usize];
let gr = input.tri_coplanar_group[tri as usize];
let el = snap.triangle_element.get(tri as usize).copied().unwrap_or(u32::MAX);
eprintln!(
" s{sidx}: tri {tri} el {el} mat {mat} prio {pr} group {gr} t {t:.4} sun {sh:.2}"
);
}
}
}

311
apps/fab/tests/viewport.rs Normal file
View file

@ -0,0 +1,311 @@
//! Lane B (L2) regression: **a viewport's composite stays inside its own walk
//! rect**, and its rect is never empty.
//!
//! The bug this pins, exactly: the composite quad was drawn before any
//! `DrawList` had been begun inside the offscreen composite pass. `begin_pass`
//! clears the pass's `main_draw_list_id`, and whatever calls `begin_always`
//! first becomes it — so anything drawn before that lands in the *enclosing*
//! draw list, which belongs to the **window**. The quad was therefore painted
//! into the window at pass-local `(0,0)`, i.e. the window origin, on top of the
//! top bar and the tool column; and the composite pass, having no draw list of
//! its own, rendered nothing, so the viewport's real rect showed an empty
//! (black) composite target. One cause, both symptoms.
//!
//! The two assertions below are the two symptoms, inverted:
//!
//! 1. the top-bar band at the top of the window is still chrome — no viewport
//! composite has been painted over it;
//! 2. every visible viewport's own rect is not one flat colour — the composite
//! pass did render, and the blit landed there.
//!
//! Headless (`MAKEPAD=headless`) runs on the CPU rasterizer at dpi 1, so no GPU
//! is touched and screenshot pixels are layout points.
use makepad_test::{makepad_test, Selector, TestApp};
use makepad_zune_png::makepad_zune_core::bytestream::ZCursor;
use makepad_zune_png::PngDecoder;
/// How far two chrome samples may drift and still count as the same paint.
/// The composite's own background gradient alone is 20 levels wide, and the
/// lit image is far further off than that.
const CHROME_TOLERANCE: i32 = 6;
struct Image {
width: usize,
height: usize,
rgba: Vec<u8>,
}
impl Image {
fn read(path: &std::path::Path) -> Image {
let bytes = std::fs::read(path)
.unwrap_or_else(|err| panic!("cannot read grab {}: {err}", path.display()));
let mut decoder = PngDecoder::new(ZCursor::new(&bytes));
let pixels = decoder
.decode_raw()
.unwrap_or_else(|err| panic!("cannot decode grab {}: {err:?}", path.display()));
let (width, height) = decoder.dimensions().expect("grab has no dimensions");
let components = decoder
.colorspace()
.expect("grab has no colorspace")
.num_components();
assert!(
components >= 3,
"grab is not a colour image ({components} components)"
);
let mut rgba = vec![0u8; width * height * 4];
for i in 0..width * height {
let src = i * components;
rgba[i * 4] = pixels[src];
rgba[i * 4 + 1] = pixels[src + 1];
rgba[i * 4 + 2] = pixels[src + 2];
rgba[i * 4 + 3] = if components == 4 { pixels[src + 3] } else { 255 };
}
Image {
width,
height,
rgba,
}
}
fn pixel(&self, x: usize, y: usize) -> [u8; 3] {
let p = (y.min(self.height - 1) * self.width + x.min(self.width - 1)) * 4;
[self.rgba[p], self.rgba[p + 1], self.rgba[p + 2]]
}
/// The most common colour in a row span — for a chrome bar that is its
/// background, with the text and icons outvoted.
fn row_mode(&self, y: usize, x0: usize, x1: usize) -> [u8; 3] {
let x1 = x1.min(self.width);
assert!(x0 < x1, "empty row span {x0}..{x1} at y={y}");
let mut tally: std::collections::HashMap<[u8; 3], usize> = std::collections::HashMap::new();
for x in x0..x1 {
*tally.entry(self.pixel(x, y)).or_default() += 1;
}
tally
.into_iter()
.max_by_key(|(_, n)| *n)
.map(|(c, _)| c)
.unwrap()
}
/// Distinct colours (5 bits per channel) inside a rect.
fn distinct_colors_in(&self, x0: usize, y0: usize, x1: usize, y1: usize) -> usize {
let mut seen = std::collections::HashSet::new();
for y in y0..y1.min(self.height) {
for x in x0..x1.min(self.width) {
let p = self.pixel(x, y);
seen.insert([p[0] >> 3, p[1] >> 3, p[2] >> 3]);
}
}
seen.len()
}
fn luma(&self, x: usize, y: usize) -> i32 {
let p = self.pixel(x, y);
(p[0] as i32 * 54 + p[1] as i32 * 183 + p[2] as i32 * 19) / 256
}
/// Mean |Δluma| of 8-pixel-aligned block boundaries over interior
/// adjacent pixels in the same span. ≈1 on a smooth face; ≫1 when the
/// composite is an un-upsampled 8×8 AO/cavity grid.
fn blockiness_8(&self, x0: usize, y0: usize, x1: usize, y1: usize) -> f32 {
let x1 = x1.min(self.width);
let y1 = y1.min(self.height);
if x1 <= x0 + 16 || y1 <= y0 + 16 {
return 0.0;
}
let mut boundary = 0i64;
let mut interior = 0i64;
let mut n = 0i64;
for y in y0..y1 {
let mut x = x0 + 8;
while x < x1 {
boundary += (self.luma(x, y) - self.luma(x - 1, y)).abs() as i64;
interior += (self.luma(x - 4, y) - self.luma(x - 5, y)).abs() as i64;
n += 1;
x += 8;
}
}
for x in x0..x1 {
let mut y = y0 + 8;
while y < y1 {
boundary += (self.luma(x, y) - self.luma(x, y - 1)).abs() as i64;
interior += (self.luma(x, y - 4) - self.luma(x, y - 5)).abs() as i64;
n += 1;
y += 8;
}
}
if n == 0 || interior == 0 {
return 0.0;
}
boundary as f32 / interior as f32
}
/// Variance of luma in a rect. Used to pick a flat lit face.
fn luma_stats(&self, x0: usize, y0: usize, x1: usize, y1: usize) -> (f32, f32) {
let x1 = x1.min(self.width);
let y1 = y1.min(self.height);
if x1 <= x0 || y1 <= y0 {
return (0.0, 0.0);
}
let mut sum = 0i64;
let mut n = 0i64;
for y in y0..y1 {
for x in x0..x1 {
sum += self.luma(x, y) as i64;
n += 1;
}
}
if n == 0 {
return (0.0, 0.0);
}
let mean = sum as f32 / n as f32;
let mut var = 0.0f32;
for y in y0..y1 {
for x in x0..x1 {
let d = self.luma(x, y) as f32 - mean;
var += d * d;
}
}
(mean, var / n as f32)
}
}
#[makepad_test]
fn viewport_composites_stay_inside_their_walk_rects(app: TestApp) {
// `main` is the shell's root view: its rect is the window's content area,
// in the same layout points every widget rect below uses.
let shell = app.locator(Selector::id("main")).wait_visible().snapshot();
assert!(
shell.width > 0 && shell.height > 0,
"the shell root has no rect: {shell:?}"
);
let viewports = Selector::id("viewport");
let count = app.locator(viewports.clone()).wait_visible().count();
assert!(
count > 0,
"no visible FabViewport in the shell — the layout changed, fix the selector"
);
let rects: Vec<_> = (0..count)
.map(|i| app.locator(viewports.clone().nth(i)).snapshot())
.collect();
let path = app.screenshot();
println!("[fab] grab: {}", path.display());
let image = Image::read(&path);
// dpi 1 headless, but derive the factor anyway so a retina visible-mode
// run (MAKEPAD_TEST_VISIBLE=1) measures the same thing.
let scale = image.width as f64 / shell.width as f64;
let to_px = |v: i64| ((v as f64) * scale).round().max(0.0) as usize;
// ---- 1. the chrome above the viewports is still chrome -----------------
// A misplaced composite is an opaque quad at pass-local (0,0) — the window
// origin — sized like its viewport, so it covers the top bar from x=0 out
// to the viewport's width and stops. Chrome further right is out of its
// reach. Comparing the two halves of the same top-bar row therefore needs
// no colour constant at all: the bar is one paint, and if the left half
// stops matching the right half, something is being painted over it.
let top = rects.iter().map(|r| to_px(r.y)).min().unwrap_or(0);
assert!(
top >= 4,
"the first viewport starts at y={top}px — there is no chrome band to test"
);
let covered = rects.iter().map(|r| to_px(r.x) + to_px(r.width)).max().unwrap_or(0);
let untouched = covered + (image.width - covered) / 2;
assert!(
untouched + 8 < image.width,
"no chrome to the right of the viewports to compare against"
);
for y in [2usize, 4, 6] {
let over = image.row_mode(y, 0, (image.width / 4).max(8));
let clear = image.row_mode(y, untouched, image.width);
let drift = (0..3)
.map(|c| (over[c] as i32 - clear[c] as i32).abs())
.max()
.unwrap();
assert!(
drift <= CHROME_TOLERANCE,
"top-bar row y={y} reads {over:?} above the viewports but {clear:?} where no \
viewport can reach a viewport composite is being painted at the window origin \
(grab {})",
path.display()
);
}
// ---- 2. every viewport painted something into its own rect -------------
for (i, r) in rects.iter().enumerate() {
let (x0, y0) = (to_px(r.x), to_px(r.y));
let (x1, y1) = (x0 + to_px(r.width), y0 + to_px(r.height));
assert!(
x1 > x0 + 8 && y1 > y0 + 8,
"viewport {i} has a degenerate rect {r:?}"
);
// Inset past the 1 px area border and any overlay chrome at the edge.
let colors = distinct_inset(&image, x0, y0, x1, y1);
assert!(
colors > 1,
"viewport {i} rect {r:?} is one flat colour — its composite pass \
rendered nothing into its own target (grab {})",
path.display()
);
}
// ---- 3. Solid (left) composite is not an 8×8 AO/cavity grid ------------
// The leftmost viewport is the realtime Solid pane. On a flat lit face
// the 8-pixel-aligned block-boundary jump must not dwarf the interior
// 1 px jump — that was the un-upsampled cavity/SSAO (or the 256-wide
// element LUT sampled as if it were spatial).
let left = rects
.iter()
.min_by_key(|r| r.x)
.expect("no viewport rect");
let (x0, y0) = (to_px(left.x), to_px(left.y));
let (x1, y1) = (x0 + to_px(left.width), y0 + to_px(left.height));
let inset_x = ((x1 - x0) / 6).max(12);
let inset_y = ((y1 - y0) / 6).max(12);
let ix0 = x0 + inset_x;
let iy0 = y0 + inset_y;
let ix1 = x1.saturating_sub(inset_x);
let iy1 = y1.saturating_sub(inset_y);
let patch = 48usize;
let mut best: Option<(f32, usize, usize)> = None;
if ix1 > ix0 + patch && iy1 > iy0 + patch {
let mut y = iy0;
while y + patch <= iy1 {
let mut x = ix0;
while x + patch <= ix1 {
let (mean, var) = image.luma_stats(x, y, x + patch, y + patch);
if mean > 28.0 && mean < 230.0 {
match best {
Some((v, _, _)) if var >= v => {}
_ => best = Some((var, x, y)),
}
}
x += 16;
}
y += 16;
}
}
if let Some((var, x, y)) = best {
let ratio = image.blockiness_8(x, y, x + patch, y + patch);
assert!(
ratio < 2.2,
"solid viewport has 8×8 blockiness {ratio:.2} (patch variance {var:.1}) \
at ({x},{y}) cavity/SSAO is being composited without a full-res \
(or bilateral) upsample (grab {})",
path.display()
);
}
}
fn distinct_inset(image: &Image, x0: usize, y0: usize, x1: usize, y1: usize) -> usize {
let inset = 6usize;
image.distinct_colors_in(
x0 + inset,
y0 + inset,
x1.saturating_sub(inset),
y1.saturating_sub(inset),
)
}

View file

@ -0,0 +1,133 @@
//! Converted-sample glazing and interior-light regression.
use fab::model::makepad_math::vec3;
use fab::model::{Loader, Scene};
use fab::render::{scene_input_from_snapshot, TrackFile};
use fab::AppState;
use makepad_fab_loader_gltf::GltfLoader;
use makepad_micro_serde::DeJson;
use makepad_raytrace::cpu_ref::cpu_tracer;
use makepad_raytrace::pack::PackedScene;
use makepad_raytrace::{Camera, Sun};
use std::path::Path;
use std::sync::Arc;
#[test]
fn woodside_interior_keys_are_lit_through_converted_glass() {
// Heavy: loads the converted house, packs the scene and runs the CPU
// tracer over the interior keys. Its peak memory reached ~120 GB on the
// 2026-08-25 build, so it runs only when asked for explicitly.
if std::env::var("FAB_HEAVY_TESTS").is_err() {
eprintln!("skipped: set FAB_HEAVY_TESTS=1 to run the interior daylight test");
return;
}
let root = Path::new(env!("CARGO_MANIFEST_DIR"))
.ancestors()
.nth(2)
.expect("app crate is inside the workspace");
let sample = root.join("local/models/samples/woodside.glb");
let track_path = root.join("local/models/tours/woodside-full-track.json");
if !sample.exists() || !track_path.exists() {
eprintln!(
"skipping: converted sample or track absent ({} / {})",
sample.display(),
track_path.display()
);
return;
}
let track_text = std::fs::read_to_string(&track_path).expect("read Woodside track");
let track = TrackFile::deserialize_json(&track_text).expect("valid Woodside track");
let document = GltfLoader
.load(&sample, &mut |_| {})
.expect("load converted Woodside GLB");
eprintln!("Woodside material dump (name, transmission, ior, double-sided):");
for material in document.materials() {
eprintln!(
" {:?}: transmission={:.3} ior={:.3} double_sided={}",
material.name, material.transmission, material.ior, material.double_sided
);
}
let glass_materials = document
.materials()
.iter()
.filter(|material| material.transmission > 0.0)
.count();
assert!(glass_materials > 0, "Woodside has no transmissive materials");
assert!(document
.materials()
.iter()
.filter(|material| material.transmission > 0.0)
.all(|material| material.double_sided));
let scene = Arc::new(Scene::from_document(document, &mut |_| {}));
let mut state = AppState::default();
state.set_scene(scene);
let mut input = scene_input_from_snapshot(state.snapshot.as_ref().expect("scene snapshot"), &state);
input.sun = Sun {
dir: vec3(0.35, 0.25, 0.9).normalize(),
turbidity: 2.5,
sky_strength: 1.0,
sun_strength: 4.0,
};
let glass_triangles = input
.tri_material
.iter()
.filter(|&&material| {
input
.materials
.get(material as usize)
.is_some_and(|material| material.transmission > 0.0)
})
.count();
assert!(glass_triangles > 0, "Woodside has no transmissive triangles");
let packed = PackedScene::pack(&input);
for key_index in [2300_usize, 2600, 2880, 2900] {
let key = track
.keys
.get(key_index)
.unwrap_or_else(|| panic!("Woodside track omitted key {key_index}"));
input.camera = Camera {
pos: vec3(key.pos[0], key.pos[1], key.pos[2]),
target: vec3(key.look_at[0], key.look_at[1], key.look_at[2]),
up: vec3(key.up[0], key.up[1], key.up[2]),
fov_y: key.fov_y_deg.to_radians(),
f_stop: 0.0,
..Default::default()
};
let tracer = cpu_tracer(&input, &packed);
let image = tracer.render(16, 10, 64, key_index as u32);
let mean = image
.iter()
.map(|pixel| (pixel[0] + pixel[1] + pixel[2]) / 3.0)
.sum::<f32>()
/ image.len() as f32;
let metered_exposure = makepad_raytrace::gpu::metered_exposure_from_rgb(
&image, 16, 10, 1.0,
);
let display_mean = |exposure| {
image
.iter()
.map(|&pixel| {
let display = makepad_raytrace::gpu::tonemap_rgb(pixel, exposure);
0.2126 * display[0] + 0.7152 * display[1] + 0.0722 * display[2]
})
.sum::<f32>()
/ image.len() as f32
};
let fixed_display = display_mean(1.0) * 255.0;
let metered_display = display_mean(metered_exposure) * 255.0;
eprintln!(
"Woodside key {key_index}: linear mean {mean:.6}, display fixed={fixed_display:.2}/255 metered={metered_display:.2}/255 at {metered_exposure:.2}x, glass {glass_materials} materials/{glass_triangles} triangles, stats {:?}",
tracer.stats()
);
assert!(mean > 0.0, "Woodside key {key_index} stayed black");
if key_index == 2880 {
assert!(
metered_display > 80.0,
"Woodside key 2880 metered mean {metered_display}/255"
);
}
}
}

18
apps/fabric/Cargo.toml Normal file
View file

@ -0,0 +1,18 @@
[package]
name = "makepad-fabric"
version = "0.1.0"
edition = "2021"
license = "MIT OR Apache-2.0"
description = "Fabric: a photo in, a fitted sewing pattern out — body model, measurements, draft, cut sheet"
[dependencies]
# The window manager's vocabulary: the bar's title and the polite close.
makepad-wm-api = { path = "../../libs/wm_api" }
makepad-widgets = { path = "../../widgets" }
makepad-fabric-measure = { path = "../../libs/fabric/measure" }
makepad-fabric-draft = { path = "../../libs/fabric/draft" }
# The body model runs in-process (Metal on the Mac, CUDA on a box); the
# hub does install, licence acknowledgement and weight location.
makepad-ai-body = { path = "../../libs/ai/models/body" }
makepad-ai-hub = { path = "../../libs/ai/hub", default-features = false, features = ["local", "body-native"] }
makepad-ai-hub-ui = { path = "../../libs/ai/hub_ui" }

View file

@ -0,0 +1,539 @@
use makepad_fabric_measure::{BodyMesh, Line, Measured, Ring};
use makepad_widgets::*;
use std::sync::Arc;
script_mod! {
use mod.prelude.widgets_internal.*
use mod.widgets.*
set_type_default() do #(DrawBodyPoint::script_shader(vm)) {
..mod.draw.DrawQuad
pixel: fn() {
let d = length(self.pos - vec2(0.5, 0.5))
let a = clamp((0.5 - d) * 7.0, 0.0, 1.0)
let near = #x80e7ff
let far = #x27435d
let c = far.mix(near, 1.0 - self.depth)
return vec4(c.xyz * a, a)
}
}
set_type_default() do #(DrawFabricLine::script_shader(vm)) {
..mod.draw.DrawQuad
pixel: fn() {
// A line as a distance field inside its bounding quad. The
// endpoints are LOCAL to the quad (the turtle may still shift
// rect_pos after the instance is written), like the chart's
// segment shader.
let p = self.pos * self.rect_size
let ab = self.p1 - self.p0
let t = clamp(dot(p - self.p0, ab) / max(dot(ab, ab), 0.0001), 0.0, 1.0)
let d = length(p - (self.p0 + ab * t))
let aa = 1.0 - smoothstep(self.half_width - 0.6, self.half_width + 0.6, d)
let alpha = aa * self.color.w
return vec4(self.color.xyz * alpha, alpha)
}
}
mod.widgets.FabricBodyViewBase = #(FabricBodyView::register_widget(vm))
mod.widgets.FabricBodyView = set_type_default() do mod.widgets.FabricBodyViewBase {
width: Fill
height: Fill
draw_bg +: {color: #x11161d}
draw_point +: {}
draw_line +: {}
draw_text +: {
color: #x9aa8b7
text_style: theme.font_regular{font_size: 9.0}
}
}
}
#[derive(Script, ScriptHook)]
#[repr(C)]
pub struct DrawBodyPoint {
#[deref]
draw_super: DrawQuad,
#[live]
depth: f32,
}
#[derive(Script, ScriptHook)]
#[repr(C)]
pub struct DrawFabricLine {
#[deref]
draw_super: DrawQuad,
#[live]
pub color: Vec4f,
#[live]
p0: Vec2f,
#[live]
p1: Vec2f,
#[live]
half_width: f32,
}
impl DrawFabricLine {
pub fn segment(&mut self, cx: &mut Cx2d, from: DVec2, to: DVec2, width: f64) {
if (to - from).length() < 0.01 {
return;
}
let half = width * 0.5;
let pad = half + 1.0;
let min = dvec2(from.x.min(to.x) - pad, from.y.min(to.y) - pad);
let max = dvec2(from.x.max(to.x) + pad, from.y.max(to.y) + pad);
self.p0 = v2f(from - min);
self.p1 = v2f(to - min);
self.half_width = half as f32;
self.draw_abs(cx, Rect { pos: min, size: max - min });
}
}
fn v2f(value: DVec2) -> Vec2f {
Vec2f {
x: value.x as f32,
y: value.y as f32,
}
}
#[derive(Clone, Copy)]
struct BodyDrag {
from: DVec2,
yaw: f64,
pitch: f64,
pan: DVec2,
panning: bool,
}
#[derive(Clone, Debug, Default)]
pub(crate) struct BodyPoseMapping {
ring_vertices: Vec<Vec<usize>>,
line_vertices: Vec<[usize; 2]>,
}
pub(crate) fn map_measurements_to_vertices(
mesh: &BodyMesh,
measured: &Measured,
) -> BodyPoseMapping {
let nearest = |point| nearest_vertex_index(&mesh.vertices, measured.scale, point);
BodyPoseMapping {
ring_vertices: measured
.rings
.iter()
.map(|ring| ring.points.iter().copied().map(nearest).collect())
.collect(),
line_vertices: measured
.lines
.iter()
.map(|line| [nearest(line.from), nearest(line.to)])
.collect(),
}
}
fn nearest_vertex_index(vertices: &[[f32; 3]], scale: f32, point: [f32; 3]) -> usize {
vertices
.iter()
.enumerate()
.map(|(index, vertex)| {
let dx = vertex[0] * scale - point[0];
let dy = vertex[1] * scale - point[1];
let dz = vertex[2] * scale - point[2];
(index, dx * dx + dy * dy + dz * dz)
})
.min_by(|left, right| left.1.total_cmp(&right.1))
.map(|(index, _)| index)
.unwrap_or(0)
}
fn mirror_x(value: f64, mirrored: bool) -> f64 {
if mirrored { -value } else { value }
}
fn bounds(points: &[[f32; 3]], scale: f32) -> Option<([f32; 3], f32)> {
let mut min = [f32::INFINITY; 3];
let mut max = [f32::NEG_INFINITY; 3];
for point in points {
for axis in 0..3 {
let value = point[axis] * scale;
min[axis] = min[axis].min(value);
max[axis] = max[axis].max(value);
}
}
if !min[0].is_finite() {
return None;
}
let centre = [
(min[0] + max[0]) * 0.5,
(min[1] + max[1]) * 0.5,
(min[2] + max[2]) * 0.5,
];
let dx = max[0] - min[0];
let dy = max[1] - min[1];
let dz = max[2] - min[2];
let radius = (dx * dx + dy * dy + dz * dz).sqrt().max(1.0) * 0.5;
Some((centre, radius))
}
#[derive(Script, ScriptHook, Widget)]
pub struct FabricBodyView {
#[uid]
uid: WidgetUid,
#[source]
source: ScriptObjectRef,
#[walk]
walk: Walk,
#[layout]
layout: Layout,
#[redraw]
#[area]
area: Area,
#[live]
draw_bg: DrawColor,
#[live]
draw_point: DrawBodyPoint,
#[live]
draw_line: DrawFabricLine,
#[live]
draw_text: DrawText,
#[rust]
mesh: Option<Arc<BodyMesh>>,
#[rust]
posed: Option<Arc<Vec<[f32; 3]>>>,
#[rust]
rings: Vec<Ring>,
#[rust]
lines: Vec<Line>,
#[rust]
pose_mapping: BodyPoseMapping,
#[rust]
mesh_scale: f32,
#[rust]
centre: [f32; 3],
#[rust]
radius: f32,
#[rust(0.35)]
yaw: f64,
#[rust(-0.06)]
pitch: f64,
#[rust(1.0)]
zoom: f64,
#[rust]
pan: DVec2,
#[rust]
drag: Option<BodyDrag>,
#[rust(true)]
mirrored: bool,
}
impl FabricBodyView {
pub fn set_body(
&mut self,
cx: &mut Cx,
mesh: Arc<BodyMesh>,
measured: &Measured,
pose_mapping: BodyPoseMapping,
) {
self.mesh_scale = measured.scale;
self.rings = measured.rings.clone();
self.lines = measured.lines.clone();
self.pose_mapping = pose_mapping;
if self.posed.is_none() {
self.fit_bounds(&mesh.vertices);
self.reset_camera();
}
self.mesh = Some(mesh);
self.redraw(cx);
}
pub fn set_pose(&mut self, cx: &mut Cx, posed: Option<Arc<Vec<[f32; 3]>>>) {
match posed {
Some(posed)
if self
.mesh
.as_ref()
.is_some_and(|mesh| mesh.vertices.len() == posed.len()) =>
{
if self.posed.is_none() {
self.fit_bounds(posed.as_slice());
self.reset_camera();
}
self.posed = Some(posed);
}
_ => {
self.posed = None;
if let Some(fit) = self
.mesh
.as_ref()
.and_then(|mesh| bounds(&mesh.vertices, self.mesh_scale))
{
(self.centre, self.radius) = fit;
self.reset_camera();
}
}
}
self.redraw(cx);
}
pub fn set_mirrored(&mut self, cx: &mut Cx, mirrored: bool) {
self.mirrored = mirrored;
self.redraw(cx);
}
fn fit_bounds(&mut self, points: &[[f32; 3]]) {
if let Some((centre, radius)) = bounds(points, self.mesh_scale) {
self.centre = centre;
self.radius = radius;
}
}
fn reset_camera(&mut self) {
self.yaw = 0.35;
self.pitch = -0.06;
self.zoom = 1.0;
self.pan = dvec2(0.0, 0.0);
}
fn project(&self, point: [f32; 3], mesh_point: bool, rect: Rect) -> Option<(DVec2, f32)> {
let scale = if mesh_point { self.mesh_scale } else { 1.0 } as f64;
let x = point[0] as f64 * scale - self.centre[0] as f64;
let y = point[1] as f64 * scale - self.centre[1] as f64;
let z = point[2] as f64 * scale - self.centre[2] as f64;
let (sy, cy) = self.yaw.sin_cos();
let (sp, cp) = self.pitch.sin_cos();
let rx = mirror_x(cy * x + sy * z, self.mirrored);
let rz = -sy * x + cy * z;
let ry = cp * y - sp * rz;
let rz = sp * y + cp * rz;
let fov = 35.0_f64.to_radians();
let fit_distance = self.radius as f64 / (fov * 0.5).tan() * 1.2;
let camera_z = fit_distance / self.zoom.max(0.08) - rz;
if camera_z <= 0.01 {
return None;
}
let focal = rect.size.y.max(1.0) * 0.5 / (fov * 0.5).tan();
let centre = rect.pos + rect.size * 0.5 + self.pan;
let screen = dvec2(centre.x + focal * rx / camera_z, centre.y - focal * ry / camera_z);
Some((screen, camera_z as f32))
}
fn projected_polyline(&self, points: &[[f32; 3]], rect: Rect) -> Vec<(DVec2, f32)> {
points
.iter()
.filter_map(|point| self.project(*point, false, rect))
.collect()
}
}
impl Widget for FabricBodyView {
fn draw_walk(&mut self, cx: &mut Cx2d, _scope: &mut Scope, walk: Walk) -> DrawStep {
let rect = cx.walk_turtle(walk);
self.draw_bg.draw_abs(cx, rect);
cx.push_clip_rect(rect);
let Some(mesh) = self.mesh.as_ref() else {
self.draw_text.color = Vec4f {
x: 0.48,
y: 0.55,
z: 0.62,
w: 1.0,
};
self.draw_text.draw_abs(
cx,
rect.pos + rect.size * 0.5 - dvec2(70.0, 5.0),
"drop a photo to start",
);
cx.pop_clip_rect();
cx.add_aligned_rect_area(&mut self.area, rect);
return DrawStep::done();
};
let posed = self
.posed
.as_deref()
.filter(|posed| posed.len() == mesh.vertices.len())
.map(Vec::as_slice);
let display_vertices = posed.unwrap_or(mesh.vertices.as_slice());
let mut points: Vec<(DVec2, f32)> = display_vertices
.iter()
.filter_map(|point| self.project(*point, true, rect))
.collect();
points.sort_by(|a, b| b.1.total_cmp(&a.1));
let min_depth = points.iter().map(|point| point.1).fold(f32::INFINITY, f32::min);
let max_depth = points
.iter()
.map(|point| point.1)
.fold(f32::NEG_INFINITY, f32::max);
let depth_span = (max_depth - min_depth).max(0.001);
self.draw_point.begin_many_instances(cx);
for (point, depth) in points {
self.draw_point.depth = (depth - min_depth) / depth_span;
self.draw_point.draw_abs(
cx,
Rect {
pos: point - dvec2(1.25, 1.25),
size: dvec2(2.5, 2.5),
},
);
}
self.draw_point.end_many_instances(cx);
let rings: Vec<(String, Vec<(DVec2, f32)>)> = self
.rings
.iter()
.enumerate()
.map(|(ring_index, ring)| {
let points = match (
posed,
self.pose_mapping.ring_vertices.get(ring_index),
) {
(Some(posed), Some(indices)) if indices.len() == ring.points.len() => indices
.iter()
.filter_map(|&index| self.project(*posed.get(index)?, true, rect))
.collect(),
_ => self.projected_polyline(&ring.points, rect),
};
(ring.key.replace('_', " "), points)
})
.collect();
let lines: Vec<(DVec2, DVec2)> = self
.lines
.iter()
.enumerate()
.filter_map(|(line_index, line)| {
let (from, to, mesh_points) = match (
posed,
self.pose_mapping.line_vertices.get(line_index),
) {
(Some(posed), Some([from, to])) =>
(*posed.get(*from)?, *posed.get(*to)?, true),
_ => (line.from, line.to, false),
};
Some((
self.project(from, mesh_points, rect)?.0,
self.project(to, mesh_points, rect)?.0,
))
})
.collect();
self.draw_line.begin_many_instances(cx);
self.draw_line.color = Vec4f {
x: 1.0,
y: 0.38,
z: 0.18,
w: 0.92,
};
for (_, points) in &rings {
for pair in points.windows(2) {
self.draw_line.segment(cx, pair[0].0, pair[1].0, 1.5);
}
if let (Some(first), Some(last)) = (points.first(), points.last()) {
self.draw_line.segment(cx, last.0, first.0, 1.5);
}
}
self.draw_line.color = Vec4f {
x: 0.42,
y: 0.78,
z: 1.0,
w: 0.9,
};
for (from, to) in lines {
self.draw_line.segment(cx, from, to, 1.25);
}
self.draw_line.end_many_instances(cx);
self.draw_text.color = Vec4f {
x: 1.0,
y: 0.66,
z: 0.48,
w: 1.0,
};
for (key, points) in rings {
if let Some(front) = points.iter().min_by(|a, b| a.1.total_cmp(&b.1)) {
self.draw_text
.draw_abs(cx, front.0 + dvec2(4.0, -5.0), &key);
}
}
cx.pop_clip_rect();
cx.add_aligned_rect_area(&mut self.area, rect);
DrawStep::done()
}
fn handle_event(&mut self, cx: &mut Cx, event: &Event, _scope: &mut Scope) {
match event.hits(cx, self.area) {
Hit::FingerDown(event) if event.device.is_primary_hit() => {
if event.tap_count >= 2 {
self.reset_camera();
self.redraw(cx);
return;
}
self.drag = Some(BodyDrag {
from: event.abs,
yaw: self.yaw,
pitch: self.pitch,
pan: self.pan,
panning: event.modifiers.shift,
});
}
Hit::FingerMove(event) => {
if let Some(drag) = self.drag {
let delta = event.abs - drag.from;
if drag.panning {
self.pan = drag.pan + delta;
} else {
self.yaw = drag.yaw - delta.x * 0.008;
self.pitch = (drag.pitch + delta.y * 0.007).clamp(-1.35, 1.35);
}
self.redraw(cx);
}
}
Hit::FingerUp(_) => self.drag = None,
Hit::FingerScroll(event) => {
self.zoom = (self.zoom * (-event.scroll.y * 0.004).exp()).clamp(0.15, 12.0);
self.redraw(cx);
}
Hit::FingerHoverIn(_) | Hit::FingerHoverOver(_) => {
cx.set_cursor(MouseCursor::Grab);
}
_ => {}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn nearest_vertex_mapping_uses_scaled_rest_mesh_positions() {
let mesh = BodyMesh {
vertices: vec![[0.0, 0.0, 0.0], [4.0, 0.0, 0.0], [9.0, 0.0, 0.0]],
faces: Vec::new(),
landmarks: None,
};
let measured = Measured {
values: makepad_fabric_measure::Measurements::sample(),
scale: 2.0,
rings: vec![Ring {
key: "test_ring",
y_cm: 0.0,
points: vec![[0.2, 0.0, 0.0], [7.5, 0.0, 0.0]],
skin_perimeter_cm: 0.0,
tape_perimeter_cm: 0.0,
}],
lines: vec![Line {
key: "test_line",
from: [7.5, 0.0, 0.0],
to: [17.0, 0.0, 0.0],
}],
};
let mapping = map_measurements_to_vertices(&mesh, &measured);
assert_eq!(mapping.ring_vertices, vec![vec![0, 1]]);
assert_eq!(mapping.line_vertices, vec![[1, 2]]);
}
#[test]
fn mirror_transform_only_flips_horizontal_view_axis() {
assert_eq!(mirror_x(12.5, false), 12.5);
assert_eq!(mirror_x(12.5, true), -12.5);
assert_eq!(mirror_x(-3.0, true), 3.0);
}
}

303
apps/fabric/src/camera.rs Normal file
View file

@ -0,0 +1,303 @@
use makepad_widgets::{
makepad_platform::video::{
CameraFrameLayout, CameraFrameRef, VideoFormatId, VideoInputId, VideoInputsEvent,
VideoPixelFormat,
},
Cx, CxMediaApi,
};
use std::{
cmp::Reverse,
sync::{
atomic::{AtomicBool, AtomicU64, Ordering},
Arc, Mutex,
},
time::{Duration, Instant},
};
pub const SEND_MAX_WIDTH: usize = 640;
const PREVIEW_MAX_WIDTH: usize = 320;
const PREVIEW_INTERVAL: Duration = Duration::from_millis(100);
#[derive(Clone, Debug, PartialEq)]
pub struct CameraRgbFrame {
pub width: u32,
pub height: u32,
pub rgb: Vec<u8>,
pub serial: u64,
}
#[derive(Default)]
struct PreviewSlot {
frame: Option<CameraRgbFrame>,
updated_at: Option<Instant>,
}
#[derive(Default)]
struct CameraMailboxInner {
want: AtomicBool,
serial: AtomicU64,
frame: Mutex<Option<CameraRgbFrame>>,
model_size: Mutex<Option<(u32, u32)>>,
preview: Mutex<PreviewSlot>,
}
/// A one-frame handoff from the camera callback to the model worker. The
/// callback only converts a model-sized frame after the worker asks for one;
/// the smaller preview is independent and limited to ten updates per second.
#[derive(Clone, Default)]
pub struct CameraMailbox {
inner: Arc<CameraMailboxInner>,
}
impl CameraMailbox {
pub fn request(&self) {
if let Ok(mut frame) = self.inner.frame.lock() {
*frame = None;
}
self.inner.want.store(true, Ordering::Release);
}
pub fn take(&self) -> Option<CameraRgbFrame> {
self.inner.frame.lock().ok()?.take()
}
pub fn peek_preview(&self) -> Option<CameraRgbFrame> {
self.inner.preview.lock().ok()?.frame.clone()
}
pub fn model_size(&self) -> Option<(u32, u32)> {
*self.inner.model_size.lock().ok()?
}
fn capture(&self, frame: &CameraFrameRef<'_>) {
let wanted = self.inner.want.swap(false, Ordering::AcqRel);
let preview_due = self
.inner
.preview
.lock()
.ok()
.map(|slot| {
slot.updated_at
.map(|updated| updated.elapsed() >= PREVIEW_INTERVAL)
.unwrap_or(true)
})
.unwrap_or(false);
if !wanted && !preview_due {
return;
}
if wanted {
if let Some((rgb, width, height)) = frame_to_rgb(frame) {
let serial = self.inner.serial.fetch_add(1, Ordering::Relaxed) + 1;
if let Ok(mut size) = self.inner.model_size.lock() {
*size = Some((width, height));
}
if let Ok(mut slot) = self.inner.frame.lock() {
*slot = Some(CameraRgbFrame {
width,
height,
rgb,
serial,
});
}
}
}
if preview_due {
if let Some((rgb, width, height)) = frame_to_rgb_max(frame, PREVIEW_MAX_WIDTH) {
let serial = self.inner.serial.fetch_add(1, Ordering::Relaxed) + 1;
if let Ok(mut slot) = self.inner.preview.lock() {
slot.frame = Some(CameraRgbFrame {
width,
height,
rgb,
serial,
});
slot.updated_at = Some(Instant::now());
}
}
}
}
}
/// Register the capture callback. Calling this also asks the platform to
/// enumerate cameras, which produces `Event::VideoInputs` on the UI thread.
pub fn install_camera(cx: &mut Cx, mailbox: CameraMailbox) {
cx.camera_frame_input(0, move |frame| mailbox.capture(&frame));
}
/// Choose the first device's smallest raw-YUV format at least 640x360.
pub fn pick_camera(event: &VideoInputsEvent) -> Option<(VideoInputId, VideoFormatId)> {
let device = event.descs.first()?;
let format = device
.formats
.iter()
.filter(|format| {
format.width >= 640
&& format.height >= 360
&& matches!(
format.pixel_format,
VideoPixelFormat::NV12 | VideoPixelFormat::YUY2
)
})
.min_by_key(|format| {
(
format.width.saturating_mul(format.height),
if format.pixel_format == VideoPixelFormat::NV12 {
0
} else {
1
},
Reverse((format.frame_rate.unwrap_or(0.0) * 1000.0) as u64),
)
})?;
Some((device.input_id, format.format_id))
}
/// Convert NV12 or YUY2 to packed RGB8, using an integer sampling step so
/// the entire frame fits within [`SEND_MAX_WIDTH`].
pub fn frame_to_rgb(frame: &CameraFrameRef<'_>) -> Option<(Vec<u8>, u32, u32)> {
frame_to_rgb_max(frame, SEND_MAX_WIDTH)
}
fn frame_to_rgb_max(
frame: &CameraFrameRef<'_>,
max_width: usize,
) -> Option<(Vec<u8>, u32, u32)> {
let (width, height) = (frame.width, frame.height);
if width == 0 || height == 0 || max_width == 0 {
return None;
}
let step = width.div_ceil(max_width).max(1);
let out_width = width.div_ceil(step);
let out_height = height.div_ceil(step);
let mut rgb = Vec::with_capacity(out_width.checked_mul(out_height)?.checked_mul(3)?);
for out_y in 0..out_height {
let source_y = (out_y * step).min(height - 1);
for out_x in 0..out_width {
let source_x = (out_x * step).min(width - 1);
let (y, u, v) = match frame.layout {
CameraFrameLayout::NV12 => nv12_pixel(frame, source_x, source_y)?,
CameraFrameLayout::YUY2 => yuy2_pixel(frame, source_x, source_y)?,
_ => return None,
};
rgb.extend_from_slice(&yuv_to_rgb(y, u, v));
}
}
Some((
rgb,
u32::try_from(out_width).ok()?,
u32::try_from(out_height).ok()?,
))
}
fn nv12_pixel(frame: &CameraFrameRef<'_>, x: usize, y: usize) -> Option<(u8, u8, u8)> {
if frame.plane_count < 2 {
return None;
}
let y_plane = frame.planes[0];
let uv_plane = frame.planes[1];
let y_index = y
.checked_mul(y_plane.row_stride)?
.checked_add(x.checked_mul(y_plane.pixel_stride)?)?;
let uv_index = (y / 2)
.checked_mul(uv_plane.row_stride)?
.checked_add((x / 2).checked_mul(uv_plane.pixel_stride)?)?;
Some((
*y_plane.bytes.get(y_index)?,
*uv_plane.bytes.get(uv_index)?,
*uv_plane.bytes.get(uv_index + 1)?,
))
}
fn yuy2_pixel(frame: &CameraFrameRef<'_>, x: usize, y: usize) -> Option<(u8, u8, u8)> {
if frame.plane_count < 1 {
return None;
}
let plane = frame.planes[0];
let pixel_stride = plane.pixel_stride.max(2);
let pair = y
.checked_mul(plane.row_stride)?
.checked_add((x / 2).checked_mul(pixel_stride.checked_mul(2)?)?)?;
Some((
*plane.bytes.get(pair + (x & 1) * pixel_stride)?,
*plane.bytes.get(pair + 1)?,
*plane.bytes.get(pair + pixel_stride + 1)?,
))
}
/// One BT.709, video-range YUV pixel to RGB8.
pub(crate) fn yuv_to_rgb(y: u8, u: u8, v: u8) -> [u8; 3] {
let c = i32::from(y) - 16;
let d = i32::from(u) - 128;
let e = i32::from(v) - 128;
let clip = |value: i32| ((value + 128) >> 8).clamp(0, 255) as u8;
[
clip(298 * c + 459 * e),
clip(298 * c - 55 * d - 136 * e),
clip(298 * c + 541 * d),
]
}
#[cfg(test)]
mod tests {
use super::*;
use makepad_widgets::makepad_platform::video::{
CameraColorMatrix, CameraFramePlaneRef,
};
#[test]
fn yuv_grey_red_and_blue() {
let grey = yuv_to_rgb(126, 128, 128);
assert!(grey.iter().all(|channel| (126i16 - i16::from(*channel)).abs() <= 2));
let red = yuv_to_rgb(81, 90, 240);
assert!(red[0] > 220 && red[1] < 60 && red[2] < 60, "{red:?}");
let blue = yuv_to_rgb(41, 240, 110);
assert!(blue[2] > 220 && blue[0] < 60 && blue[1] < 60, "{blue:?}");
}
#[test]
fn nv12_integer_downsample_keeps_the_whole_frame() {
const WIDTH: usize = 64;
const HEIGHT: usize = 32;
let mut y_plane = vec![0u8; WIDTH * HEIGHT];
for y in 0..HEIGHT {
for x in 0..WIDTH {
y_plane[y * WIDTH + x] = 16 + ((x + y) % 200) as u8;
}
}
let uv_plane = vec![128u8; WIDTH * HEIGHT / 2];
let frame = CameraFrameRef {
timestamp_ns: 0,
width: WIDTH,
height: HEIGHT,
layout: CameraFrameLayout::NV12,
matrix: CameraColorMatrix::BT709,
plane_count: 2,
planes: [
CameraFramePlaneRef {
bytes: &y_plane,
row_stride: WIDTH,
pixel_stride: 1,
},
CameraFramePlaneRef {
bytes: &uv_plane,
row_stride: WIDTH,
pixel_stride: 2,
},
CameraFramePlaneRef::empty(),
],
};
let (rgb, width, height) = frame_to_rgb_max(&frame, 16).unwrap();
assert_eq!((width, height), (16, 8));
assert_eq!(rgb.len(), 16 * 8 * 3);
assert_eq!(&rgb[..3], &yuv_to_rgb(y_plane[0], 128, 128));
let last_source = y_plane[28 * WIDTH + 60];
assert_eq!(&rgb[rgb.len() - 3..], &yuv_to_rgb(last_source, 128, 128));
}
}

View file

@ -0,0 +1,73 @@
use makepad_ai_hub::{
local::{InstallState, LocalModels},
registry::LicenseRestriction,
};
use makepad_ai_hub_ui::{ModelRowInstallState, ModelRowState};
pub const BODY_MODEL_ID: &str = "sam3dbody";
pub const BODY_MODEL_ROLE: &str = "native-body";
pub fn body_model_row(models: &LocalModels) -> ModelRowState {
let spec = models.spec(BODY_MODEL_ID);
let bytes_from_spec = spec
.map(|spec| spec.files.iter().filter_map(|file| file.size).sum())
.unwrap_or(0);
let (bytes_done, bytes_total, state) = match models.install_state(BODY_MODEL_ID) {
InstallState::NotInstalled { bytes_total } => {
(0, bytes_total.max(bytes_from_spec), ModelRowInstallState::NotInstalled)
}
InstallState::Partial {
bytes_done,
bytes_total,
} => (bytes_done, bytes_total, ModelRowInstallState::NotInstalled),
InstallState::Installed => (
bytes_from_spec,
bytes_from_spec,
ModelRowInstallState::Installed,
),
};
let license = spec.and_then(|spec| spec.license.as_ref());
ModelRowState {
model_id: BODY_MODEL_ID.to_string(),
name: "SAM 3D Body".to_string(),
bytes_total,
bytes_done,
state,
license_name: license
.map(|license| license.name.clone())
.unwrap_or_else(|| "Licence unavailable".to_string()),
restriction: license
.map(|license| restriction_name(license.restriction).to_string())
.unwrap_or_else(|| "restricted".to_string()),
}
}
pub fn body_model_status(models: &LocalModels, downloading: bool) -> String {
if !models.license_acknowledged(BODY_MODEL_ID) {
return "licence not accepted".to_string();
}
match models.install_state(BODY_MODEL_ID) {
InstallState::Installed => "installed · 2.8 GB · Metal".to_string(),
InstallState::NotInstalled { .. } => "not installed · 2.8 GB".to_string(),
InstallState::Partial {
bytes_done,
bytes_total,
} => {
let percent = bytes_done.saturating_mul(100) / bytes_total.max(1);
if downloading {
format!("downloading {percent} %")
} else {
format!("not installed · {percent} % downloaded")
}
}
}
}
fn restriction_name(restriction: LicenseRestriction) -> &'static str {
match restriction {
LicenseRestriction::None => "none",
LicenseRestriction::NonCommercial => "non-commercial",
LicenseRestriction::Community => "community",
LicenseRestriction::Restricted => "restricted",
}
}

1802
apps/fabric/src/main.rs Normal file

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,398 @@
use crate::body_view::DrawFabricLine;
use makepad_fabric_draft::{flatten, nest, offset, Layout as PatternLayout, Part, Pattern, Point};
use makepad_widgets::*;
script_mod! {
use mod.prelude.widgets_internal.*
use mod.widgets.*
mod.widgets.FabricPatternViewBase = #(FabricPatternView::register_widget(vm))
mod.widgets.FabricPatternView = set_type_default() do mod.widgets.FabricPatternViewBase {
width: Fill
height: Fill
draw_bg +: {color: #x0d1218}
draw_line +: {}
draw_text +: {
color: #xc5d0dc
text_style: theme.font_regular{font_size: 8.5}
}
}
}
#[derive(Clone, Copy)]
struct PatternDrag {
from: DVec2,
pan: DVec2,
}
#[derive(Clone, Copy, Default)]
struct Bounds {
min: DVec2,
max: DVec2,
valid: bool,
}
impl Bounds {
fn include(&mut self, point: DVec2) {
if !self.valid {
self.min = point;
self.max = point;
self.valid = true;
} else {
self.min.x = self.min.x.min(point.x);
self.min.y = self.min.y.min(point.y);
self.max.x = self.max.x.max(point.x);
self.max.y = self.max.y.max(point.y);
}
}
fn size(self) -> DVec2 {
let size = self.max - self.min;
dvec2(size.x.max(1.0), size.y.max(1.0))
}
fn centre(self) -> DVec2 {
(self.min + self.max) * 0.5
}
}
#[derive(Script, ScriptHook, Widget)]
pub struct FabricPatternView {
#[uid]
uid: WidgetUid,
#[source]
source: ScriptObjectRef,
#[walk]
walk: Walk,
#[layout]
layout: Layout,
#[redraw]
#[area]
area: Area,
#[live]
draw_bg: DrawColor,
#[live]
draw_line: DrawFabricLine,
#[live]
draw_text: DrawText,
#[rust]
pattern: Option<Pattern>,
#[rust]
nested: Option<PatternLayout>,
#[rust]
error: String,
#[rust]
bounds: Bounds,
#[rust(1.0)]
zoom: f64,
/// The view aspect the current nest was chosen for.
#[rust(1.0)]
nest_aspect: f64,
#[rust]
pan: DVec2,
#[rust]
drag: Option<PatternDrag>,
}
impl FabricPatternView {
pub fn set_pattern(&mut self, cx: &mut Cx, pattern: Pattern) {
self.pattern = Some(pattern);
// Nested on the next draw, for the pane's shape.
self.nested = None;
self.error.clear();
self.zoom = 1.0;
self.pan = dvec2(0.0, 0.0);
self.redraw(cx);
}
pub fn set_error(&mut self, cx: &mut Cx, error: impl Into<String>) {
self.pattern = None;
self.nested = None;
self.error = error.into();
self.redraw(cx);
}
/// Nest onto the fabric width whose finished layout has the pane's
/// aspect ratio, so the pieces fill the view instead of a tall strip.
fn ensure_nest(&mut self, rect: Rect) {
let target = (rect.size.x - 24.0).max(1.0) / (rect.size.y - 24.0).max(1.0);
if self.nested.is_some() && (self.nest_aspect / target).ln().abs() < 0.12 {
return;
}
const WIDTHS: [f64; 9] = [
900.0, 1200.0, 1500.0, 2000.0, 2500.0, 3000.0, 4000.0, 5000.0, 6500.0,
];
let best = {
let Some(pattern) = &self.pattern else { return };
let mut best: Option<(f64, PatternLayout, Bounds)> = None;
for width in WIDTHS {
let layout = nest(pattern, width);
let bounds = pattern_bounds(pattern, &layout);
let size = bounds.size();
let aspect = size.x.max(1.0) / size.y.max(1.0);
let score = (aspect / target).ln().abs();
if best.as_ref().map_or(true, |(other, _, _)| score < *other) {
best = Some((score, layout, bounds));
}
}
best
};
if let Some((_, layout, bounds)) = best {
self.nested = Some(layout);
self.bounds = bounds;
self.nest_aspect = target;
}
}
fn to_screen(&self, point: DVec2, rect: Rect) -> DVec2 {
let size = self.bounds.size();
let fit = ((rect.size.x - 24.0) / size.x)
.min((rect.size.y - 24.0) / size.y)
.max(0.0001);
rect.pos + rect.size * 0.5
+ self.pan
+ (point - self.bounds.centre()) * fit * self.zoom
}
fn part_offset<'a>(
&'a self,
layout: &'a PatternLayout,
part_index: usize,
) -> (Point, f64) {
layout
.placements
.iter()
.find(|placement| placement.part == part_index)
.map(|placement| (placement.offset, placement.rotation_deg))
.unwrap_or((Point::default(), 0.0))
}
fn path_segments(
&self,
path: &makepad_fabric_draft::Path,
offset_by: Point,
rotation: f64,
rect: Rect,
) -> Vec<(DVec2, DVec2)> {
let points = flatten(path, 1.0);
let transformed: Vec<DVec2> = points
.iter()
.map(|point| self.to_screen(place(*point, offset_by, rotation), rect))
.collect();
let mut segments: Vec<_> = transformed
.windows(2)
.map(|pair| (pair[0], pair[1]))
.collect();
if path.closed {
if let (Some(first), Some(last)) = (transformed.first(), transformed.last()) {
segments.push((*last, *first));
}
}
segments
}
}
impl Widget for FabricPatternView {
fn draw_walk(&mut self, cx: &mut Cx2d, _scope: &mut Scope, walk: Walk) -> DrawStep {
let rect = cx.walk_turtle(walk);
self.draw_bg.draw_abs(cx, rect);
cx.push_clip_rect(rect);
self.ensure_nest(rect);
let (Some(pattern), Some(layout)) = (&self.pattern, &self.nested) else {
let message = if self.error.is_empty() {
"pattern preview"
} else {
&self.error
};
self.draw_text.color = Vec4f {
x: 0.52,
y: 0.58,
z: 0.65,
w: 1.0,
};
self.draw_text
.draw_abs(cx, rect.pos + dvec2(14.0, 16.0), message);
cx.pop_clip_rect();
cx.add_aligned_rect_area(&mut self.area, rect);
return DrawStep::done();
};
struct Stroke {
from: DVec2,
to: DVec2,
color: Vec4f,
width: f64,
}
let mut strokes = Vec::new();
let mut labels = Vec::new();
let cut_color = Vec4f {
x: 0.93,
y: 0.96,
z: 0.99,
w: 1.0,
};
let seam_color = Vec4f {
x: 0.40,
y: 0.49,
z: 0.58,
w: 0.72,
};
let mark_color = Vec4f {
x: 1.0,
y: 0.40,
z: 0.18,
w: 0.95,
};
for (part_index, part) in pattern.parts.iter().enumerate() {
let (part_offset, rotation) = self.part_offset(layout, part_index);
let cut_path = offset(&part.outline, part.seam_allowance_mm);
for (from, to) in self.path_segments(&cut_path, part_offset, rotation, rect) {
strokes.push(Stroke {
from,
to,
color: cut_color,
width: 1.25,
});
}
for (from, to) in self.path_segments(&part.outline, part_offset, rotation, rect) {
strokes.push(Stroke {
from,
to,
color: seam_color,
width: 0.75,
});
}
for path in &part.internal {
for (from, to) in self.path_segments(path, part_offset, rotation, rect) {
strokes.push(Stroke {
from,
to,
color: seam_color,
width: 0.75,
});
}
}
for notch in &part.notches {
let at = self.to_screen(place(*notch, part_offset, rotation), rect);
strokes.push(Stroke {
from: at + dvec2(-3.0, -3.0),
to: at + dvec2(3.0, 3.0),
color: mark_color,
width: 1.0,
});
strokes.push(Stroke {
from: at + dvec2(-3.0, 3.0),
to: at + dvec2(3.0, -3.0),
color: mark_color,
width: 1.0,
});
}
let grain_from = self.to_screen(place(part.grainline.0, part_offset, rotation), rect);
let grain_to = self.to_screen(place(part.grainline.1, part_offset, rotation), rect);
strokes.push(Stroke {
from: grain_from,
to: grain_to,
color: mark_color,
width: 1.0,
});
let label_at = part
.labels
.first()
.map(|label| label.at)
.unwrap_or(part.outline.start);
labels.push((
self.to_screen(place(label_at, part_offset, rotation), rect),
format!(
"{} · {}",
part.name,
if part.on_fold {
"cut on fold".to_string()
} else {
format!("cut {}", part.cut_count)
}
),
));
}
self.draw_line.begin_many_instances(cx);
for stroke in strokes {
self.draw_line.color = stroke.color;
self.draw_line
.segment(cx, stroke.from, stroke.to, stroke.width);
}
self.draw_line.end_many_instances(cx);
self.draw_text.color = Vec4f {
x: 0.78,
y: 0.84,
z: 0.90,
w: 1.0,
};
for (at, text) in labels {
self.draw_text.draw_abs(cx, at + dvec2(4.0, -5.0), &text);
}
cx.pop_clip_rect();
cx.add_aligned_rect_area(&mut self.area, rect);
DrawStep::done()
}
fn handle_event(&mut self, cx: &mut Cx, event: &Event, _scope: &mut Scope) {
match event.hits(cx, self.area) {
Hit::FingerDown(event) if event.device.is_primary_hit() => {
self.drag = Some(PatternDrag {
from: event.abs,
pan: self.pan,
});
}
Hit::FingerMove(event) => {
if let Some(drag) = self.drag {
self.pan = drag.pan + event.abs - drag.from;
self.redraw(cx);
}
}
Hit::FingerUp(_) => self.drag = None,
Hit::FingerScroll(event) => {
self.zoom = (self.zoom * (-event.scroll.y * 0.004).exp()).clamp(0.1, 30.0);
self.redraw(cx);
}
Hit::FingerHoverIn(_) | Hit::FingerHoverOver(_) => {
cx.set_cursor(MouseCursor::Grab)
}
_ => {}
}
}
}
fn place(point: Point, offset: Point, rotation_deg: f64) -> DVec2 {
let angle = rotation_deg.to_radians();
let (sin, cos) = angle.sin_cos();
dvec2(
point.x * cos - point.y * sin + offset.x,
point.x * sin + point.y * cos + offset.y,
)
}
fn pattern_bounds(pattern: &Pattern, layout: &PatternLayout) -> Bounds {
let mut bounds = Bounds::default();
for (part_index, part) in pattern.parts.iter().enumerate() {
let (part_offset, rotation) = layout
.placements
.iter()
.find(|placement| placement.part == part_index)
.map(|placement| (placement.offset, placement.rotation_deg))
.unwrap_or((Point::default(), 0.0));
include_part(&mut bounds, part, part_offset, rotation);
}
if !bounds.valid && layout.width_mm > 0.0 && layout.height_mm > 0.0 {
bounds.include(dvec2(0.0, 0.0));
bounds.include(dvec2(layout.width_mm, layout.height_mm));
}
bounds
}
fn include_part(bounds: &mut Bounds, part: &Part, part_offset: Point, rotation: f64) {
let cut = offset(&part.outline, part.seam_allowance_mm);
for point in flatten(&cut, 1.0) {
bounds.include(place(point, part_offset, rotation));
}
}

676
apps/fabric/src/pipeline.rs Normal file
View file

@ -0,0 +1,676 @@
use crate::{
body_view::{map_measurements_to_vertices, BodyPoseMapping},
camera::CameraMailbox,
};
use makepad_ai_body::model::BodyModel;
use makepad_fabric_measure::{measure, BodyMesh, MeasureOptions, Measured};
use makepad_widgets::image_cache::ImageBuffer;
use makepad_widgets::makepad_platform::thread::SignalToUI;
use std::{
collections::VecDeque,
path::{Path, PathBuf},
sync::{
atomic::{AtomicBool, Ordering},
mpsc::{self, Receiver, Sender},
Arc,
},
thread,
time::{Duration, Instant},
};
const PHOTO_CROP_SIZE: usize = 512;
const LIVE_CROP_SIZE: usize = 384;
const SHAPE_ALPHA: f32 = 0.35;
const POSE_ALPHA: f32 = 0.6;
const SHAPE_RESET_GAP: Duration = Duration::from_secs(1);
const FRAME_TIMEOUT: Duration = Duration::from_secs(2);
const FRAME_POLL: Duration = Duration::from_millis(5);
pub enum PipelineMessage {
Stage(String),
LiveFrame {
fps: f32,
model_ms: f32,
pose_ms: f32,
person: bool,
bbox: Option<[f32; 4]>,
},
Done {
measured: Box<Measured>,
mesh: Arc<BodyMesh>,
posed: Option<Arc<Vec<[f32; 3]>>>,
pose_mapping: BodyPoseMapping,
reset_pose: bool,
},
Failed(String),
}
struct RunRequest {
photo: PathBuf,
weights: PathBuf,
height_cm: Option<f32>,
}
struct LiveRequest {
weights: PathBuf,
height_cm: Option<f32>,
mailbox: CameraMailbox,
}
enum WorkerRequest {
Photo(RunRequest),
Live(LiveRequest),
}
pub struct Pipeline {
request_tx: Sender<WorkerRequest>,
message_rx: Receiver<PipelineMessage>,
live_stop: Arc<AtomicBool>,
}
impl Pipeline {
pub fn new() -> Self {
let (request_tx, request_rx) = mpsc::channel();
let (message_tx, message_rx) = mpsc::channel();
let live_stop = Arc::new(AtomicBool::new(true));
let worker_stop = live_stop.clone();
thread::Builder::new()
.name("fabric-body-pipeline".to_string())
.spawn(move || worker(request_rx, message_tx, worker_stop))
.expect("spawn fabric body worker");
Self {
request_tx,
message_rx,
live_stop,
}
}
pub fn run(
&self,
photo: PathBuf,
weights: PathBuf,
height_cm: Option<f32>,
) -> Result<(), String> {
self.request_tx
.send(WorkerRequest::Photo(RunRequest {
photo,
weights,
height_cm,
}))
.map_err(|_| "the body model worker stopped".to_string())
}
pub fn start_live(
&self,
weights: PathBuf,
height_cm: Option<f32>,
mailbox: CameraMailbox,
) -> Result<(), String> {
self.live_stop.store(false, Ordering::Release);
if self
.request_tx
.send(WorkerRequest::Live(LiveRequest {
weights,
height_cm,
mailbox,
}))
.is_err()
{
self.live_stop.store(true, Ordering::Release);
return Err("the body model worker stopped".to_string());
}
Ok(())
}
pub fn stop_live(&self) {
self.live_stop.store(true, Ordering::Release);
}
pub fn poll(&self) -> Vec<PipelineMessage> {
self.message_rx.try_iter().collect()
}
}
fn emit(sender: &Sender<PipelineMessage>, message: PipelineMessage) -> bool {
if sender.send(message).is_err() {
return false;
}
SignalToUI::set_ui_signal();
true
}
fn worker(
requests: Receiver<WorkerRequest>,
messages: Sender<PipelineMessage>,
live_stop: Arc<AtomicBool>,
) {
let mut loaded: Option<(PathBuf, BodyModel)> = None;
while let Ok(request) = requests.recv() {
let result = match request {
WorkerRequest::Photo(request) => run_one(&request, &messages, &mut loaded),
WorkerRequest::Live(request) => {
run_live(&request, &messages, &mut loaded, &live_stop)
}
};
if let Err(error) = result {
if !emit(&messages, PipelineMessage::Failed(error)) {
return;
}
}
}
}
fn run_one(
request: &RunRequest,
messages: &Sender<PipelineMessage>,
loaded: &mut Option<(PathBuf, BodyModel)>,
) -> Result<(), String> {
emit(messages, PipelineMessage::Stage("decoding photo…".to_string()));
let (rgb, width, height) = decode_rgb(&request.photo)?;
ensure_model(&request.weights, messages, loaded)?;
emit(messages, PipelineMessage::Stage("inferring…".to_string()));
let model = &mut loaded.as_mut().expect("model was loaded above").1;
model
.set_crop_size(PHOTO_CROP_SIZE)
.map_err(|error| format!("could not configure the body model: {error}"))?;
let packet = model
.infer(&rgb, width, height, None)
.map_err(|error| format!("body inference failed: {error}"))?;
let person = packet
.people
.first()
.ok_or_else(|| "no person found".to_string())?;
let posed = Arc::new(posed_vertices(
model,
&person.shape,
&person.expr,
person.mhr,
person.global_rot,
)?);
let vertices = model.rig().rest_vertices(&person.shape, &person.expr);
let face_indices = model
.weights
.i64_shaped("head_pose.faces", &[36_874, 3])
.map_err(|error| format!("could not read the body mesh faces: {error}"))?;
let mesh = Arc::new(body_mesh_from_flat(&vertices, &face_indices)?);
emit(messages, PipelineMessage::Stage("measuring…".to_string()));
let measured = measure(
&mesh,
&MeasureOptions {
height_cm: request.height_cm,
},
)
.map_err(|error| error.to_string())?;
let pose_mapping = map_measurements_to_vertices(&mesh, &measured);
emit(
messages,
PipelineMessage::Done {
measured: Box::new(measured),
mesh,
posed: Some(posed),
pose_mapping,
reset_pose: true,
},
);
Ok(())
}
fn ensure_model(
weights: &Path,
messages: &Sender<PipelineMessage>,
loaded: &mut Option<(PathBuf, BodyModel)>,
) -> Result<(), String> {
if loaded
.as_ref()
.map(|(path, _)| path != weights)
.unwrap_or(true)
{
emit(
messages,
PipelineMessage::Stage("loading model 2.8 GB…".to_string()),
);
let model = BodyModel::load(weights)
.map_err(|error| format!("could not load the body model: {error}"))?;
*loaded = Some((weights.to_path_buf(), model));
}
Ok(())
}
fn run_live(
request: &LiveRequest,
messages: &Sender<PipelineMessage>,
loaded: &mut Option<(PathBuf, BodyModel)>,
stop: &AtomicBool,
) -> Result<(), String> {
if stop.load(Ordering::Acquire) {
return Ok(());
}
ensure_model(&request.weights, messages, loaded)?;
let model = &mut loaded.as_mut().expect("model was loaded above").1;
model
.set_crop_size(LIVE_CROP_SIZE)
.map_err(|error| format!("could not configure live body inference: {error}"))?;
let face_indices = model
.weights
.i64_shaped("head_pose.faces", &[36_874, 3])
.map_err(|error| format!("could not read the body mesh faces: {error}"))?;
let started = Instant::now();
let mut previous_bbox = None;
let mut smoother = ShapeSmoother::default();
let mut pose_smoother = PoseSmoother::default();
let mut fps = FpsCounter::default();
let mut reset_pose = true;
while !stop.load(Ordering::Acquire) {
request.mailbox.request();
let wait_started = Instant::now();
let frame = loop {
if stop.load(Ordering::Acquire) {
return Ok(());
}
if let Some(frame) = request.mailbox.take() {
break Some(frame);
}
if wait_started.elapsed() >= FRAME_TIMEOUT {
break None;
}
thread::sleep(FRAME_POLL);
};
let Some(frame) = frame else {
if !emit(
messages,
PipelineMessage::Stage("no camera frames".to_string()),
) {
return Ok(());
}
continue;
};
let model_started = Instant::now();
let packet = model
.infer(&frame.rgb, frame.width, frame.height, previous_bbox)
.map_err(|error| format!("live body inference failed: {error}"))?;
let model_ms = model_started.elapsed().as_secs_f32() * 1000.0;
let now = started.elapsed();
let person = packet.people.first();
let bbox = person.map(|person| person.bbox);
let mut pose_ms = 0.0;
let mut posed = None;
let smoothed_shape = if let Some(person) = person {
previous_bbox = Some(expand_bbox(
person.bbox,
frame.width,
frame.height,
0.15,
));
let (mhr, global_rot) = pose_smoother.observe(person.mhr, person.global_rot);
let pose_started = Instant::now();
posed = Some(Arc::new(posed_vertices(
model,
&person.shape,
&person.expr,
mhr,
global_rot,
)?));
pose_ms = pose_started.elapsed().as_secs_f32() * 1000.0;
Some(smoother.observe_person(person.shape, now))
} else {
previous_bbox = None;
smoother.observe_miss(now);
pose_smoother.reset();
None
};
let current_fps = fps.tick(now);
if !emit(
messages,
PipelineMessage::LiveFrame {
fps: current_fps,
model_ms,
pose_ms,
person: person.is_some(),
bbox,
},
) {
return Ok(());
}
if stop.load(Ordering::Acquire) {
return Ok(());
}
let Some(shape) = smoothed_shape else {
continue;
};
let expression = [0.0f32; 72];
let vertices = model.rig().rest_vertices(&shape, &expression);
let mesh = Arc::new(body_mesh_from_flat(&vertices, &face_indices)?);
let measured = measure(
&mesh,
&MeasureOptions {
height_cm: request.height_cm,
},
)
.map_err(|error| error.to_string())?;
let pose_mapping = map_measurements_to_vertices(&mesh, &measured);
if !emit(
messages,
PipelineMessage::Done {
measured: Box::new(measured),
mesh,
posed,
pose_mapping,
reset_pose,
},
) {
return Ok(());
}
reset_pose = false;
}
Ok(())
}
#[derive(Default)]
struct PoseSmoother {
mhr: Option<[f32; 204]>,
global_rot: Option<[f32; 3]>,
}
impl PoseSmoother {
fn observe(
&mut self,
mhr: [f32; 204],
global_rot: [f32; 3],
) -> ([f32; 204], [f32; 3]) {
let global_rot = match self.global_rot {
Some(previous) => std::array::from_fn(|index| {
previous[index] + POSE_ALPHA * (global_rot[index] - previous[index])
}),
None => global_rot,
};
let mut mhr = match self.mhr {
Some(previous) => std::array::from_fn(|index| {
previous[index] + POSE_ALPHA * (mhr[index] - previous[index])
}),
None => mhr,
};
// The packet's 204 values are exactly the rig's [pose 136 | scales 68]
// input. Global rotation is pose slots 3..6; keep the separately
// smoothed copy authoritative before MhrRig::forward pads to 249.
mhr[3..6].copy_from_slice(&global_rot);
self.mhr = Some(mhr);
self.global_rot = Some(global_rot);
(mhr, global_rot)
}
fn reset(&mut self) {
self.mhr = None;
self.global_rot = None;
}
}
#[derive(Default)]
struct ShapeSmoother {
shape: Option<[f32; 45]>,
last_person: Option<Duration>,
}
impl ShapeSmoother {
fn observe_person(&mut self, shape: [f32; 45], now: Duration) -> [f32; 45] {
if self
.last_person
.is_some_and(|last| now.saturating_sub(last) > SHAPE_RESET_GAP)
{
self.shape = None;
}
self.last_person = Some(now);
let smoothed = match self.shape {
Some(previous) => std::array::from_fn(|index| {
previous[index] + SHAPE_ALPHA * (shape[index] - previous[index])
}),
None => shape,
};
self.shape = Some(smoothed);
smoothed
}
fn observe_miss(&mut self, now: Duration) {
if self
.last_person
.is_some_and(|last| now.saturating_sub(last) > SHAPE_RESET_GAP)
{
self.shape = None;
self.last_person = None;
}
}
}
#[derive(Default)]
struct FpsCounter {
samples: VecDeque<Duration>,
}
impl FpsCounter {
fn tick(&mut self, now: Duration) -> f32 {
self.samples.push_back(now);
while self.samples.len() > 10 {
self.samples.pop_front();
}
let Some(first) = self.samples.front().copied() else {
return 0.0;
};
let seconds = now.saturating_sub(first).as_secs_f32();
if seconds <= f32::EPSILON {
0.0
} else {
(self.samples.len().saturating_sub(1)) as f32 / seconds
}
}
}
pub(crate) fn expand_bbox(
bbox: [f32; 4],
width: u32,
height: u32,
amount: f32,
) -> [f32; 4] {
let box_width = (bbox[2] - bbox[0]).max(0.0);
let box_height = (bbox[3] - bbox[1]).max(0.0);
[
(bbox[0] - box_width * amount).clamp(0.0, width as f32),
(bbox[1] - box_height * amount).clamp(0.0, height as f32),
(bbox[2] + box_width * amount).clamp(0.0, width as f32),
(bbox[3] + box_height * amount).clamp(0.0, height as f32),
]
}
fn posed_vertices(
model: &BodyModel,
shape: &[f32; 45],
expression: &[f32; 72],
mut mhr: [f32; 204],
global_rot: [f32; 3],
) -> Result<Vec<[f32; 3]>, String> {
// BodyPerson::mhr is already model_params(): pose 0..136 followed by
// scales 136..204. The root translation remains zero and the model's
// global rotation occupies 3..6. forward() pads the 45 identity slots
// to the rig's 249-wide internal vector, then returns rig-space cm.
mhr[3..6].copy_from_slice(&global_rot);
let output = model.rig().forward(shape, &mhr, expression, true);
vertices_from_flat(&output.verts)
}
fn decode_rgb(path: &Path) -> Result<(Vec<u8>, u32, u32), String> {
let bytes = std::fs::read(path)
.map_err(|error| format!("could not read {}: {error}", path.display()))?;
let extension = path
.extension()
.and_then(|extension| extension.to_str())
.unwrap_or_default()
.to_ascii_lowercase();
let image = match extension.as_str() {
"jpg" | "jpeg" => ImageBuffer::from_jpg(&bytes),
"png" => ImageBuffer::from_png(&bytes),
_ => return Err("choose a JPG or PNG photo".to_string()),
}
.map_err(|error| format!("could not decode {}: {error}", path.display()))?;
let pixel_count = image
.width
.checked_mul(image.height)
.ok_or_else(|| "photo dimensions are too large".to_string())?;
if image.data.len() < pixel_count {
return Err("decoded photo has too few pixels".to_string());
}
let width = u32::try_from(image.width).map_err(|_| "photo is too wide".to_string())?;
let height = u32::try_from(image.height).map_err(|_| "photo is too tall".to_string())?;
Ok((argb_to_rgb(&image.data[..pixel_count]), width, height))
}
pub(crate) fn argb_to_rgb(pixels: &[u32]) -> Vec<u8> {
let mut rgb = Vec::with_capacity(pixels.len() * 3);
for pixel in pixels {
rgb.push((pixel >> 16) as u8);
rgb.push((pixel >> 8) as u8);
rgb.push(*pixel as u8);
}
rgb
}
pub(crate) fn faces_i64_to_u32(
values: &[i64],
vertex_count: usize,
) -> Result<Vec<[u32; 3]>, String> {
if values.len() % 3 != 0 {
return Err("body face index buffer is not made of triangles".to_string());
}
values
.chunks_exact(3)
.enumerate()
.map(|(face_index, triangle)| {
let mut face = [0; 3];
for corner in 0..3 {
let index = usize::try_from(triangle[corner]).map_err(|_| {
format!("body face {face_index} contains a negative vertex index")
})?;
if index >= vertex_count {
return Err(format!(
"body face {face_index} references vertex {index}, but there are {vertex_count} vertices"
));
}
face[corner] = u32::try_from(index)
.map_err(|_| format!("body vertex index {index} exceeds u32"))?;
}
Ok(face)
})
.collect()
}
pub(crate) fn body_mesh_from_flat(
vertices: &[f32],
face_indices: &[i64],
) -> Result<BodyMesh, String> {
let vertices = vertices_from_flat(vertices)?;
let faces = faces_i64_to_u32(face_indices, vertices.len())?;
Ok(BodyMesh {
vertices,
faces,
landmarks: None,
})
}
fn vertices_from_flat(vertices: &[f32]) -> Result<Vec<[f32; 3]>, String> {
if vertices.is_empty() || vertices.len() % 3 != 0 {
return Err("body vertex buffer is empty or malformed".to_string());
}
Ok(vertices
.chunks_exact(3)
.map(|point| [point[0], point[1], point[2]])
.collect())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn converts_aarrggbb_to_rgb() {
assert_eq!(
argb_to_rgb(&[0xff_12_34_56, 0x00_ab_cd_ef]),
vec![0x12, 0x34, 0x56, 0xab, 0xcd, 0xef]
);
}
#[test]
fn builds_a_tiny_checked_mesh() {
let mesh = body_mesh_from_flat(
&[0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0],
&[0, 1, 2],
)
.unwrap();
assert_eq!(mesh.vertices.len(), 3);
assert_eq!(mesh.faces, vec![[0, 1, 2]]);
assert!(faces_i64_to_u32(&[0, 1, 3], 3).is_err());
assert!(faces_i64_to_u32(&[-1, 1, 2], 3).is_err());
}
#[test]
fn live_shape_ema_converges_and_resets_after_a_gap() {
let mut smoother = ShapeSmoother::default();
assert_eq!(
smoother.observe_person([0.0; 45], Duration::ZERO),
[0.0; 45]
);
let first = smoother.observe_person([10.0; 45], Duration::from_millis(250));
assert!((first[0] - 3.5).abs() < 0.0001);
let second = smoother.observe_person([10.0; 45], Duration::from_millis(500));
assert!((second[0] - 5.775).abs() < 0.0001);
smoother.observe_miss(Duration::from_millis(1_501));
let reset = smoother.observe_person([8.0; 45], Duration::from_millis(1_750));
assert_eq!(reset, [8.0; 45]);
}
#[test]
fn live_pose_ema_smooths_parameters_and_keeps_global_slots_in_sync() {
let mut smoother = PoseSmoother::default();
let mut initial_mhr = [0.0; 204];
initial_mhr[20] = 2.0;
let (initial, initial_rot) = smoother.observe(initial_mhr, [1.0, 2.0, 3.0]);
assert_eq!(initial[3..6], initial_rot);
let mut next_mhr = [10.0; 204];
next_mhr[20] = 12.0;
let (smoothed, smoothed_rot) = smoother.observe(next_mhr, [3.0, 4.0, 5.0]);
assert!((smoothed[20] - 8.0).abs() < 0.0001);
assert_eq!(smoothed_rot, [2.2, 3.2, 4.2]);
assert_eq!(smoothed[3..6], smoothed_rot);
smoother.reset();
let (reset, _) = smoother.observe([7.0; 204], [0.5, 1.0, 1.5]);
assert_eq!(reset[20], 7.0);
}
#[test]
fn bbox_expansion_is_fifteen_percent_and_clamped() {
let expanded = expand_bbox([100.0, 50.0, 300.0, 250.0], 640, 360, 0.15);
for (actual, expected) in expanded.into_iter().zip([70.0, 20.0, 330.0, 280.0]) {
assert!((actual - expected).abs() < 0.0001, "{expanded:?}");
}
assert_eq!(
expand_bbox([5.0, 10.0, 635.0, 350.0], 640, 360, 0.15),
[0.0, 0.0, 640.0, 360.0]
);
}
#[test]
fn fps_counter_tracks_recent_frame_cadence() {
let mut counter = FpsCounter::default();
assert_eq!(counter.tick(Duration::ZERO), 0.0);
for quarter in 1..=4 {
let fps = counter.tick(Duration::from_millis(quarter * 250));
assert!((fps - 4.0).abs() < 0.0001, "{fps}");
}
}
}

25
apps/files/Cargo.toml Normal file
View file

@ -0,0 +1,25 @@
[package]
name = "makepad-files"
version = "0.1.0"
edition = "2021"
[[bin]]
name = "files"
path = "src/main.rs"
[features]
default = ["chat"]
chat = ["dep:makepad-ai-hub"]
demo = []
[dependencies]
makepad-widgets = { path = "../../widgets" }
makepad-wm-theme = { path = "../../libs/wm_theme" }
makepad-wm-api = { path = "../../libs/wm_api" }
# The AI services wire: bounded read and confirmed mutation tools on the bus
# (hosted port, id-correlated runner — see ai_service.rs).
makepad-ai-services = { path = "../../libs/ai/services" }
makepad-strict-json = { path = "../../libs/strict_json" }
# The ask panel's local model: an in-process Qwen GGUF on makepad-ggml, loaded
# only when the panel is first opened.
makepad-ai-hub = { path = "../../libs/ai/hub", default-features = false, features = ["llm"], optional = true }

64
apps/files/build.rs Normal file
View file

@ -0,0 +1,64 @@
#[path = "../../libs/ai/models/paint/src/png.rs"]
#[allow(dead_code)]
mod png;
use std::{env, fs, path::Path};
const WIDTH: u32 = 256;
const HEIGHT: u32 = 128;
const PALETTES: [(&str, [u8; 3], [u8; 3], [u8; 3]); 6] = [
("aurora-vignette.png", [28, 31, 78], [69, 175, 170], [227, 151, 232]),
("canyon-vignette.png", [86, 35, 38], [221, 119, 70], [255, 211, 128]),
("lagoon-vignette.png", [12, 55, 77], [28, 154, 166], [151, 232, 207]),
("meadow-vignette.png", [36, 68, 45], [124, 167, 79], [234, 213, 126]),
("twilight-vignette.png", [36, 25, 65], [103, 71, 141], [238, 144, 116]),
("cinema-still.png", [19, 24, 39], [56, 75, 105], [235, 176, 91]),
];
fn mix(a: u8, b: u8, amount: u32) -> u32 {
(u32::from(a) * (255 - amount) + u32::from(b) * amount) / 255
}
fn picture(top: [u8; 3], bottom: [u8; 3], accent: [u8; 3], seed: u32) -> Vec<u8> {
let mut pixels = Vec::with_capacity((WIDTH * HEIGHT * 3) as usize);
let glow_x = 36 + seed * 31;
let glow_y = 22 + (seed * 17) % 54;
for y in 0..HEIGHT {
for x in 0..WIDTH {
let blend = (y * 190 / (HEIGHT - 1) + x * 65 / (WIDTH - 1)).min(255);
let edge_x = (x as i32 * 2 - (WIDTH - 1) as i32).unsigned_abs();
let edge_y = (y as i32 * 2 - (HEIGHT - 1) as i32).unsigned_abs();
let vignette = edge_x * edge_x * 18 / ((WIDTH - 1) * (WIDTH - 1))
+ edge_y * edge_y * 24 / ((HEIGHT - 1) * (HEIGHT - 1));
let glow_dx = (x as i32 - glow_x as i32).unsigned_abs();
let glow_dy = (y as i32 - glow_y as i32).unsigned_abs();
let glow = 72u32.saturating_sub(glow_dx * glow_dx / 360 + glow_dy * glow_dy / 90);
let texture = ((x * 17 + y * 31 + seed * 43) & 7) as i32 - 3;
for channel in 0..3 {
let base = mix(top[channel], bottom[channel], blend);
let lit = (base * (72 - glow) + u32::from(accent[channel]) * glow) / 72;
pixels.push((lit as i32 - vignette as i32 + texture).clamp(0, 255) as u8);
}
}
}
pixels
}
fn write_if_changed(path: &Path, bytes: &[u8]) {
if fs::read(path).ok().as_deref() != Some(bytes) {
fs::write(path, bytes).expect("write generated files demo picture");
}
}
fn main() {
println!("cargo:rerun-if-changed=../../libs/ai/models/paint/src/png.rs");
println!("cargo:rerun-if-changed=build.rs");
let out_dir = env::var_os("OUT_DIR").expect("Cargo did not set OUT_DIR");
for (seed, (name, top, bottom, accent)) in PALETTES.iter().copied().enumerate() {
let pixels = picture(top, bottom, accent, seed as u32);
let encoded = png::encode_png(WIDTH, HEIGHT, png::PngColor::Rgb, &pixels);
write_if_changed(&Path::new(&out_dir).join(name), &encoded);
}
}

View file

Before

Width:  |  Height:  |  Size: 307 KiB

After

Width:  |  Height:  |  Size: 307 KiB

Before After
Before After

Binary file not shown.

After

Width:  |  Height:  |  Size: 374 KiB

View file

Before

Width:  |  Height:  |  Size: 143 KiB

After

Width:  |  Height:  |  Size: 143 KiB

Before After
Before After

View file

@ -0,0 +1,6 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 56">
<path fill="#e0af68" d="M7 2h22l12 12v38c0 1.1-.9 2-2 2H7c-1.1 0-2-.9-2-2V4c0-1.1.9-2 2-2z"/>
<path fill="#f0d2a6" d="M29 2v12h12z"/>
<path fill="#3a2c14" d="M20 2h5v5h-5zm5 5h5v5h-5zm-5 5h5v5h-5zm5 5h5v5h-5zm-5 5h5v5h-5zm5 5h5v5h-5zm-6 6h7a2 2 0 0 1 2 2v8a2 2 0 0 1-2 2h-7a2 2 0 0 1-2-2v-8a2 2 0 0 1 2-2z"/>
<path fill="#f0d2a6" d="M22 37h4v5h-4z"/>
</svg>

After

Width:  |  Height:  |  Size: 427 B

View file

@ -0,0 +1,5 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 56">
<path fill="#bb9af7" d="M7 2h22l12 12v38c0 1.1-.9 2-2 2H7c-1.1 0-2-.9-2-2V4c0-1.1.9-2 2-2z"/>
<path fill="#d5c0fb" d="M29 2v12h12z"/>
<path fill="#2a2140" d="M32 22v18a6 5 0 1 1-4-4.7V27l-11 2.6v13.9a6 5 0 1 1-4-4.7V27z"/>
</svg>

After

Width:  |  Height:  |  Size: 297 B

View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="square" stroke-linejoin="miter" d="m14 5-7 7 7 7M7 12h11"/></svg>

After

Width:  |  Height:  |  Size: 199 B

View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M6 2h12a1 1 0 0 1 1 1v19l-7-4.4L5 22V3a1 1 0 0 1 1-1"/></svg>

After

Width:  |  Height:  |  Size: 130 B

View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M2 3h20v13H10l-5 5v-5H2zM4 5v9h3v3l3-3h10V5zM6.5 8h2v2h-2zm4 0h2v2h-2zm4 0h2v2h-2z"/></svg>

After

Width:  |  Height:  |  Size: 160 B

View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.3 19.4 1.6 11.7l2.8-2.8 4.9 4.9L19.6 3.5l2.8 2.8z"/></svg>

After

Width:  |  Height:  |  Size: 130 B

View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><circle cx="12" cy="12" r="9" fill="none" stroke="currentColor" stroke-width="2"/><path d="M12 6v6l4 2" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="square"/></svg>

After

Width:  |  Height:  |  Size: 247 B

View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="m5.6 3.5 6.4 6.4 6.4-6.4 2.1 2.1-6.4 6.4 6.4 6.4-2.1 2.1-6.4-6.4-6.4 6.4-2.1-2.1 6.4-6.4-6.4-6.4z"/></svg>

After

Width:  |  Height:  |  Size: 175 B

View file

@ -0,0 +1,5 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 56">
<path fill="#7aa2f7" d="M7 2h22l12 12v38c0 1.1-.9 2-2 2H7c-1.1 0-2-.9-2-2V4c0-1.1.9-2 2-2z"/>
<path fill="#a9c2fb" d="M29 2v12h12z"/>
<path fill="none" stroke="#16161e" stroke-width="3" stroke-linecap="round" stroke-linejoin="round" d="m19 27-6 7 6 7m10-14 6 7-6 7"/>
</svg>

After

Width:  |  Height:  |  Size: 342 B

View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 4h2.5v2.5H3zm4.5 0H21v2.5H7.5zM3 10.75h2.5v2.5H3zm4.5 0H21v2.5H7.5zM3 17.5h2.5V20H3zm4.5 0H21V20H7.5z"/></svg>

After

Width:  |  Height:  |  Size: 183 B

View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M5 7h14l-1 14H6zM3 4h18v3H3zm6-3h6l1 3H8zM8.43 11.31 11.12 14l-2.69 2.69.88.88L12 14.88l2.69 2.69.88-.88L12.88 14l2.69-2.69-.88-.88L12 13.12 9.31 10.43z"/></svg>

After

Width:  |  Height:  |  Size: 230 B

View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 5c5 0 9 4.2 10 7-1 2.8-5 7-10 7S3 14.8 2 12c1-2.8 5-7 10-7zm0 3a4 4 0 1 0 0 8 4 4 0 0 0 0-8zm0 2.5a1.5 1.5 0 1 1 0 3 1.5 1.5 0 0 1 0-3z"/></svg>

After

Width:  |  Height:  |  Size: 218 B

View file

@ -0,0 +1,5 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 56">
<path fill="#8992ad" d="M7 2h22l12 12v38c0 1.1-.9 2-2 2H7c-1.1 0-2-.9-2-2V4c0-1.1.9-2 2-2z"/>
<path fill="#b4bdd2" d="M29 2v12h12z"/>
<path stroke="#5f6880" stroke-width="3" stroke-linecap="round" d="M13 25h20M13 33h20M13 41h14"/>
</svg>

After

Width:  |  Height:  |  Size: 305 B

View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 4h18l-7 8.5V20l-4-2.8v-4.7z"/></svg>

After

Width:  |  Height:  |  Size: 109 B

Some files were not shown because too many files have changed in this diff Show more