Squash of 59 work commits (Sep 1–12):
b0380ba image-tiles: the picture-wall engine as a library — tape atlases, a baker CLI, and the TileGrid widget
83d8eac mpsheets: demo feature and the SheetDocs seam
daff390 finance: demo feature — generated ledger, SQLite target-gated, import hidden
13b4c48 mpfiles: demo build — procedural fake filesystem, still-image thumbnails, chat feature
58e1f72 mpsheets: review fixes for the demo seam
3203e32 finance: review fixes — id remapping on persist, structural determinism tests, one cfg seam
0537687 mpfiles: distinct repo-owned pictures for the demo thumbnail pool
fe920c8 files: review fixes — demo scan exclusions, Zipf sizes, trash root guarded, depth bound, tests
4144dc4 files: the spy test filesystem implements the clock
1940f3d fonts: leave apps/wm untouched — its font-set declaration waits for the aichat lane to land
b0e34c2 wm: the AI pane and its bus — the aichat child seated in the slot, F10, the os service with the typed open
de937ac route: the web build is a one-to-one recompile — native UI, service seams behind it
2f5a984 files: the file browser on the AI bus — four read-only tools through a correlated, cancellable runner
2b035ff route: the navigation session keeps time on the platform clock — the std clock traps on wasm
1b70899 files: the space view is where a tab starts
412b3c9 files: the space view rescans through the virtual filesystem — the web has no other disk
0105fa5 files: the space view opens in 2.5D
5d007e0 files: an unset projection preference means 2.5D
6857d86 files: the projected treemap clips boxes at the near plane and keeps the camera above them
d3cd233 wm: one desk for every target — the host seam, the assistant seated in-process, the web profile, and the assistant up at boot
6d47c3f wm: the omarchy themes moved to omacom/omarchy — the importer follows
82def6b wm: every app under the desk — the chat keeps the keyboard through an automatic refocus, a pool that gives up, polite closes for browser, sheets and aichat
6105b61 sheets + files: the assistant reads and writes cells, makes folders, renames and trashes
82ac768 photos: the picture wall as an app — the SMBC archive on the tile engine, a module from its first line; fabric in the launcher
190062e route: the maps app exposes its tool table to the desktop assistant
3768653 image_tiles: the wall is box-packed — justified rows keep every picture's own shape and fill the width
3691875 wm: launcher icons — a photo for Photos, a shirt for Fabric, a globe for Route, a play badge for Video, a pulse for the task manager
8625076 aichat + photos + image_tiles: make me a picture and it lands on the wall; the wall filters as you type, tiles flying to their places
1dcca41 wm: a theme with no wallpapers fetches them on its own
ff6cfec wm: minimise, maximise and close in the bar's top right on Windows and Linux
62f38a5 route: --hour=N pins the theme hour for harness grabs
362ac59 wm: the checkout is found from the working directory too, and a sibling binary is an .exe on Windows
0fdfba7 files: the web's makepad home is a fixed virtual path — std's temp_dir panics there
daf88e1 route demo: the tiles come from the repacked world archive at its own never-cached path
15c3b11 ai-services: what the pubsub review found — modules get a subscription seam and a publish sink through both WM paths, lease end flushes unsubscribes to hosted services, the subscription cap counts closing rows, an endpoint's queue is dropped after Unregister, the prompt drops a subscription on final
d852699 route: no tile-source dropdown — makepad.nl is the tile source through the local range cache; a world.mkmap in the saved maps folder is the only override; the stale preference file is removed on start
aa271c3 map + route + geodata: the Terrain layer through the archive plane — TerrainSource (hosted .mkmap elevation shards fetched by range through the shared reader, a local MBTiles only as a dev override), the hillshade rendered on the pool's heavy lane with reused TerrainScratch buffers (web capped at 2048×1536 at DPR 1, native DPR-aware), one request in flight; the demo checkbox now renders terrain like native
dbc7838 route: the maps-folder field and its save button are gone from the settings panel; the maps root is automatic
a53d9b6 route: no clock-driven night theme — the theme is the user's toggle, remembered in cx.storage, default day (the hour rule fired on the web for the first time once wasm had a clock and darkened the map after 19:00)
ca92ff9 route: the open sea on the web — the ocean-low/ocean-high overlays are an always-on group in the shared hosted overlay table, read through the archive plane (makepad.nl, cached); native keeps the local ocean files only as a dev override; the native-only OCEAN_MBTILES path is gone
4cd24c4 route demo: the first location fix flies the map to the user at zoom 14, as native does; Amsterdam stays the default until a fix arrives
2e869f8 route: the ocean-high overlay comes from the re-sharded archive (58 shards of at most 16 MB, one small leaf directory each) instead of two shards whose directories decoded to 407 MiB
8021bb0 Add Clock and Weather with shared mobile tile views and persistent alarms
77b8309 Make application layouts and custom widgets follow desktop and mobile themes
b357164 Add desktop and mobile compositor shells with OS switching and dock transitions
11dec2e Keep hosted app output readers off the shared task pool
c4002c5 Round complete desktop surfaces and add Windows snap layouts
a5f8dd8 Frame home photo tiles and clip the picture wall below app controls
678721d Give mobile app libraries real search focus and hosted keyboard input
4794cdf Keep window title fills opaque at the application surface join
2cc584e Match Windows 2000 and NeXTSTEP window chrome to original screenshots
4a876a8 Give Files navigation and storage tools a clearer hierarchy
7821c90 Keep prewarmed browsers in the active desktop appearance
5bfe694 Compact the Files toolbar and repair navigation and selection actions
44fef36 Preserve Photos subjects and zoom across host view and aspect changes
ae20efc Draw larger macOS traffic lights with centered hover symbols
2bc4d89 wm: the WM is a library plus a desktop binary, Linux controls, hosted tick pacing
d643eb4 apps: the in-process app wave — mail, notes, calendar, reminders, calculator; clock, weather, finance, photos and route as modules; civil time, read-only sqlite, Linux CEF
56c1dee wm: a timer beat missed while the child renders is serviced on its acknowledgement
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Apps ask the hub for a recognizer or a voice and get one; where it runs is
the hub's decision. AiHub::start_stt / start_tts return poll-driven
sessions shaped like the chat session. The Auto ladder is Whisper/Kokoro in
this process (weights present, machine election), on the machine node over
loopback, on a LAN node, else the OS engine; SpeechReach::Local is the
"don't reach out" knob. Audio always comes back as PCM: the app owns the
device.
Three layers:
- makepad-ai-speech is the whole speech model family, engines only.
libs/voice (Whisper + Silero VAD) folds in as the `whisper` and `vad`
modules next to kokoro and indextts, each a cargo feature; the Apple
bridges and the Speaker/VoiceTranscriber selection leave it.
- makepad-system-speech (new) is the OS speech services as blocking fns:
Apple SpeechAnalyzer/AVSpeechSynthesizer via Swift, Windows.Media.Speech*
on the vendored bindings, Android SpeechRecognizer/TextToSpeech through
MakepadSpeech.java (API 26 floor), espeak-ng on Linux. It models the two
STT shapes honestly: PCM in (Whisper, Apple) versus an engine that owns
the microphone (Android, Windows), with capabilities the caller reads.
- the hub grows speech sessions, in-process Whisper/Kokoro workers with the
residency election, a `whisper` wire backend (stt domain, registry entry
pinned to ggerganov/whisper.cpp) so a Mac can serve a Quest, and a
`language` field on the generate request.
Consumers: the Window voice input runs on an STT session and switches to
engine-mic mode when the recognizer owns the microphone; converse's
SpeechOutput is a lazily started TTS session plus a pump thread; route
drops its private speech copy for converse; vj's lyrics fallback and the
alignment bakes call the engines directly.
Verified here: speech-roundtrip through the real sessions (Apple voice in,
in-process Whisper on Metal out, 4.3% WER); system-speech-test TTS->STT
verbatim; hub/converse/system-speech unit tests; msvc, aarch64-android and
linux-gnu cross-checks; Java against android-34. Windows, Android and Linux
bridges are compile-checked only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Squashed from work:
- asset-ai: FastH3 4-step fast video backend; clip keyframes on the wire
- asset-ui: loop video chains — text→image→video that ends where it began
- h3: safetensors -> pruned-Q4_K GGUF quantizer for the 24GB DiT tiers
- h3_quant_gguf verify: row-error gates calibrated to the measured Q4_K floor
- asset-ai realtime: the feedback loop — the source anchors, the drifted frame inits
- asset-ai realtime: a feedback loop survives a resize and travels by default
- asset-ai realtime: the feedback loop frees itself from the feed handshake and pauses for its listener
- asset-ai realtime: the outbound encode leaves the loop's critical path
- asset-ai ocr: the ocr domain — Chandra 2 at page resolution, and the tower goes planner-owned
- llm slots: a lane can hold an image span — embedding prefill and a rope cursor of its own
- vision tower on CUDA: the encode leg gets its two missing kernels
- llm/ocr: one M-RoPE grid encoder for both image paths, and a livelock made an error
- vision tower on CUDA: the f16 GEMM keeps the precision it was throwing away
- live: a feed that moves box takes its trip with it — one seed image
- vision tower on CUDA: the tiled attention becomes bit-exact, and tensor cores go
- llm prefill on CUDA: the MMA attention kernel gets the tile a 4-to-1 model needs
- asset-ai ocr: the CUDA encode lane joins the integration — vision-parity sits beside run's three arms, and the kernels
- Merge branch 'ocr-perf-integration' into work
- asset-ai: the live anchor can follow the trip, and text leaves the 5090
- asset-ai: the camera moves the world, and the world starts still
- asset-import: the EA strategy classics, in the one 2D contract
- rtsmap: one seeded generator for tiled strategy maps
- asset-ui: one card for the strategy classics, with a pack dropdown
- asset-ai: music3 reference-audio path, ocr/h3 backends, registry
- asset: mp4 sample index for range-streaming, chat tools, import profiles
- cnc: tiberium is twelve growth frames, not twelve empty variants
- platform: native file and save dialogs, in-house on all three desktops
- chat: the scan holds out for a lane home
- chat: a full home queues you — take the free lane
- chat: the preload has a percentage, and the boundless cap stops showing
- llm cuda: the 32x2 attention tile — even GQA ratios stay on MMA
- sa3 gets a bake path: the sfx model's tables precomputed by a diffusion-side bin
- sqlite_query: anti-join regression test
- td import: HARV's second frame block is its harvesting cycle, not a turret
- asset-ui: sprite enhancement runs on the 32B dev DiT — distillation, not the prompt, was the ceiling
- ai-hub: makepad-asset-ai becomes makepad-ai-hub at libs/ai/hub, the chat pane becomes makepad-chat-ui, the service bin
- asset-ui: test health fixtures grow the realtime field they were born without
- ai-hub: one home at ~/.makepad — weights/ run/ cache/ logs/, the service cache migrates from ai_content by a single re
- ai-hub: subprocess workers die with the node — process groups everywhere, PDEATHSIG on linux, one KILL_ON_JOB_CLOSE Jo
- ai-hub: the hub object — AiHub::in_process, pipes vocabulary, and the local LLM engine generalized out of mpfiles (aic
- strict-json: the dependency-free JSON module gets its own crate; asset-client re-exports it so nothing downstream move
- ai-hub: the machine layer — node entries, the 0600 machine token, and the residency election that IS the lock (aicore
- ai-hub: MPHUB1 — the fabric beacon only dedicated nodes can send (aicore §4)
- ai-hub: job leases — work lives only while it is renewed (aicore §8)
- asset-creator: the pipeline library is born — specs, the deps gate, and the derived-state law (aicore §9)
- ai-hub: RAM residency facts — the CPU-side twin of residency.rs (aicore §3)
- ai-hub: ETA placement primitives — relative GPU throughput, the four-term estimate, and an observable breakdown (aicor
- ai-hub: leases go live on the wire — origin fields on submit, /job/<id>/keepalive, /bye, and the reaper that cancels w
- ai-hub: the chat providers move in — fleet qwen, openai, grok, claude/codex/grok CLIs, the responses driver, and the w
- asset-creator: the engine — one pipeline run against the hub, deps-gated, spliced, cancellable, resumable-by-construct
- ai-hub: the machine node mode — --machine binds loopback, registers in ~/.makepad/run, and exits on its own once idle
- asset-creator: makepad-creator-run — the detached client for runs that must outlive a window (aicore §9)
- ai-hub: a native Claude Messages-API provider — API-key or Claude Code OAuth, bounded SSE streaming, injected tools (a
- route + converse: off makepad_ai — the Agent seam moves to converse, route's cloud dispatcher rides the hub's Claude p
- asset-creator: the preset tables move in — fifteen chain-policy constants shared by every creator app (aicore §9 / P6)
- makepad_ai is deleted — every backend is a hub pipe, the agent seam lives with its consumers (aicore §14, decided 2026
- ai-hub: loads hold the machine residency election — set_model_state claims on Loaded and publishes the service port (a
- ai-hub: chats run the machine election — route to a serving holder, wait on a loading one, claim and publish when open
- ai-hub: pick_for_domain_eta — ETA-ranked placement over the shared hard-filter core (aicore §6 / P4)
- asset-creator: the engine picks a provider per stage at dispatch time — a chain's later stages see fresh fleet state (
- ai-hub: the fabric secret gates the service HTTP surface — bearer on everything but /health and the ticketed peer path
- vj: DREAM runs execute in the app — pipelines.rs becomes the run it used to watch (aicore §9 / F1)
- asset-creator: the runner — generate one thing and put it in the catalog, one implementation for every surface (aicore
- chat-ui: the session runs in the app — no broker anywhere on the chat path (aicore P8 / F5)
- asset-store: assets.query is a first-class query endpoint — the bounded SQL surface outlives the broker (aicore P8 / F
- asset-creator: CreatorTools — the chat tool pack for a store that only stores (aicore §9 / P8)
- asset-store: the shrink — the store stores (aicore P7)
- importer + asset-server host: the coordination era ends (aicore P7)
- store config purge + asset-ui goes fleet-direct; the derive protocol gets its route proof (aicore P7)
- client + chat dispatcher: the dead wire comes out (aicore P7/P8)
- ai-hub: 0.3.0 — the health version says which era a node runs
- ai-hub: the default fleet is 'gen' — apps hear the LAN without env plumbing
- ai-hub: the preload note percents the prefill, not the job bar
- ai-hub: conversations keep their KV — the wire mirror, the lane identity, the in-turn dynamic context (aicore §7)
- ai-hub: an open-think model is thinking from its first token
- libs: the zero-warning sweep — stitch casts say what they mean, xatlas keeps upstream's surface quietly
- zero-warning sweep, round two — the first full-workspace pass
- zero-warning sweep, round three — the model lanes and the deep examples
- zero-warning sweep, round four — the last stragglers
- zero-warning sweep, round five — vj and chat-ui
- zero-warning sweep, round six — three cascades
Co-authored-by: Claude <info@makepad.nl>
- README: a build quick-start for macOS and Windows, the honest Linux
story, what CUDA is for and how to install the separation model.
- AGENTS.md: the `--remote` control surface protocol, so the harness it
documents is usable without reading platform/src/remote.rs.
- Cargo.toml: workspace membership for the crates this series adds and
removes.
- tools/: the Windows box scripts (wincmd, winps and friends, winrun) and
remote_smoke.sh — how a build gets driven on a remote machine.
- apps/asset-server: the standalone server binary and its README.
- Small follow-ups in libs/{windows,apple_sys,makepad_test,mbtile_reader,
converse} and apps/route, plus .gitignore and makepad.splash.
Drop the archived old/ tree, root AI notes, splashgame.md (Arcade lives
in the private sandbox), widgets-dll leftovers, and LAN/oracle helper
scripts. Move download_*.sh into tools/, and move sim/math out of
libs/game/ so the public repo no longer has a game/ directory.
Also quiet first-party compile noise and skip asset-ui tests that need
sandbox kits or uncommitted fixtures.
libs/converse gains a default-on `tts` feature so gamemaker and route are
unaffected; Arcade takes default-features = false and re-enables it through
its existing `voice` feature. With tts off, SpeechOutput still exists and its
worker drains the queue silently, so no Arcade source changes were needed —
which also avoided colliding with concurrent edits to those files. No
capability is lost: speech is fully reachable under --features voice.
Arcade before 25.11 MB
Arcade after (default) 15.01 MB (-10.10 MB, -40%)
Arcade --features voice 26.18 MB
hello_world 13.46 MB
Isolated by A/B in a throwaway worktree before making the real change. Of the
10.1 MB, 2.78 MB is us_lexicon.bin embedded via include_bytes!; __const alone
was 11.2 MB, larger than __text.
Also: Kokoro construction is now lazy, deferring ~327 MB until first
utterance. That already closed the RSS gap this pass was chasing — Arcade's
max RSS measures 190 MB against hello_world's 222 MB on the same method, i.e.
BELOW the baseline app, so the premised memory problem no longer exists and
was not invented into one. The lean-isolate prelude was likewise dropped after
instrumentation showed Arcade allocates no script isolate on the demo path at
all; it should be re-measured against a loaded script game before anyone
spends effort there.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New makepad-converse crate: SpeechOutput/Playback moved out of gamemaker
(mix_into for custom audio callbacks, install_audio_output convenience);
TranscriptFilter trait + PassthroughFilter + FilterWorker thread (filters
built on the worker via factory since LlamaSession is not Send);
ConversePipeline wiring filter -> makepad_ai agent backend -> speech with a
ConverseAction stream, llm leg pluggable (claude-code / acp /
openai-compatible base_url for local servers). QwenFilter behind the
local-llm feature: chatml prompt, non-thinking prefill, greedy, SEND:/SKIP:
line protocol, fail-open parse; filter-repl bin for interactive testing.
Gamemaker's speech leg now uses the crate; its agent plumbing stays local.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Admin
