Apps ask the hub for a recognizer or a voice and get one; where it runs is
the hub's decision. AiHub::start_stt / start_tts return poll-driven
sessions shaped like the chat session. The Auto ladder is Whisper/Kokoro in
this process (weights present, machine election), on the machine node over
loopback, on a LAN node, else the OS engine; SpeechReach::Local is the
"don't reach out" knob. Audio always comes back as PCM: the app owns the
device.
Three layers:
- makepad-ai-speech is the whole speech model family, engines only.
libs/voice (Whisper + Silero VAD) folds in as the `whisper` and `vad`
modules next to kokoro and indextts, each a cargo feature; the Apple
bridges and the Speaker/VoiceTranscriber selection leave it.
- makepad-system-speech (new) is the OS speech services as blocking fns:
Apple SpeechAnalyzer/AVSpeechSynthesizer via Swift, Windows.Media.Speech*
on the vendored bindings, Android SpeechRecognizer/TextToSpeech through
MakepadSpeech.java (API 26 floor), espeak-ng on Linux. It models the two
STT shapes honestly: PCM in (Whisper, Apple) versus an engine that owns
the microphone (Android, Windows), with capabilities the caller reads.
- the hub grows speech sessions, in-process Whisper/Kokoro workers with the
residency election, a `whisper` wire backend (stt domain, registry entry
pinned to ggerganov/whisper.cpp) so a Mac can serve a Quest, and a
`language` field on the generate request.
Consumers: the Window voice input runs on an STT session and switches to
engine-mic mode when the recognizer owns the microphone; converse's
SpeechOutput is a lazily started TTS session plus a pump thread; route
drops its private speech copy for converse; vj's lyrics fallback and the
alignment bakes call the engines directly.
Verified here: speech-roundtrip through the real sessions (Apple voice in,
in-process Whisper on Metal out, 4.3% WER); system-speech-test TTS->STT
verbatim; hub/converse/system-speech unit tests; msvc, aarch64-android and
linux-gnu cross-checks; Java against android-34. Windows, Android and Linux
bridges are compile-checked only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
libs/windows/windows-rs/src/Windows/mod.rs was a hand-pruned snapshot with
no recipe behind it; adding a WinRT namespace meant hand-copying vtables.
Now tools/windows_bindgen (standalone, windows-bindgen 0.62.1) regenerates
it from filter.txt: package mode with the COM `_Impl` traits, Win32 imports
linked through windows_core, upstream's 800-column rustfmt (the repo-root
rustfmt.toml disables formatting and has to be overridden), the filter
closed over every dependency the generator would otherwise skip a member
for, the namespace tree folded into one file with every feature gate kept,
and the two spots where the published 0.62.1 generator predates the vendored
core 0.62.2 normalized (Error::from_thread, imp::array_proxy).
The filter is the old file's item list plus Windows.Globalization.Language,
Windows.Media.SpeechRecognition and Windows.Media.SpeechSynthesis for the
OS speech engines. The generated `deprecated` gate is declared as a feature.
Checked on x86_64-pc-windows-msvc: platform, video, network, mpterm,
system-speech.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Squashed from work:
- rtsmap: one seeded generator for tiled strategy maps
- map: bake a runnable Amsterdam test map from inside the route app
- route: the first-run test map starts itself, and its buttons are guarded
- map: the test map bakes its road faces too, from the same shared pass
- map_tiles: mkmap-extract — the weave is reversible, world-cells reconstructible from world.mkmap
- route: the local dispatcher rides the hub — its copied engine is deleted, limits preserved (aicore P1)
- route + converse: off makepad_ai — the Agent seam moves to converse, route's cloud dispatcher rides the hub's Claude p
- ai-hub: chats run the machine election — route to a serving holder, wait on a loading one, claim and publish when open
- libs: the zero-warning sweep — stitch casts say what they mean, xatlas keeps upstream's surface quietly
Squashed from work:
- platform: native file and save dialogs, in-house on all three desktops
- platform: file dialogs on Android and iOS, and unbreak the Android build
Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- vj: the thumbnail pipeline becomes one honest machine, and effects go livecodable
- widgets: boxed labels center on their ink, not on the font's line box
- metal: a fresh texture forgets nothing it never had — reallocated vec textures upload whole
- fab: a 3D creation shell and the viewer built on it
- texcomp: the block codec and the container every texture will travel in
- mixer: a live LR-Mix surface for the XR18 — strips paired the way the desk is run, auto-connect, EQ that bends its own curve, and a sweep paced so the console drops nothing
- README: a build quick-start for macOS and Windows, the honest Linux
story, what CUDA is for and how to install the separation model.
- AGENTS.md: the `--remote` control surface protocol, so the harness it
documents is usable without reading platform/src/remote.rs.
- Cargo.toml: workspace membership for the crates this series adds and
removes.
- tools/: the Windows box scripts (wincmd, winps and friends, winrun) and
remote_smoke.sh — how a build gets driven on a remote machine.
- apps/asset-server: the standalone server binary and its README.
- Small follow-ups in libs/{windows,apple_sys,makepad_test,mbtile_reader,
converse} and apps/route, plus .gitignore and makepad.splash.
Drop the archived old/ tree, root AI notes, splashgame.md (Arcade lives
in the private sandbox), widgets-dll leftovers, and LAN/oracle helper
scripts. Move download_*.sh into tools/, and move sim/math out of
libs/game/ so the public repo no longer has a game/ directory.
Also quiet first-party compile noise and skip asset-ui tests that need
sandbox kits or uncommitted fixtures.
Point the workspace at libs/asset and libs/render. Drop Arcade
frontend, gamemaker, rig, pack registry, physics, and the old
libs/game gameplay crates. Those live in the private sandbox repo.
Android builds passed `-C prefer-dynamic` unconditionally, so Rust shipped
`std` as a separate `libstd-<hash>.so`. That library is a rustup prebuilt whose
LOAD segments are only 4 KB-page aligned (`p_align 0x1000`), so it can't be
mapped on the 16 KB-page devices Android 15 allows. Google Play requires apps
targeting API 35+ to run there, so release apks were failing that bar even
though `libmakepad.so` itself was already linked with 16 KB alignment.
Only debug builds keep `prefer-dynamic` now, where the faster incremental
relink is worth having and nothing ships. Everything else links `std`
statically, which drops the separate library entirely and leaves a single
NDK-linked, 16 KB-aligned `.so`. The aab path already did this.
* android oes video zero-copy and gles shader fixes
* regenerate windows-rs by windows-strip
* fix windows video freezes with MF on an MTA worker
* regenerate windows-rs by windows-strip
* MTA MF video, COM notify, YUV texture reuse
* add local video file playback support to video-player example
* NV12 Metal present, seek warm-up gate, YUV full range
* fix Linux GStreamer video: A/V mute, HLS prepare, async teardown
* no-op SelectVideoTrack/SelectAudioTrack on non-Linux backends
* Linux ALSA/Pulse: bigger buffers, fix resample setup, report device rate
* fix some warnings
* Move XInput/DirectInput device discovery off the UI thread
* fix GLSL unpack4u8 for GLES 3.0 (#version 300 es)
* Linux GStreamer: DMA-Buf NV12 OES zero-copy, optional GLMemory path
* Linux MediaPlugin: DMA-Buf NV12 and GLMemory zero-copy present APIs
* Add Windows SourceReader DXGI NV12 zero-copy video path
* fix build error
* fix GLES: add highp precision for sampler2DArray in Linux shaders
* playback speed support for android
---------
Co-authored-by: jasonqiu <jasonqiuchen@outlook.com>
Co-authored-by: jasonqiu <jasonqiu@futunn.com>
Two ways HEAD failed to build for a fresh checkout, both from work landing
across concurrent streams:
- 623ee745e renamed skin_to_pbr -> skin_to_packed, but arcade_view.rs was
being edited by another stream at the time, so its call site fix stayed
uncommitted while the rename landed
- Cargo.toml listed tools/arcade_eval as a workspace member while the crate
itself was untracked, so loading the workspace failed outright
Both verified: the crate builds, and a stash-everything check now leaves a
working tree that compiles.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pass 4 collects relation jobs, bbox-scans them against the RAM stores,
sorts by NL-spiral distance and publishes store/spool-frontier.txt (key
of the first unfinished relation; spool flushed before every publish).
pbf-base slices a LIVE store behind that gate (--bbox only, torn-tail-
tolerant block reads, pid-scoped sort chunks so concurrent slices never
clobber shared edge blocks), and mapfleet claims block per cell on the
frontier instead of waiting for the whole spool.
Pass-granular resume: passes 2/3 stamp exact spool block lengths +
counters + stats; a restart rolls the spool back to the newest stamp
and reruns only the remaining passes (unit tests + crash drill).
Weave fixes from the Luxembourg end-to-end smoke run: bake outputs land
via temp+rename so the ledger name never exists half-written, and
transmux verification now checks the union of ALL sources with first-
source-wins ownership (it compared source 0 alone and failed every
multi-source weave).
Also: pass-4 worker clamp 12→14, dead paged NodeStore/WayStore and
emit_lines/emit_polygons deleted (flat stores/prepare_* replaced them),
mapfleet --pbf + startup chunk sweep + end-of-spiral drain and final
weave, world-build.sh refreshes the -run binary copies.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
prepare_polygons clipped the FULL ring (and holes) against every tile in
the bbox: a continent-scale boundary relation (millions of points x
millions of tiles) pinned one core for hours at the planet spool's tail.
Halves clip once per split and recurse (geojson-vt scheme); per-tile
output is geometrically identical (equivalence test: same tiles, equal
areas, identical interior vertices — direct's zero-area boundary spikes
are collapsed, which is strictly cleaner).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
map_bake --recompress re-encodes EVERY tile (both pass-through sites) at
the target quality — compression parallelizes across the whole fleet and
the final dataset is uniformly q11. The slicer ships q2 intermediates
(fast, disk-lean). Bake mode also skips runtime-only work per tile:
building/tree vertex extrusion and POI point parsing never reach the
sink and no longer run on workers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tools/remote joins the root workspace (default-run makepad-remote, so
'cargo run -p makepad-remote -- --server' works from the checkout root
on worker boxes). Libraries stay in the build graph as path deps; the
runnable-package list is the things a human actually launches.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Protocol extracted to a shared module + TAG_FILE_PULL (server returns a
file from its cwd, escape-guarded). mapfleet dispatches the NL-spiral:
local slicing against the store (serialized), remote bakes via cargo-run
on each box (bootstrap build cached), pull-back, coalesced weave with
atomic world.mkmap swap. Resume ledger (cell-NNN-baked.mbtiles, empty =
ocean) is shared with world-slabs.sh — run one driver at a time. Failed
cells requeue with backoff; a dead box just stops claiming.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
