Apps ask the hub for a recognizer or a voice and get one; where it runs is
the hub's decision. AiHub::start_stt / start_tts return poll-driven
sessions shaped like the chat session. The Auto ladder is Whisper/Kokoro in
this process (weights present, machine election), on the machine node over
loopback, on a LAN node, else the OS engine; SpeechReach::Local is the
"don't reach out" knob. Audio always comes back as PCM: the app owns the
device.
Three layers:
- makepad-ai-speech is the whole speech model family, engines only.
libs/voice (Whisper + Silero VAD) folds in as the `whisper` and `vad`
modules next to kokoro and indextts, each a cargo feature; the Apple
bridges and the Speaker/VoiceTranscriber selection leave it.
- makepad-system-speech (new) is the OS speech services as blocking fns:
Apple SpeechAnalyzer/AVSpeechSynthesizer via Swift, Windows.Media.Speech*
on the vendored bindings, Android SpeechRecognizer/TextToSpeech through
MakepadSpeech.java (API 26 floor), espeak-ng on Linux. It models the two
STT shapes honestly: PCM in (Whisper, Apple) versus an engine that owns
the microphone (Android, Windows), with capabilities the caller reads.
- the hub grows speech sessions, in-process Whisper/Kokoro workers with the
residency election, a `whisper` wire backend (stt domain, registry entry
pinned to ggerganov/whisper.cpp) so a Mac can serve a Quest, and a
`language` field on the generate request.
Consumers: the Window voice input runs on an STT session and switches to
engine-mic mode when the recognizer owns the microphone; converse's
SpeechOutput is a lazily started TTS session plus a pump thread; route
drops its private speech copy for converse; vj's lyrics fallback and the
alignment bakes call the engines directly.
Verified here: speech-roundtrip through the real sessions (Apple voice in,
in-process Whisper on Metal out, 4.3% WER); system-speech-test TTS->STT
verbatim; hub/converse/system-speech unit tests; msvc, aarch64-android and
linux-gnu cross-checks; Java against android-34. Windows, Android and Linux
bridges are compile-checked only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Inhouse i_overlay 7.0.3 and its i_float/i_shape/i_key_sort/i_tree deps
(used by the map road union mesh under the widgets 'maps' feature) as
workspace-excluded path crates, stripped of tests/locks/registry crud.
i_float's no_std libm dependency is removed by switching it to std
float math, so the maps feature no longer pulls any external crates.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Runaway root cause, captured via self-sampling memory watchdog + boolean
input dumps: simplify_shape on 17-20k-ring soups (dense tiers at high
overzoom) spins in contour extraction for minutes while allocating
unboundedly (both i_overlay 4 and 7). Chunked dissolve (3000-ring chunks,
union of clean outlines) turns the captured hangs into 91/140 ms; the
full pan/tilt torture that blasted to 100 GB now holds 5-6 GB flat.
- i_overlay 4.5 -> 7.0.3 (API-compatible, booleans ~1.7x faster).
- Boolean input hygiene: 1/64-unit snap + degenerate ring drop.
- DzField: guard degenerate terminal-segment cap extension, clamp grid
walks (a NaN/huge coord can no longer explode cell insertion).
- AA fringe v1: miter skirts along boolean output rings, u 0.5->0 over
1px, emitted after each face at its ladder slot.
- Level-straddling rings join both grounded+lifted parts (transition
seams overlap instead of sharing an epsilon-fragile edge).
- Oneway arrows foreshorten with tilt (ground-plane glyphs).
- Debug: in-app memory watchdog (14 GB: sample own stacks -> abort),
MP_TILE_PROFILE via /tmp/mp_tile_profile, boolean repro test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bridge elevation (bridge.md M1+M2): tools/map_tiles bridge-bake solves
per-vertex road/rail dz over the OSM graph (crossing clearance, grade
ramps, deck holds, junction consensus) with AHN DSM-DTM measured deck
profiles (BigTIFF reader in geodata, WGS84->RD), baked as bridge_dz +
per-base-tile base_dz (L/F/P join to exact renderer geometry). Renderer
joins dz through parse into TileWay.dz; strokes/arrows/fills lift off
their own profiles; tunnels never deck.
Road overlay unifier: painter's algorithm as geometry — per-way segment
rects + vertex discs, i_overlay top-down subtraction cascade into
DISJOINT faces (overlay_paint_groups), triangulated once; flat render is
pixel-equal to paint order (proven 0-diff on the abstract unit case) and
tilt cannot reorder it. Legacy strokes interleave by rank; plazas join
the cascade at true alpha. @roads 0/1 A/B toggle, @abtest headless
CPU-raster diff harness (1.50% vs 2D reference on the Raampoort tile),
@cam debug camera, bridge_eval.sh aerial-vs-render loop.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
