Packages are makepad-<name> with the short name as the binary. Env vars
follow (MAKEPAD_WM_*, MAKEPAD_FILES_*, MAKEPAD_TERMINAL_*), config moves
under ~/.makepad/<app>/, the theme namespaces are mod.wm_theme and
mod.browser_theme, the hosted AI envelope key is wm_ai. platform/video
becomes makepad-platform-video so the video app can be makepad-video.
Carries the Score entries that were pending in the WM's curated table.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
LocalModels runs registry models in-process (install state, resumable downloads, recorded licence acknowledgements at $MAKEPAD_HOME/license_acks.json, weight paths by file role) and libs/ai/hub_ui is the install panel + licence modal every app can embed. New native ports: Beat This! (beats + downbeats) and Basic Pitch (note transcription) with their registry entries; the Salamander Drumkit samples (CC BY-SA 3.0, 37 files pinned by size and sha256) as a sample bank the downloader fetches like a model.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The last two hub commits staged whole files and carried uncommitted hunks
of another lane (beats-native, notes-native, the local runner, new
domains and license keys) that reference files not yet in the tree. This
restores those files to the body changes only; the other lane's edits
stay in its working tree.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The native port covers everything the Python reference worker did (body,
hands, mask prompt, multi-person), so the subprocess backend, its fake
worker harness, the sam3dbody-ref registry entry and the
MAKEPAD_SAM3DBODY_* environment go. The packet validator moves to the
native backend.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Apps ask the hub for a recognizer or a voice and get one; where it runs is
the hub's decision. AiHub::start_stt / start_tts return poll-driven
sessions shaped like the chat session. The Auto ladder is Whisper/Kokoro in
this process (weights present, machine election), on the machine node over
loopback, on a LAN node, else the OS engine; SpeechReach::Local is the
"don't reach out" knob. Audio always comes back as PCM: the app owns the
device.
Three layers:
- makepad-ai-speech is the whole speech model family, engines only.
libs/voice (Whisper + Silero VAD) folds in as the `whisper` and `vad`
modules next to kokoro and indextts, each a cargo feature; the Apple
bridges and the Speaker/VoiceTranscriber selection leave it.
- makepad-system-speech (new) is the OS speech services as blocking fns:
Apple SpeechAnalyzer/AVSpeechSynthesizer via Swift, Windows.Media.Speech*
on the vendored bindings, Android SpeechRecognizer/TextToSpeech through
MakepadSpeech.java (API 26 floor), espeak-ng on Linux. It models the two
STT shapes honestly: PCM in (Whisper, Apple) versus an engine that owns
the microphone (Android, Windows), with capabilities the caller reads.
- the hub grows speech sessions, in-process Whisper/Kokoro workers with the
residency election, a `whisper` wire backend (stt domain, registry entry
pinned to ggerganov/whisper.cpp) so a Mac can serve a Quest, and a
`language` field on the generate request.
Consumers: the Window voice input runs on an STT session and switches to
engine-mic mode when the recognizer owns the microphone; converse's
SpeechOutput is a lazily started TTS session plus a pump thread; route
drops its private speech copy for converse; vj's lyrics fallback and the
alignment bakes call the engines directly.
Verified here: speech-roundtrip through the real sessions (Apple voice in,
in-process Whisper on Metal out, 4.3% WER); system-speech-test TTS->STT
verbatim; hub/converse/system-speech unit tests; msvc, aarch64-android and
linux-gnu cross-checks; Java against android-34. Windows, Android and Linux
bridges are compile-checked only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The recut in a6341981d applied its patch against the wrong directory
and restored the six shared files to the previous tree without the
body-native hunks. This commit adds exactly those: the `body-native`
feature and optional dependency, the pinned `sam3dbody` registry entry
and its test, the backend arms and the module declaration. Working
tree untouched.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
66e5e2f11 committed the working tree of the shared hub files and with it
another lane's uncommitted edits (a new domain, request fields, a
backend arm, manifest lines). This commit restores those files to the
previous tree plus only the body-native hunks. The working tree is
untouched: the peer's edits stay on disk as their uncommitted work,
exactly as before.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
model.rs closes the loop: crop -> backbone -> ray-conditioned context ->
six decoder steps, each through the pose head, the rig, the camera and
the projection, then the packet the sandbox already reads (kp3d/kp2d in
camera axes, the 204 rig parameters, global rotation, camera translation,
joint positions). Against the reference on the oracle image, on Metal:
3D keypoints within 1.7 mm, 2D within 0.4 px, rig parameters, camera
and rotation within 2e-3. packet.rs writes the JSON by hand with the
reference worker's rounding and field order.
The hub gains the `body-native` feature (default on): registry entry
`sam3dbody` pinned to the Comfy-Org repack by revision, size and sha,
body_native_backend.rs beside the subprocess reference backend with the
same live_step contract, the `body` capability advertised when the
feature is compiled, and a stubbed test double for the CPU-only tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
New `body` backend (sam3dbody-ref): a persistent length-prefixed-PNG /
JSON-lines worker subprocess seam with ready handshake, per-frame timeout,
bounded restarts. LiveFrameOut grows aux_json — structured per-frame JSON
sent to the client before the frame — and output_encoding "none" makes a
session pose-only (refused with loop_mode feedback, also on control flips,
which upgraded apply_control to Result). Worker code+model stay
box-provisioned via MAKEPAD_SAM3DBODY_WORKER; the repo carries only the
MIT seam. Codex lane + Fable review (ready handshake, spawn timeout).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0165w1ZL1f1TruX5u2qC7mSX
Squashed from work:
- asset-ai: FastH3 4-step fast video backend; clip keyframes on the wire
- asset-ui: loop video chains — text→image→video that ends where it began
- h3: safetensors -> pruned-Q4_K GGUF quantizer for the 24GB DiT tiers
- h3_quant_gguf verify: row-error gates calibrated to the measured Q4_K floor
- asset-ai realtime: the feedback loop — the source anchors, the drifted frame inits
- asset-ai realtime: a feedback loop survives a resize and travels by default
- asset-ai realtime: the feedback loop frees itself from the feed handshake and pauses for its listener
- asset-ai realtime: the outbound encode leaves the loop's critical path
- asset-ai ocr: the ocr domain — Chandra 2 at page resolution, and the tower goes planner-owned
- llm slots: a lane can hold an image span — embedding prefill and a rope cursor of its own
- vision tower on CUDA: the encode leg gets its two missing kernels
- llm/ocr: one M-RoPE grid encoder for both image paths, and a livelock made an error
- vision tower on CUDA: the f16 GEMM keeps the precision it was throwing away
- live: a feed that moves box takes its trip with it — one seed image
- vision tower on CUDA: the tiled attention becomes bit-exact, and tensor cores go
- llm prefill on CUDA: the MMA attention kernel gets the tile a 4-to-1 model needs
- asset-ai ocr: the CUDA encode lane joins the integration — vision-parity sits beside run's three arms, and the kernels
- Merge branch 'ocr-perf-integration' into work
- asset-ai: the live anchor can follow the trip, and text leaves the 5090
- asset-ai: the camera moves the world, and the world starts still
- asset-import: the EA strategy classics, in the one 2D contract
- rtsmap: one seeded generator for tiled strategy maps
- asset-ui: one card for the strategy classics, with a pack dropdown
- asset-ai: music3 reference-audio path, ocr/h3 backends, registry
- asset: mp4 sample index for range-streaming, chat tools, import profiles
- cnc: tiberium is twelve growth frames, not twelve empty variants
- platform: native file and save dialogs, in-house on all three desktops
- chat: the scan holds out for a lane home
- chat: a full home queues you — take the free lane
- chat: the preload has a percentage, and the boundless cap stops showing
- llm cuda: the 32x2 attention tile — even GQA ratios stay on MMA
- sa3 gets a bake path: the sfx model's tables precomputed by a diffusion-side bin
- sqlite_query: anti-join regression test
- td import: HARV's second frame block is its harvesting cycle, not a turret
- asset-ui: sprite enhancement runs on the 32B dev DiT — distillation, not the prompt, was the ceiling
- ai-hub: makepad-asset-ai becomes makepad-ai-hub at libs/ai/hub, the chat pane becomes makepad-chat-ui, the service bin
- asset-ui: test health fixtures grow the realtime field they were born without
- ai-hub: one home at ~/.makepad — weights/ run/ cache/ logs/, the service cache migrates from ai_content by a single re
- ai-hub: subprocess workers die with the node — process groups everywhere, PDEATHSIG on linux, one KILL_ON_JOB_CLOSE Jo
- ai-hub: the hub object — AiHub::in_process, pipes vocabulary, and the local LLM engine generalized out of mpfiles (aic
- strict-json: the dependency-free JSON module gets its own crate; asset-client re-exports it so nothing downstream move
- ai-hub: the machine layer — node entries, the 0600 machine token, and the residency election that IS the lock (aicore
- ai-hub: MPHUB1 — the fabric beacon only dedicated nodes can send (aicore §4)
- ai-hub: job leases — work lives only while it is renewed (aicore §8)
- asset-creator: the pipeline library is born — specs, the deps gate, and the derived-state law (aicore §9)
- ai-hub: RAM residency facts — the CPU-side twin of residency.rs (aicore §3)
- ai-hub: ETA placement primitives — relative GPU throughput, the four-term estimate, and an observable breakdown (aicor
- ai-hub: leases go live on the wire — origin fields on submit, /job/<id>/keepalive, /bye, and the reaper that cancels w
- ai-hub: the chat providers move in — fleet qwen, openai, grok, claude/codex/grok CLIs, the responses driver, and the w
- asset-creator: the engine — one pipeline run against the hub, deps-gated, spliced, cancellable, resumable-by-construct
- ai-hub: the machine node mode — --machine binds loopback, registers in ~/.makepad/run, and exits on its own once idle
- asset-creator: makepad-creator-run — the detached client for runs that must outlive a window (aicore §9)
- ai-hub: a native Claude Messages-API provider — API-key or Claude Code OAuth, bounded SSE streaming, injected tools (a
- route + converse: off makepad_ai — the Agent seam moves to converse, route's cloud dispatcher rides the hub's Claude p
- asset-creator: the preset tables move in — fifteen chain-policy constants shared by every creator app (aicore §9 / P6)
- makepad_ai is deleted — every backend is a hub pipe, the agent seam lives with its consumers (aicore §14, decided 2026
- ai-hub: loads hold the machine residency election — set_model_state claims on Loaded and publishes the service port (a
- ai-hub: chats run the machine election — route to a serving holder, wait on a loading one, claim and publish when open
- ai-hub: pick_for_domain_eta — ETA-ranked placement over the shared hard-filter core (aicore §6 / P4)
- asset-creator: the engine picks a provider per stage at dispatch time — a chain's later stages see fresh fleet state (
- ai-hub: the fabric secret gates the service HTTP surface — bearer on everything but /health and the ticketed peer path
- vj: DREAM runs execute in the app — pipelines.rs becomes the run it used to watch (aicore §9 / F1)
- asset-creator: the runner — generate one thing and put it in the catalog, one implementation for every surface (aicore
- chat-ui: the session runs in the app — no broker anywhere on the chat path (aicore P8 / F5)
- asset-store: assets.query is a first-class query endpoint — the bounded SQL surface outlives the broker (aicore P8 / F
- asset-creator: CreatorTools — the chat tool pack for a store that only stores (aicore §9 / P8)
- asset-store: the shrink — the store stores (aicore P7)
- importer + asset-server host: the coordination era ends (aicore P7)
- store config purge + asset-ui goes fleet-direct; the derive protocol gets its route proof (aicore P7)
- client + chat dispatcher: the dead wire comes out (aicore P7/P8)
- ai-hub: 0.3.0 — the health version says which era a node runs
- ai-hub: the default fleet is 'gen' — apps hear the LAN without env plumbing
- ai-hub: the preload note percents the prefill, not the job bar
- ai-hub: conversations keep their KV — the wire mirror, the lane identity, the in-turn dynamic context (aicore §7)
- ai-hub: an open-think model is thinking from its first token
- libs: the zero-warning sweep — stitch casts say what they mean, xatlas keeps upstream's surface quietly
- zero-warning sweep, round two — the first full-workspace pass
- zero-warning sweep, round three — the model lanes and the deep examples
- zero-warning sweep, round four — the last stragglers
- zero-warning sweep, round five — vj and chat-ui
- zero-warning sweep, round six — three cascades
Co-authored-by: Claude <info@makepad.nl>
