Account for missing and partial model files per volume, reserve headroom, and reject disk-constrained workers before accepting a job. Preserve typed admission failures so callers can choose another peer, and make activity gating and cancellation recover cleanly.
Validation: 592 release hub tests passed, one ignored; required-CUDA builds deployed to six idle Windows workers.
Clocked piano/ironfish/drum rack, program-bus mix, splat/mixer/music
updates. Silence the unused warnings that show up in `cargo check -p
makepad-vj --release`.
The 5090 kept about 1.5 GB of CUDA allocator pool after evicting every
model, so a card that fits flux2-dev refused it (30510 MB free reported,
32090 MB in a fresh process). Admission now trims the pool when the last
resident is gone and re-measures before refusing; usable VRAM is measured
after the same trim; the refusal names the pool it released.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Measured on the RTX 5090 (2026-09-04, 1024x1024, 8 steps, nvidia-smi 250 ms
samples): 30.5 GB used at peak, the run completed in 52 s. The old 29 GB
was a pre-measurement guess that under-reported the model by 1.5 GB.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
flux2-dev cannot fit on the 5090 at the default reserve, and the role
table barred the only card that can hold it. The user opened image on
10.0.0.165 ("let the rtx serve images too"); the role test and the
flow's role-aware listing test follow.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The 5090 listed flux2-dev ready (total VRAM passed the gate) then refused
every job (29696 + 2048 reserve > ~30510 usable); the flow picked it by
domain ETA and never retried. Nodes now publish vram_usable_mb and mark
un-admittable models too_small; the fleet gate uses usable VRAM; the
flow's gen executor picks admitted nodes for the requested model (ready
first), retries up to three nodes after an admission refusal, and when no
node can take the model its error says why per node (role, too small with
the numbers, waiting for VRAM). The flow's model listing drops (model,
node) pairs the node's fleet role bars, so the picker no longer counts the
chat-only PRO 6000 as ready for image; its label reads ready/absent/too
small with the GPUs named.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The hub chat session's election no longer stops at this machine: a
co-located serving holder first, then a fleet chat node heard by
discovery and role-allowed for chat (a short patience for beacons),
then the weights on this machine — and an honest answer naming both
misses and where weights may be put when none of those exists. Tool
packs ride every route: the system text carries the tool table as the
node's chat_system, one splitter strips thinking and collects
<tool_call> bodies across deltas, the in-process worker's parser is the
one parser, and tool results go back as tool turns. A node that fails
mid-turn ends that turn with the node named; the person's next line
re-elects, served first on the new route. The tools-only guard on the
proxy is gone. The session exposes its route, and the panel's chip
shows it.
The local lookup is independent of the working directory: the env
override, then the makepad home's weights (Qwen preferred, largest),
then the checkout the binary came from, then the cwd — so an assistant
launched from a binary copy finds the same weights as one run from the
checkout, and the Local provider always builds even with none.
Proven live with no local weights: from the sheets overlay a plain
line reached the fleet's 27B (the node with the model resident won the
pick), the model called sheets.summary and answered in 12 s; the app
stayed at 254 MB. hub_chat + local_llm 11, services 34.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A headless box's service log showed only the startup banner for a whole
session; now it records the model and wire encodings on open, and on close the
elapsed time, frames in/out, fps, and the dropped/undecodable/unencoded counts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The AI runtime crates read their precision, activation and kernel
choices from environment variables; the code path taken with none set
is the one that ships, so every knob that selected it is now an explicit
argument and every losing path is gone. GemmPrecision { f16_accumulate,
f16_activations } is passed by each caller: the default {true, true} is
the old unset Flux route; H3's DiT and text encoder pass {false, false}
(H3's >1e4 activations saturate f16 — the policy H3 used to set through
FLUX_GEMM_F16ACC=0 on itself), its VAE {true, false}; DA3's StrictF32
selects f32 packed attention in code; Hy-Motion carries an explicit
f16_attention_operands flag through its text refiner, its double and
single blocks and the CUDA backend (true in production, false only in
its full validator). The libs/diffusion bins — a separate workspace —
are migrated to the same shapes.
Benches and validators no longer set variables on themselves: llama's
skip-logits is a session option (the CUDA bench turns it on), OCR takes
explicit use_f16_gemm and tiled_roformer options, the lane speculative
probe reads its CLI. The live gates the first cut had deleted are back
as explicit-config tests: MMQ M=129, the strided-f32 MMV path, the
RMS+MUL CPU oracle. The loader's THREADS and CHUNK_MB stay real settings.
The Metal quantized-matmul experiment (metal_qmm and its vendored MLX
kernels) was reachable only through a knob and goes with it.
Reviewed in three rounds by the delegate reviewer (the last round
accepted everything but one Hy-Motion call site, fixed in round four
and reviewed here), and gated on the Windows CUDA box: lib checks of
common/paint/loader/cuda/llm/motion/vision, motion 24 and vision 23
tests, the hub check, the diffusion bins, llm 253 passed / 1 ignored.
On this Mac: the same checks plus the motion and vision tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The live decoder never produced a frame on the body node: MF_E_TRANSFORM_STREAM_CHANGE
was hand-derived as 0xC00D6D60, which is MF_E_TRANSFORM_TYPE_NOT_SET; the real stream
change (0xC00D6D61) was treated as a hard error, the output type was only negotiated
lazily, and without MF_LOW_LATENCY the decoder holds a reorder window a 2-3 frame live
pipeline never fills. Set MF_LOW_LATENCY on the transform, commit NV12 before the first
ProcessOutput, re-negotiate on TYPE_NOT_SET/STREAM_CHANGE/BUFFERTOOSMALL, drain on
NOTACCEPTING, and stamp packets with monotonic 100 ns timestamps. MAKEPAD_H264_DEBUG=<file>
traces packets, HRESULTs, negotiations and frames for headless services.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Packages are makepad-<name> with the short name as the binary. Env vars
follow (MAKEPAD_WM_*, MAKEPAD_FILES_*, MAKEPAD_TERMINAL_*), config moves
under ~/.makepad/<app>/, the theme namespaces are mod.wm_theme and
mod.browser_theme, the hosted AI envelope key is wm_ai. platform/video
becomes makepad-platform-video so the video app can be makepad-video.
Carries the Score entries that were pending in the WM's curated table.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
LocalModels runs registry models in-process (install state, resumable downloads, recorded licence acknowledgements at $MAKEPAD_HOME/license_acks.json, weight paths by file role) and libs/ai/hub_ui is the install panel + licence modal every app can embed. New native ports: Beat This! (beats + downbeats) and Basic Pitch (note transcription) with their registry entries; the Salamander Drumkit samples (CC BY-SA 3.0, 37 files pinned by size and sha256) as a sample bank the downloader fetches like a model.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The last two hub commits staged whole files and carried uncommitted hunks
of another lane (beats-native, notes-native, the local runner, new
domains and license keys) that reference files not yet in the tree. This
restores those files to the body changes only; the other lane's edits
stay in its working tree.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The native port covers everything the Python reference worker did (body,
hands, mask prompt, multi-person), so the subprocess backend, its fake
worker harness, the sam3dbody-ref registry entry and the
MAKEPAD_SAM3DBODY_* environment go. The packet validator moves to the
native backend.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The sam3dbody backend reads its options from the request's prompt string
(`hands`, `detect`, `persons=N`): `hands` runs the full mode and the
packet carries which hands were fused and their boxes; `detect` finds up
to N persons with SAM 3.1 (an optional native-segment role on the body
entry, the same artifact the segment entry pins) and runs one body pass
per person with its box and mask, so the packet's people array grows.
The body crate shares one body pass between the packet, mask and hands
entry points, and infer_full takes the mask prompt too.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Apps ask the hub for a recognizer or a voice and get one; where it runs is
the hub's decision. AiHub::start_stt / start_tts return poll-driven
sessions shaped like the chat session. The Auto ladder is Whisper/Kokoro in
this process (weights present, machine election), on the machine node over
loopback, on a LAN node, else the OS engine; SpeechReach::Local is the
"don't reach out" knob. Audio always comes back as PCM: the app owns the
device.
Three layers:
- makepad-ai-speech is the whole speech model family, engines only.
libs/voice (Whisper + Silero VAD) folds in as the `whisper` and `vad`
modules next to kokoro and indextts, each a cargo feature; the Apple
bridges and the Speaker/VoiceTranscriber selection leave it.
- makepad-system-speech (new) is the OS speech services as blocking fns:
Apple SpeechAnalyzer/AVSpeechSynthesizer via Swift, Windows.Media.Speech*
on the vendored bindings, Android SpeechRecognizer/TextToSpeech through
MakepadSpeech.java (API 26 floor), espeak-ng on Linux. It models the two
STT shapes honestly: PCM in (Whisper, Apple) versus an engine that owns
the microphone (Android, Windows), with capabilities the caller reads.
- the hub grows speech sessions, in-process Whisper/Kokoro workers with the
residency election, a `whisper` wire backend (stt domain, registry entry
pinned to ggerganov/whisper.cpp) so a Mac can serve a Quest, and a
`language` field on the generate request.
Consumers: the Window voice input runs on an STT session and switches to
engine-mic mode when the recognizer owns the microphone; converse's
SpeechOutput is a lazily started TTS session plus a pump thread; route
drops its private speech copy for converse; vj's lyrics fallback and the
alignment bakes call the engines directly.
Verified here: speech-roundtrip through the real sessions (Apple voice in,
in-process Whisper on Metal out, 4.3% WER); system-speech-test TTS->STT
verbatim; hub/converse/system-speech unit tests; msvc, aarch64-android and
linux-gnu cross-checks; Java against android-34. Windows, Android and Linux
bridges are compile-checked only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The recut in a6341981d applied its patch against the wrong directory
and restored the six shared files to the previous tree without the
body-native hunks. This commit adds exactly those: the `body-native`
feature and optional dependency, the pinned `sam3dbody` registry entry
and its test, the backend arms and the module declaration. Working
tree untouched.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
66e5e2f11 committed the working tree of the shared hub files and with it
another lane's uncommitted edits (a new domain, request fields, a
backend arm, manifest lines). This commit restores those files to the
previous tree plus only the body-native hunks. The working tree is
untouched: the peer's edits stay on disk as their uncommitted work,
exactly as before.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
model.rs closes the loop: crop -> backbone -> ray-conditioned context ->
six decoder steps, each through the pose head, the rig, the camera and
the projection, then the packet the sandbox already reads (kp3d/kp2d in
camera axes, the 204 rig parameters, global rotation, camera translation,
joint positions). Against the reference on the oracle image, on Metal:
3D keypoints within 1.7 mm, 2D within 0.4 px, rig parameters, camera
and rotation within 2e-3. packet.rs writes the JSON by hand with the
reference worker's rounding and field order.
The hub gains the `body-native` feature (default on): registry entry
`sam3dbody` pinned to the Comfy-Org repack by revision, size and sha,
body_native_backend.rs beside the subprocess reference backend with the
same live_step contract, the `body` capability advertised when the
feature is compiled, and a stubbed test double for the CPU-only tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A realtime feed session parked in its wait-for-a-frame loop forever when
the client died without sending stop (a sandbox quit left job-2 live on
.123 for five minutes holding the GPU slot). The wait loop now returns
to the top of the session loop once no socket is left, where the idle
timeout counts a socketless session down. Test covers it.
SkinnedModel gains node_parent/node_count, joint_skinned_centroid (the
direction a leaf limb actually runs, from the flesh it skins) and
from_nodes (a mesh-less rig for hierarchy-maths tests) — what the
sandbox's webcam mocap retarget needs.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B626urtY1Xo4hQdLzvSK6F
New `body` backend (sam3dbody-ref): a persistent length-prefixed-PNG /
JSON-lines worker subprocess seam with ready handshake, per-frame timeout,
bounded restarts. LiveFrameOut grows aux_json — structured per-frame JSON
sent to the client before the frame — and output_encoding "none" makes a
session pose-only (refused with loop_mode feedback, also on control flips,
which upgraded apply_control to Result). Worker code+model stay
box-provisioned via MAKEPAD_SAM3DBODY_WORKER; the repo carries only the
MIT seam. Codex lane + Fable review (ready handshake, spawn timeout).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0165w1ZL1f1TruX5u2qC7mSX
the vision and ocr backends' ContentBackend machinery lives behind the
llm feature; the imports and the lane-refill helpers now do too, so a
client crate compiling the hub with a leaner feature set (vj, chat-ui)
builds without warnings. pipeline_submit_collect stays feature-free for
its tests and says so.
Squashed from work:
- asset-ai: FastH3 4-step fast video backend; clip keyframes on the wire
- asset-ui: loop video chains — text→image→video that ends where it began
- h3: safetensors -> pruned-Q4_K GGUF quantizer for the 24GB DiT tiers
- h3_quant_gguf verify: row-error gates calibrated to the measured Q4_K floor
- asset-ai realtime: the feedback loop — the source anchors, the drifted frame inits
- asset-ai realtime: a feedback loop survives a resize and travels by default
- asset-ai realtime: the feedback loop frees itself from the feed handshake and pauses for its listener
- asset-ai realtime: the outbound encode leaves the loop's critical path
- asset-ai ocr: the ocr domain — Chandra 2 at page resolution, and the tower goes planner-owned
- llm slots: a lane can hold an image span — embedding prefill and a rope cursor of its own
- vision tower on CUDA: the encode leg gets its two missing kernels
- llm/ocr: one M-RoPE grid encoder for both image paths, and a livelock made an error
- vision tower on CUDA: the f16 GEMM keeps the precision it was throwing away
- live: a feed that moves box takes its trip with it — one seed image
- vision tower on CUDA: the tiled attention becomes bit-exact, and tensor cores go
- llm prefill on CUDA: the MMA attention kernel gets the tile a 4-to-1 model needs
- asset-ai ocr: the CUDA encode lane joins the integration — vision-parity sits beside run's three arms, and the kernels
- Merge branch 'ocr-perf-integration' into work
- asset-ai: the live anchor can follow the trip, and text leaves the 5090
- asset-ai: the camera moves the world, and the world starts still
- asset-import: the EA strategy classics, in the one 2D contract
- rtsmap: one seeded generator for tiled strategy maps
- asset-ui: one card for the strategy classics, with a pack dropdown
- asset-ai: music3 reference-audio path, ocr/h3 backends, registry
- asset: mp4 sample index for range-streaming, chat tools, import profiles
- cnc: tiberium is twelve growth frames, not twelve empty variants
- platform: native file and save dialogs, in-house on all three desktops
- chat: the scan holds out for a lane home
- chat: a full home queues you — take the free lane
- chat: the preload has a percentage, and the boundless cap stops showing
- llm cuda: the 32x2 attention tile — even GQA ratios stay on MMA
- sa3 gets a bake path: the sfx model's tables precomputed by a diffusion-side bin
- sqlite_query: anti-join regression test
- td import: HARV's second frame block is its harvesting cycle, not a turret
- asset-ui: sprite enhancement runs on the 32B dev DiT — distillation, not the prompt, was the ceiling
- ai-hub: makepad-asset-ai becomes makepad-ai-hub at libs/ai/hub, the chat pane becomes makepad-chat-ui, the service bin
- asset-ui: test health fixtures grow the realtime field they were born without
- ai-hub: one home at ~/.makepad — weights/ run/ cache/ logs/, the service cache migrates from ai_content by a single re
- ai-hub: subprocess workers die with the node — process groups everywhere, PDEATHSIG on linux, one KILL_ON_JOB_CLOSE Jo
- ai-hub: the hub object — AiHub::in_process, pipes vocabulary, and the local LLM engine generalized out of mpfiles (aic
- strict-json: the dependency-free JSON module gets its own crate; asset-client re-exports it so nothing downstream move
- ai-hub: the machine layer — node entries, the 0600 machine token, and the residency election that IS the lock (aicore
- ai-hub: MPHUB1 — the fabric beacon only dedicated nodes can send (aicore §4)
- ai-hub: job leases — work lives only while it is renewed (aicore §8)
- asset-creator: the pipeline library is born — specs, the deps gate, and the derived-state law (aicore §9)
- ai-hub: RAM residency facts — the CPU-side twin of residency.rs (aicore §3)
- ai-hub: ETA placement primitives — relative GPU throughput, the four-term estimate, and an observable breakdown (aicor
- ai-hub: leases go live on the wire — origin fields on submit, /job/<id>/keepalive, /bye, and the reaper that cancels w
- ai-hub: the chat providers move in — fleet qwen, openai, grok, claude/codex/grok CLIs, the responses driver, and the w
- asset-creator: the engine — one pipeline run against the hub, deps-gated, spliced, cancellable, resumable-by-construct
- ai-hub: the machine node mode — --machine binds loopback, registers in ~/.makepad/run, and exits on its own once idle
- asset-creator: makepad-creator-run — the detached client for runs that must outlive a window (aicore §9)
- ai-hub: a native Claude Messages-API provider — API-key or Claude Code OAuth, bounded SSE streaming, injected tools (a
- route + converse: off makepad_ai — the Agent seam moves to converse, route's cloud dispatcher rides the hub's Claude p
- asset-creator: the preset tables move in — fifteen chain-policy constants shared by every creator app (aicore §9 / P6)
- makepad_ai is deleted — every backend is a hub pipe, the agent seam lives with its consumers (aicore §14, decided 2026
- ai-hub: loads hold the machine residency election — set_model_state claims on Loaded and publishes the service port (a
- ai-hub: chats run the machine election — route to a serving holder, wait on a loading one, claim and publish when open
- ai-hub: pick_for_domain_eta — ETA-ranked placement over the shared hard-filter core (aicore §6 / P4)
- asset-creator: the engine picks a provider per stage at dispatch time — a chain's later stages see fresh fleet state (
- ai-hub: the fabric secret gates the service HTTP surface — bearer on everything but /health and the ticketed peer path
- vj: DREAM runs execute in the app — pipelines.rs becomes the run it used to watch (aicore §9 / F1)
- asset-creator: the runner — generate one thing and put it in the catalog, one implementation for every surface (aicore
- chat-ui: the session runs in the app — no broker anywhere on the chat path (aicore P8 / F5)
- asset-store: assets.query is a first-class query endpoint — the bounded SQL surface outlives the broker (aicore P8 / F
- asset-creator: CreatorTools — the chat tool pack for a store that only stores (aicore §9 / P8)
- asset-store: the shrink — the store stores (aicore P7)
- importer + asset-server host: the coordination era ends (aicore P7)
- store config purge + asset-ui goes fleet-direct; the derive protocol gets its route proof (aicore P7)
- client + chat dispatcher: the dead wire comes out (aicore P7/P8)
- ai-hub: 0.3.0 — the health version says which era a node runs
- ai-hub: the default fleet is 'gen' — apps hear the LAN without env plumbing
- ai-hub: the preload note percents the prefill, not the job bar
- ai-hub: conversations keep their KV — the wire mirror, the lane identity, the in-turn dynamic context (aicore §7)
- ai-hub: an open-think model is thinking from its first token
- libs: the zero-warning sweep — stitch casts say what they mean, xatlas keeps upstream's surface quietly
- zero-warning sweep, round two — the first full-workspace pass
- zero-warning sweep, round three — the model lanes and the deep examples
- zero-warning sweep, round four — the last stragglers
- zero-warning sweep, round five — vj and chat-ui
- zero-warning sweep, round six — three cascades
Co-authored-by: Claude <info@makepad.nl>
