Account for missing and partial model files per volume, reserve headroom, and reject disk-constrained workers before accepting a job. Preserve typed admission failures so callers can choose another peer, and make activity gating and cancellation recover cleanly.
Validation: 592 release hub tests passed, one ignored; required-CUDA builds deployed to six idle Windows workers.
Clocked piano/ironfish/drum rack, program-bus mix, splat/mixer/music
updates. Silence the unused warnings that show up in `cargo check -p
makepad-vj --release`.
The 5090 kept about 1.5 GB of CUDA allocator pool after evicting every
model, so a card that fits flux2-dev refused it (30510 MB free reported,
32090 MB in a fresh process). Admission now trims the pool when the last
resident is gone and re-measures before refusing; usable VRAM is measured
after the same trim; the refusal names the pool it released.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Measured on the RTX 5090 (2026-09-04, 1024x1024, 8 steps, nvidia-smi 250 ms
samples): 30.5 GB used at peak, the run completed in 52 s. The old 29 GB
was a pre-measurement guess that under-reported the model by 1.5 GB.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
flux2-dev cannot fit on the 5090 at the default reserve, and the role
table barred the only card that can hold it. The user opened image on
10.0.0.165 ("let the rtx serve images too"); the role test and the
flow's role-aware listing test follow.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The 5090 listed flux2-dev ready (total VRAM passed the gate) then refused
every job (29696 + 2048 reserve > ~30510 usable); the flow picked it by
domain ETA and never retried. Nodes now publish vram_usable_mb and mark
un-admittable models too_small; the fleet gate uses usable VRAM; the
flow's gen executor picks admitted nodes for the requested model (ready
first), retries up to three nodes after an admission refusal, and when no
node can take the model its error says why per node (role, too small with
the numbers, waiting for VRAM). The flow's model listing drops (model,
node) pairs the node's fleet role bars, so the picker no longer counts the
chat-only PRO 6000 as ready for image; its label reads ready/absent/too
small with the GPUs named.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
gen.image{prompt} is a service inside the assistant: on a worker it
runs the creator pipeline for the image domain against the node the
runner picks (LAN discovery, ETA-ranked, the resident flux model wins),
streams the node's progress into the tool card, and writes the picture
under the makepad home's gen/ dir; the creator library gains
generate_bytes — the request, poll and fetch that generate_and_publish
now shares — so no second pipeline exists. photos.add{path} bakes that
one file into the open library (only from the gen dir or the person's
home), re-opens the wall and glides onto the new picture; the module
executor answers it later through the host's reply sink. The model
chains them, launching photos in between. A registry seam found on the
way: a call dispatched in the same event a link was adopted reached the
port before its Registered, so the registry now answers a leading
Register at once.
The wall gained a search box: every keystroke re-cuts the packing over
the matches (every word in a title or link; title beats link, a whole
word beats a substring), each picture flies from its drawn rect to its
target over 450 ms, dropped pictures shrink and fade, returning ones
grow back; Esc clears; ⌘F or a bare slash focuses the box; the status
reads "35 of 293 · ro". photos.filter{query} sets the same from the
pane; instances are rebuilt only while a flight runs, and the clock is
the platform's.
Proven on the fleet: "make me a picture of a red bicycle on the moon
and put it in photos" → the file in 40 s, photos launched, added and
shown among the comics; typing r, ro, rob re-packs the wall live; the
filter through the pane in a process tile and as a module. ai-services
39, aichat 5, photos 9, image_tiles 12, creator 9; aichat and photos
check for wasm32.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
An ok os.launch result is held while the launched app is not on the
bus and delivered when it registers (within 8 s), rewritten as "<App>
is running now. Its tools: … Call them directly." with the tool table
rebound mid-turn where the model can (the local model prepends the
update to the tool result; Claude keeps its native table until the next
turn and gets the names in the text); past the patience the result says
the app has not connected yet. An assistant entry that never got
visible text is dropped before a tool call and at the end of a turn,
and the panel draws an empty one as nothing — the blank block above
the first card is gone. The model's context now opens with the running
apps and their tool names and lists the others as os.launch candidates;
the WM's launch description says a running app's tools are already
there, and its answer for one that is already running says so and
brings it to the front.
Proven on the fleet model: "open the photos app and find the pictures
about dogs" → launch (running 4 s later) → search, summary, search →
the answer, one turn, the tile beside the pane. ai-services 38, wm 153,
aichat 2; aichat checks for wasm32.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The hub chat session's election no longer stops at this machine: a
co-located serving holder first, then a fleet chat node heard by
discovery and role-allowed for chat (a short patience for beacons),
then the weights on this machine — and an honest answer naming both
misses and where weights may be put when none of those exists. Tool
packs ride every route: the system text carries the tool table as the
node's chat_system, one splitter strips thinking and collects
<tool_call> bodies across deltas, the in-process worker's parser is the
one parser, and tool results go back as tool turns. A node that fails
mid-turn ends that turn with the node named; the person's next line
re-elects, served first on the new route. The tools-only guard on the
proxy is gone. The session exposes its route, and the panel's chip
shows it.
The local lookup is independent of the working directory: the env
override, then the makepad home's weights (Qwen preferred, largest),
then the checkout the binary came from, then the cwd — so an assistant
launched from a binary copy finds the same weights as one run from the
checkout, and the Local provider always builds even with none.
Proven live with no local weights: from the sheets overlay a plain
line reached the fleet's 27B (the node with the model resident won the
pick), the model called sheets.summary and answered in 12 s; the app
stayed at 254 MB. hub_chat + local_llm 11, services 34.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The per-family model crates (flux, h3, paint, music, speech, stems,
vision, sfx, rife, trellis, beats, common) and the libs/diffusion
research binaries read some 190 environment variables that were
research knobs: tensor dumps, per-stage timing, oracle-parity and
fixture rigs, experiment toggles. The path taken with none of them set
is the one that ships; every such knob is deleted with the code it
gated, and every losing branch of an experiment toggle goes with its
toggle — dead kernels, fields and functions included. What remains are
the real configuration variables (the FLUX_*_MODE family, FLUX_GRAPH,
the VAE pool cap, the FLUX2 text-encoder residency, H3_VAE_BATCH, the
music3 caches and official modes, the stems/beats f16 switches, the
weight and data roots) and the build-script variables.
Rebased on the runtime cleanup: precision stays explicit everywhere
(GemmPrecision, f16_attention_operands, the H3 text precision, DA3's
StrictF32 in code); no act16, no H3_ACT_F16, no FLUX_ATTN_F16 or
FLUX_VAE_CONV_GEMM reads survive.
Reviewed by the delegate reviewer (APPLY, no findings) and gated on the
Windows CUDA box: all seventeen model crates check, motion and vision
tests, the hub and the diffusion bins — the gate caught one CUDA-only
tap marker the Mac never compiles, removed here. On this Mac: the same
checks plus motion 24, paint 158 and vision 23 tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A headless box's service log showed only the startup banner for a whole
session; now it records the model and wire encodings on open, and on close the
elapsed time, frames in/out, fps, and the dropped/undecodable/unencoded counts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The AI runtime crates read their precision, activation and kernel
choices from environment variables; the code path taken with none set
is the one that ships, so every knob that selected it is now an explicit
argument and every losing path is gone. GemmPrecision { f16_accumulate,
f16_activations } is passed by each caller: the default {true, true} is
the old unset Flux route; H3's DiT and text encoder pass {false, false}
(H3's >1e4 activations saturate f16 — the policy H3 used to set through
FLUX_GEMM_F16ACC=0 on itself), its VAE {true, false}; DA3's StrictF32
selects f32 packed attention in code; Hy-Motion carries an explicit
f16_attention_operands flag through its text refiner, its double and
single blocks and the CUDA backend (true in production, false only in
its full validator). The libs/diffusion bins — a separate workspace —
are migrated to the same shapes.
Benches and validators no longer set variables on themselves: llama's
skip-logits is a session option (the CUDA bench turns it on), OCR takes
explicit use_f16_gemm and tiled_roformer options, the lane speculative
probe reads its CLI. The live gates the first cut had deleted are back
as explicit-config tests: MMQ M=129, the strided-f32 MMV path, the
RMS+MUL CPU oracle. The loader's THREADS and CHUNK_MB stay real settings.
The Metal quantized-matmul experiment (metal_qmm and its vendored MLX
kernels) was reachable only through a knob and goes with it.
Reviewed in three rounds by the delegate reviewer (the last round
accepted everything but one Hy-Motion call site, fixed in round four
and reviewed here), and gated on the Windows CUDA box: lib checks of
common/paint/loader/cuda/llm/motion/vision, motion 24 and vision 23
tests, the hub check, the diffusion bins, llm 253 passed / 1 ignored.
On this Mac: the same checks plus the motion and vision tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every Window now carries an AiChatSlot beside the tweaker: zero cost
while off, inert when the window manager hosts the process (the WM's
pane is the chat then), and on the first bare F10 it instantiates the
chat's module root BY NAME — mod.widgets.AiChatOverlay — which exists
when the app links makepad-aichat and calls its script_mod; an app that
does not gets one log line. The overlay slides in from the right over
the body with the WM pane's motion, owns the pointer inside its rect and
the keyboard through its composer, keeps ticking while hidden, and draws
in the window's retained overlay list so it composites over the deferred
body. Requests from outside the tree ride Cx globals, never statics.
An app exposes itself with one call: AiServicePort::open(cx, manifest)
is the hosted transport under the WM and, standalone, an in-process link
parked on Cx (PendingServiceLinks) that whichever chat root is up adopts
into its registry — the overlay today, the superbuild's pane later. The
registry wakes the UI when it sends, so an in-process call is answered
without waiting for a pointer event. NoModel leaves the engine feature
so a build without a model runtime still answers honestly and keeps the
tool console; aichat's engine is a default-on feature.
The bridge: Cx::ai_callback beside tweak_callback, /ai?on=1|0, /ai?say=…
and /ai/transcript (the overlay publishes the transcript as JSON after
each state change), installed by makepad_aichat::script_mod.
sheets links aichat and exposes sheets.summary (name, used range, header
row, selection) through the port: standalone the overlay answers it,
under the WM the bus does.
The local model no longer loads on a registration: its session starts
on the first user line, so an app joining the bus costs nothing.
Verified by hidden grabs: sheets standalone /ai?on=1 → the overlay with
the summary card and no model; F10 closes it; under the WM the sheets
tile shows no overlay and F10 toggles the WM pane. widgets 135 + 4
(two widget_tree find_within tests fail before and after this change),
aichat 33, sheets 83 + 2, services 29; widgets and aichat check for
wasm32.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The live decoder never produced a frame on the body node: MF_E_TRANSFORM_STREAM_CHANGE
was hand-derived as 0xC00D6D60, which is MF_E_TRANSFORM_TYPE_NOT_SET; the real stream
change (0xC00D6D61) was treated as a hard error, the output type was only negotiated
lazily, and without MF_LOW_LATENCY the decoder holds a reorder window a 2-3 frame live
pipeline never fills. Set MF_LOW_LATENCY on the transform, commit NV12 before the first
ProcessOutput, re-negotiate on TYPE_NOT_SET/STREAM_CHANGE/BUFFERTOOSMALL, drain on
NOTACCEPTING, and stamp packets with monotonic 100 ns timestamps. MAKEPAD_H264_DEBUG=<file>
traces packets, HRESULTs, negotiations and frames for headless services.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Source (webcam by device, a still for tests), one VideoToolbox encoder
session per run for the H.264 wire with raw as the fallback, the hub
realtime session on a node chosen by domain, typed events (status,
connected, aux, rate, ended) through a mailbox to any consumer. The
sandbox's mocap is the first; fabric is next.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
apps/aichat is a lib and a bin. The panel widget owns the registry, the
model and the transcript, draws EngineState as user lines, markdown
answers, tool cards with progress and confirm buttons, and system lines
over a composer; every event pumps the engine and a busy turn asks for
the next frame. The bus client turns the window manager's stamped
studio frames into registry links under the WM's own endpoint ids
(registry.register_as). Settings persist under ~/.makepad/aichat with
the cloud lock kept in code. The binary is both the standalone window
and the WM's special child.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
LocalModel drives the hub chat session; a changed tool table or system
prompt rides the next turn as an update block instead of restarting an
append-only context. ClaudeModel owns the history and rebuilds the
per-turn provider with the new native tool array, dots mapped to the
double underscore Anthropic allows. NoModel answers nothing but says so,
so the web demo's pane, services and tool console work without a
backend. Provider rows report honest availability, the local model file
policy is one function, and the crate no longer needs the converse seam.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The registry issues endpoints, answers each port's registration, keeps
where every instance lives, builds the tool table (each app once,
canonical names) and the briefs, pumps the links and forgets dead ones.
The core drives a Cx-free Model seam: routes each tool call to the
instance the model named (or the focused one), stripping the selector;
holds destructive calls for a confirm card, with host risk floors that a
service can raise but never lower; times out silent services, extended
by progress up to a hard cap; answers unknown names with the real ones;
reconfigures the model when the registry moves and restarts it when it
cannot rebind; honours result dispositions (end the turn, reset the
conversation); caps tool rounds; and offers a local tool console that
drives services without a model. Time is host-supplied seconds, never
Instant, so the same core runs on the web.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
An app id says what a service is; a host-issued EndpointId says which
instance. Ports register with a nonce, learn their endpoint from the
host's Registered answer, take only frames addressed to it, and never
stamp themselves on the way up. Hosted frames are refused over a size
cap before deserialization and every variant is validated on receipt;
tool schemas must describe an argument object. Results carry bounded
structured data and a disposition (continue, end the turn, reset the
conversation); Cancelled and TimedOut are outcomes of their own.
InstanceMeta records where an instance lives for the model.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
apps/files, apps/sheets, libs/wm_api, libs/wm_theme and the renamed env vars and script
namespaces meet the demo seams; added demo files move to the renamed paths; the files app
keeps its own makepad_home so demo builds without the chat feature do not link the AI hub.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
Packages are makepad-<name> with the short name as the binary. Env vars
follow (MAKEPAD_WM_*, MAKEPAD_FILES_*, MAKEPAD_TERMINAL_*), config moves
under ~/.makepad/<app>/, the theme namespaces are mod.wm_theme and
mod.browser_theme, the hosted AI envelope key is wm_ai. platform/video
becomes makepad-platform-video so the video app can be makepad-video.
Carries the Score entries that were pending in the WM's curated table.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
libs/ai/loader, libs/sqlite_query, libs/video_flow: 32-bit const overflows and unix-only
file locking/bulk reads gated so the app crates type-check for wasm32-unknown-unknown.
platform web: ToWasmInit arrived before the app's Window widget allocated window zero and
indexed an empty pool; the browser geometry is now stored and applied once the window
exists, and the other id_zero index sites are guarded.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV6BzHQiJEvvK9EPc1d4ks
An app exposes an AI service (manifest, tools with a risk class, a port
that receives calls and answers them later); a host aggregates every
connected service into one chat. Hosted by mpwm the port rides the
studio protocol's Custom frames under the mpwm_ai envelope; embedded, it
is a channel pair. The engine behind the feature flag and the panel that
draws EngineState land in the next lanes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
LocalModels runs registry models in-process (install state, resumable downloads, recorded licence acknowledgements at $MAKEPAD_HOME/license_acks.json, weight paths by file role) and libs/ai/hub_ui is the install panel + licence modal every app can embed. New native ports: Beat This! (beats + downbeats) and Basic Pitch (note transcription) with their registry entries; the Salamander Drumkit samples (CC BY-SA 3.0, 37 files pinned by size and sha256) as a sample bank the downloader fetches like a model.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The last two hub commits staged whole files and carried uncommitted hunks
of another lane (beats-native, notes-native, the local runner, new
domains and license keys) that reference files not yet in the tree. This
restores those files to the body changes only; the other lane's edits
stay in its working tree.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The native port covers everything the Python reference worker did (body,
hands, mask prompt, multi-person), so the subprocess backend, its fake
worker harness, the sam3dbody-ref registry entry and the
MAKEPAD_SAM3DBODY_* environment go. The packet validator moves to the
native backend.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The sam3dbody backend reads its options from the request's prompt string
(`hands`, `detect`, `persons=N`): `hands` runs the full mode and the
packet carries which hands were fused and their boxes; `detect` finds up
to N persons with SAM 3.1 (an optional native-segment role on the body
entry, the same artifact the segment entry pins) and runs one body pass
per person with its box and mask, so the packet's people array grows.
The body crate shares one body pass between the packet, mask and hands
entry points, and infer_full takes the mask prompt too.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
BodyModel::infer_masked takes a provided person box and its full-frame 0/1
mask: the mask is warped with the crop's own affine (bilinear, rounded back
to 0/1 like the reference's uint8 warp), encoded by the prompt encoder's
mask CNN (four stride-2 convolutions with channel LayerNorm and erf GELU,
then a 1x1 to 1280) and added to the backbone tokens before the ray
conditioning, replacing the folded no-mask term. Oracle parity on the
mask fixture: warp exact, every CNN stage within f32 accumulation noise,
conditioned context 8e-4, end to end kp3d 2.2 mm / kp2d 0.25 px. 41 tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
BodyModel::infer_full runs the reference's full mode natively: the body
pass's hand boxes become two 512 crops (the left one cut from the mirrored
image, padding 0.9), each goes through the backbone, its own ray
conditioning and the hand decoder (the body decoder with the *_hand
tensors), the hand-mode rig pre-transform (local_to_world_wrist, wrist and
root offsets, non-hand parameters zeroed) and the hand camera head's
scale factor of 10; the left result is un-mirrored. The fusion gates each
hand (wrist angle, box size, keypoint spread, wrist distance), re-prompts
the body decoder with the trusted wrists and elbows as point prompts (the
decoder now takes N prompt tokens and a previous estimate), and writes the
fused wrist angles, hand parameters and hand scale/shape back before the
final rig pass. pose.rs gains the roma xyz (extrinsic) and XZY (intrinsic)
euler pairs; mhr.rs returns joint global rotations.
Oracle parity on both full-mode fixtures: every hand-decoder stage per
step (tokens 7e-3, heads 4e-4, rig params 1e-4), fusion reports as the
reference (one hand trusted on the standing photo, both on the crop), and
end to end kp3d 1.6 mm / kp2d 0.85 px. 36 tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
One backend call per SAM-style decoder layer (PE norms, token
self-attention, token-to-image cross-attention, erf-GELU feed-forward,
final norm) inside one command buffer, with the layer's f32 weights cached
on the device under their content identity and pooled transients.
gpu_two_way_layer_resident in the common backend; CUDA declines and the
per-op path stays. Body decoder loop on the M3 Max: layers 100 -> 9 ms,
frame 405 -> 266 ms, oracle parity unchanged (kp3d 1.5 mm).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
gpu_linear_f32_resident promised a resident weight but the Metal tensor
backend re-uploaded it every call, which for the MHR rig's 664 MB pose
corrective matrix cost 50 ms a step. Metal tensors now carry a content
identity (fresh at creation and after every in-place write, so a cache
keyed by it can never serve a stale weight), and the resident linear caches
the device buffer under it. Body frame on the M3 Max: 705 -> 405 ms (rig
302 -> 20 ms, refine 41 -> 13 ms, heads 10 -> 6 ms).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The Metal tensor backend is host-Vec based: every op copied its inputs up
and its result back, which put the DINOv3 backbone at 2.1 s a frame on an
M3 Max. try_vit_backbone_resident_f32 runs a whole pre-norm ViT stack
(LayerNorm, q/k/v, rotate-half rope from tables, flash attention, out,
LayerNorm, SwiGLU, down; residuals in place) inside one command buffer
against cached bf16 weights, with a new kernel_makepad_rope_half_tables_f32
kernel for the rope. The precompiled metallib now carries the bf16 GEMM
kernels the runtime source compile already enabled on bfloat devices.
The common backend exposes it as gpu_vit_backbone_resident (CUDA declines;
its per-op path is already resident) and the body backbone tries it first:
233 ms a frame on the M3 Max, same oracle parity (kp3d 1.5 mm).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
