Taken from vjroger/work. A full merge of that fork fights this work branch
in widgets; this is the stems crate, vocal mel-band, span-cache lanes,
log_ring, output fence, midi inject, effect_doc, mp3 sniff, mp4 audio
edit, and audio-only file open. Stage on origin/main cargo-checks again.
Squash of 54 work commits (Sep 1–12):
6251f7c ai-hub: body domain — live pose packets ride the realtime session
ea50c77 chat_ui: the feed's session gets its profile brief back
f51b5f3 ai-body: the crate for the native SAM 3D Body port, with its weights reader
8211ae6 ai-body: the MHR rig and the pose head's parameter decoding, oracle-exact
9e343a8 ai-body: the DINOv3 ViT-H+/16 backbone, crop and ray conditioning; Metal gains rope-half and affine layer norm
69d842c ai-body: the promptable pose decoder and its refinement loop, oracle-matched on Metal
66e5e2f ai-hub: SAM 3D Body runs natively — `sam3dbody` on the body domain, oracle-matched end to end
a634198 ai-hub: the body-native commit carried a peer's in-flight hub hunks; put them back where they were
9ff44e8 ai-hub: the body-native wiring, this time only the lane's hunks
6a1c16b ai-body: third-party notices — what the port is implemented after, and what it is not
d78411a ai-body: the per-step work moves to the GPU
b22259b ai-body: the context stays on the GPU; only the pose token leaves the loop
346f31f ai-body: flash attention for the head-dim-64 blocks
45b5b98 ai-body: the crop size is a runtime knob, and the loop reports where its time goes
4be6d19 ai-body: the test modules import the grid constants they still use
7598346 ai-body: tensor-core GEMMs for the backbone, and the rig's correctives only where they count
a9ce596 ai-body: the crop warp runs across cores
8964ba6 ai-body: an FP8 backbone mode, off by default, measured against the oracle
a2aaa8f ai-body: the FP8 bias rides a column-broadcast add on the device
d53c77d metal: a device-resident ViT stack, and the body backbone rides it
d006d0a metal: resident f32 linears keep their weight on the device
525ba1c metal: a device-resident two-way decoder layer, and the body decoder rides it
c9e6d88 ai-body: the hands pass — hand crops, the hand decoder, the hand-mode rig and the wrist fusion
62dff26 ai-body: the mask prompt — a person's segmentation mask conditions the body pass
a648cf8 ai-hub: body session options — hands, detect, persons=N
8c568df ai-hub: drop the SAM 3D Body reference worker backend
7ff875a ai-hub: keep a peer's in-flight beats/notes/local work out of the body commits
31e5faa ai-hub: local model runner, licence acknowledgements, a shared install panel; Beat This!, Basic Pitch and the Salamander drum-kit entries
b94bc58 ai-services: the wire, the app port and the panel state — one conversation, many apps
2acb798 ai-services: wire v2 — endpoints, receiver-side caps, result disposition
8ae0ffb ai-services: the engine core — registry, router and conversation, tested against a scripted model
2308736 ai-services: the real models behind the engine feature — local through the hub, Claude, and none
c3f631d livepipe: one reusable pipe from a camera to a fleet node and back
ff62db3 ai libs: the runtime env-var cleanup — precision is a per-caller policy, not an environment side channel
04a94ef realtime: one service-log line when a live session opens and one when it closes
0ecb81c ai models: the model-crates env-var cleanup — 172 research knobs gone, the unset default is the code
4ca36c1 ai hub + services: the assistant's model comes from wherever it is resident — the fleet chat box, with tools, then the local weights
432121e aichat engine + wm: launch, then use — the assistant continues in the same turn once the app it started is on the bus
7a5bf69 ai-hub registry: the Salamander drumkit samples come from the makepad.nl mirror — the GitHub repo only carries the .sfz files
102ffc5 ai-services: messages on the bus — a manifest declares topics, the engine subscribes on a tool's behalf or by ToolResult.subscribe, a service publishes Message frames, an idle conversation wakes on a message as an event turn under rate laws; the WM bus forwards the new frames; every app that matches the wire gets its arm
a837792 hub + flow: a whitespace-only chat completion is retried once and then fails instead of passing as an answer; a flow's model is a fleet model id unless it names a weight file on disk; chat models show under the text domain in /v1/models
bc6c620 hub + flow: what the chat review found — the in-process route retries an empty completion too, a node says whether its prefill opened thinking so a brief-mode answer is never discarded, a preferred model falls back to normal election when no node has it, discovery keeps looking for the preferred model until patience runs out
75c3441 hub: the PRO 6000 serves image as well as chat and text
ad5e98b hub registry: flux2-dev's VRAM estimate is its measured peak, 30 GB
c7241e0 hub: a node that evicted every resident releases its cached allocator pool before refusing a load or publishing usable VRAM
30575f0 flow: route generation by request workload
1be1e21 ai-hub: gate downloads by disk capacity and recover fleet admission
df6b394 filesystem_watcher, bounded_http, ai services: live and tool prerequisites
79ebdb9 ai-hub: add a native Pixal3D image-to-3D backend
0ba0d74 ai-hub: propagate typed refusals under reject queue policy
cc6c872 Speed up H3 conditioning and video decoding
e512059 Fix Qwen vision residency and generated material colors
2864f68 ai-hub http client: bound every plain TCP connect to 3 s per address
3d93229 ai: CUDA is a Linux/Windows-only dependency; the hub library defaults to llm + stt
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Squashed from work; the fine-grained history is under tag archive/work-2026-08-29:
- mp* wave: mpwm window manager + the mp app family, WM API, theme bridge, PDF engine fix
- work: land the sources the last commits reference
- kenney: catalogue all 50 free 3D kits; Modal dismissed() never fired
- asset store: central vision-annotation queue; Kenney donate prompt
- asset-ai: vision domain — image + prompt -> text on every fleet node
- annotation runs the fleet's vision services through the normal job queue
- vj: responsive DJ mixer + Windows drag-and-drop, cherry-picked from PR #1199 (vjroger)
- store search: WordNet synonym expansion, query-side; per-term seeks
- video scheduling: a cold model pin never downloads past a warm one, and a dedicated box only serves its role
- an expansion can never lose a run — and `expand: true` finally means something
- fleet panel: a slow box is not a missing box
- h3: first+last keyframe conditioning (the weights were always FL2VA)
- registry: the three H3 FL2VA tiers name their real conditioning
- fleet scheduling: spread before stacking, and a stuck job moves
- a download never steals a job from a box that has the weights
- the faster GPU takes the tie: 6000 > 5090 > 4090
- an evicted flux model gives the card its VRAM back
- a job now records what each of its stages was handed
- open a stage in RUNS and read what it sent
- every expanded song was an instrumental: the lyrics had nowhere to go
- the writer never overwrites words the person wrote
- store: a dependent job's body is spliced from its deps' results at claim
- a fleet box's job row says what that job was asked for
- the store runs a whole pipeline, and one record says how far it got
- a client can declare a whole run, watch it, and stop it
- 100% now means published, and the expander is a job you can queue
- one card says what a spawned task is doing, everywhere
- a run that ends says so, instead of being noticed later
- when every box holding the model is busy, buy another copy
The model code was spread across eight crates that had grown into each other:
ggml and cuda and mlx each owned part of a tensor runtime, llama and tts and
voice2 each owned part of a model, and libs/diffusion owned everything else.
They are now one tree with an explicit shape:
libs/ai/cuda — kernels and launch surface
libs/ai/metal — Metal shaders and the shim
libs/ai/llm — the language-model runtime (sessions, lanes, contexts,
the CUDA and Metal executors, the compiled Metal path)
libs/ai/models/ — common, flux, h3, music, paint, speech, stems, vision
libs/diffusion is not deleted but demoted: what remains is the VALIDATOR
crate — several dozen `*_validate.rs` oracles that check a native
implementation against a reference, which is where they belong now that the
implementations live next door.
The functional work inside the move is mostly in the LLM runtime: N lanes that
draft while one verify batch serves all of them, per-slot prefill over a shared
folded attention arena, speculation that survives batching, and a scheduler
that reports rather than publishes. And in the CUDA build: a machine without
usable CUDA must still LINK (and say so), the default kernel arch is the
building machine's GPU, `NO_CUDA` forces the stub even where the toolkit
exists, and kernels compile in parallel with progress.
libs/video_flow is new here: classical optical flow estimation and the `mkfl`
motion-field payload — a flow field measured from a clip without a model,
which is what drives free-rate bounce-looping playback and the uprez/tween
enhance pipe.
Admin