Squashed from work; the fine-grained history is under tag archive/work-2026-08-26:
- vj: thumbnails become mp4 — hardware-coded sheets at measured-4K cells, and the bake stops racing the GPU
- vj: the console grows real transports, and the deck stops lying about reverse
- vj: reverse earns a memory, and the effects stop aging
- vj: video goes NV12 end to end, and the GPU does the unpacking
- vj: the GPU learns to see motion — realtime frame tweening on every deck
- vj: the tweener learns — RIFE runs on the Mac and feeds the warp
- vj: the classical tweener grows up, and every deck gets a tween chip
- vj: the tween clock tells presented time, not producer time
- vj: the transport becomes a platter — velocity in, position out, one map
- vj: the tween presenter reads the platter — one clock per deck, cued once at the frame on screen
- vj: the OFF tier joins the platter — a resident clip's picture is cache[nearest(pos)]
- vj: the media thread loses its second clock — resident clips park the decoder
- vj: the producer gets a contract — keyed ladders, deadlines, and a capacity law
- vj: two decks, one law — identical inputs are bit-identical, and the warp agrees to the byte
- vj: the presenter switch lands without its scaffolding
- vj: the next pair's fields are fetched ahead of the change under the capacity law — a pair change costs an ordinary beat; macos: the layer's own display link paces the frame when the system offers it, the old path stays as fallback
- vj: AI3 subdivides adaptively — one, three or seven neural frames per pair, chosen from measured synth time against the pair's own period, with classical flow between them and a 7-3-1-FL fallback; the deck shows the depth
- video_flow: the flow debug bins, declared behind the convert feature so --no-default-features skips them instead of failing
The model code was spread across eight crates that had grown into each other:
ggml and cuda and mlx each owned part of a tensor runtime, llama and tts and
voice2 each owned part of a model, and libs/diffusion owned everything else.
They are now one tree with an explicit shape:
libs/ai/cuda — kernels and launch surface
libs/ai/metal — Metal shaders and the shim
libs/ai/llm — the language-model runtime (sessions, lanes, contexts,
the CUDA and Metal executors, the compiled Metal path)
libs/ai/models/ — common, flux, h3, music, paint, speech, stems, vision
libs/diffusion is not deleted but demoted: what remains is the VALIDATOR
crate — several dozen `*_validate.rs` oracles that check a native
implementation against a reference, which is where they belong now that the
implementations live next door.
The functional work inside the move is mostly in the LLM runtime: N lanes that
draft while one verify batch serves all of them, per-slot prefill over a shared
folded attention arena, speculation that survives batching, and a scheduler
that reports rather than publishes. And in the CUDA build: a machine without
usable CUDA must still LINK (and say so), the default kernel arch is the
building machine's GPU, `NO_CUDA` forces the stub even where the toolkit
exists, and kernels compile in parallel with progress.
libs/video_flow is new here: classical optical flow estimation and the `mkfl`
motion-field payload — a flow field measured from a clip without a model,
which is what drives free-rate bounce-looping playback and the uprez/tween
enhance pipe.
