- ggml: read-only mmap module (unix-gated) + two-region Context (mapped
weights / dirty caches) + segmented Metal buffer binding; llama loads
GGUF weights as file-backed clean pages (jetsam-exempt) with owned-arena
fallback (MAKEPAD_LLAMA_NO_MMAP=1). Route app dirty footprint 12GB -> 4-7GB;
model load becomes lazy page-in; A/B byte-identical on 4B + 9B.
- llama: fix graph-cache keying corruption — a graph keyed wider than the
KV cache corrupted attention for any prefill batch >= 2 (flash op reads
permute-node dims baked at build; view reconfigure never reached the
kernel; masks were written cache-narrow). Masks now always fill the full
graph key width and graphs key by 1024-buckets; reconfigure path removed.
Verified byte-exact vs per-length reference across batch 1/2/8/64/512,
short+long prompts, mmap on/off, plus a two-session concurrency probe.
- voice: passive VoiceWaves no longer register the global audio-input
callback (the invisible caption-bar wave stole mic audio — last
registrant wins — and spawned duplicate whisper workers); whisper back
to F16 default (voice Metal library has no quantized kernels; q5_0
failed every GPU matmul); raw transcripts render immediately.
- route: kokoro TTS voice output (speaker toggle; streams reply sentences,
announces nav maneuvers + arrival) with barge-in — voice activity on the
mic stops playback instantly; dispatcher context 8k -> 32k (hybrid KV is
12/48 layers, ~48KB/token); window caption bar suppressed under studio.
- llama-generate: --max-context/--prefill-batch-size + state fingerprints;
new llama_concurrent_probe bin.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LlamaSession gains append_image_embeddings: precomputed vision embeddings
prefill through a second Embeddings-input graph spec sharing the same
cache tensors (SessionGraphParams grows an embeddings_input discriminant).
HybridDecodeBatchLayout gains an optional pre-expanded rope_positions
override, threaded to encode_rope_positions — cache indices and attention
masks stay linear while image tokens get qwen-vl 2D positions
[pos0, pos0+y, pos0+x, 0] and the span advances rope position by
max(w, h). Text after an image continues from the shifted position; pure
text paths are byte-identical to before (verified via llama-generate).
vlm-probe runs the whole thing: ppm -> vision tower -> chatml with
vision_start/end -> greedy. Output is token-for-token identical to
llama-mtmd-cli --temp 0 on both test images (radar scene description and
'A red circle.'). 229-token mixed prefill 0.98s, generation 27.9 tok/s
on the 9B UD-Q4_K_XL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New vision module: mmproj GGUF load (reuses the arch-agnostic loader),
exact-port preprocessing (calc_size_preserved_ratio, f64 bilinear resize to
match the reference binary's double promotion, block-major patch unfold),
and the 27-block ViT graph: patchify as two matmuls vs the flattened conv
weights, interpolated learned pos-embd (gpu bilinear+antialias, gathered to
block order), 2D vision rope, full bidirectional flash attention (f16 k/v,
f32 prec), layernorm composed as norm-mul-add, and the qwen3vl_merger
2-layer MLP into the LLM's 4096-dim space.
vlm-vision-probe validates against clip.cpp dumps: preproc bit-exact on
aligned images (1e-7), embeddings rms 3e-4..9e-4 / cosine 0.99999+ on all
three test images — inside the oracle's own flash-vs-composed spread
(rms 8.9e-4). 64-token encode 38ms, 192-token 103ms after graph compile.
Also fixes a stale DeltaNetRecurrentBlockSpec test initializer that broke
cargo test compilation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Grid width for the non-flat unary path was ne01 while the kernel decomposes
tgpig.x into (ne0-chunk, row) — rows past ntg were silently dropped, masked
below the 32768-element flat-path threshold; silu on [8192, n] crossed it at
prefill batch 4 and corrupted qwen35 batched prefill. Grid is now
ne01 * ceil(ne0/nth). Adds metal unary/gated-delta-net/ssm_conv regression
tests, the llama-batch-probe bisect harness, prefill batch default 32, and
keeps the graph cache until a reserve-retry actually needs eviction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
