LlamaSession gains append_image_embeddings: precomputed vision embeddings
prefill through a second Embeddings-input graph spec sharing the same
cache tensors (SessionGraphParams grows an embeddings_input discriminant).
HybridDecodeBatchLayout gains an optional pre-expanded rope_positions
override, threaded to encode_rope_positions — cache indices and attention
masks stay linear while image tokens get qwen-vl 2D positions
[pos0, pos0+y, pos0+x, 0] and the span advances rope position by
max(w, h). Text after an image continues from the shifted position; pure
text paths are byte-identical to before (verified via llama-generate).
vlm-probe runs the whole thing: ppm -> vision tower -> chatml with
vision_start/end -> greedy. Output is token-for-token identical to
llama-mtmd-cli --temp 0 on both test images (radar scene description and
'A red circle.'). 229-token mixed prefill 0.98s, generation 27.9 tok/s
on the 9B UD-Q4_K_XL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New vision module: mmproj GGUF load (reuses the arch-agnostic loader),
exact-port preprocessing (calc_size_preserved_ratio, f64 bilinear resize to
match the reference binary's double promotion, block-major patch unfold),
and the 27-block ViT graph: patchify as two matmuls vs the flattened conv
weights, interpolated learned pos-embd (gpu bilinear+antialias, gathered to
block order), 2D vision rope, full bidirectional flash attention (f16 k/v,
f32 prec), layernorm composed as norm-mul-add, and the qwen3vl_merger
2-layer MLP into the LLM's 4096-dim space.
vlm-vision-probe validates against clip.cpp dumps: preproc bit-exact on
aligned images (1e-7), embeddings rms 3e-4..9e-4 / cosine 0.99999+ on all
three test images — inside the oracle's own flash-vs-composed spread
(rms 8.9e-4). 64-token encode 38ms, 192-token 103ms after graph compile.
Also fixes a stale DeltaNetRecurrentBlockSpec test initializer that broke
cargo test compilation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Grid width for the non-flat unary path was ne01 while the kernel decomposes
tgpig.x into (ne0-chunk, row) — rows past ntg were silently dropped, masked
below the 32768-element flat-path threshold; silu on [8192, n] crossed it at
prefill batch 4 and corrupted qwen35 batched prefill. Grid is now
ne01 * ceil(ne0/nth). Adds metal unary/gated-delta-net/ssm_conv regression
tests, the llama-batch-probe bisect harness, prefill batch default 32, and
keeps the graph cache until a reserve-retry actually needs eviction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
