Commit graph

2 commits

Author SHA1 Message Date
Admin
6ba9b7da65 llama: qwen3.5 vision tower on makepad-ggml — oracle parity achieved
New vision module: mmproj GGUF load (reuses the arch-agnostic loader),
exact-port preprocessing (calc_size_preserved_ratio, f64 bilinear resize to
match the reference binary's double promotion, block-major patch unfold),
and the 27-block ViT graph: patchify as two matmuls vs the flattened conv
weights, interpolated learned pos-embd (gpu bilinear+antialias, gathered to
block order), 2D vision rope, full bidirectional flash attention (f16 k/v,
f32 prec), layernorm composed as norm-mul-add, and the qwen3vl_merger
2-layer MLP into the LLM's 4096-dim space.

vlm-vision-probe validates against clip.cpp dumps: preproc bit-exact on
aligned images (1e-7), embeddings rms 3e-4..9e-4 / cosine 0.99999+ on all
three test images — inside the oracle's own flash-vs-composed spread
(rms 8.9e-4). 64-token encode 38ms, 192-token 103ms after graph compile.

Also fixes a stale DeltaNetRecurrentBlockSpec test initializer that broke
cargo test compilation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 14:23:26 +02:00
Admin
9f851e2d45 vlm: clip.cpp oracle tool — dump qwen3.5 mmproj preproc + embeddings for parity testing
Links the local llama.cpp April build (libmtmd exports clip_* internals);
generates deterministic PPM test images; dumps preprocessed f32 tensor and
projected [n_tokens x 4096] embeddings. Reference stats recorded: radar
512x384 -> 192 tokens, cpu-vs-metal spread ~2e-4 abs on embd values.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 14:07:18 +02:00