makepad/tools/vlm_oracle
Admin 6ba9b7da65 llama: qwen3.5 vision tower on makepad-ggml — oracle parity achieved
New vision module: mmproj GGUF load (reuses the arch-agnostic loader),
exact-port preprocessing (calc_size_preserved_ratio, f64 bilinear resize to
match the reference binary's double promotion, block-major patch unfold),
and the 27-block ViT graph: patchify as two matmuls vs the flattened conv
weights, interpolated learned pos-embd (gpu bilinear+antialias, gathered to
block order), 2D vision rope, full bidirectional flash attention (f16 k/v,
f32 prec), layernorm composed as norm-mul-add, and the qwen3vl_merger
2-layer MLP into the LLM's 4096-dim space.

vlm-vision-probe validates against clip.cpp dumps: preproc bit-exact on
aligned images (1e-7), embeddings rms 3e-4..9e-4 / cosine 0.99999+ on all
three test images — inside the oracle's own flash-vs-composed spread
(rms 8.9e-4). 64-token encode 38ms, 192-token 103ms after graph compile.

Also fixes a stale DeltaNetRecurrentBlockSpec test initializer that broke
cargo test compilation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 14:23:26 +02:00
..
build.sh vlm: clip.cpp oracle tool — dump qwen3.5 mmproj preproc + embeddings for parity testing 2026-07-28 14:07:18 +02:00
clip_dump vlm: clip.cpp oracle tool — dump qwen3.5 mmproj preproc + embeddings for parity testing 2026-07-28 14:07:18 +02:00
clip_dump.cpp llama: qwen3.5 vision tower on makepad-ggml — oracle parity achieved 2026-07-28 14:23:26 +02:00
gen_test_image.py vlm: clip.cpp oracle tool — dump qwen3.5 mmproj preproc + embeddings for parity testing 2026-07-28 14:07:18 +02:00
grad_256x256.ppm vlm: clip.cpp oracle tool — dump qwen3.5 mmproj preproc + embeddings for parity testing 2026-07-28 14:07:18 +02:00
radar_500x375.ppm vlm: clip.cpp oracle tool — dump qwen3.5 mmproj preproc + embeddings for parity testing 2026-07-28 14:07:18 +02:00
radar_512x384.ppm vlm: clip.cpp oracle tool — dump qwen3.5 mmproj preproc + embeddings for parity testing 2026-07-28 14:07:18 +02:00