723b7928bb
- ggml: read-only mmap module (unix-gated) + two-region Context (mapped weights / dirty caches) + segmented Metal buffer binding; llama loads GGUF weights as file-backed clean pages (jetsam-exempt) with owned-arena fallback (MAKEPAD_LLAMA_NO_MMAP=1). Route app dirty footprint 12GB -> 4-7GB; model load becomes lazy page-in; A/B byte-identical on 4B + 9B. - llama: fix graph-cache keying corruption — a graph keyed wider than the KV cache corrupted attention for any prefill batch >= 2 (flash op reads permute-node dims baked at build; view reconfigure never reached the kernel; masks were written cache-narrow). Masks now always fill the full graph key width and graphs key by 1024-buckets; reconfigure path removed. Verified byte-exact vs per-length reference across batch 1/2/8/64/512, short+long prompts, mmap on/off, plus a two-session concurrency probe. - voice: passive VoiceWaves no longer register the global audio-input callback (the invisible caption-bar wave stole mic audio — last registrant wins — and spawned duplicate whisper workers); whisper back to F16 default (voice Metal library has no quantized kernels; q5_0 failed every GPU matmul); raw transcripts render immediately. - route: kokoro TTS voice output (speaker toggle; streams reply sentences, announces nav maneuvers + arrival) with barge-in — voice activity on the mic stops playback instantly; dispatcher context 8k -> 32k (hybrid KV is 12/48 layers, ~48KB/token); window caption bar suppressed under studio. - llama-generate: --max-context/--prefill-batch-size + state fingerprints; new llama_concurrent_probe bin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| src | ||
| build.rs | ||
| Cargo.toml | ||