7f59912916
The model code was spread across eight crates that had grown into each other:
ggml and cuda and mlx each owned part of a tensor runtime, llama and tts and
voice2 each owned part of a model, and libs/diffusion owned everything else.
They are now one tree with an explicit shape:
libs/ai/cuda — kernels and launch surface
libs/ai/metal — Metal shaders and the shim
libs/ai/llm — the language-model runtime (sessions, lanes, contexts,
the CUDA and Metal executors, the compiled Metal path)
libs/ai/models/ — common, flux, h3, music, paint, speech, stems, vision
libs/diffusion is not deleted but demoted: what remains is the VALIDATOR
crate — several dozen `*_validate.rs` oracles that check a native
implementation against a reference, which is where they belong now that the
implementations live next door.
The functional work inside the move is mostly in the LLM runtime: N lanes that
draft while one verify batch serves all of them, per-slot prefill over a shared
folded attention arena, speculation that survives batching, and a scheduler
that reports rather than publishes. And in the CUDA build: a machine without
usable CUDA must still LINK (and say so), the default kernel arch is the
building machine's GPU, `NO_CUDA` forces the stub even where the toolkit
exists, and kernels compile in parallel with progress.
libs/video_flow is new here: classical optical flow estimation and the `mkfl`
motion-field payload — a flow field measured from a clip without a model,
which is what drives free-rate bounce-looping playback and the uprez/tween
enhance pipe.
77 lines
1.8 KiB
TOML
77 lines
1.8 KiB
TOML
[package]
|
|
name = "makepad-ai-llm"
|
|
version = "0.1.0"
|
|
edition = "2021"
|
|
description = "GGUF LLM family (Qwen). Graph builders + Metal/CUDA exec. Was libs/llama (aiarch.md §1)."
|
|
license = "MIT"
|
|
|
|
[dependencies]
|
|
# Bake the metallib at build time. Without this, MetalRuntime::new() compiles
|
|
# the 13k-line shader source on the GPU and can stall WindowServer.
|
|
makepad-ai-loader = { path = "../loader" }
|
|
makepad-ai-cuda = { path = "../cuda" }
|
|
makepad-ai-metal = { path = "../metal", features = ["metal-precompile"] }
|
|
|
|
[[bin]]
|
|
name = "llama-load"
|
|
path = "src/bin/llama_load.rs"
|
|
|
|
[[bin]]
|
|
name = "llama-load-time"
|
|
path = "src/bin/llama_load_time.rs"
|
|
|
|
[[bin]]
|
|
name = "llama-graph-census"
|
|
path = "src/bin/llama_graph_census.rs"
|
|
|
|
[[bin]]
|
|
name = "llama-cuda-canary"
|
|
path = "src/bin/llama_cuda_canary.rs"
|
|
|
|
[[bin]]
|
|
name = "llama-compare"
|
|
path = "src/bin/llama_compare.rs"
|
|
|
|
[[bin]]
|
|
name = "llama-generate"
|
|
path = "src/bin/llama_generate.rs"
|
|
|
|
[[bin]]
|
|
name = "llama-tokenize"
|
|
path = "src/bin/llama_tokenize.rs"
|
|
|
|
[[bin]]
|
|
name = "llama-slot-probe"
|
|
path = "src/bin/llama_slot_probe.rs"
|
|
|
|
[[bin]]
|
|
name = "llama-lane-spec-probe"
|
|
path = "src/bin/llama_lane_spec_probe.rs"
|
|
|
|
[[bin]]
|
|
name = "llama-batch-probe"
|
|
path = "src/bin/llama_batch_probe.rs"
|
|
|
|
# The decode column-cost curve: what an extra batch column actually costs,
|
|
# which is what sets how many chats one card can serve.
|
|
[[bin]]
|
|
name = "llama-batch-bench"
|
|
path = "src/bin/llama_batch_bench.rs"
|
|
|
|
[[bin]]
|
|
name = "vlm-vision-probe"
|
|
path = "src/bin/vlm_vision_probe.rs"
|
|
|
|
[[bin]]
|
|
name = "vlm-probe"
|
|
path = "src/bin/vlm_probe.rs"
|
|
|
|
# Batch executor for the asset-annotation pass (libs/asset/annotate): one
|
|
# resident model, many sheets, line-oriented job/result files.
|
|
[[bin]]
|
|
name = "vlm-annotate"
|
|
path = "src/bin/vlm_annotate.rs"
|
|
|
|
[[bin]]
|
|
name = "llama-reset-probe"
|
|
path = "src/bin/llama_reset_probe.rs"
|