Squashed from work: - asset-ai: FastH3 4-step fast video backend; clip keyframes on the wire - asset-ui: loop video chains — text→image→video that ends where it began - h3: safetensors -> pruned-Q4_K GGUF quantizer for the 24GB DiT tiers - h3_quant_gguf verify: row-error gates calibrated to the measured Q4_K floor - asset-ai realtime: the feedback loop — the source anchors, the drifted frame inits - asset-ai realtime: a feedback loop survives a resize and travels by default - asset-ai realtime: the feedback loop frees itself from the feed handshake and pauses for its listener - asset-ai realtime: the outbound encode leaves the loop's critical path - asset-ai ocr: the ocr domain — Chandra 2 at page resolution, and the tower goes planner-owned - llm slots: a lane can hold an image span — embedding prefill and a rope cursor of its own - vision tower on CUDA: the encode leg gets its two missing kernels - llm/ocr: one M-RoPE grid encoder for both image paths, and a livelock made an error - vision tower on CUDA: the f16 GEMM keeps the precision it was throwing away - live: a feed that moves box takes its trip with it — one seed image - vision tower on CUDA: the tiled attention becomes bit-exact, and tensor cores go - llm prefill on CUDA: the MMA attention kernel gets the tile a 4-to-1 model needs - asset-ai ocr: the CUDA encode lane joins the integration — vision-parity sits beside run's three arms, and the kernels - Merge branch 'ocr-perf-integration' into work - asset-ai: the live anchor can follow the trip, and text leaves the 5090 - asset-ai: the camera moves the world, and the world starts still - asset-import: the EA strategy classics, in the one 2D contract - rtsmap: one seeded generator for tiled strategy maps - asset-ui: one card for the strategy classics, with a pack dropdown - asset-ai: music3 reference-audio path, ocr/h3 backends, registry - asset: mp4 sample index for range-streaming, chat tools, import profiles - cnc: tiberium is twelve growth frames, not twelve empty variants - platform: native file and save dialogs, in-house on all three desktops - chat: the scan holds out for a lane home - chat: a full home queues you — take the free lane - chat: the preload has a percentage, and the boundless cap stops showing - llm cuda: the 32x2 attention tile — even GQA ratios stay on MMA - sa3 gets a bake path: the sfx model's tables precomputed by a diffusion-side bin - sqlite_query: anti-join regression test - td import: HARV's second frame block is its harvesting cycle, not a turret - asset-ui: sprite enhancement runs on the 32B dev DiT — distillation, not the prompt, was the ceiling - ai-hub: makepad-asset-ai becomes makepad-ai-hub at libs/ai/hub, the chat pane becomes makepad-chat-ui, the service bin - asset-ui: test health fixtures grow the realtime field they were born without - ai-hub: one home at ~/.makepad — weights/ run/ cache/ logs/, the service cache migrates from ai_content by a single re - ai-hub: subprocess workers die with the node — process groups everywhere, PDEATHSIG on linux, one KILL_ON_JOB_CLOSE Jo - ai-hub: the hub object — AiHub::in_process, pipes vocabulary, and the local LLM engine generalized out of mpfiles (aic - strict-json: the dependency-free JSON module gets its own crate; asset-client re-exports it so nothing downstream move - ai-hub: the machine layer — node entries, the 0600 machine token, and the residency election that IS the lock (aicore - ai-hub: MPHUB1 — the fabric beacon only dedicated nodes can send (aicore §4) - ai-hub: job leases — work lives only while it is renewed (aicore §8) - asset-creator: the pipeline library is born — specs, the deps gate, and the derived-state law (aicore §9) - ai-hub: RAM residency facts — the CPU-side twin of residency.rs (aicore §3) - ai-hub: ETA placement primitives — relative GPU throughput, the four-term estimate, and an observable breakdown (aicor - ai-hub: leases go live on the wire — origin fields on submit, /job/<id>/keepalive, /bye, and the reaper that cancels w - ai-hub: the chat providers move in — fleet qwen, openai, grok, claude/codex/grok CLIs, the responses driver, and the w - asset-creator: the engine — one pipeline run against the hub, deps-gated, spliced, cancellable, resumable-by-construct - ai-hub: the machine node mode — --machine binds loopback, registers in ~/.makepad/run, and exits on its own once idle - asset-creator: makepad-creator-run — the detached client for runs that must outlive a window (aicore §9) - ai-hub: a native Claude Messages-API provider — API-key or Claude Code OAuth, bounded SSE streaming, injected tools (a - route + converse: off makepad_ai — the Agent seam moves to converse, route's cloud dispatcher rides the hub's Claude p - asset-creator: the preset tables move in — fifteen chain-policy constants shared by every creator app (aicore §9 / P6) - makepad_ai is deleted — every backend is a hub pipe, the agent seam lives with its consumers (aicore §14, decided 2026 - ai-hub: loads hold the machine residency election — set_model_state claims on Loaded and publishes the service port (a - ai-hub: chats run the machine election — route to a serving holder, wait on a loading one, claim and publish when open - ai-hub: pick_for_domain_eta — ETA-ranked placement over the shared hard-filter core (aicore §6 / P4) - asset-creator: the engine picks a provider per stage at dispatch time — a chain's later stages see fresh fleet state ( - ai-hub: the fabric secret gates the service HTTP surface — bearer on everything but /health and the ticketed peer path - vj: DREAM runs execute in the app — pipelines.rs becomes the run it used to watch (aicore §9 / F1) - asset-creator: the runner — generate one thing and put it in the catalog, one implementation for every surface (aicore - chat-ui: the session runs in the app — no broker anywhere on the chat path (aicore P8 / F5) - asset-store: assets.query is a first-class query endpoint — the bounded SQL surface outlives the broker (aicore P8 / F - asset-creator: CreatorTools — the chat tool pack for a store that only stores (aicore §9 / P8) - asset-store: the shrink — the store stores (aicore P7) - importer + asset-server host: the coordination era ends (aicore P7) - store config purge + asset-ui goes fleet-direct; the derive protocol gets its route proof (aicore P7) - client + chat dispatcher: the dead wire comes out (aicore P7/P8) - ai-hub: 0.3.0 — the health version says which era a node runs - ai-hub: the default fleet is 'gen' — apps hear the LAN without env plumbing - ai-hub: the preload note percents the prefill, not the job bar - ai-hub: conversations keep their KV — the wire mirror, the lane identity, the in-turn dynamic context (aicore §7) - ai-hub: an open-think model is thinking from its first token - libs: the zero-warning sweep — stitch casts say what they mean, xatlas keeps upstream's surface quietly - zero-warning sweep, round two — the first full-workspace pass - zero-warning sweep, round three — the model lanes and the deep examples - zero-warning sweep, round four — the last stragglers - zero-warning sweep, round five — vj and chat-ui - zero-warning sweep, round six — three cascades Co-authored-by: Claude <info@makepad.nl>
179 lines
6.3 KiB
Rust
179 lines
6.3 KiB
Rust
//! One real turn, end to end, on the in-app session (aicore P8): a scripted
|
|
//! PROVIDER through the feed's factory seam, the worker thread, a tool the
|
|
//! `gen` profile parks on the app — executed and answered by function call —
|
|
//! and the transcript that comes out the other side. No broker anywhere:
|
|
//! that is the point.
|
|
|
|
use makepad_asset_chat::wire::ToolOutcome;
|
|
use makepad_ai_hub::chat_wire::{ProviderAvailability, ProviderKind, ServingFacts};
|
|
use makepad_ai_hub::providers::provider::{ChatProvider, ProviderEvent, TurnInput};
|
|
use makepad_chat_ui::feed::{ChatFeed, ClientTools, FeedConfig};
|
|
use makepad_chat_ui::transcript::{ChatData, ChatRole, CHAT};
|
|
use makepad_asset_client::json::{self, Value};
|
|
use makepad_asset_client::ApiEndpoints;
|
|
use std::sync::mpsc::{self, Receiver, Sender};
|
|
use std::sync::{Arc, Mutex};
|
|
use std::time::{Duration, Instant};
|
|
|
|
/// Scripted provider: each `begin_turn` shifts the next event script.
|
|
/// Send-safe (the feed's worker owns it across threads).
|
|
struct Scripted {
|
|
scripts: Mutex<Vec<Vec<ProviderEvent>>>,
|
|
pending: Mutex<Vec<ProviderEvent>>,
|
|
}
|
|
|
|
impl Scripted {
|
|
fn new(scripts: Vec<Vec<ProviderEvent>>) -> Scripted {
|
|
Scripted { scripts: Mutex::new(scripts), pending: Mutex::new(Vec::new()) }
|
|
}
|
|
}
|
|
|
|
impl ChatProvider for Scripted {
|
|
fn kind(&self) -> ProviderKind {
|
|
ProviderKind::FleetQwen
|
|
}
|
|
fn availability(&mut self) -> ProviderAvailability {
|
|
ProviderAvailability::Available { model: "scripted".into(), detail: "test".into() }
|
|
}
|
|
fn begin_turn(&mut self, _input: &TurnInput) -> Result<(), String> {
|
|
let mut scripts = self.scripts.lock().unwrap();
|
|
if scripts.is_empty() {
|
|
return Err("script exhausted".to_string());
|
|
}
|
|
*self.pending.lock().unwrap() = scripts.remove(0);
|
|
Ok(())
|
|
}
|
|
fn poll(&mut self) -> Vec<ProviderEvent> {
|
|
std::mem::take(&mut *self.pending.lock().unwrap())
|
|
}
|
|
fn cancel(&mut self) {
|
|
self.pending.lock().unwrap().clear();
|
|
}
|
|
fn continue_function(&mut self, _call_id: &str, _output: &str) -> Result<(), String> {
|
|
let mut scripts = self.scripts.lock().unwrap();
|
|
if scripts.is_empty() {
|
|
return Err("script exhausted".to_string());
|
|
}
|
|
*self.pending.lock().unwrap() = scripts.remove(0);
|
|
Ok(())
|
|
}
|
|
}
|
|
|
|
/// The app under test: records the parked call, answers ok.
|
|
struct RecordingTools {
|
|
calls: Sender<(String, Value)>,
|
|
}
|
|
|
|
impl ClientTools for RecordingTools {
|
|
fn execute(&mut self, name: &str, args: &Value) -> ToolOutcome {
|
|
let _ = self.calls.send((name.to_string(), args.clone()));
|
|
ToolOutcome::Ok { value: json::obj(vec![("queued", Value::Bool(true))]) }
|
|
}
|
|
}
|
|
|
|
fn wait_for(what: &str, mut done: impl FnMut() -> bool) {
|
|
let deadline = Instant::now() + Duration::from_secs(30);
|
|
while !done() {
|
|
assert!(Instant::now() < deadline, "timed out waiting for {what}");
|
|
std::thread::sleep(Duration::from_millis(25));
|
|
}
|
|
}
|
|
|
|
fn serving() -> ServingFacts {
|
|
ServingFacts {
|
|
gen_tokens: 4,
|
|
lanes_active: None,
|
|
slots_total: None,
|
|
think_tokens: None,
|
|
visible_tokens: Some(4),
|
|
prefix_ingested: None,
|
|
prefix_resumed: None,
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn a_turn_streams_runs_the_apps_tool_and_lands() {
|
|
let (calls_tx, calls_rx): (Sender<(String, Value)>, Receiver<(String, Value)>) =
|
|
mpsc::channel();
|
|
// Turn script: stream, park image.generate on the app, then finish.
|
|
let scripts = Arc::new(Mutex::new(Some(vec![
|
|
vec![
|
|
ProviderEvent::Delta("Making a trawler…".to_string()),
|
|
ProviderEvent::Serving(serving()),
|
|
ProviderEvent::FunctionCall {
|
|
call_id: "call_1".to_string(),
|
|
name: "image_generate".to_string(),
|
|
arguments: json::obj(vec![
|
|
("prompt", json::s("a rusty trawler")),
|
|
("width", Value::Int(768)),
|
|
("height", Value::Int(512)),
|
|
])
|
|
.to_json(),
|
|
},
|
|
],
|
|
vec![
|
|
ProviderEvent::Delta("Queued the trawler image.".to_string()),
|
|
ProviderEvent::Done { text: String::new() },
|
|
],
|
|
])));
|
|
// Endpoints nobody answers: the executor half degrades to honest
|
|
// "unreachable" capability text; parked tools never need it.
|
|
let endpoints = ApiEndpoints {
|
|
control: "127.0.0.1:1".parse().unwrap(),
|
|
data: "127.0.0.1:1".parse().unwrap(),
|
|
};
|
|
let mut cfg = FeedConfig::new(
|
|
endpoints,
|
|
None,
|
|
std::env::temp_dir().join(format!("mp_chat_ui_feed_{}", std::process::id())),
|
|
"gen",
|
|
"gen",
|
|
);
|
|
cfg.provider_factory = Some(Arc::new(move || {
|
|
let scripts = scripts
|
|
.lock()
|
|
.unwrap()
|
|
.take()
|
|
.expect("one session per test");
|
|
Box::new(Scripted::new(scripts))
|
|
}));
|
|
let feed = ChatFeed::start(cfg, Box::new(RecordingTools { calls: calls_tx }));
|
|
|
|
// The app owns the user's bubble — exactly as a host does it.
|
|
ChatData::push(ChatRole::User, "make me a trawler");
|
|
feed.send("make me a trawler".into(), Vec::new());
|
|
|
|
// The session parked image.generate on us and the worker executed it.
|
|
let (name, args) = match calls_rx.recv_timeout(Duration::from_secs(10)) {
|
|
Ok(pair) => pair,
|
|
Err(_) => {
|
|
let data = CHAT.read().unwrap();
|
|
let dump: Vec<String> = data
|
|
.messages
|
|
.iter()
|
|
.map(|m| format!("{:?}: {}", m.role, m.text))
|
|
.collect();
|
|
panic!("tool never called; transcript: {dump:?} status={} activity={}",
|
|
data.status, data.activity);
|
|
}
|
|
};
|
|
assert_eq!(name, "image.generate");
|
|
assert_eq!(args.get("width").and_then(Value::as_i64), Some(768));
|
|
|
|
wait_for("the turn to land", || !ChatData::is_streaming());
|
|
|
|
let data = CHAT.read().unwrap();
|
|
let roles: Vec<ChatRole> = data.messages.iter().map(|m| m.role).collect();
|
|
assert!(roles.contains(&ChatRole::User));
|
|
assert!(
|
|
roles.contains(&ChatRole::Assistant),
|
|
"the streamed reply landed as an assistant bubble: {roles:?}"
|
|
);
|
|
let text: String = data
|
|
.messages
|
|
.iter()
|
|
.map(|m| m.text.as_str())
|
|
.collect::<Vec<_>>()
|
|
.join("\n");
|
|
assert!(text.contains("trawler"), "{text}");
|
|
}
|