makepad/libs/asset/chat/AI_BACKENDS.md

AI backends: one broker, six providers, one lock

The asset server's chat broker (libs/asset/chat + libs/asset/store/src/host/chat.rs) is the ONLY place a model is reached from. Every app — the sandbox game, the VJ, the Asset UI — is a thin client of /v1/chat/*: it opens a session on a named provider, sends text, and renders the event stream. No app holds a key, a fleet address, or a CLI.

Providers (GET /v1/chat/providers)

slug	what runs	locality	tools contract
fleet-qwen	Qwen on our asset-ai fleet (LAN beacons)	local	<<tool>> text markers
openai	OpenAI Responses API (OPENAI_API_KEY on the server)	cloud	native functions
grok	xAI Responses API (XAI_API_KEY on the server)	cloud	native functions
claude-cli	the claude CLI logged in on the server host	cloud	<<tool>> text markers
codex-cli	the codex CLI on the server host	cloud	<<tool>> text markers
grok-cli	the grok CLI on the server host	cloud	<<tool>> text markers

Each row is {kind, locality, state: available{model} | unavailable{reason}}. locality is the server's word: local means our own fleet, cloud means a vendor — by key or by a logged-in CLI, the client does not care which. A frontend that promises "Local AI only" filters on that field and nothing else, so adding a provider never needs a client release to stay honest.

The CLI providers (cli.rs, claude.rs, codex_cli.rs, grok_cli.rs)

• One process per turn, in its own process group, cwd = an empty scratch dir under the temp dir; cancel kills the group. Conversation continuity is the CLI's own session id (--resume / codex exec resume), so only the new tail of the history is sent on later turns.
• Chat-only by construction: claude --tools "" + empty strict MCP set; grok --disallowed-tools <all it advertises> + --permission-mode dontAsk
  • --max-turns 1; codex --sandbox read-only, user config and rules ignored, shell_environment_policy.inherit=none. The only tools any of them can express are the broker's content tools, executed by the dispatcher against the asset server (same as the fleet lane).
• Reasoning is forwarded inside <think>…</think> text like the fleet lane, so every client renders it one way and none of it enters the history.
• Availability = "is the executable there" (CLAUDE_CODE_PATH / CODEX_CLI_PATH / GROK_CLI_PATH override, then $PATH, then the usual install dirs). The reason string never carries a path to the wire.
• Verified live (2026-08-26, macOS): claude 2.1.246 needs the prompt on stdin (a trailing positional after --tools "" is swallowed as a tool name); grok 1.0.5 ignores --tools ""/none but honours --disallowed-tools; codex 0.149.1 exec --json emits whole agent_message items, no deltas.

Fleet discovery (fleet_discovery.rs)

The fleet lane listens for asset-ai beacons on UDP 41830 and keeps only the fleet named by the server config, else MAKEPAD_AI_FLEET, else default. The embedded server inside the Asset UI pins this to the fleet the UI's own panel shows (gen); an empty name used to force default and silently drop every beacon ("no fleet nodes configured" while the panel showed 2/2 up). The listener starts with the broker and a probe in its first seconds waits for the first beacon instead of reporting an empty fleet.

Durable per-game conversations (keyed sessions)

The server keeps the chat STATE for its clients. A session created with both client_key and context_key is one durable conversation per (principal, client_key, context_key):

• client_key = who is talking — an opaque, display-safe id the app chooses (the sandbox sends ip:<lan-ip> today; a multiplayer player id later). context_key = what it is about — the GAME asset id. One conversation per (client, game), never shared across games: a game's context is context we cannot pollute.
• Key shape (enforced identically by the client crate, wire::chat_key_ok, and the route): 1..=64 bytes of [A-Za-z0-9._:@-] with at least one alnum. Never a secret — the bearer token is the authorization.

Wire

• POST /v1/chat/sessions body gains the optional client_key and context_key (both or neither; one alone is a 400). With both it is CREATE-OR-RESUME: a LIVE session for that key answers 200 with itself (same session id, state as it is); a PERSISTED one is rebuilt from its transcript under the SAME id on a fresh provider and answers 200; nothing known answers 201 with a fresh session. Without keys: today's behaviour (201, ephemeral). A live keyed session asked for on a different provider/namespace/profile is rebuilt on the requested one while idle (its transcript kept), and is 409 busy mid-turn.
• The session document (POST/GET/list/cancel) carries client_key and context_key on keyed sessions, omitted otherwise.
• GET /v1/chat/sessions/{id}/transcript → {"session":…, "provider":"<slug>", "turn":N, "truncated":bool, "messages":[{"role":"user|assistant|system|tool","text":"…"}]} — the conversation as the client should render it: what the session feeds its provider, minus the prompt plumbing (thinking stripped, the tool reminder and trained call text folded away); each executed tool is one tool row whose text is a short chip title (world.set_source · ok) with tool (dotted name) and outcome (ok|unavailable|denied|refused|failed) alongside. Bounded: the LAST 128 rows within 192 KiB of text, each row's text clipped to 8 KiB; truncated says older rows were dropped. Works for unkeyed sessions too (memory only). 404 for a session that is not live — a client always create-or-resumes first, then reads.
• DELETE /v1/chat/sessions/{id} on a keyed session is the client's Clear: the worker AND the persisted transcript go, synchronously; the next create-or-resume is a fresh conversation (new id).
• Retiring an asset (DELETE /v1/assets/{id}) drops every keyed conversation whose context_key is that asset — live or on disk, whoever's — and logs the count.

Persistence (libs/asset/store/src/host/chat_store.rs)

• <root>/chat/<principal>/<client_key>/<context_key>.jsonl, keys path-encoded ([A-Za-z0-9._-] verbatim, else %XX: ip:10.0.0.7 is ip%3A10.0.0.7). Header line {"k":"h","v":1,"session","provider", "namespace","profile","client_key","context_key","created_ms"}, then one {"k":"m","role","text","turn"} per history row exactly as the provider sees it (the transcript is rendered on read), and an optional {"k":"p","resume":…} slot for a provider-native resume id (unused today: the threaded provider wrapper does not expose one; a resumed session simply replays its history to a fresh provider, which every lane — CLIs included — accepts).
• The session worker appends the new tail after every publish (write_all + sync_data); a crash costs at most one torn last line, which load drops. A resume rewrites the file (temp + rename) with the current binding and the history trimmed to the newest 64 rows, so a resumed conversation always has room to continue.
• Workers are bounded: a keyed session idle 30 min is evicted to disk; when an owner's cap (or the server's) is full of idle keyed sessions, the longest-idle one is evicted to make room. Unkeyed sessions are never evicted (nowhere to go) and refuse as before.
• One writer per file (the worker) and Clear wipes under the same lock, so a late append can never resurrect a cleared transcript.

world.new_level (game profile)

Advertised next to world.set_source, client-executed: {"title", "source", "note"?} — "create a NEW game from this source and switch the player to it". The game publishes, switches, and answers the tool result {asset_id, alias, title}; the broker records the round and ends the turn with done (no further model round — the player is in another game, which has its own conversation). world.set_source stays the in-place edit of the CURRENT game.

The sandbox (apps/sandbox)

• fleet_chat.rs holds ChatPrefs { provider, local_only }, persisted in local/sandbox/ai-prefs, default {fleet-qwen, true}. The lock is enforced in the worker, not just hidden in the UI: selecting a cloud provider while locked is refused with a chat line; turning the lock on while on a cloud provider drops back to the fleet.
• Settings = one dropdown of what the server listed (cloud rows removed while locked; unavailable rows shown as such) + the "Local AI only" checkbox. No keys, no pairing, no device-local backends.