6ba1b9425c
Squash of 54 work commits (Sep 1–12):6251f7cai-hub: body domain — live pose packets ride the realtime sessionea50c77chat_ui: the feed's session gets its profile brief backf51b5f3ai-body: the crate for the native SAM 3D Body port, with its weights reader8211ae6ai-body: the MHR rig and the pose head's parameter decoding, oracle-exact9e343a8ai-body: the DINOv3 ViT-H+/16 backbone, crop and ray conditioning; Metal gains rope-half and affine layer norm69d842cai-body: the promptable pose decoder and its refinement loop, oracle-matched on Metal66e5e2fai-hub: SAM 3D Body runs natively — `sam3dbody` on the body domain, oracle-matched end to enda634198ai-hub: the body-native commit carried a peer's in-flight hub hunks; put them back where they were9ff44e8ai-hub: the body-native wiring, this time only the lane's hunks6a1c16bai-body: third-party notices — what the port is implemented after, and what it is notd78411aai-body: the per-step work moves to the GPUb22259bai-body: the context stays on the GPU; only the pose token leaves the loop346f31fai-body: flash attention for the head-dim-64 blocks45b5b98ai-body: the crop size is a runtime knob, and the loop reports where its time goes4be6d19ai-body: the test modules import the grid constants they still use7598346ai-body: tensor-core GEMMs for the backbone, and the rig's correctives only where they counta9ce596ai-body: the crop warp runs across cores8964ba6ai-body: an FP8 backbone mode, off by default, measured against the oraclea2aaa8fai-body: the FP8 bias rides a column-broadcast add on the deviced53c77dmetal: a device-resident ViT stack, and the body backbone rides itd006d0ametal: resident f32 linears keep their weight on the device525ba1cmetal: a device-resident two-way decoder layer, and the body decoder rides itc9e6d88ai-body: the hands pass — hand crops, the hand decoder, the hand-mode rig and the wrist fusion62dff26ai-body: the mask prompt — a person's segmentation mask conditions the body passa648cf8ai-hub: body session options — hands, detect, persons=N8c568dfai-hub: drop the SAM 3D Body reference worker backend7ff875aai-hub: keep a peer's in-flight beats/notes/local work out of the body commits31e5faaai-hub: local model runner, licence acknowledgements, a shared install panel; Beat This!, Basic Pitch and the Salamander drum-kit entriesb94bc58ai-services: the wire, the app port and the panel state — one conversation, many apps2acb798ai-services: wire v2 — endpoints, receiver-side caps, result disposition8ae0ffbai-services: the engine core — registry, router and conversation, tested against a scripted model2308736ai-services: the real models behind the engine feature — local through the hub, Claude, and nonec3f631dlivepipe: one reusable pipe from a camera to a fleet node and backff62db3ai libs: the runtime env-var cleanup — precision is a per-caller policy, not an environment side channel04a94efrealtime: one service-log line when a live session opens and one when it closes0ecb81cai models: the model-crates env-var cleanup — 172 research knobs gone, the unset default is the code4ca36c1ai hub + services: the assistant's model comes from wherever it is resident — the fleet chat box, with tools, then the local weights432121eaichat engine + wm: launch, then use — the assistant continues in the same turn once the app it started is on the bus7a5bf69ai-hub registry: the Salamander drumkit samples come from the makepad.nl mirror — the GitHub repo only carries the .sfz files102ffc5ai-services: messages on the bus — a manifest declares topics, the engine subscribes on a tool's behalf or by ToolResult.subscribe, a service publishes Message frames, an idle conversation wakes on a message as an event turn under rate laws; the WM bus forwards the new frames; every app that matches the wire gets its arma837792hub + flow: a whitespace-only chat completion is retried once and then fails instead of passing as an answer; a flow's model is a fleet model id unless it names a weight file on disk; chat models show under the text domain in /v1/modelsbc6c620hub + flow: what the chat review found — the in-process route retries an empty completion too, a node says whether its prefill opened thinking so a brief-mode answer is never discarded, a preferred model falls back to normal election when no node has it, discovery keeps looking for the preferred model until patience runs out75c3441hub: the PRO 6000 serves image as well as chat and textad5e98bhub registry: flux2-dev's VRAM estimate is its measured peak, 30 GBc7241e0hub: a node that evicted every resident releases its cached allocator pool before refusing a load or publishing usable VRAM30575f0flow: route generation by request workload1be1e21ai-hub: gate downloads by disk capacity and recover fleet admission df6b394 filesystem_watcher, bounded_http, ai services: live and tool prerequisites 79ebdb9 ai-hub: add a native Pixal3D image-to-3D backend 0ba0d74 ai-hub: propagate typed refusals under reject queue policy cc6c872 Speed up H3 conditioning and video decoding e512059 Fix Qwen vision residency and generated material colors 2864f68 ai-hub http client: bound every plain TCP connect to 3 s per address 3d93229 ai: CUDA is a Linux/Windows-only dependency; the hub library defaults to llm + stt Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
85 lines
4.4 KiB
Markdown
85 lines
4.4 KiB
Markdown
# Third-party notices — `makepad-ai-body`
|
|
|
|
The code in this crate is our own and is licensed MIT, like the rest of the
|
|
repository (see the `license` field in `Cargo.toml` and the repository
|
|
`LICENSE`). This file records the third-party work our port is *implemented
|
|
after*, and the attribution those upstreams ask for.
|
|
|
|
Nothing listed here is vendored. No third-party source file is copied into
|
|
this crate, and no model weights are redistributed by this repository —
|
|
operators pull the checkpoint at runtime under the checkpoint's own license.
|
|
|
|
---
|
|
|
|
## SAM 3D Body — `src/model.rs`, `src/decoder.rs`, `src/heads.rs`, `src/pose.rs`, `src/preprocess.rs`, `src/condition.rs`
|
|
|
|
**Architecture implemented after the published paper.** SAM 3D Body: Robust
|
|
Full-Body Human Mesh Recovery (Meta AI, arXiv:2602.15989,
|
|
<https://arxiv.org/abs/2602.15989>). The promptable decoder is described
|
|
there as a Segment-Anything-style transformer decoder over image tokens
|
|
with pose, prompt and keypoint tokens, refined layer by layer through the
|
|
rig; the crop, CLIFF-style camera conditioning, ray conditioning and
|
|
perspective camera follow the paper's description of the pipeline.
|
|
|
|
The reference inference code published with the model
|
|
(<https://github.com/facebookresearch/sam-3d-body>) is released under the
|
|
SAM License and is **not** part of this crate: it was not copied, adapted or
|
|
vendored. The port was written from an architecture specification (tensor
|
|
inventory, operation order, formulas) and validated against reference
|
|
intermediate dumps produced by running that reference code on its own,
|
|
outside the repository, on a public-domain photograph.
|
|
|
|
## DINOv3 ViT-H+/16 — `src/dino.rs`
|
|
|
|
**Architecture implemented after: Hugging Face `transformers`, Apache
|
|
License 2.0.**
|
|
|
|
> The vision backbone in `src/dino.rs` is implemented after the `DINOv3ViT`
|
|
> implementation in Hugging Face `transformers`,
|
|
> <https://github.com/huggingface/transformers/tree/main/src/transformers/models/dinov3_vit>
|
|
> (`modeling_dinov3_vit.py`, `configuration_dinov3_vit.py`), the same source
|
|
> the TRELLIS conditioner in `makepad-ai-trellis` follows, whose copyright
|
|
> header reads:
|
|
>
|
|
> > Copyright 2025 The Meta AI Authors and The HuggingFace Team. All rights reserved.
|
|
> > Licensed under the Apache License, Version 2.0 (the "License");
|
|
> > you may not use this file except in compliance with the License.
|
|
> > You may obtain a copy of the License at
|
|
> > <http://www.apache.org/licenses/LICENSE-2.0>
|
|
|
|
Every architectural constant of the backbone — patch 16, width 1280, depth
|
|
32, 20 heads, the SwiGLU feed-forward of width 5120, the four register
|
|
tokens, the axial rotate-half rotary embedding with base 100, layer scale on
|
|
both residuals, LayerNorm with eps 1e-5 — is present in that Apache-2.0
|
|
source. The checkpoint's tensor names follow the same implementation.
|
|
|
|
## Momentum Human Rig (MHR) — `src/mhr.rs`
|
|
|
|
**Rig semantics implemented after: facebookresearch/MHR, Apache License 2.0**
|
|
(<https://github.com/facebookresearch/MHR>), which describes the rig's
|
|
parameterisation (identity and expression blendshapes, the model-parameter
|
|
to joint-parameter transform, the joint hierarchy with per-joint translation
|
|
offsets, pre-rotations, XYZ euler rotations and log2 scales, the pose
|
|
corrective MLP, linear blend skinning against an inverse bind pose) and
|
|
whose downloadable rig assets are Apache-2.0. The kinematics follow
|
|
Momentum (<https://github.com/facebookresearch/momentum>, MIT). The rig data
|
|
this crate loads (`mhr.*`) ships inside the model checkpoint below.
|
|
|
|
## Weights — SAM License
|
|
|
|
The production checkpoint is the Comfy-Org repack `Comfy-Org/sam-3d-body`,
|
|
`detection/sam_3d_body_dinov3_bf16.safetensors`, pinned in the hub registry
|
|
by revision, byte size and SHA-256. It is released under the SAM License
|
|
(see the model card at <https://huggingface.co/Comfy-Org/sam-3d-body>);
|
|
operators must review and comply with those terms. This repository never
|
|
downloads `facebook/*` checkpoints and never redistributes weights.
|
|
|
|
### Reference numerics
|
|
|
|
The port reproduces the reference pipeline's numerics for the body decoder
|
|
path (`inference_type = "body"`): rig vertices within 1e-4 cm, keypoints
|
|
within 1e-6 m from identical rig parameters, and end to end from an image
|
|
within 2 mm on 3D keypoints and 0.5 px on 2D keypoints, the residue being
|
|
bf16 accumulation-order noise in the backbone. The full path
|
|
(`inference_type = "full"`: both hand crops, the hand decoder, the wrist
|
|
fusion and the keypoint re-prompt) matches end to end within 2 mm and 1 px.
|