andodeki/makepad
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
makepad/libs/ai/models/body/THIRD_PARTY_NOTICES.md
Admin 6ba1b9425c libs: ai hub/models/cuda, speech, chat_ui 
Squash of 54 work commits (Sep 1–12):
  6251f7c  ai-hub: body domain — live pose packets ride the realtime session
  ea50c77  chat_ui: the feed's session gets its profile brief back
  f51b5f3  ai-body: the crate for the native SAM 3D Body port, with its weights reader
  8211ae6  ai-body: the MHR rig and the pose head's parameter decoding, oracle-exact
  9e343a8  ai-body: the DINOv3 ViT-H+/16 backbone, crop and ray conditioning; Metal gains rope-half and affine layer norm
  69d842c  ai-body: the promptable pose decoder and its refinement loop, oracle-matched on Metal
  66e5e2f  ai-hub: SAM 3D Body runs natively — `sam3dbody` on the body domain, oracle-matched end to end
  a634198  ai-hub: the body-native commit carried a peer's in-flight hub hunks; put them back where they were
  9ff44e8  ai-hub: the body-native wiring, this time only the lane's hunks
  6a1c16b  ai-body: third-party notices — what the port is implemented after, and what it is not
  d78411a  ai-body: the per-step work moves to the GPU
  b22259b  ai-body: the context stays on the GPU; only the pose token leaves the loop
  346f31f  ai-body: flash attention for the head-dim-64 blocks
  45b5b98  ai-body: the crop size is a runtime knob, and the loop reports where its time goes
  4be6d19  ai-body: the test modules import the grid constants they still use
  7598346  ai-body: tensor-core GEMMs for the backbone, and the rig's correctives only where they count
  a9ce596  ai-body: the crop warp runs across cores
  8964ba6  ai-body: an FP8 backbone mode, off by default, measured against the oracle
  a2aaa8f  ai-body: the FP8 bias rides a column-broadcast add on the device
  d53c77d  metal: a device-resident ViT stack, and the body backbone rides it
  d006d0a  metal: resident f32 linears keep their weight on the device
  525ba1c  metal: a device-resident two-way decoder layer, and the body decoder rides it
  c9e6d88  ai-body: the hands pass — hand crops, the hand decoder, the hand-mode rig and the wrist fusion
  62dff26  ai-body: the mask prompt — a person's segmentation mask conditions the body pass
  a648cf8  ai-hub: body session options — hands, detect, persons=N
  8c568df  ai-hub: drop the SAM 3D Body reference worker backend
  7ff875a  ai-hub: keep a peer's in-flight beats/notes/local work out of the body commits
  31e5faa  ai-hub: local model runner, licence acknowledgements, a shared install panel; Beat This!, Basic Pitch and the Salamander drum-kit entries
  b94bc58  ai-services: the wire, the app port and the panel state — one conversation, many apps
  2acb798  ai-services: wire v2 — endpoints, receiver-side caps, result disposition
  8ae0ffb  ai-services: the engine core — registry, router and conversation, tested against a scripted model
  2308736  ai-services: the real models behind the engine feature — local through the hub, Claude, and none
  c3f631d  livepipe: one reusable pipe from a camera to a fleet node and back
  ff62db3  ai libs: the runtime env-var cleanup — precision is a per-caller policy, not an environment side channel
  04a94ef  realtime: one service-log line when a live session opens and one when it closes
  0ecb81c  ai models: the model-crates env-var cleanup — 172 research knobs gone, the unset default is the code
  4ca36c1  ai hub + services: the assistant's model comes from wherever it is resident — the fleet chat box, with tools, then the local weights
  432121e  aichat engine + wm: launch, then use — the assistant continues in the same turn once the app it started is on the bus
  7a5bf69  ai-hub registry: the Salamander drumkit samples come from the makepad.nl mirror — the GitHub repo only carries the .sfz files
  102ffc5  ai-services: messages on the bus — a manifest declares topics, the engine subscribes on a tool's behalf or by ToolResult.subscribe, a service publishes Message frames, an idle conversation wakes on a message as an event turn under rate laws; the WM bus forwards the new frames; every app that matches the wire gets its arm
  a837792  hub + flow: a whitespace-only chat completion is retried once and then fails instead of passing as an answer; a flow's model is a fleet model id unless it names a weight file on disk; chat models show under the text domain in /v1/models
  bc6c620  hub + flow: what the chat review found — the in-process route retries an empty completion too, a node says whether its prefill opened thinking so a brief-mode answer is never discarded, a preferred model falls back to normal election when no node has it, discovery keeps looking for the preferred model until patience runs out
  75c3441  hub: the PRO 6000 serves image as well as chat and text
  ad5e98b  hub registry: flux2-dev's VRAM estimate is its measured peak, 30 GB
  c7241e0  hub: a node that evicted every resident releases its cached allocator pool before refusing a load or publishing usable VRAM
  30575f0  flow: route generation by request workload
  1be1e21  ai-hub: gate downloads by disk capacity and recover fleet admission
  df6b394  filesystem_watcher, bounded_http, ai services: live and tool prerequisites
  79ebdb9  ai-hub: add a native Pixal3D image-to-3D backend
  0ba0d74  ai-hub: propagate typed refusals under reject queue policy
  cc6c872  Speed up H3 conditioning and video decoding
  e512059  Fix Qwen vision residency and generated material colors
  2864f68  ai-hub http client: bound every plain TCP connect to 3 s per address
  3d93229  ai: CUDA is a Linux/Windows-only dependency; the hub library defaults to llm + stt

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-15 13:40:31 +02:00

4.4 KiB
Raw Permalink Blame History

Third-party notices — makepad-ai-body

The code in this crate is our own and is licensed MIT, like the rest of the repository (see the license field in Cargo.toml and the repository LICENSE). This file records the third-party work our port is implemented after, and the attribution those upstreams ask for.

Nothing listed here is vendored. No third-party source file is copied into this crate, and no model weights are redistributed by this repository — operators pull the checkpoint at runtime under the checkpoint's own license.

SAM 3D Body — src/model.rs, src/decoder.rs, src/heads.rs, src/pose.rs, src/preprocess.rs, src/condition.rs

Architecture implemented after the published paper. SAM 3D Body: Robust Full-Body Human Mesh Recovery (Meta AI, arXiv:2602.15989, https://arxiv.org/abs/2602.15989). The promptable decoder is described there as a Segment-Anything-style transformer decoder over image tokens with pose, prompt and keypoint tokens, refined layer by layer through the rig; the crop, CLIFF-style camera conditioning, ray conditioning and perspective camera follow the paper's description of the pipeline.

The reference inference code published with the model (https://github.com/facebookresearch/sam-3d-body) is released under the SAM License and is not part of this crate: it was not copied, adapted or vendored. The port was written from an architecture specification (tensor inventory, operation order, formulas) and validated against reference intermediate dumps produced by running that reference code on its own, outside the repository, on a public-domain photograph.

DINOv3 ViT-H+/16 — src/dino.rs

Architecture implemented after: Hugging Face transformers, Apache License 2.0.

The vision backbone in src/dino.rs is implemented after the DINOv3ViT implementation in Hugging Face transformers, https://github.com/huggingface/transformers/tree/main/src/transformers/models/dinov3_vit (modeling_dinov3_vit.py, configuration_dinov3_vit.py), the same source the TRELLIS conditioner in makepad-ai-trellis follows, whose copyright header reads:

Copyright 2025 The Meta AI Authors and The HuggingFace Team. All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Every architectural constant of the backbone — patch 16, width 1280, depth 32, 20 heads, the SwiGLU feed-forward of width 5120, the four register tokens, the axial rotate-half rotary embedding with base 100, layer scale on both residuals, LayerNorm with eps 1e-5 — is present in that Apache-2.0 source. The checkpoint's tensor names follow the same implementation.

Momentum Human Rig (MHR) — src/mhr.rs

Rig semantics implemented after: facebookresearch/MHR, Apache License 2.0 (https://github.com/facebookresearch/MHR), which describes the rig's parameterisation (identity and expression blendshapes, the model-parameter to joint-parameter transform, the joint hierarchy with per-joint translation offsets, pre-rotations, XYZ euler rotations and log2 scales, the pose corrective MLP, linear blend skinning against an inverse bind pose) and whose downloadable rig assets are Apache-2.0. The kinematics follow Momentum (https://github.com/facebookresearch/momentum, MIT). The rig data this crate loads (mhr.*) ships inside the model checkpoint below.

Weights — SAM License

The production checkpoint is the Comfy-Org repack Comfy-Org/sam-3d-body, detection/sam_3d_body_dinov3_bf16.safetensors, pinned in the hub registry by revision, byte size and SHA-256. It is released under the SAM License (see the model card at https://huggingface.co/Comfy-Org/sam-3d-body); operators must review and comply with those terms. This repository never downloads facebook/* checkpoints and never redistributes weights.

Reference numerics

The port reproduces the reference pipeline's numerics for the body decoder path (inference_type = "body"): rig vertices within 1e-4 cm, keypoints within 1e-6 m from identical rig parameters, and end to end from an image within 2 mm on 3D keypoints and 0.5 px on 2D keypoints, the residue being bf16 accumulation-order noise in the backbone. The full path (inference_type = "full": both hand crops, the hand decoder, the wrist fusion and the keypoint re-prompt) matches end to end within 2 mm and 1 px.