Commit graph

239 commits

Author SHA1 Message Date
Kevin Boos
d0fe5f2b74
cargo_makepad: link std statically for shippable android builds (#1177)
Android builds passed `-C prefer-dynamic` unconditionally, so Rust shipped
`std` as a separate `libstd-<hash>.so`. That library is a rustup prebuilt whose
LOAD segments are only 4 KB-page aligned (`p_align 0x1000`), so it can't be
mapped on the 16 KB-page devices Android 15 allows. Google Play requires apps
targeting API 35+ to run there, so release apks were failing that bar even
though `libmakepad.so` itself was already linked with 16 KB alignment.

Only debug builds keep `prefer-dynamic` now, where the faster incremental
relink is worth having and nothing ships. Everything else links `std`
statically, which drops the separate library entirely and leaves a single
NDK-linked, 16 KB-aligned `.so`. The aab path already did this.
2026-08-13 10:04:11 +02:00
Jason Yau
947f815181
Video playback fixes and improvements (#1155)
* android oes video zero-copy and gles shader fixes

* regenerate windows-rs by windows-strip

* fix windows video freezes with MF on an MTA worker

* regenerate windows-rs by windows-strip

* MTA MF video, COM notify, YUV texture reuse

* add local video file playback support to video-player example

* NV12 Metal present, seek warm-up gate, YUV full range

* fix Linux GStreamer video: A/V mute, HLS prepare, async teardown

* no-op SelectVideoTrack/SelectAudioTrack on non-Linux backends

* Linux ALSA/Pulse: bigger buffers, fix resample setup, report device rate

* fix some warnings

* Move XInput/DirectInput device discovery off the UI thread

* fix GLSL unpack4u8 for GLES 3.0 (#version 300 es)

* Linux GStreamer: DMA-Buf NV12 OES zero-copy, optional GLMemory path

* Linux MediaPlugin: DMA-Buf NV12 and GLMemory zero-copy present APIs

* Add Windows SourceReader DXGI NV12 zero-copy video path

* fix build error

* fix GLES: add highp precision for sampler2DArray in Linux shaders

* playback speed support for android

---------

Co-authored-by: jasonqiu <jasonqiuchen@outlook.com>
Co-authored-by: jasonqiu <jasonqiu@futunn.com>
2026-08-10 12:29:33 +02:00
Admin
51cf444bf2 bake: ungate the shadow sweep — sink-gated, the baker captured deck-only shadow sets, invisible while the sig bug masked it; fingerprint now also hashes shadow shape/point counts so signature-neutral capture bugs change it
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 10:45:44 +02:00
Admin
f883c689dd The three fixes the AMS single-tile test demanded: shadow-sig sink filter (baker stored a jobs-less signature no runtime could match — shadow bake was dead weight), transmux merges below-z14 duplicate border tiles (protobuf layer-union; first-wins blanked every cell boundary), mapfleet --rebake-from + stale-field strip (rebake straight from baked cells, no respool)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 09:56:05 +02:00
Admin
776b9f6ca1 fleet version handshake: bake fingerprint on a fixture tile — mapfleet refuses workers whose cascade signatures drift from the dispatcher (run #1 baked the planet with mixed checkout vintages; every fleet bake's cascade was signature-dead)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 09:31:25 +02:00
Admin
0e20c4e2e7 bake: NaN-height guards (nan-tagged buildings panicked 3 cells) + bounded reorder window + MAPBAKE_TRACE
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 09:25:48 +02:00
Admin
f35acc03e8 Fix HEAD: skin_to_packed call site + track tools/arcade_eval
Two ways HEAD failed to build for a fresh checkout, both from work landing
across concurrent streams:

- 623ee745e renamed skin_to_pbr -> skin_to_packed, but arcade_view.rs was
  being edited by another stream at the time, so its call site fix stayed
  uncommitted while the rename landed
- Cargo.toml listed tools/arcade_eval as a workspace member while the crate
  itself was untracked, so loading the workspace failed outright

Both verified: the crate builds, and a stash-everything check now leaves a
working tree that compiles.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 09:20:47 +02:00
Admin
0fd9545cb4 mapfleet: --local-workers N + dead-host parking — a decoupled LAN finishes the run on the dispatcher's own cores
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 08:13:16 +02:00
Admin
3ab364f44a mapfleet: in-flight claim set + writer-unique tmp names — requeue rewinds re-issued mid-bake cells, and the remote pull and local bake shared one tmp path, interleaving writes into 52 corrupt ledger entries
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 07:39:25 +02:00
Admin
348c7cfb98 mkmap: weave preflight requires full summed source size — blocks mid-run weaves while the store holds the disk, self-resumes when the endgame frees it
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 07:37:03 +02:00
Admin
462f95e38f mkmap transmux: skip corrupt sources with a warning + disk preflight — one bad cell mbtiles was aborting every weave, and a too-full disk burned 67GB per failed attempt
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 07:33:26 +02:00
Admin
a929dafa18 world grid 2.5 degrees (9792 cells) — dense slices fit the 16GB M4 workers' in-RAM frame receive
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 00:45:56 +02:00
Admin
a015fdbf82 mapfleet: atomic slice outputs + twice-failed-on-host cells route to the local worker
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 00:42:21 +02:00
Admin
536e0e2be2 mapfleet: 5-degree world grid + chunked socket writes — dense 15-degree cells sliced to 6-23GB, over macOS's 2GB single-write ceiling and worker RAM; oversize slices (>3.5GB) stay on the local worker
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 00:33:13 +02:00
Admin
cb6cec91ca mapfleet: drop world.mkmap.prev after successful swap — a retained prev is a full extra world copy against a tight disk budget
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 23:02:52 +02:00
Admin
f2f4b1bb50 Spool v2: streaming frontier — the fleet bakes while pass 4 runs
Pass 4 collects relation jobs, bbox-scans them against the RAM stores,
sorts by NL-spiral distance and publishes store/spool-frontier.txt (key
of the first unfinished relation; spool flushed before every publish).
pbf-base slices a LIVE store behind that gate (--bbox only, torn-tail-
tolerant block reads, pid-scoped sort chunks so concurrent slices never
clobber shared edge blocks), and mapfleet claims block per cell on the
frontier instead of waiting for the whole spool.

Pass-granular resume: passes 2/3 stamp exact spool block lengths +
counters + stats; a restart rolls the spool back to the newest stamp
and reruns only the remaining passes (unit tests + crash drill).

Weave fixes from the Luxembourg end-to-end smoke run: bake outputs land
via temp+rename so the ledger name never exists half-written, and
transmux verification now checks the union of ALL sources with first-
source-wins ownership (it compared source 0 alone and failed every
multi-source weave).

Also: pass-4 worker clamp 12→14, dead paged NodeStore/WayStore and
emit_lines/emit_polygons deleted (flat stores/prepare_* replaced them),
mapfleet --pbf + startup chunk sweep + end-of-spiral drain and final
weave, world-build.sh refreshes the -run binary copies.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 22:59:15 +02:00
Admin
1e7cf7723c spool: --no-tiles skips pass 5 (nothing consumes it); chain uses it
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 22:15:31 +02:00
Admin
c7f1dbed8e spool: recursive-bisection polygon tiling — continental boundaries clip in O(n log t)
prepare_polygons clipped the FULL ring (and holes) against every tile in
the bbox: a continent-scale boundary relation (millions of points x
millions of tiles) pinned one core for hours at the planet spool's tail.
Halves clip once per split and recurse (geojson-vt scheme); per-tile
output is geometrically identical (equivalence test: same tiles, equal
areas, identical interior vertices — direct's zero-area boundary spikes
are collapsed, which is strictly cleaner).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 22:08:21 +02:00
Admin
be9d49ceb7 spool.md: largest-first scheduling + clip_ring segment prefilter (ideas 8-9)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 20:54:12 +02:00
Admin
0529960542 spool.md: pass-granular resume (idea 7)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 20:05:02 +02:00
Admin
85b292b593 spool.md: v2 bottleneck analysis — spiral-ordered finalization kills the fleet gate
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 18:55:26 +02:00
Admin
b30ff3ede3 Fleet compression architecture: q2 throwaway slices, workers deliver uniform q11
map_bake --recompress re-encodes EVERY tile (both pass-through sites) at
the target quality — compression parallelizes across the whole fleet and
the final dataset is uniformly q11. The slicer ships q2 intermediates
(fast, disk-lean). Bake mode also skips runtime-only work per tile:
building/tree vertex extrusion and POI point parsing never reach the
sink and no longer run on workers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 18:42:21 +02:00
Admin
f202b9d212 mapfleet: slice at brotli q10 — matches bake re-encode, keeps ten bakers fed
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 18:42:21 +02:00
Admin
dbfd3844cf mapfleet: slice gate 3-wide for the ten-worker fleet
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 18:42:21 +02:00
Admin
bfb0816816 mapfleet: bootstrap retries every 5min — boxes join when healed
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 18:42:21 +02:00
Admin
a5919cd2a4 mapfleet: workers self-gate on the spool marker — hot-joined boxes bootstrap immediately
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 18:42:21 +02:00
Admin
12e296ac77 mapfleet: eager worker bootstrap during the spool wait
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 18:42:21 +02:00
admin2
943a5a2a77 Fix partial Android NDK DMG extraction 2026-08-02 18:29:14 +02:00
Admin
acf5442efe Workspace: members = app + examples + studio + necessary tools + tests; makepad-remote runnable from root
tools/remote joins the root workspace (default-run makepad-remote, so
'cargo run -p makepad-remote -- --server' works from the checkout root
on worker boxes). Libraries stay in the build graph as path deps; the
runnable-package list is the things a human actually launches.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:55:44 +02:00
Admin
4c81087010 mapfleet: live-joining compute nodes (--hosts-file polled 30s) + dual slice gate
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:44 +02:00
Admin
05f40f2ed9 mapfleet: distributed world-cell bake over makepad-remote boxes
Protocol extracted to a shared module + TAG_FILE_PULL (server returns a
file from its cwd, escape-guarded). mapfleet dispatches the NL-spiral:
local slicing against the store (serialized), remote bakes via cargo-run
on each box (bootstrap build cached), pull-back, coalesced weave with
atomic world.mkmap swap. Resume ledger (cell-NNN-baked.mbtiles, empty =
ocean) is shared with world-slabs.sh — run one driver at a time. Failed
cells requeue with backoff; a dead box just stops claiming.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:44 +02:00
Admin
a8b52d737d Map: world chain hardening — pipefail (tee masked spool failure), instance locks, mercator-clamped cells
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:44 +02:00
Admin
cf1b977d00 Map: NL-spiral slab driver — slice/bake/weave per cell, live-growing world shards
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:44 +02:00
Admin
990f63de26 mkmap: multi-source weave transmux — first-source-wins, spiral-cell assembly
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:44 +02:00
Admin
489d5aa6a1 Map: world-build chain script — planet download-wait -> spool -> base -> bake -> shards
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:44 +02:00
Admin
8785b98a25 Map: finalcut runs aliased binaries — dodge the unhunted makepad-map-* name killer
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:44 +02:00
Admin
0375e344f7 Map: finalcut v6 — integer keyframes 15-18 + v4 dissolved-buildings streams
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:44 +02:00
Admin
e61b9af7fc Fix all compile warnings across map crates (unused mut/import, key visibility)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:43 +02:00
Admin
19d231405b Map: finalcut v5 — actually two keyframes (14,16); v4 edit had not applied
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:42 +02:00
Admin
d0f92ed090 Map: finalcut v4 — two-keyframe morph cut (buckets 14,16)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:42 +02:00
Admin
781dd2c83e Map: bucket ceiling 17 + bake buckets 14-17 (rz18+ magnifies bucket 17)
5-bucket streams priced at ~2.6x (=130GB archive) — clamped instead:
rz18 shows bucket-17 geometry like any mid-gesture frame, GPU strokes
re-derive widths, nav icons survive. Consumption window 10..=18 stays
(harmless; ceiling makes >17 unreachable).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:42 +02:00
Admin
5a0839cb50 Map: finalcut outputs europe-base-br-v3 (shadow-baked cut)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:42 +02:00
Admin
627f0bb04b Map: app reads .mkmap shards — MkmapReader + TileArchiveReader, varint-KV index v2
libs/mbtile_reader gains the .mkmap consumer: root.mkidx parse (Hilbert
tile ids, root ranges -> brotli leaf directories -> shard/offset/len),
positioned shard reads, same get_metadata/get_tile_decoded surface as
MbtilesReader; TileArchiveReader sniffs the path so mbtiles_path can
point at either. Index format v2 drops JSON: the metadata section is now
varint KV like the leaves (writer + reader; no serde in the container).
Loader, zoom-range probe and the headless harness go through the enum;
bridge-dz/overlay sidecars stay mbtiles. Harness parity via shards:
worst AMS rz16 164.7ms, z12 native 51.3ms — identical to mbtiles. App
repointed to local/maps/europe-base-br.mkmap (111 shards + root.mkidx,
the exact bytes a CDN would serve).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:42 +02:00
Admin
e8c21892dd Map: bake bucket 15 (rz15 is a cruising zoom, not a transient band) + status.md
In-app at z15 every tile ran the 460ms runtime cascade (buckets were
14+16 only); NL smoke: worst rz15 key 600 -> 53ms. Europe final cut
rerunning with 14,15,16.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:42 +02:00
Admin
9b4d871075 Map: repoint app to europe-base-br-v2 (theme-independent final cut) + pipeline script
Final cut: 9.38M tiles, 133,431 baked (9.6GB streams, zooms 10-14,
threshold 100ms), 107 shards under cap, 253,525 sampled byte-identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:42 +02:00
Admin
fce6c68835 Map: repoint app to europe-base-br-faces-mz (mid-zoom baked) + midzoom pipeline script
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:42 +02:00
Admin
68b950f3e3 Map: mid-zoom cascade bake + O(n^2) road-join prefilters — z12 city 661 -> 100ms
The zoom-out pyramid was never baked: z11-13 city tiles ran the full
painter cascade at runtime (z12 AMS 340ms cascade, 661ms build). map_bake
gains --zooms (native bucket below z14) and the renderer accepts baked
buckets 10-16; NL smoke bake: 861 tiles/55MB/31s, z13 166->19ms,
z11 252->28ms. Remaining z12 cost was two quadratic endpoint passes over
4017 join ways: bbox+reach prefilters and an endpoint hash grid cut
grade-through 103->33ms and flush-ends 98->3ms, bit-identical (every gate
is half-width-bounded; buffer sizes byte-equal across the sweep). Also
sp-* sub-laps in stroke-prep.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:42 +02:00
Admin
80d5967198 Map: Europe single-origin archive shipped — 51.8GB base + faces bake + .mkmap shards
Overnight run 2026-08-01: 9,380,390 tiles z2-14 in 3.2h, faces bake 1h
(+8GB, worker-side brotli q10), transmux to 99 shards all <510MB.
App repointed to europe-base-br-faces.mbtiles. nl-bridge-dz verified
byte-identical against the new base (deterministic feature indices).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:42 +02:00
Admin
9bddc458b9 Map: painter-cascade per-bucket bake (v2-faces-1) + z14 streets merge
- The boolean union cascade (59-77% of heavy tile builds) is now baked
  per (tile, bucket 14/15/16) as additive field 101: cascade output made
  deterministic (1/64 grid snap, canonical ring order, sorted tiers) and
  bit-reproducible; renderer consumes guarded by structure signature +
  coordinate checksum with runtime fallback; MAKEPAD_NO_BAKED_FACES=1.
  Roundtrip bit-identical on all keys x buckets x dz configs. New
  tools/map_bake producer (worker-pool incl. worker-side brotli).
- z14 streets merging never ran (early-out at DETAIL_ZOOM): fixed —
  1957->1285 ways (-34%), payload -20MB, cascade input shrunk at source.
- Slow-tile log now prints per-stage ms + a ready-to-paste headless repro
  command; harness honors archive/detail/dz/overlay/rz/3D envs.
- Worst AMS tile: 2-3s cold -> 175ms rz16 (61ms native); NL bake data
  882MB at 60ms threshold; Europe needs threshold 150ms + 2 buckets to
  fit shard budget (+2-4GB est).
- App -> nl-base-br3-faces archive; bridge-dz rebake required (joins
  fail closed against br3 feature indices).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 17:48:42 +02:00
Kevin Boos
d6c3d5c322 cargo_makepad: don't include resources/android/ios on every platform (#1157)
* TextFlow: fix inline `<code>` spans sagging below the baseline

finish_row_center centered every walk by its own height, so a code run's
shorter walk got a larger downward shift than the surrounding prose under
`RowAlign.Center`. That undid TextFlow's baseline_shift: inline `<code>`
spans sat a few px below the line's baseline, and their descenders poked
out of the bottom of the code box.

* FinishedWalk now carries an optional `align_height` that text runs set
  to their line style's height, so every text run on a row receives the
  same centering shift and stays on the baseline. Also fixes sub- and
  superscripts drifting under `RowAlign.Center`.
* The per-style metrics probe now caches descenders too, since we need
  the full line height (ascender + descender) to compute `align_height`.

* Html/Markdown: make the fixed/code font size scale configurable

Replaces the hardcoded 0.85 `FIXED_FONT_SIZE_SCALE` consts with a
`fixed_font_size_scale` live property on TextFlow (default 0.85), so
apps can tune how much smaller `<code>` text renders than the prose.

* cargo_makepad: don't include `resources/android`/`ios` on every platform

Runtime asset bundling copied each crate's entire `resources/` tree into
every package, so `resources/android/` (manifest template, launcher icon
mipmaps) shipped as dead weight inside APK assets and Apple bundles, and
any `resources/ios/` content would ship on Android too. Those dirs are
platform-specific, so only their own platform should bundle them.

* Add `cp_all_skip_top()` to `makepad-shell`: like `cp_all()`, but skips
  top-level entries by exact name.
* Android (APK and AAB staging): bundle `resources/android/` minus the
  packaging inputs `AndroidManifest.xml.template` and `res/`, which
  already reach the package via the generated manifest and the aapt
  `res/` pipeline; skip `resources/ios/` entirely.
* Apple bundles: skip `resources/android/`, keep `resources/ios/`.
2026-08-01 10:26:31 +02:00