f2f4b1bb50
Pass 4 collects relation jobs, bbox-scans them against the RAM stores, sorts by NL-spiral distance and publishes store/spool-frontier.txt (key of the first unfinished relation; spool flushed before every publish). pbf-base slices a LIVE store behind that gate (--bbox only, torn-tail- tolerant block reads, pid-scoped sort chunks so concurrent slices never clobber shared edge blocks), and mapfleet claims block per cell on the frontier instead of waiting for the whole spool. Pass-granular resume: passes 2/3 stamp exact spool block lengths + counters + stats; a restart rolls the spool back to the newest stamp and reruns only the remaining passes (unit tests + crash drill). Weave fixes from the Luxembourg end-to-end smoke run: bake outputs land via temp+rename so the ledger name never exists half-written, and transmux verification now checks the union of ALL sources with first- source-wins ownership (it compared source 0 alone and failed every multi-source weave). Also: pass-4 worker clamp 12→14, dead paged NodeStore/WayStore and emit_lines/emit_polygons deleted (flat stores/prepare_* replaced them), mapfleet --pbf + startup chunk sweep + end-of-spiral drain and final weave, world-build.sh refreshes the -run binary copies. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
44 lines
1.9 KiB
Bash
Executable file
44 lines
1.9 KiB
Bash
Executable file
#!/bin/zsh
|
|
# World (planet) build chain — runs DETACHED, survives session death.
|
|
# Waits for the planet download, then: spool -> base -> bake -> shards.
|
|
# Progress: tail -f local/maps/world-build.log ; status per phase below.
|
|
set -e
|
|
set -o pipefail
|
|
# Single-instance lock: a raced duplicate corrupted the spool once.
|
|
LOCK="/tmp/$(basename $0).lock"
|
|
if ! mkdir "$LOCK" 2>/dev/null; then echo "another instance holds $LOCK — exiting"; exit 1; fi
|
|
trap 'rmdir "$LOCK"' EXIT
|
|
cd "$(dirname "$0")/../.."
|
|
PBF=local/maps/pbf/planet-latest.osm.pbf
|
|
STORE=local/maps/world-detail.store
|
|
DETAIL_TMP=local/maps/world-detail-tmp.mbtiles
|
|
BASE=local/maps/world-base-br.mbtiles
|
|
BAKED=local/maps/world-base-br-v1.mbtiles
|
|
MKMAP=local/maps/world-base-br.mkmap
|
|
LOG=local/maps/world-build.log
|
|
BIN=./target/release/mptiles-run
|
|
BAKE=./target/release/mpbake-run
|
|
# The -run names are copies (instance-lock/watchdog isolation). Refresh
|
|
# them from the canonical binaries so a launch never runs stale code.
|
|
cp -f ./target/release/makepad-map-tiles "$BIN"
|
|
cp -f ./target/release/makepad-map-bake "$BAKE"
|
|
|
|
phase() { echo "==== $(date '+%F %T') PHASE: $1 ====" | tee -a "$LOG"; }
|
|
|
|
phase "wait-for-download"
|
|
# Download done = curl gone AND file over 70GB AND size stable 60s.
|
|
while pgrep -f "planet-latest.osm.pbf" | grep -qv $$; do sleep 60; done
|
|
SIZE1=$(stat -f %z "$PBF"); sleep 60; SIZE2=$(stat -f %z "$PBF")
|
|
if [ "$SIZE1" != "$SIZE2" ] || [ "$SIZE1" -lt 70000000000 ]; then
|
|
echo "download incomplete/unstable ($SIZE1 vs $SIZE2) — aborting" | tee -a "$LOG"
|
|
exit 1
|
|
fi
|
|
echo "planet pbf: $SIZE1 bytes" | tee -a "$LOG"
|
|
|
|
phase "spool (pbf-detail)"
|
|
"$BIN" pbf-detail "$PBF" "$DETAIL_TMP" --store "$STORE" --sort-memory-mib 2048 --no-tiles 2>&1 | tee -a "$LOG"
|
|
|
|
phase "spool-complete"
|
|
# The slab driver (world-slabs.sh) takes over from here: Europe-outward
|
|
# pbf-base --bbox slabs, per-slab bake, weave into the serving shard set.
|
|
touch local/maps/world-detail.store/SPOOL_COMPLETE
|