nigig-org/workflow.md

Nigig-org implementation workflow

This document records the reproducible workflow used in this session from repository acquisition through validation and Gitdab push. It deliberately excludes credentials, access tokens, and any other secret material.

0. Mandatory pre-task checks for any new user or AI agent

Every task, in any crate, starts with the two checks below. Do not begin implementation work until both are satisfied.

0.1 Reconcile the Makepad fork with upstream dev

The workspace depends on the Gitdab fork https://gitdab.com/andodeki/makepad (pinned by rev = in crates/apps/pdf/pdf-makepad/Cargo.toml and crates/apps/map/Cargo.toml). The fork is based on the upstream Makepad dev branch (the improved-Makepad-paradigm codebase) plus two fork-only branches:

• portallist_flow_adaptive_view — the default/synced branch we build against.
• nigig-dev-reexports — re-exports optional Makepad sibling crates (map, pdf, …) that upstream dev does not re-export, needed by nigig-map / nigig-pdf.

Upstream dev advances frequently. A stale fork silently leaves us behind on platform fixes (Windows WASAPI, macOS present-gate watchdog, iOS panics, headless shader rebuilds, portal-list/animation, etc.). Before any task:

1. Compare the pinned rev with upstream dev:

   # fork default branch HEAD (what Cargo.toml `rev =` should point at)
   curl -sL "https://gitdab.com/api/v1/repos/andodeki/makepad/branches" | \
     python3 -c "import json,sys;[print(b['name'], b['commit']['id'][:12]) for b in json.load(sys.stdin)]"
   # upstream dev HEAD (what the fork should have synced to)
   curl -sL "https://api.github.com/repos/makepad/makepad/commits?sha=dev&per_page=5" | \
     python3 -c "import json,sys;[print(c['sha'][:12], c['commit']['committer']['date'][:10], c['commit']['message'].splitlines()[0][:70]) for c in json.load(sys.stdin)]"
   # rev actually pinned in this workspace
   grep -h "makepad-widgets" crates/apps/pdf/pdf-makepad/Cargo.toml crates/apps/map/Cargo.toml
   

2. If upstream dev is ahead (or has been ahead since last sync), update the fork before continuing:

   # from a local clone of the fork
   git fetch upstream dev
   git push origin <synced-branch>:portallist_flow_adaptive_view
   git push origin upstream/dev:nigig-dev-reexports   # merge re-export changes back
   

3. Reconcile any conflicts between upstream dev and the fork-only changes (the nigig-dev-reexports re-exports and portallist_flow_adaptive_view map/adaptive-view work). Resolve them in the fork, never by quietly dropping the re-exports the workspace depends on.
4. Bump the rev = in both Cargo.toml files to the new fork commit and verify the whole workspace still builds and the test suite still passes (see section 5). Check for breaking API changes between Makepad versions.
5. Record the sync in the commit message (e.g. chore(makepad): sync fork with upstream dev <sha>).

0.2 nigig-pdf crate context (read before touching any pdf-* crate)

crates/apps/pdf/ contains the four-crate nigig-pdf stack:

• nigig-pdf-cos — PDF file-format core (lexer, parser, objects, xref, writer, filters, encryption, incremental updates).
• nigig-pdf-document — document semantics (page tree, annotations, AcroForm, form actions, signatures/reading, structure tree, save/revisions).
• nigig-pdf-graphics — content-stream interpreter, PdfDevice trait, RenderCommand recording, font engine (SFNT/Type1/CID/CMap), colour spaces, ICC, PDF functions, transparency, cache, async worker, text extraction.
• nigig-pdf-makepad — Makepad GPU rendering (PdfRenderer, MakepadPdfDevice), PdfPageWidget viewer, form-field interaction.

Relationship to the other two PDF codebases in this project:

• Makepad upstream dev ships libs/pdf_parse (a minimal parse-only library) and widgets/src/pdf_view.rs (a view-only widget with text selection). It can read and display PDFs only.
• nigig-pdf is the Rust improvement/re-implementation of that Makepad PDF code. It already exceeds the upstream dev viewer (device-trait rendering, golden/fuzz testing, colour spaces, forms interaction, incremental save).
• dart-pdf (DavBfr / ben-milanko, Apache-2.0) is the reference feature target. The goal is feature parity for creating and editing PDFs, not just viewing. Its packages (pdf_cos, pdf_document, pdf_graphics, dart_pdf_editor) map 1:1 onto the four nigig crates.

When working on any pdf-* crate, carry this context forward:

• The 4-crate boundary is strict: cos → document → graphics → makepad. No Makepad imports in the three core crates; UI never mutates COS dictionaries directly.
• The current feature-gap plan against dart-pdf lives in NIGIG_PDF_FEATURE_PARITY_PLAN.md at the repo root. Check it before starting work; add phase notes there rather than only in commit messages.
• Provenance: nigig-pdf draws on dart-pdf (Apache-2.0) and Makepad (MIT OR Apache-2.0). Keep THIRD_PARTY_NOTICES.md accurate when porting patterns.
• Never claim a feature is supported without an integration test. The corpus lives in crates/apps/pdf/tests/corpus/, fuzz targets under pdf-cos/fuzz/, golden renders in pdf-graphics/tests/golden_render.rs.

1. Obtain and inspect the repository

git clone https://gitdab.com/andodeki/nigig-org.git
cd nigig-org
git log -1 --format='%H %cs %s'
git status --short

Initial review inputs were read from REVIEWS/, especially:

• REVIEWS/NIGIG_PAY_CONSOLIDATED_REVIEW.md
• REVIEWS/PAY_CAD_IMPLEMENTATION_STATUS.md
• the map, spreadsheet, CAD, and cost-estimator review documents.

The implementation work started from the review priority order: contain unsafe payment behavior first, establish a pure domain and durable storage boundary, then migrate legacy paths incrementally.

2. Preserve local work while pulling upstream changes

Do not pull over uncommitted implementation work directly. Fetch first, inspect changes, then use rebase/autostash only after confirming the target branch.

git fetch origin
git log --oneline HEAD..origin/main
git pull --rebase --autostash origin main
git status --short

This session pulled upstream commit e9616c3 (added missing crates) and preserved local changes through Git’s autostash mechanism.

3. Repair vendored crate paths after upstream import

Upstream added Robius crates under crates/, while dependent manifests still pointed to old sibling-repository paths.

The local crate paths were repaired for:

• robius-sms
• robius-ussd
• robius-fingerprinting
• robius-contacts
• robius-trigger

The root workspace list was also repaired to add commas, remove a duplicate member, and include each imported crate once.

Validation used Python’s TOML parser and direct path checks:

python3 - <<'PY'
import tomllib
from pathlib import Path
for manifest in [Path('Cargo.toml'), *Path('crates').rglob('Cargo.toml')]:
    tomllib.loads(manifest.read_text())
print('all Cargo manifests parse')
PY

The full workspace is not claimed buildable in a standalone checkout until all external Makepad and remaining Robius sibling dependencies are available or vendored.

4. Use isolated, self-cleaning Rust tests

The repository contains:

tools/test-rust-clean.sh

It creates a temporary directory containing:

• isolated RUSTUP_HOME;
• isolated CARGO_HOME;
• isolated build target;
• copied pure crate(s), outside the incomplete workspace dependency graph;
• a minimal Rust toolchain and rustfmt.

It removes the entire environment through a shell trap on success, failure, interruption, or termination.

Run pure domain tests:

TEST_TARGET=domain ./tools/test-rust-clean.sh

Run storage tests:

TEST_TARGET=storage ./tools/test-rust-clean.sh

To inspect a failed temporary environment only:

KEEP_TEST_ENV=1 TEST_TARGET=storage ./tools/test-rust-clean.sh

Then delete the retained temporary path manually after diagnosis. Do not retain toolchains/caches inside the repository or user home directory.

5. Validate each implementation tranche

Before committing each feature:

git diff --check
TEST_TARGET=domain ./tools/test-rust-clean.sh
TEST_TARGET=storage ./tools/test-rust-clean.sh

Use only relevant isolated tests when a feature changes one pure crate. Be explicit if full application compilation cannot run because external GUI/platform dependencies are absent.

6. Commit focused changes

Use one focused commit for one tested implementation tranche:

git add <specific files>
git commit -m '<type>(pay): concise description'
git log -1 --format='%H %s'

Repository-local identity used in this session:

git config user.name 'andodeki'
git config user.email 'andodeki@noreply.gitdab.com'

Do not set a global identity unless the developer explicitly wants that behavior.

7. Push to Gitdab

Push only after validation and a successful focused commit:

git push origin main

If HTTPS credentials are unavailable in a non-interactive environment, configure authentication through a secure credential helper, SSH deploy key, or an access token supplied through a secure mechanism. Never commit, log, store, or copy a token into repository files, workflow documents, shell history, or Git remote configuration.

After pushing:

git status --short
git log -1 --format='%H %s'

The expected result is a clean working tree and the pushed commit at origin/main.

8. Current payment migration order

The current safe migration sequence is:

1. Keep default builds tracker-only.
2. Block legacy PIN UI, PIN request creation, USSD dispatch, automatic retry, and bulk dispatch outside explicit demo mode.
3. Use exact-domain PaymentIntent, Money, validation and fee calculation.
4. Persist payment intent transitions through SQLite and a single storage worker.
5. Import legacy JSON pending records conservatively as reconciliation-required where outcome is ambiguous.
6. Wire startup migration once, then move authoritative UI reads from legacy JSON to SQLite.
7. Only then replace the legacy thread-local PayFlowHandler with the coordinator/application-service path.
8. Add SQLCipher/Keystore integration and provider-authorized gateway before enabling production payment dispatch.

9. Non-negotiable rules

• No production default path may collect or retain an M-Pesa PIN.
• No ambiguous session/SMS outcome may be called a known payment failure or success.
• No automatic replay of a financial dispatch after an ambiguous outcome.
• No silent unknown fee fallback to zero.
• No git push before test/check results are known.
• No access token or credential in committed files or workflow documentation.