nigig-org/workflow.md

# Nigig-org implementation workflow

This document records the reproducible workflow used in this session from repository acquisition through validation and Gitdab push. It deliberately excludes credentials, access tokens, and any other secret material.

## 0. Mandatory pre-task checks for any new user or AI agent

Every task, in any crate, starts with the two checks below. Do **not** begin
implementation work until both are satisfied.

### 0.1 Reconcile the Makepad fork with upstream `dev`

The workspace depends on the Gitdab fork `https://gitdab.com/andodeki/makepad`
(pinned by `rev =` in `crates/apps/pdf/pdf-makepad/Cargo.toml` and
`crates/apps/map/Cargo.toml`). The fork is based on the upstream Makepad `dev`
branch (the improved-Makepad-paradigm codebase) plus two fork-only branches:

- `portallist_flow_adaptive_view` — the default/synced branch we build against.
- `nigig-dev-reexports` — re-exports optional Makepad sibling crates (map,
  pdf, …) that upstream `dev` does not re-export, needed by `nigig-map` /
  `nigig-pdf`.

Upstream `dev` advances frequently. A stale fork silently leaves us behind on
platform fixes (Windows WASAPI, macOS present-gate watchdog, iOS panics, headless
shader rebuilds, portal-list/animation, etc.). Before any task:

1. Compare the pinned rev with upstream `dev`:
   ```bash
   # fork default branch HEAD (what Cargo.toml `rev =` should point at)
   curl -sL "https://gitdab.com/api/v1/repos/andodeki/makepad/branches" | \
     python3 -c "import json,sys;[print(b['name'], b['commit']['id'][:12]) for b in json.load(sys.stdin)]"
   # upstream dev HEAD (what the fork should have synced to)
   curl -sL "https://api.github.com/repos/makepad/makepad/commits?sha=dev&per_page=5" | \
     python3 -c "import json,sys;[print(c['sha'][:12], c['commit']['committer']['date'][:10], c['commit']['message'].splitlines()[0][:70]) for c in json.load(sys.stdin)]"
   # rev actually pinned in this workspace
   grep -h "makepad-widgets" crates/apps/pdf/pdf-makepad/Cargo.toml crates/apps/map/Cargo.toml
   ```
2. If upstream `dev` is ahead (or has been ahead since last sync), update the
   fork **before** continuing:
   ```bash
   # from a local clone of the fork
   git fetch upstream dev
   git push origin <synced-branch>:portallist_flow_adaptive_view
   git push origin upstream/dev:nigig-dev-reexports   # merge re-export changes back
   ```
3. Reconcile any conflicts between upstream `dev` and the fork-only changes
   (the `nigig-dev-reexports` re-exports and `portallist_flow_adaptive_view`
   map/adaptive-view work). Resolve them in the fork, never by quietly dropping
   the re-exports the workspace depends on.
4. Bump the `rev =` in both `Cargo.toml` files to the new fork commit and verify
   the whole workspace still builds and the test suite still passes (see
   section 5). Check for breaking API changes between Makepad versions.
5. Record the sync in the commit message (e.g.
   `chore(makepad): sync fork with upstream dev <sha>`).

### 0.2 nigig-pdf crate context (read before touching any `pdf-*` crate)

`crates/apps/pdf/` contains the four-crate `nigig-pdf` stack:

- `nigig-pdf-cos` — PDF file-format core (lexer, parser, objects, xref,
  writer, filters, encryption, incremental updates).
- `nigig-pdf-document` — document semantics (page tree, annotations, AcroForm,
  form actions, signatures/reading, structure tree, save/revisions).
- `nigig-pdf-graphics` — content-stream interpreter, `PdfDevice` trait,
  `RenderCommand` recording, font engine (SFNT/Type1/CID/CMap), colour spaces,
  ICC, PDF functions, transparency, cache, async worker, text extraction.
- `nigig-pdf-makepad` — Makepad GPU rendering (`PdfRenderer`,
  `MakepadPdfDevice`), `PdfPageWidget` viewer, form-field interaction.

Relationship to the other two PDF codebases in this project:

- **Makepad upstream `dev`** ships `libs/pdf_parse` (a minimal parse-only
  library) and `widgets/src/pdf_view.rs` (a view-only widget with text
  selection). It can **read and display** PDFs only.
- **nigig-pdf** is the Rust improvement/re-implementation of that Makepad PDF
  code. It already exceeds the upstream `dev` viewer (device-trait rendering,
  golden/fuzz testing, colour spaces, forms interaction, incremental save).
- **dart-pdf (DavBfr / ben-milanko, Apache-2.0)** is the reference feature
  target. The goal is feature parity for **creating and editing** PDFs, not
  just viewing. Its packages (`pdf_cos`, `pdf_document`, `pdf_graphics`,
  `dart_pdf_editor`) map 1:1 onto the four nigig crates.

When working on any `pdf-*` crate, carry this context forward:

- The 4-crate boundary is strict: `cos` → `document` → `graphics` → `makepad`.
  No Makepad imports in the three core crates; UI never mutates COS
  dictionaries directly.
- The current feature-gap plan against dart-pdf lives in
  `NIGIG_PDF_FEATURE_PARITY_PLAN.md` at the repo root. Check it before starting
  work; add phase notes there rather than only in commit messages.
- Provenance: nigig-pdf draws on dart-pdf (Apache-2.0) and Makepad
  (MIT OR Apache-2.0). Keep `THIRD_PARTY_NOTICES.md` accurate when porting
  patterns.
- Never claim a feature is supported without an integration test. The corpus
  lives in `crates/apps/pdf/tests/corpus/`, fuzz targets under
  `pdf-cos/fuzz/`, golden renders in `pdf-graphics/tests/golden_render.rs`.

## 1. Obtain and inspect the repository

```bash
git clone https://gitdab.com/andodeki/nigig-org.git
cd nigig-org
git log -1 --format='%H %cs %s'
git status --short
```

Initial review inputs were read from `REVIEWS/`, especially:

- `REVIEWS/NIGIG_PAY_CONSOLIDATED_REVIEW.md`
- `REVIEWS/PAY_CAD_IMPLEMENTATION_STATUS.md`
- the map, spreadsheet, CAD, and cost-estimator review documents.

The implementation work started from the review priority order: contain unsafe payment behavior first, establish a pure domain and durable storage boundary, then migrate legacy paths incrementally.

## 2. Preserve local work while pulling upstream changes

Do not pull over uncommitted implementation work directly. Fetch first, inspect changes, then use rebase/autostash only after confirming the target branch.

```bash
git fetch origin
git log --oneline HEAD..origin/main
git pull --rebase --autostash origin main
git status --short
```

This session pulled upstream commit `e9616c3` (`added missing crates`) and preserved local changes through Git’s autostash mechanism.

## 3. Repair vendored crate paths after upstream import

Upstream added Robius crates under `crates/`, while dependent manifests still pointed to old sibling-repository paths.

The local crate paths were repaired for:

- `robius-sms`
- `robius-ussd`
- `robius-fingerprinting`
- `robius-contacts`
- `robius-trigger`

The root workspace list was also repaired to add commas, remove a duplicate member, and include each imported crate once.

Validation used Python’s TOML parser and direct path checks:

```bash
python3 - <<'PY'
import tomllib
from pathlib import Path
for manifest in [Path('Cargo.toml'), *Path('crates').rglob('Cargo.toml')]:
    tomllib.loads(manifest.read_text())
print('all Cargo manifests parse')
PY
```

The full workspace is not claimed buildable in a standalone checkout until all external Makepad and remaining Robius sibling dependencies are available or vendored.

## 4. Use isolated, self-cleaning Rust tests

The repository contains:

```text
tools/test-rust-clean.sh
```

It creates a temporary directory containing:

- isolated `RUSTUP_HOME`;
- isolated `CARGO_HOME`;
- isolated build target;
- copied pure crate(s), outside the incomplete workspace dependency graph;
- a minimal Rust toolchain and `rustfmt`.

It removes the entire environment through a shell trap on success, failure, interruption, or termination.

Run pure domain tests:

```bash
TEST_TARGET=domain ./tools/test-rust-clean.sh
```

Run storage tests:

```bash
TEST_TARGET=storage ./tools/test-rust-clean.sh
```

To inspect a failed temporary environment only:

```bash
KEEP_TEST_ENV=1 TEST_TARGET=storage ./tools/test-rust-clean.sh
```

Then delete the retained temporary path manually after diagnosis. Do not retain toolchains/caches inside the repository or user home directory.

## 5. Validate each implementation tranche

Before committing each feature:

```bash
git diff --check
TEST_TARGET=domain ./tools/test-rust-clean.sh
TEST_TARGET=storage ./tools/test-rust-clean.sh
```

Use only relevant isolated tests when a feature changes one pure crate. Be explicit if full application compilation cannot run because external GUI/platform dependencies are absent.

## 6. Commit focused changes

Use one focused commit for one tested implementation tranche:

```bash
git add <specific files>
git commit -m '<type>(pay): concise description'
git log -1 --format='%H %s'
```

Repository-local identity used in this session:

```bash
git config user.name 'andodeki'
git config user.email 'andodeki@noreply.gitdab.com'
```

Do not set a global identity unless the developer explicitly wants that behavior.

## 7. Push to Gitdab

Push only after validation and a successful focused commit:

```bash
git push origin main
```

If HTTPS credentials are unavailable in a non-interactive environment, configure authentication through a secure credential helper, SSH deploy key, or an access token supplied through a secure mechanism. Never commit, log, store, or copy a token into repository files, workflow documents, shell history, or Git remote configuration.

After pushing:

```bash
git status --short
git log -1 --format='%H %s'
```

The expected result is a clean working tree and the pushed commit at `origin/main`.

## 8. Current payment migration order

The current safe migration sequence is:

1. Keep default builds tracker-only.
2. Block legacy PIN UI, PIN request creation, USSD dispatch, automatic retry, and bulk dispatch outside explicit demo mode.
3. Use exact-domain `PaymentIntent`, `Money`, validation and fee calculation.
4. Persist payment intent transitions through SQLite and a single storage worker.
5. Import legacy JSON pending records conservatively as reconciliation-required where outcome is ambiguous.
6. Wire startup migration once, then move authoritative UI reads from legacy JSON to SQLite.
7. Only then replace the legacy thread-local PayFlowHandler with the coordinator/application-service path.
8. Add SQLCipher/Keystore integration and provider-authorized gateway before enabling production payment dispatch.

## 9. Non-negotiable rules

- No production default path may collect or retain an M-Pesa PIN.
- No ambiguous session/SMS outcome may be called a known payment failure or success.
- No automatic replay of a financial dispatch after an ambiguous outcome.
- No silent unknown fee fallback to zero.
- No `git push` before test/check results are known.
- No access token or credential in committed files or workflow documentation.