Reuse shared review and confirmation state. Android tile/setup and volatile OCR handoff route into Makepad; native floating controls remain. Android overlay rendering host is not implemented. Prior isolated checks: macOS ARM64 and Android ARM64 library checks, native Linux build/run and tests; native mobile/macOS runtime validation remains outstanding.
117 lines
12 KiB
Markdown
117 lines
12 KiB
Markdown
# Shared Makepad capture UI migration
|
|
|
|
## Shared access controls (published as 2bf18d8)
|
|
|
|
- The app's Capture product entry opens the existing Makepad capture page on every target instead of immediately opening Android's Java setup activity.
|
|
- That page contains shared access-status presentation, an explicit refresh action, Android overlay/notification permission requests, and enable/stop controls using the Rust capture facade.
|
|
- Requests are distinguished from grants. MediaProjection consent remains per-capture, never a persisted grant.
|
|
- Desktop capture and editable review continue to use the existing Makepad page. Desktop access wording does not pretend Android permissions apply to desktop platforms.
|
|
- The isolated native validation runner now recreates the manifest's actual sibling Makepad directory layout.
|
|
|
|
## Not implemented yet — do not call the migration complete
|
|
|
|
- Android Quick Settings now opens the Makepad capture page (see Android increment below).
|
|
- Android access explanations/requests are now Makepad-owned. The consent activity contains only Android system-consent lifecycle glue.
|
|
- Android floating controls and the small handoff/error prompt remain native Java views; editing and confirmation are Makepad-owned.
|
|
- A Makepad service-hosted Android rendering surface, input/focus routing, and lifecycle synchronization are not implemented.
|
|
- Android OCR delivery into the shared Makepad review model is implemented with a bounded volatile mailbox; Share to Budget remains independent and unchanged.
|
|
- macOS floating controls are implemented as described below. Inline review remains in the main Makepad window. Windows/Linux floating controls are not yet implemented.
|
|
- HarmonyOS PC remains unavailable pending the native SDK bridge and validation. Wayland must not promise unrestricted overlay placement.
|
|
- The missing dependency DEX packager fix discussed in chat is not applied to an SDK checkout here. It remains necessary for Android class loading.
|
|
|
|
## Required next implementation steps
|
|
|
|
1. Restore the exact owner Makepad SDK checkout, including local SDK 33/35 changes and dependency DEX packaging.
|
|
2. Build and run the app through app/tests/run-clean.sh before committing/publishing these UI changes.
|
|
3. Implement and test Android service-hosted Makepad surface lifecycle; do not reuse a single activity EGL surface for two live windows without backend support.
|
|
4. Route tile/overlay actions into a shared Rust state machine; keep consent tokens in the native capture backend and never persist them.
|
|
5. Deliver ephemeral OCR results to the existing Makepad review/transaction path, then remove duplicated Java presentation only after device tests pass.
|
|
6. Validate desktop floating-window behavior individually, exposing capability limitations (especially Wayland and HarmonyOS PC) to shared UI.
|
|
|
|
## Historical validation of the access-controls commit
|
|
|
|
- Existing Python Android contract tests: 2 passed.
|
|
- Existing Python app packaging tests: 2 passed.
|
|
- Native build/run attempted with the corrected isolated runner: blocked before toolchain setup because ../nigig-org/makepad-native-glue/makepad is absent.
|
|
- No Rust compilation, rendered UI test, Android device test, or native macOS/Windows/HarmonyOS test passed for this increment. Existing Python checks do not validate Makepad DSL or Rust type correctness.
|
|
- Published on the existing Budget main branch at the owner's explicit request. Publication does not imply Rust compilation or platform validation; the blockers above remain outstanding.
|
|
|
|
|
|
## macOS floating panel increment
|
|
|
|
`app/src/capture_panel.rs` adapts Makepad's `examples/floating_panel`:
|
|
|
|
- A macOS-only second Makepad window, configured as a borderless, non-activating floating NSPanel, with a draggable header.
|
|
- Capture, cancel/discard, and open-review controls reuse the existing BudgetApp capture state. No second job, workbook, or review model is created.
|
|
- Capture is disabled while a worker is active or a review is pending. Requests from the main page and panel share the same guard.
|
|
- All editing and financial confirmation remain on the existing main Makepad review page. The panel cannot confirm a purchase.
|
|
- Before starting a worker, the native backend hides the panel with AppKit orderOut. If it cannot find/hide the panel on the UI thread, capture does not start. Borderless panels are not assumed to be miniaturizable.
|
|
- The panel is restored on completion with orderFrontRegardless; tray open will not restore it while a worker is active.
|
|
- Android and other platforms retain their original single-window shell. No Android overlay support is claimed by this increment.
|
|
|
|
The native helper in budget-capture only controls window visibility. It draws no widgets. It identifies the single app-owned panel by its reserved title `Old Mutual Budget — Capture Panel`.
|
|
|
|
### Owner macOS acceptance test
|
|
|
|
With the matching Makepad SDK available:
|
|
|
|
```sh
|
|
cargo run -p budget
|
|
cargo test -p budget --test ui floating_capture_panel_opens_shared_review -- --nocapture
|
|
```
|
|
|
|
Manually verify: header drag; no unwanted app activation on panel click; hide before screenshot; restore on success/error/cancel; review blocks another capture; cancel adds nothing; tray reopen; Screen Recording denial; quitting with a worker active. A cross-target check does not prove any of these native runtime properties.
|
|
|
|
### Validation of this increment
|
|
|
|
- Exact upstream Makepad revision: 44ecb822b48106fa76bde4dfd309e5a8a7ff1932, which contains the floating_panel example. This is not the owner's locally modified Android packager.
|
|
- Full app macOS ARM64 cross-target cargo check: passed (2m01s). No macOS app was launched in this Linux environment.
|
|
- Python source/packaging contracts: 7 passed. These are structural checks, not rendered-panel tests.
|
|
- Native Linux isolated build and 15-second cargo run smoke: passed. Release UI suite: 7 passed in 16.96s, including Linux-rendered macOS/Omarchy style regression checks. These are not native macOS panel tests.
|
|
- Rust test summaries: 101 app library + 6 capture library + 5 review + 9 models + 4 desktop + 3 OCR + 7 UI = 135 passed. Opt-in live capture tests returned without running their live-capture branches (LIVE_X11_CAPTURE was not enabled).
|
|
- Python checks: 7 passed; Java model tests: 12 passed.
|
|
- Native macOS panel smoke test was added but not executed here. No Android, Windows, or HarmonyOS runtime test was performed.
|
|
- First Linux attempt failed for missing libclang. Installed the native dependency and reran successfully. No project code workaround was needed.
|
|
- Temporary SDKs, Rust toolchains/caches/build products and the temporary 3 GiB swap file were removed. System-native development libraries remain installed in the sandbox.
|
|
- Detailed logs are under /home/user/panel-validation; the resolved Linux validation lock is panel-validation/validated.lock. No dependency pins or repository lockfile were changed.
|
|
- This increment is included in the owner-authorized Budget main publication together with the Android setup/review increment.
|
|
|
|
|
|
## Android Makepad setup/review increment
|
|
|
|
### Implemented
|
|
|
|
1. BudgetTileService queues an in-process open request and launches the main Budget activity. It does not start screen capture.
|
|
2. The Makepad capture page consumes that request on startup/resume/actions and presents the shared access UI. Enabling floating capture checks overlay access and requests the native service; it no longer opens a Java setup form.
|
|
3. Tapping the existing floating control opens CapturePermissionActivity, which has no custom layout/buttons. It requests a fresh Android MediaProjection grant and passes that grant only to the native capture service.
|
|
4. OCR processing still stops projection before inference. Its native handoff prompt offers Open review in Budget or Discard.
|
|
5. CaptureReviewInbox holds at most one OCR string (16,384 UTF-16 units) in process memory. Reads/offers reject entries five minutes old. It uses no disk, clipboard, intent text extras, uploads, screenshot retention, or consent-token persistence. Process death loses the handoff; TTL cleanup is lazy on access.
|
|
6. JNI consumes the text only when Makepad has no pending review/job. Main-window fields, validation, UUID identity, persistence, and explicit confirmation use the same Review/product transaction path as desktop. Empty OCR allows manual entry on Android.
|
|
7. Cancel/discard stops the native capture generation and clears pending handoff text. Native stop, close, task removal and screen-off also clear unconsumed handoffs where the service is active. Successful handoff teardown deliberately does not clear text before Makepad consumes it.
|
|
8. Existing ShareToBudget Java activity and its external sharing workflow are unchanged.
|
|
|
|
### Still not a Makepad-rendered Android floating surface
|
|
|
|
The Android renderer inspected at Makepad 44ecb822 has one activity-owned rendering surface and global surface JNI callbacks; CreateWindow does not supply the independent service-window hosting used by macOS NSPanel. This increment does NOT reparent that surface, create a second Android renderer, or pretend the macOS configuration works on Android.
|
|
|
|
BudgetOverlayService still draws the floating +/close controls and handoff/error prompt with Android views. Those require a separate SDK-level rendering/lifecycle integration before they can be replaced safely. No claim of full Android floating-UI migration should be made.
|
|
|
|
### Packaging and device acceptance
|
|
|
|
The final APK must include CaptureReviewInbox plus the existing native classes. app/tests/verify_apk.py now checks it. The owner's dependency-DEX bundling fix remains required; this increment does not modify cargo-makepad.
|
|
|
|
On an Android device verify: tile opens Makepad setup; deny/grant overlay access; enable/stop; system capture denial; one successful capture; Open review routes into Makepad; edits survive reopening; another result cannot overwrite active edits; discard adds nothing; confirmation is idempotent; task/process death discards unconsumed handoff; Share to Budget still works. Fresh consent is mandatory on every capture.
|
|
|
|
### Validation
|
|
|
|
- Pure Java inbox/extraction tests: 18 passed.
|
|
- Python source/packaging contract tests: 9 passed.
|
|
- All production Java compiled against the real Android SDK 35 android.jar; the capture build script also generated DEX during the Android check.
|
|
- Android ARM64 `cargo check -p budget --lib --target aarch64-linux-android`: passed in 1m06s using upstream Makepad 44ecb822. This is type checking, not native linking or APK assembly.
|
|
- Linux workspace library tests (101 + 6), shared review (5), and models (9): 121 passed. Native cargo build and 15-second cargo run smoke passed.
|
|
- No fresh rendered UI suite was run for the Android increment; the prior macOS increment's Linux UI results are recorded above.
|
|
- First validation command incorrectly checked the desktop executable target on Android. It failed because Android exports the library entry point rather than shell::app_main. The runner was corrected to --lib; no app entrypoint workaround was introduced.
|
|
- JNI calls now use bounded local frames so repeated handoff/access polling does not leak local references on the permanently attached Rust thread.
|
|
- Logs and the resolved validation lock are in /home/user/android-review-validation. Temporary SDK archives, Rust toolchains, caches, and build products were removed. No repository lockfile was changed.
|
|
- The macOS and Android increments are included in the owner-authorized Budget main publication. The proposed Makepad-rendered Android overlay host is not implemented; no Makepad SDK changes are included.
|
|
- No Android device/emulator was available; no full APK installation or runtime behavior is claimed.
|