Compare commits
4 commits
site-02-rc
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 97f03fd2bd | |||
| 834755205b | |||
| 9b07b8f7d5 | |||
| f535296b43 |
19 changed files with 2771 additions and 214 deletions
|
|
@ -3,8 +3,8 @@
|
|||
**Plan date:** 2026-09-12
|
||||
**Audit baseline:** `e899a271c69efca0e11ae274b879378d2f26485f` (`main`)
|
||||
**Scope:** `crates/apps/nigig-site`, its domain/storage/media/export code, and its protocol boundary with `nimanyatta`
|
||||
**Status:** Execution in progress. SITE-00 and SITE-01 are complete and published; SITE-02 through SITE-19 remain incomplete.
|
||||
**Published evidence:** SITE-00 `f49d8b16ac8f1245ba1e80ac19588825f91a79f4`; SITE-01 `5d2d890f701444a3ff7e9c227e41e4608406ee89`.
|
||||
**Status:** Execution in progress. SITE-00, SITE-01 and SITE-03 are complete and published; SITE-02 is a blocked candidate (setup/migration hard-locked pending independent review); SITE-04 through SITE-13 and SITE-15 through SITE-19 hardening modules are implemented in the worktree and pending verification; SITE-14 follows the scanner-owned OCR design below (no whole-PDF dependency). Product scope (`construction-site-app-scope.md` + `construction-site-app-scope2.md`) is mapped to feature tranches SITE-20–SITE-32 with an R1–R5 release roadmap (§13); anything in the scope docs without a tranche row in §13 is not covered — report it as a plan gap.
|
||||
**Published evidence:** SITE-00 `f49d8b16ac8f1245ba1e80ac19588825f91a79f4`; SITE-01 `5d2d890f701444a3ff7e9c227e41e4608406ee89`; SITE-03 `834755205b6c0cc63f506435cd8edbdec2422024`.
|
||||
**Release posture:** **Do not enable current multi-device sync. Do not call current storage confidential. Do not claim real OCR, safe media processing, or lossless photo reports.**
|
||||
|
||||
This is the authoritative forward execution plan for this crate. `README.md` and `NIGIG_SITE_ASSESSMENT_AND_PLAN.md` contain useful history but do not satisfy the security, migration, and release gates here.
|
||||
|
|
@ -211,6 +211,12 @@ These are initial hard safety ceilings. Lower platform-specific quotas are allow
|
|||
| Meeting transcript | 5 MiB | Ingest/editor/AI boundary. |
|
||||
| PDF/DOCX | 500 pages and 100 MiB output | Paginated counting writer. |
|
||||
| AI request/response | 64 KiB / 16 KiB; 30 s; 1 active/site | AI coordinator. |
|
||||
| OCR input (grayscale) | 20 MiB / <= 8,192 edge / <= 40 MP; <= 100,000 components | Scanner engine preflight before allocation. |
|
||||
| OCR suggestion | confidence >= 0.80 and exact field-charset, else `None` | Scanner engine + site confirmation gate. |
|
||||
| Photo sync rendition | <= 1 MB per photo; original retained until synced | Client media pipeline (SITE-32). |
|
||||
| Capture → saved | < 1 s on mid-range Android | No decode/export in handler (SITE-32). |
|
||||
| Cold start | < 3 s on mid-range Android | Deferred content load (SITE-32). |
|
||||
| Monthly pack generation | < 60 s | Streaming export (SITE-21/32). |
|
||||
| Undo/offline journal | 10,000 ops or 64 MiB/site before compaction | Sync/repository. |
|
||||
| UI event | p95 < 8 ms desktop / 12 ms mobile | No disk/network/decode/export in handler. |
|
||||
| Site switch | p95 < 100 ms for metadata; content loads incrementally | Runtime benchmark. |
|
||||
|
|
@ -559,26 +565,33 @@ Each tranche is a separately tested commit and push. Security containment must n
|
|||
|
||||
**Rollback:** AI stays disabled; manual text remains fully functional.
|
||||
|
||||
### SITE-14 — Real OCR or truthful capture-only flow
|
||||
### SITE-14 — Scanner-owned on-device OCR (no whole-PDF dependency)
|
||||
|
||||
**Priority:** P1 privacy
|
||||
**Effort:** 6–12 person-days plus platform/license review
|
||||
**Depends on:** SITE-05, SITE-07
|
||||
**Owns:** `crates/apps/nigig_doc_scanner` (`ocr` cargo feature). **Will consume (next chunk):** `nigig-site` via `nigig_doc_scanner/ocr` only (buffer API, `default-features = false`; site bridge + UI enablement land separately — until then the site production graph contains no OCR path). **Explicitly not shipped:** `nigig-pdf-document` or any other `pdf-*` crate — the PDF OCR seam (`pdf-document/src/ocr.rs`, an engine trait only) is gated behind its own `ocr` cargo feature so no consumer pulls `rsa`/`x509`/`cms` and the document model just to recognize text.
|
||||
**Related parallel effort:** `crates/apps/nigig-ocr` (desktop OCR-tool port; OS-native Vision/WinRT providers, stub fallback on Linux). It must not become the site recognition path: its Linux fallback should delegate to the scanner engine here, and its `nigig-pdf-document` dependency must stay inside that app, never inside `nigig-site` production.
|
||||
|
||||
**Change**
|
||||
|
||||
- Rename current behavior to crop/enhance if no OCR engine is shipped.
|
||||
- If OCR is enabled, integrate a maintained on-device engine with pinned model/version/license, language coverage, confidence/field extraction, and no network by default.
|
||||
- Process managed `AssetId` under media budgets in a bounded worker; correlate result to site/worker/base revision.
|
||||
- Ship the engine in `nigig_doc_scanner` behind an explicit `ocr` cargo feature (`default = ["app", "ocr"]`; downstream device crates enable `nigig_doc_scanner/ocr` with `default-features = false` so Makepad UI, `nigig-core`/`nigig-uikit`, and `robius-location` never enter their production graph for text recognition).
|
||||
- The engine is on-device, pure-Rust, no model download and no network: grayscale → Otsu binarization → connected-component segmentation → line grouping → embedded-glyph template match with per-word confidence. Language coverage is exactly the embedded glyph set (ASCII digits plus uppercase Latin for ID numbers/names); anything outside it must lower confidence, never invent text.
|
||||
- The engine core takes raw 8-bit grayscale buffers (`&[u8], width, height`) so `nigig-site` production needs no `image` dependency; `DynamicImage` glue stays in the scanner crate beside `scanner_core`.
|
||||
- Process managed `AssetId` bytes under media budgets in a bounded worker; correlate result to operation/site/worker/base revision; suggestions below the confidence floor return `None` and force manual entry.
|
||||
- Present suggestions with confidence for user confirmation; never auto-approve identity.
|
||||
- Define ID-image deletion immediately after verified extraction unless retention has explicit lawful purpose/consent.
|
||||
- The scanner app itself becomes functional through the same `ocr` feature (same engine, same vectors); `nigig-site` transfers the functionality by calling the scanner crate, never by copying the engine or by depending on PDF crates.
|
||||
|
||||
**Tests / exit**
|
||||
|
||||
- Representative permitted ID fixtures measure field precision/recall and false-positive behavior; synthetic fixtures alone are insufficient.
|
||||
- Rotated, blurred, glare, Unicode names, no text, adversarial image, cancel, and stale/site-switch tests.
|
||||
- Synthetic determinism fixtures (text rendered from the embedded glyph set at multiple scales, plus blank/rotated/blurred/glare/adversarial/no-text inputs) prove the pipeline end to end in CI without camera hardware.
|
||||
- Rotated, blurred, glare, Unicode names, no text, adversarial image, cancel, budget-overflow, and stale/site-switch tests.
|
||||
- UI never says OCR found data when suggestions are `None`.
|
||||
- Privacy/license/security review passes and no image leaves device absent explicit remote-OCR consent.
|
||||
- `cargo tree` (or lockfile inspection) proves `nigig-site` production enables no `pdf-*` crate for OCR and `nigig_doc_scanner` without `app` enables no UI/location crates.
|
||||
- Privacy/license/security review passes and no image leaves device absent explicit remote-OCR consent. License impact is nil (no new dependency; engine is first-party code).
|
||||
|
||||
**Rollback:** capture/manual entry only, with no OCR claim.
|
||||
|
||||
|
|
@ -702,6 +715,260 @@ Each tranche is a separately tested commit and push. Security containment must n
|
|||
|
||||
**Rollback:** release is blocked or the failing capability remains disabled through SITE-01.
|
||||
|
||||
### SITE-20 — Organisation, full RBAC, site registry, settings and signatures
|
||||
|
||||
**Priority:** P0 product foundation (scope Phase 0: FR-0.1, FR-0.2, FR-0.3, FR-0.8)
|
||||
**Effort:** 8–12 person-days
|
||||
**Depends on:** SITE-03, SITE-04, SITE-05
|
||||
|
||||
**Change**
|
||||
|
||||
- Add `Organization` aggregate: company creation, invite onboarding (email/phone/share link), per-site role assignment, consultant/client scoping strictly to shared scope.
|
||||
- Extend the SITE-05 role set to the §4.2 product matrix: Org Admin, Overall Supervisor, Report Master, Site Supervisor, Engineer/Foreman, Procurement Officer, HSE Officer, External Consultant (assigned-tasks-only), Client viewer (digest-only). Capabilities remain deny-by-default; matrix defaults are per-organisation configurable.
|
||||
- Extend site registry: per-site working hours, team assignment, geofence polygon, project type (already `SiteNature`), emergency info pointer (owned by SITE-28).
|
||||
- Settings: user profile, drawn/typed signature capture (stored as a managed `AssetId`, never raw path), language English/Swahili, dark mode, notification preferences, storage management. Signatures bind to approvals per SITE-21 (identity + device + timestamp + document hash).
|
||||
- No biometric attendance/verification: explicit non-goal (open question Q7 resolved as photo/QR per scope; biometrics would add DPA-2019 biometric-data obligations).
|
||||
|
||||
**Tests / exit**
|
||||
|
||||
- Invite/onboarding, role matrix (9 roles × capabilities, deny-by-default), consultant/client scope isolation, geofence in/out, working-hours validation, signature round-trip and tamper tests.
|
||||
- Swahili string coverage for new surfaces; no hardcoded English in product flows.
|
||||
|
||||
**Rollback:** organisation features stay local single-user; never weaken SITE-05 denial.
|
||||
|
||||
### SITE-21 — Report richness, signatures, versioning and monthly packs
|
||||
|
||||
**Priority:** P1 (scope Phase 1: FR-1.2–1.5, FR-1.7, FR-1.14; escalation FR-1.6)
|
||||
**Effort:** 10–15 person-days
|
||||
**Depends on:** SITE-06, SITE-16, SITE-20
|
||||
|
||||
**Change**
|
||||
|
||||
- Per-entry category/tags (e.g. concrete works, plumbing) for aggregation; predefined quick-tags; voice-dictation hook (STT arrives in SITE-27; the entry editor reserves the affordance without claiming it).
|
||||
- Report numbering (`DR-<SITE>-<YYYY>-<MM>-<DD>` with uniqueness guard), branded cover page, table of contents, customisable templates; monthly accumulation auto-builds from approved dailies plus HSE/procurement/progress stats (stats engines arrive with SITE-25/26/28; the accumulator consumes their query interfaces).
|
||||
- Collaboration: per-entry status (pending/accepted/returned with comment), @mentions/comments, rejection reason + resubmission loop; approval captures drawn/typed signature bound to user/device/timestamp/document hash; approved reports lock and version (resubmission creates a new version with change log — SITE-06 immutability preserved).
|
||||
- Multi-site compilation flags missing sites and sends chase notifications; escalation nudge 30 minutes after shift end when no report started (new SITE-17 triggers).
|
||||
- Archive: status workflow `Draft → In Review → Approved → Locked`, full-text search, filter by site/date/author/status, batch export. PowerPoint and charts belong to SITE-29 (presentation layer), not here.
|
||||
|
||||
**Tests / exit**
|
||||
|
||||
- Numbering uniqueness under concurrency, template rendering golden files, signature binding/tamper, version-chain integrity, missing-site/chase/escalation journeys, archive search precision/recall on fixtures, batch-export failure atomicity.
|
||||
|
||||
**Rollback:** text-only export with explicit omissions (SITE-16) remains; never print paths.
|
||||
|
||||
### SITE-22 — Site-diary data: weather, plant, deliveries, delays, visitors
|
||||
|
||||
**Priority:** P1 (scope Phase 1: FR-1.9, FR-1.10, FR-1.11, FR-1.12, FR-1.13)
|
||||
**Effort:** 6–10 person-days
|
||||
**Depends on:** SITE-04
|
||||
|
||||
**Change**
|
||||
|
||||
- New aggregates (all site-scoped, validated, audited): `WeatherSnapshot` (auto-fetch + manual override; feeds delay justification), `PlantItem` (equipment/machinery with hours operated), `DelayRecord` (structured reason: weather/labour/materials/design/access + lost-time estimate), `VisitorRecord` (who/purpose/time in-out).
|
||||
- Manpower auto-fills from Phase 2 attendance (SITE-23 query interface); materials-received links deliveries against orders (SITE-26 interface; unresolved link = explicit pending state, never silent).
|
||||
- Weather fetch is the only network call in this tranche: explicit consent, cached offline, provider timeout treated as absent-data (manual override), never blocking report save.
|
||||
|
||||
**Tests / exit**
|
||||
|
||||
- Override-wins-fetch, offline-no-weather journeys, delay roll-up into monthly fixtures, plant-hour arithmetic (checked, finite), visitor overlap validation.
|
||||
|
||||
**Rollback:** diary sections degrade to manual-entry-only; never fabricate weather.
|
||||
|
||||
### SITE-23 — Workforce depth: consent, attendance, QR, register, payroll data
|
||||
|
||||
**Priority:** P1 privacy + payroll correctness (scope Phase 2: FR-2.1–2.6)
|
||||
**Effort:** 8–12 person-days
|
||||
**Depends on:** SITE-05, SITE-07, SITE-14, SITE-20
|
||||
|
||||
**Change**
|
||||
|
||||
- Consent flag required on every ID capture before storage (DPA 2019 lawful basis); no-consent scans stay in volatile memory and are discarded with an explicit notice.
|
||||
- `AttendanceRecord`: clock-in/clock-out per worker per day with optional geofence check; hours and overtime computed with checked finite arithmetic; payroll-ready CSV export (per department/day/week/month). Full payroll and statutory deductions stay out of scope (export-only, §17 of scope).
|
||||
- QR badges: printable per-worker QR for repeat check-in (scan, not full ID capture); badge IDs are opaque, revocable, and distinct from national ID numbers.
|
||||
- `WorkerRegister`: per-site status (active/inactive), skill/trade tags, organisation-level blocklist with appeal/audit trail.
|
||||
- Registration photo at enrolment (anti buddy-punching) as managed asset with the same retention policy as ID images; PPE compliance checklist at sign-in (helmet/boots/vest) feeding HSE stats (SITE-28).
|
||||
- Labour-cost summary (headcount × day rate by trade) for budget tracking; Excel/PDF export of workers table (CSV now, Excel via SITE-16 export work).
|
||||
- Auto-purge job for expired ID data under the retention policy; full deletion on offboarding propagating to indexes, assets, sync ops and backups per stated policy.
|
||||
|
||||
**Tests / exit**
|
||||
|
||||
- No-consent-never-stored, double clock-in/out rejection, overnight-shift arithmetic, geofence edge, QR revocation, blocklist enforcement + audit, purge verification (no residual bytes), payroll CSV golden files, PPE roll-up.
|
||||
|
||||
**Rollback:** attendance stays single-scan daily table; never invent hours.
|
||||
|
||||
### SITE-24 — Chat completeness on the decided transport
|
||||
|
||||
**Priority:** P1 (scope Phase 3: FR-3.1–3.6)
|
||||
**Effort:** 8–12 person-days
|
||||
**Depends on:** SITE-08 through SITE-12 transport decision, SITE-20, SITE-31 server
|
||||
|
||||
**Change**
|
||||
|
||||
- Channel model: per-site channels (e.g. Announcements, General), cross-site management channel, direct messages; membership tied to site teams and SITE-20 roles; clients excluded from channels entirely.
|
||||
- Essentials: read receipts, @mentions with push, pinned messages, message search, offline send-queue with delivery on reconnect, file/document library with inline preview and pinned approvals/drawings, read receipts on critical documents.
|
||||
- Bot notices for key events (report approved, meeting scheduled, PO issued, inspection due); Overall Supervisor broadcast across sites; channel admin roles, mute, archive, retention policy.
|
||||
- Until the transport/server decision lands, chat remains the current local 50-line cache with no composer, no send path, and no sync claims (SITE-01 containment holds).
|
||||
|
||||
**Tests / exit**
|
||||
|
||||
- Membership isolation (cross-site DM refusal), receipt/mention/broadcast journeys, offline queue ordering and dedup, retention-purge verification, moderation actions audited.
|
||||
|
||||
**Rollback:** local cache only; never transmit chat as sync snapshots (SITE-P0-05 stays closed).
|
||||
|
||||
### SITE-25 — Programme depth: dependencies, Gantt, checklists, snags, RFIs, variations
|
||||
|
||||
**Priority:** P1 (scope Phase 4: FR-4.1, FR-4.3–4.9)
|
||||
**Effort:** 12–18 person-days
|
||||
**Depends on:** SITE-06, SITE-20
|
||||
|
||||
**Change**
|
||||
|
||||
- Scheduling core: task dependencies (predecessor/successor), milestones, baseline vs actual tracking with slippage alerts, percentage complete, daily-entry ↔ task linking (entries reference `TaskId`; dangling links rejected at command time).
|
||||
- Views by reuse, not reimplementation: extract `crates/apps/nigig-build/src/construction_frame/pages/workspace/project_management/` (`GanttTask`/`TaskType`, `GanttRenderer`, `GanttHistory`, `persistence`, scheduling `logic`) into its own crate (proposed `nigig-gantt`) with the pure scheduling core separated from Makepad views; extend with dependency/critical-path computation and baseline capture. `nigig-site` depends on that crate and implements only site-specific wiring. Gantt, calendar and board views with status/owner/department/consultant filters.
|
||||
- Inspection checklists (pass/fail/NA with notes + evidence photos), NCRs, inspection request forms and approval certificates as document hooks (SITE-30); AI tailored task breakdown from project type + scope description (SITE-13 pipeline, advisory only).
|
||||
- Registers (all site-scoped, validated, revisioned): snag/defect list (location, photo, assignee, due date; closure requires closure photo + sign-off; stats feed monthly), RFI log (response tracking, due dates, delay-risk highlighting), variation register (justification, cost/time impact, approval trail, affected-task links).
|
||||
|
||||
**Tests / exit**
|
||||
|
||||
- Dependency-cycle rejection, critical-path golden schedules, baseline/slippage fixtures, checklist evidence requirements, snag-closure photo enforcement, RFI overdue escalation, variation impact arithmetic (fixed-decimal), extracted-crate independence (site builds without `nigig-build`).
|
||||
|
||||
**Rollback:** list/board views only; never show uncomputed dates as a schedule.
|
||||
|
||||
### SITE-26 — Procurement depth: LPO workflow, deliveries, budgets, payments
|
||||
|
||||
**Priority:** P1 (scope Phase 5: FR-5.1–5.6)
|
||||
**Effort:** 10–14 person-days
|
||||
**Depends on:** SITE-06, SITE-14, SITE-25
|
||||
|
||||
**Change**
|
||||
|
||||
- Required-vs-delivered auto-computation from daily-report deliveries against the material schedule; remaining-to-procure surfaced per task.
|
||||
- Supplier rating (price, timeliness, quality) from history with price comparison per material; ratings are computed, never manually inflated without an audit event.
|
||||
- Requisition → numbered LPO (PDF) workflow with approval chain (Supervisor → Procurement → Admin), drawn/typed signatures, status `Requested → Approved → Ordered → Delivered → Closed`.
|
||||
- Delivery capture: photo + OCR of delivery notes (SITE-14 engine), quantity verification against the LPO, shortfall/damage flags; verified receipts update schedule and budget automatically.
|
||||
- Budget vs actual per material and per task with variance highlights for the monthly report; BOQ linkage tying material requirements to SITE-25 tasks so procurement timing follows the schedule; on-site inventory levels with shortage alerts.
|
||||
- Optional payment records with receipt images and mobile-money reference capture (e.g. M-Pesa). Full accounting stays out of scope (API export instead).
|
||||
|
||||
**Tests / exit**
|
||||
|
||||
- LPO numbering uniqueness, chain-of-approval enforcement, delivery shortfall arithmetic, budget variance fixtures, BOQ explosion against schedule changes, payment-reference format validation, OCR-misread quarantine (unverified quantities never update the schedule).
|
||||
|
||||
**Rollback:** schedule + directory only; never mark ordered/delivered without evidence.
|
||||
|
||||
### SITE-27 — Meetings depth: RSVP, consent-gated audio, actions, cross-links
|
||||
|
||||
**Priority:** P1 privacy (scope Phase 6: FR-6.1, FR-6.3–6.9)
|
||||
**Effort:** 10–16 person-days plus STT evaluation
|
||||
**Depends on:** SITE-05, SITE-13, SITE-20
|
||||
|
||||
**Change**
|
||||
|
||||
- Scheduling: RSVP per invitee, agenda templates, pre-read attachments (monthly report, programme update), attendance register (auto or manual) attached to minutes.
|
||||
- Recording-consent gate: audio capture activates only after explicit recorded consent from attendees (jurisdictional notice); without it the meeting is minutes-manual-only. Audio never leaves the device without a second explicit processing consent.
|
||||
- Speech-to-text with speaker identification (English and Swahili) via an evaluated explicit-consent cloud STT (scope2 build order keeps meetings last for this reason — on-device STT is not viable in the Makepad ecosystem today); transcript export alongside minutes; AI minutes draft (discussions → decisions → actions with owners and deadlines) through the SITE-13 pipeline, always advisory.
|
||||
- Action-item tracker: tracked items with due-date reminders, status, automatic carry-forward of unresolved items; cross-links from minutes to reports, tasks, RFIs, snags and variations discussed.
|
||||
|
||||
**Tests / exit**
|
||||
|
||||
- No-consent-no-capture (microphone never opens), consent audit trail, RSVP/quorum rules, action carry-forward chains, cross-link dangling rejection, transcript redaction of non-consented segments, STT provider data-retention contract on file.
|
||||
|
||||
**Rollback:** manual minutes only with no recording claim; never transcribe without consent.
|
||||
|
||||
### SITE-28 — Safety, Health & Environment (HSE)
|
||||
|
||||
**Priority:** P1 (scope Phase 7: FR-7.1–7.5; scope2 safety/incident addendum)
|
||||
**Effort:** 8–12 person-days
|
||||
**Depends on:** SITE-04, SITE-05, SITE-23
|
||||
|
||||
**Change**
|
||||
|
||||
- New aggregates (site-scoped, validated, audited): `IncidentRecord` (incidents and near misses with photos, severity, persons involved, immediate action; escalation to supervisor + HSE officer), `ToolboxTalk` (topic, date, attendees pulled from the worker register), `SafetyInspection` (scheduled PPE/scaffolding/housekeeping/electrical checklists with corrective actions, assignees, closure evidence).
|
||||
- Per-site emergency information (contacts, assembly points, procedures) pinned for offline availability; synced as read-mostly reference data with version stamps.
|
||||
- HSE statistics engine: days since last incident, toolbox-talk counts, open corrective actions, PPE compliance roll-up (from SITE-23 sign-in checklists) — consumed by monthly reports (SITE-21) and dashboards (SITE-29) through query interfaces, never by copying.
|
||||
|
||||
**Tests / exit**
|
||||
|
||||
- Severity-escalation matrix, corrective-action closure-evidence enforcement, offline emergency-info availability, statistics golden fixtures (incident-free streaks, reopen handling).
|
||||
|
||||
**Rollback:** incident log manual-only; never auto-close corrective actions.
|
||||
|
||||
### SITE-29 — Dashboards, analytics, charts and client digest
|
||||
|
||||
**Priority:** P1/P2 (scope Phase 8: FR-8.1–8.5; FR-1.7 charts and PowerPoint)
|
||||
**Effort:** 8–12 person-days
|
||||
**Depends on:** SITE-21, SITE-25, SITE-26, SITE-28
|
||||
|
||||
**Change**
|
||||
|
||||
- Site dashboard: programme progress %, workers on site today, report status (submitted/missing), open snags/RFIs, delays this month, spend vs budget — all from the query interfaces of their owning tranches.
|
||||
- Portfolio view for the Overall Supervisor: all sites side by side with drill-down and traffic-light health; trends (manpower over time, task burn-down, materials consumed, report timeliness).
|
||||
- Monthly-pack charts (progress trend, manpower trend, incident count) embedded in the PDF; PowerPoint export for the meeting presentation; any dashboard/chart exportable as PDF/image.
|
||||
- Client digest: automated weekly/monthly progress digest (summary + photo highlights) shared read-only; optional lightweight web-portal access (portal timing is open question Q5 — digest-by-share ships first, portal follows the server decision in SITE-31).
|
||||
|
||||
**Tests / exit**
|
||||
|
||||
- Dashboard golden fixtures (missing-data states render honestly, never zero-filled as real), chart data-point audits against source aggregates, digest scope test (client sees shared-only), export fidelity for charts.
|
||||
|
||||
**Rollback:** no dashboard; monthly pack without charts rather than with wrong charts.
|
||||
|
||||
### SITE-30 — Document and drawing control
|
||||
|
||||
**Priority:** P1 (scope Phase 9: FR-9.1–9.4; scope2 drawing-revision addendum)
|
||||
**Effort:** 8–12 person-days
|
||||
**Depends on:** SITE-05, SITE-07
|
||||
|
||||
**Change**
|
||||
|
||||
- Permissioned central `DocumentLibrary`: drawings, contracts, approvals, permits, insurance certificates, survey reports; upload from camera or files; in-app PDF/image viewer; search by name, type, discipline.
|
||||
- Drawing revision control: revision register per drawing (Rev A/B/C…), supersede/withdraw with automatic team notification; offline access to current revisions on device; task-level latest-approved-revision tracking (SITE-25 hook) so crews never build from superseded drawings.
|
||||
- Transmittals: who received which revision and when, with read receipts (SITE-24 hook where chat exists).
|
||||
|
||||
**Tests / exit**
|
||||
|
||||
- Supersede-notification journeys, offline-current-revision guarantee, withdrawn-revision refusal at task level, transmittal completeness audit, viewer fuzz over malformed PDFs/images (panic-free, budgeted).
|
||||
|
||||
**Rollback:** flat file list with no revision claims; never serve a withdrawn revision as current.
|
||||
|
||||
### SITE-31 — Integrations, API, portability and the server decision
|
||||
|
||||
**Priority:** P1/P2 (scope Phase 10: FR-10.1–10.5; scope §5 backend; scope §6 sync)
|
||||
**Effort:** 10–16 person-days plus backend build
|
||||
**Depends on:** SITE-12 protocol work, SITE-17, SITE-20
|
||||
|
||||
**Change**
|
||||
|
||||
- Server decision (recorded in an ADR): evaluate extending the existing server at `/Users/aok/Projects/rustdev/CratesCode/nimanyatta/src` (auth, db, broadcast, client exist) into the scope §5 backend — Rust API service + PostgreSQL + S3-compatible storage — covering delta sync with resumable media upload, server arbitration for shared-document conflicts (flagged for the Report Master), server-side approval/signature/permission enforcement, data-residency option per organisation, and organisation-level backup/export archives.
|
||||
- Architectural divergence resolved explicitly: scope §6 requires server-side authority for approvals, signatures, locking and roles; the SITE-08/09 E2EE group protocol remains the only permitted path for payloads the server must not read, and only after its independent review. The ADR states per-data-class authority (server-arbitrated vs end-to-end) so the two designs compose instead of contradicting.
|
||||
- Integrations: two-way calendar sync (meetings + reminders), email/WhatsApp share targets for reports/minutes/POs, map view of site registry, weather-service wiring (SITE-22 provider behind the same consent/timeout contract).
|
||||
- REST API + webhooks for accounting/ERP integration (timesheet and PO push) with the SITE-20 capability matrix enforced on every call.
|
||||
|
||||
**Tests / exit**
|
||||
|
||||
- Arbitration golden conflicts (both orders converge, loser flagged), resumable-upload interruption suite, webhook delivery/retry/idempotency, API authorization matrix (server-side, independent of client checks), residency configuration test, backup-restore drill.
|
||||
|
||||
**Rollback:** local encrypted operation journal remains intact (SITE-10); no sync claims without the server.
|
||||
|
||||
### SITE-32 — Product NFRs, scale proof and feature-release evidence
|
||||
|
||||
**Priority:** P1 release gate for the feature-rich app
|
||||
**Effort:** 6–10 person-days plus device lab
|
||||
**Depends on:** every enabled-feature tranche (SITE-20–31)
|
||||
|
||||
**Change**
|
||||
|
||||
- Latency budgets (scope §10, added to §7): cold start < 3 s on mid-range Android; photo capture → entry saved < 1 s; monthly pack generation < 60 s; draw/event and site-switch budgets from §7 unchanged.
|
||||
- Media pipeline target: client-side compression to ≤ 1 MB per photo for sync; originals retained on device until synced with storage display and cleanup tools; evidence watermarks (timestamp + GPS + site name) applied at capture and verifiable thereafter.
|
||||
- Localisation: Swahili + English UI with complete string coverage gates (no hardcoded product strings); accessibility: dynamic text sizing, contrast modes, large touch targets (glove operation), sunlight-glare contrast checks.
|
||||
- Scale fixtures: 100+ sites and 1,000+ workers per organisation without redesign (virtualized lists per SITE-18, paginated queries per SITE-03, repository budgets per §7).
|
||||
- Compatibility matrix: Android 9+, iOS 15+, Windows 10+, macOS 12+, modern browsers; battery discipline for GPS/camera/batched sync.
|
||||
- Feature-release evidence per R1–R5 (§13 roadmap): capability matrix, FR traceability run, performance table, known limitations — published with commit, toolchain, server schema and device list.
|
||||
|
||||
**Tests / exit**
|
||||
|
||||
- Startup/capture/generation timing on reference devices (recorded, not asserted on CI runners), photo-size distribution audit, i18n completeness lint, accessibility traversal, 100-site/1,000-worker soak within §7 memory budgets.
|
||||
|
||||
**Rollback:** features missing their NFR evidence stay out of the release notes; never claim performance without device data.
|
||||
|
||||
---
|
||||
|
||||
## 9. Detailed legacy migration and recovery policy
|
||||
|
|
@ -764,12 +1031,22 @@ Release CI launches the pinned server itself; `NIMANYATTA_E2E_URL` cannot be an
|
|||
- [ ] Domain transitions, references, finite/range rules, roles, revisions, and audit are centrally enforced.
|
||||
- [ ] Worker PII and ID-image lifecycle policy is approved and tested.
|
||||
- [ ] All enabled media uses managed assets and passes encoded/pixel/frame/output/peak-memory limits.
|
||||
- [ ] OCR is real and measured or labelled capture-only.
|
||||
- [ ] OCR is the scanner-owned on-device engine (measured, confidence-gated, confirmation-required, no PDF dependency) or the flow is labelled capture-only.
|
||||
- [ ] PDF/DOCX are Unicode-safe, embed authorized photos, return errors, and pass independent consumers.
|
||||
- [ ] AI is disabled or passes provider/privacy/correlation/acceptance gates.
|
||||
- [ ] Reminders pass time-zone/durability/idempotency/reconciliation gates or remain in-app only.
|
||||
- [ ] Site-owned CI has a definitive green result with real runtime assertions.
|
||||
|
||||
### Feature-rich product release gates (R1–R5, §13)
|
||||
|
||||
- [ ] **R1 — Verified reporting core:** SITE-20 (org/RBAC/registry/settings) + SITE-21 (report richness, signatures, versioning, monthly packs) on top of the local/offline gates above. Approval signatures bind identity/device/timestamp/document hash; approved reports lock and version.
|
||||
- [ ] **R2 — Field operations complete:** SITE-22 (site-diary data) + SITE-23 (consent, attendance, QR, register, payroll CSV, auto-purge) + SITE-24 chat only if its transport gates pass, otherwise chat stays local-cache-only with no send claims.
|
||||
- [ ] **R3 — Planning and materials:** SITE-25 (dependencies, Gantt via the extracted crate, checklists, snags, RFIs, variations) + SITE-26 (LPO workflow, delivery verification, budgets). No ordered/delivered state without evidence.
|
||||
- [ ] **R4 — Governance and safety:** SITE-27 (RSVP, consent-gated audio, action tracker) + SITE-28 (HSE records and stats). No recording or transcription without recorded consent.
|
||||
- [ ] **R5 — Oversight and ecosystem:** SITE-29 (dashboards, charts, digest) + SITE-30 (document/drawing control, no withdrawn revision served as current) + SITE-31 (server ADR, arbitration, API auth, backup drill).
|
||||
- [ ] **NFR evidence (SITE-32):** startup/capture/generation timings on reference devices, photo-size audit, i18n completeness (EN/SW), accessibility traversal, 100-site/1,000-worker soak. Features missing NFR evidence stay out of release notes.
|
||||
- [ ] **No biometric verification ships** (photo/QR sufficiency per scope Q7); full payroll, accounting, BIM/CAD editing, telematics and offline client-side AI stay out of scope with export/API paths instead.
|
||||
|
||||
### Additional sync release gates
|
||||
|
||||
- [ ] Shared protocol and threat-model ADR are approved.
|
||||
|
|
@ -814,6 +1091,62 @@ All enabled paths + SITE-00 → SITE-18 → SITE-19
|
|||
```
|
||||
|
||||
- Build/Traffic/CAD work does not justify delaying Site's emergency containment.
|
||||
- OCR is owned by `nigig_doc_scanner` (`ocr` feature); `nigig-site` consumes `nigig_doc_scanner/ocr` with `default-features = false` and never depends on `pdf-*` for text recognition. The PDF `ocr` seam stays engine-free and feature-gated.
|
||||
- Future integration with `nigig-build` requires an authenticated mapping between Build project ID and Site ID; never sync one app's global store into the other.
|
||||
- Shared chat rooms may coexist with sync, but sync payloads must use dedicated opaque protocol events and keys, not visible text bodies.
|
||||
- A safe release may omit sync, AI, OCR, video, DOCX, or OS reminders. It may not ship unsafe substitutes for them.
|
||||
|
||||
```text
|
||||
SITE-20 org/RBAC/registry ─┬→ SITE-21 reports ─┬→ SITE-29 dashboards ─┐
|
||||
│ └→ SITE-30 documents ─┤
|
||||
├→ SITE-22 site-diary ────────────────────┤
|
||||
├→ SITE-23 workforce ─┬→ SITE-28 HSE ──────┤→ SITE-32 NFRs
|
||||
├→ SITE-24 chat (needs SITE-08..12+31) ───┤ + release
|
||||
├→ SITE-25 programme ─┬→ SITE-26 procure ──┤ evidence
|
||||
└→ SITE-27 meetings ─┘ ┘
|
||||
SITE-31 server/ADR underpins SITE-24 + multi-device sync + API + backup.
|
||||
```
|
||||
|
||||
### Feature release roadmap (scope §16: R1–R5)
|
||||
|
||||
| Release | Tranches | Value |
|
||||
|---|---|---|
|
||||
| **R1 — MVP** | SITE-00–07 + SITE-20 + SITE-21 (FR-1.1–1.8, 1.14 core) | Verified daily reports, multi-site compilation, monthly packs |
|
||||
| **R2** | SITE-22 + SITE-23 + SITE-24 (transport permitting) | Complete daily field operations |
|
||||
| **R3** | SITE-25 + SITE-26 | Planning and materials control |
|
||||
| **R4** | SITE-27 + SITE-28 | Governance and safety |
|
||||
| **R5** | SITE-29 + SITE-30 + SITE-31 | Oversight, control and ecosystem |
|
||||
|
||||
R1 is the critical path; R2–R5 proceed in parallel streams once SITE-20 exists. SITE-32 gates every release's NFR claims.
|
||||
|
||||
### FR → tranche traceability (scope `construction-site-app-scope.md` + `construction-site-app-scope2.md`)
|
||||
|
||||
| Scope requirement | Tranche |
|
||||
|---|---|
|
||||
| FR-0.1 organisation & invites, FR-0.2 RBAC matrix, FR-0.3 site registry, FR-0.8 settings/signatures/language | SITE-20 |
|
||||
| FR-1.1 media (+annotation, voice note, watermark), FR-1.2 categories/tags/dictation/captioning, FR-1.3 numbering/cover/TOC/templates | SITE-21 (+ SITE-07 assets, SITE-15 media, SITE-27 STT) |
|
||||
| FR-1.4 entry status/comments/signatures/lock+version, FR-1.5 chase, FR-1.6 escalation, FR-1.14 lifecycle/search/archive/batch | SITE-21 (+ SITE-17 triggers) |
|
||||
| FR-1.7 monthly accumulation (+AI summary), charts, PowerPoint | SITE-21 (accumulation) + SITE-29 (charts/PPT) |
|
||||
| FR-1.9 weather, FR-1.10 manpower/plant, FR-1.11 deliveries, FR-1.12 delays, FR-1.13 visitors | SITE-22 |
|
||||
| FR-2.1 consent, FR-2.2 Excel export, FR-2.3 attendance/payroll CSV, FR-2.4 QR, FR-2.5 register/blocklist, FR-2.6 auto-purge, registration photo, PPE checklist, labour cost | SITE-23 |
|
||||
| FR-3.1–3.6 chat, channels/DMs, receipts, bots, broadcast, moderation | SITE-24 (needs transport + server) |
|
||||
| FR-4.1 AI breakdown/library, FR-4.3 deps/milestones/baseline, FR-4.4 checklists/NCRs, FR-4.5 statuses/%/links | SITE-25 (+ SITE-13 AI) |
|
||||
| FR-4.6 Gantt/calendar/board via `nigig-build/.../project_management` extraction | SITE-25 (new `nigig-gantt`-style crate) |
|
||||
| FR-4.7 snags, FR-4.8 RFIs, FR-4.9 variations, drawing-register hooks | SITE-25 (+ SITE-30) |
|
||||
| FR-5.1 req-vs-delivered, FR-5.2 ratings/compare, FR-5.3 LPO workflow, FR-5.4 delivery verify, FR-5.5 budgets, FR-5.6 payments/M-Pesa, BOQ, inventory | SITE-26 |
|
||||
| FR-6.1 RSVP, FR-6.3 speaker ID, FR-6.4 action owners, FR-6.6 agendas/pre-reads, FR-6.7 tracker, FR-6.8 cross-links, FR-6.9 STT/transcript, attendance register, recording consent | SITE-27 |
|
||||
| FR-7.1–7.5 HSE + scope2 incident addendum | SITE-28 |
|
||||
| FR-8.1–8.5 dashboards/digest/portal | SITE-29 |
|
||||
| FR-9.1–9.4 documents/drawings + scope2 revision control | SITE-30 |
|
||||
| FR-10.1 calendar, FR-10.2 share, FR-10.3 maps/weather, FR-10.4 API/webhooks, FR-10.5 backup | SITE-31 |
|
||||
| Scope2 §0 foundations (offline, RBAC, audit, switcher, signatures, CSV/Excel, i18n, a11y) | SITE-00–05, SITE-16, SITE-18, SITE-20, SITE-32 |
|
||||
| AI summary table (refine, STT, captioning, templates, minutes, anomaly nudges) | SITE-13 + SITE-21 + SITE-25 + SITE-27 |
|
||||
| Notifications table (escalation, chase, inspection-due, approvals, snag/RFI, meeting, incident, LPO) | SITE-17 extended per tranche |
|
||||
|
||||
### Out of scope v1 (scope §17, binding on this plan)
|
||||
|
||||
Full payroll processing and statutory deductions (payroll-ready export only); accounting/bookkeeping (API integration); BIM/CAD authoring or editing (documents viewed, not edited); equipment telematics/IoT; client-side internet-free AI (cloud AI with graceful offline degradation); biometric worker verification.
|
||||
|
||||
### Open decisions to record (scope §18)
|
||||
|
||||
Retention period for worker ID data (needs legal/HR guidance — blocks SITE-23 purge defaults); Word parity vs PDF-first on day one; client portal in R1 vs R5 (default: R5, digest-by-share first); AI provider and budget; hosting model and data-residency offering (blocks SITE-31 ADR).
|
||||
|
|
|
|||
737
crates/apps/nigig-site/src/aggregates.rs
Normal file
737
crates/apps/nigig-site/src/aggregates.rs
Normal file
|
|
@ -0,0 +1,737 @@
|
|||
//! SITE-03 — site-scoped versioned aggregates with explicit context.
|
||||
//!
|
||||
//! The legacy `SiteStore` is one monolithic blob: every mutation clones and
|
||||
//! re-serializes all sites and their PII. This module splits a store into
|
||||
//! typed, versioned, per-site aggregates plus device-local preferences:
|
||||
//!
|
||||
//! - Every site-scoped query, mutation payload and export builds from ONE
|
||||
//! aggregate, whose plaintext provably contains no other site's records.
|
||||
//! - `PreferencesLocal` (selection, chat presentation cache, transient UI)
|
||||
//! is typed separately and EXCLUDED from the replicated set — device state
|
||||
//! never replicates as domain state.
|
||||
//! - The legacy supplier directory has no `site_id` and stays a shared
|
||||
//! aggregate under explicit quarantine until assigned per site.
|
||||
//! - Per-aggregate revisions with compare-and-swap back the command layer's
|
||||
//! stale-write rejection (SITE-04); the repository writer keeps the single
|
||||
//! durable revision counter.
|
||||
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use zeroize::Zeroizing;
|
||||
|
||||
use crate::ids::SiteId;
|
||||
use crate::site_context::{ContextError, Page, SiteContext};
|
||||
use crate::store::SiteStore;
|
||||
|
||||
/// Aggregate format version. Unknown versions are rejected, never coerced.
|
||||
pub const AGGREGATE_FORMAT_VERSION: u32 = 1;
|
||||
/// §7 per-aggregate plaintext ceiling, enforced before sealing.
|
||||
pub const MAX_AGGREGATE_BYTES: usize = 16 * 1024 * 1024;
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
|
||||
pub enum AggregateKind {
|
||||
ProfileSites,
|
||||
Reports,
|
||||
Workers,
|
||||
Tasks,
|
||||
Procurement,
|
||||
SuppliersShared,
|
||||
Meetings,
|
||||
Reminders,
|
||||
Directory,
|
||||
PreferencesLocal,
|
||||
}
|
||||
|
||||
impl AggregateKind {
|
||||
/// `false` only for device-local state that must never replicate.
|
||||
pub const fn is_replicated(self) -> bool {
|
||||
!matches!(self, Self::PreferencesLocal)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub enum AggregateError {
|
||||
UnknownKind,
|
||||
UnsupportedVersion,
|
||||
DuplicateAggregate,
|
||||
CrossSiteRecord,
|
||||
TooLarge(&'static str),
|
||||
StaleRevision,
|
||||
UnknownSite,
|
||||
Quarantined(&'static str),
|
||||
}
|
||||
|
||||
impl std::fmt::Display for AggregateError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
Self::UnknownKind => write!(f, "SITE-AGG-KIND"),
|
||||
Self::UnsupportedVersion => write!(f, "SITE-AGG-VERSION"),
|
||||
Self::DuplicateAggregate => write!(f, "SITE-AGG-DUPLICATE"),
|
||||
Self::CrossSiteRecord => write!(f, "SITE-AGG-CROSS-SITE"),
|
||||
Self::TooLarge(s) => write!(f, "SITE-AGG-LIMIT:{s}"),
|
||||
Self::StaleRevision => write!(f, "SITE-AGG-STALE"),
|
||||
Self::UnknownSite => write!(f, "SITE-AGG-UNKNOWN-SITE"),
|
||||
Self::Quarantined(s) => write!(f, "SITE-AGG-QUARANTINE:{s}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for AggregateError {}
|
||||
|
||||
/// One versioned, site-bound plaintext unit. `payload` is canonical JSON of
|
||||
/// ONLY this aggregate's records — proven by sentinel tests, never by trust.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct SiteAggregate {
|
||||
pub kind: AggregateKind,
|
||||
pub site_id: Option<SiteId>,
|
||||
pub version: u32,
|
||||
pub revision: u64,
|
||||
pub payload: Zeroizing<Vec<u8>>,
|
||||
}
|
||||
|
||||
impl SiteAggregate {
|
||||
pub fn contains(&self, sentinel: &str) -> bool {
|
||||
String::from_utf8_lossy(&self.payload).contains(sentinel)
|
||||
}
|
||||
}
|
||||
|
||||
/// Ambiguous records that need explicit human assignment before they may
|
||||
/// join a site aggregate. Nothing quarantined is silently dropped: split
|
||||
/// round-trips it, queries exclude it loudly.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Eq)]
|
||||
pub struct QuarantineReport {
|
||||
pub unassigned_suppliers: Vec<String>,
|
||||
pub unsited_reminders: Vec<String>,
|
||||
pub dangling_records: Vec<String>,
|
||||
}
|
||||
|
||||
impl QuarantineReport {
|
||||
pub fn is_clear(&self) -> bool {
|
||||
self.unassigned_suppliers.is_empty()
|
||||
&& self.unsited_reminders.is_empty()
|
||||
&& self.dangling_records.is_empty()
|
||||
}
|
||||
}
|
||||
|
||||
fn seal(
|
||||
kind: AggregateKind,
|
||||
site_id: Option<SiteId>,
|
||||
revision: u64,
|
||||
value: &impl Serialize,
|
||||
) -> Result<SiteAggregate, AggregateError> {
|
||||
let bytes = serde_json::to_vec(value).map_err(|_| AggregateError::TooLarge("encode"))?;
|
||||
if bytes.len() > MAX_AGGREGATE_BYTES {
|
||||
return Err(AggregateError::TooLarge("aggregate"));
|
||||
}
|
||||
Ok(SiteAggregate {
|
||||
kind,
|
||||
site_id,
|
||||
version: AGGREGATE_FORMAT_VERSION,
|
||||
revision,
|
||||
payload: Zeroizing::new(bytes),
|
||||
})
|
||||
}
|
||||
|
||||
/// Split a legacy store into per-site aggregates plus shared/local ones.
|
||||
/// `revision` stamps every aggregate with the durable revision the split was
|
||||
/// read at; mutations bump only affected aggregates via the ledger.
|
||||
pub fn split_store(
|
||||
store: &SiteStore,
|
||||
revision: u64,
|
||||
) -> Result<(Vec<SiteAggregate>, QuarantineReport), AggregateError> {
|
||||
let known: BTreeSet<&str> = store.sites.iter().map(|s| s.id.as_str()).collect();
|
||||
let mut out = Vec::new();
|
||||
let mut quarantine = QuarantineReport::default();
|
||||
|
||||
out.push(seal(
|
||||
AggregateKind::ProfileSites,
|
||||
None,
|
||||
revision,
|
||||
&store.sites,
|
||||
)?);
|
||||
|
||||
for site in &store.sites {
|
||||
let id = SiteId::parse(&site.id).map_err(|_| AggregateError::UnknownSite)?;
|
||||
let reports: Vec<_> = store
|
||||
.reports
|
||||
.iter()
|
||||
.filter(|r| r.site_id == site.id)
|
||||
.collect();
|
||||
let workers: Vec<_> = store
|
||||
.workers
|
||||
.iter()
|
||||
.filter(|t| t.site_id == site.id)
|
||||
.collect();
|
||||
let tasks: Vec<_> = store
|
||||
.tasks
|
||||
.iter()
|
||||
.filter(|t| t.site_id == site.id)
|
||||
.collect();
|
||||
let procurement: Vec<_> = store
|
||||
.procurement
|
||||
.iter()
|
||||
.filter(|p| p.site_id == site.id)
|
||||
.collect();
|
||||
let meetings: Vec<_> = store
|
||||
.meetings
|
||||
.iter()
|
||||
.filter(|m| m.site_id == site.id)
|
||||
.collect();
|
||||
let reminders: Vec<_> = store
|
||||
.reminders
|
||||
.iter()
|
||||
.filter(|r| r.site_id.as_deref() == Some(site.id.as_str()))
|
||||
.collect();
|
||||
let directory = store.directories.iter().find(|d| d.site_id == site.id);
|
||||
out.push(seal(
|
||||
AggregateKind::Reports,
|
||||
Some(id.clone()),
|
||||
revision,
|
||||
&reports,
|
||||
)?);
|
||||
out.push(seal(
|
||||
AggregateKind::Workers,
|
||||
Some(id.clone()),
|
||||
revision,
|
||||
&workers,
|
||||
)?);
|
||||
out.push(seal(
|
||||
AggregateKind::Tasks,
|
||||
Some(id.clone()),
|
||||
revision,
|
||||
&tasks,
|
||||
)?);
|
||||
out.push(seal(
|
||||
AggregateKind::Procurement,
|
||||
Some(id.clone()),
|
||||
revision,
|
||||
&procurement,
|
||||
)?);
|
||||
out.push(seal(
|
||||
AggregateKind::Meetings,
|
||||
Some(id.clone()),
|
||||
revision,
|
||||
&meetings,
|
||||
)?);
|
||||
out.push(seal(
|
||||
AggregateKind::Reminders,
|
||||
Some(id.clone()),
|
||||
revision,
|
||||
&reminders,
|
||||
)?);
|
||||
out.push(seal(
|
||||
AggregateKind::Directory,
|
||||
Some(id),
|
||||
revision,
|
||||
&directory,
|
||||
)?);
|
||||
}
|
||||
|
||||
// Shared supplier directory: no site_id anywhere, so it stays shared and
|
||||
// every supplier id is reported for explicit assignment (SITE-03 exit).
|
||||
out.push(seal(
|
||||
AggregateKind::SuppliersShared,
|
||||
None,
|
||||
revision,
|
||||
&store.suppliers,
|
||||
)?);
|
||||
quarantine.unassigned_suppliers = store
|
||||
.suppliers
|
||||
.suppliers
|
||||
.iter()
|
||||
.map(|s| s.id.clone())
|
||||
.collect();
|
||||
|
||||
// Dangling references: records naming a site absent from the profile.
|
||||
let mut dangling: BTreeSet<String> = BTreeSet::new();
|
||||
for id in store
|
||||
.reports
|
||||
.iter()
|
||||
.map(|r| r.site_id.as_str())
|
||||
.chain(store.workers.iter().map(|t| t.site_id.as_str()))
|
||||
.chain(store.tasks.iter().map(|t| t.site_id.as_str()))
|
||||
.chain(store.procurement.iter().map(|p| p.site_id.as_str()))
|
||||
.chain(store.meetings.iter().map(|m| m.site_id.as_str()))
|
||||
.chain(store.directories.iter().map(|d| d.site_id.as_str()))
|
||||
{
|
||||
if !known.contains(id) {
|
||||
dangling.insert(id.to_string());
|
||||
}
|
||||
}
|
||||
for reminder in &store.reminders {
|
||||
match reminder.site_id.as_deref() {
|
||||
Some(id) if !known.contains(id) => {
|
||||
dangling.insert(id.to_string());
|
||||
}
|
||||
None => quarantine.unsited_reminders.push(reminder.id.clone()),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
quarantine.dangling_records = dangling.into_iter().collect();
|
||||
|
||||
// Device-local preferences: selection + chat presentation cache. Typed
|
||||
// apart and excluded from `replicated_aggregates` by construction.
|
||||
out.push(seal(
|
||||
AggregateKind::PreferencesLocal,
|
||||
None,
|
||||
revision,
|
||||
&(&store.selected_site_id, &store.chat_threads),
|
||||
)?);
|
||||
|
||||
Ok((out, quarantine))
|
||||
}
|
||||
|
||||
/// Aggregates that may replicate. Device-local preferences are excluded by
|
||||
/// type, not by caller discipline.
|
||||
pub fn replicated_aggregates(all: &[SiteAggregate]) -> Vec<&SiteAggregate> {
|
||||
all.iter().filter(|a| a.kind.is_replicated()).collect()
|
||||
}
|
||||
|
||||
/// Reassemble a store from aggregates with strict validation. Unknown
|
||||
/// kinds/versions, duplicate scopes, and records bound to a different site
|
||||
/// than their envelope are rejected; quarantined and dangling records
|
||||
/// round-trip through their named aggregates for explicit resolution.
|
||||
pub fn assemble_store(aggregates: &[SiteAggregate]) -> Result<SiteStore, AggregateError> {
|
||||
let mut seen: BTreeSet<(AggregateKind, Option<String>)> = BTreeSet::new();
|
||||
for aggregate in aggregates {
|
||||
if aggregate.version != AGGREGATE_FORMAT_VERSION {
|
||||
return Err(AggregateError::UnsupportedVersion);
|
||||
}
|
||||
let key = (
|
||||
aggregate.kind,
|
||||
aggregate.site_id.as_ref().map(|id| id.as_str().to_string()),
|
||||
);
|
||||
if !seen.insert(key) {
|
||||
return Err(AggregateError::DuplicateAggregate);
|
||||
}
|
||||
}
|
||||
let find =
|
||||
|kind: AggregateKind, site: Option<&str>| -> Result<&SiteAggregate, AggregateError> {
|
||||
aggregates
|
||||
.iter()
|
||||
.find(|a| a.kind == kind && a.site_id.as_ref().map(|id| id.as_str()) == site)
|
||||
.ok_or(AggregateError::UnknownKind)
|
||||
};
|
||||
let sites: Vec<crate::domain::site::Site> =
|
||||
serde_json::from_slice(&find(AggregateKind::ProfileSites, None)?.payload)
|
||||
.map_err(|_| AggregateError::TooLarge("decode"))?;
|
||||
let mut all_reports = Vec::new();
|
||||
let mut all_workers = Vec::new();
|
||||
let mut all_tasks = Vec::new();
|
||||
let mut all_procurement = Vec::new();
|
||||
let mut all_meetings = Vec::new();
|
||||
let mut all_reminders = Vec::new();
|
||||
let mut all_directories = Vec::new();
|
||||
|
||||
for site in &sites {
|
||||
let id = SiteId::parse(&site.id).map_err(|_| AggregateError::UnknownSite)?;
|
||||
let scope = id.as_str();
|
||||
let mut reports: Vec<crate::domain::daily_report::DailyReport> =
|
||||
serde_json::from_slice(&find(AggregateKind::Reports, Some(scope))?.payload)
|
||||
.map_err(|_| AggregateError::TooLarge("decode"))?;
|
||||
let mut workers: Vec<crate::domain::workers::WorkersDailyTable> =
|
||||
serde_json::from_slice(&find(AggregateKind::Workers, Some(scope))?.payload)
|
||||
.map_err(|_| AggregateError::TooLarge("decode"))?;
|
||||
let mut tasks: Vec<crate::domain::approvals::ConstructionTask> =
|
||||
serde_json::from_slice(&find(AggregateKind::Tasks, Some(scope))?.payload)
|
||||
.map_err(|_| AggregateError::TooLarge("decode"))?;
|
||||
let mut procurement: Vec<crate::domain::procurement::ProcurementSchedule> =
|
||||
serde_json::from_slice(&find(AggregateKind::Procurement, Some(scope))?.payload)
|
||||
.map_err(|_| AggregateError::TooLarge("decode"))?;
|
||||
let mut meetings: Vec<crate::domain::meetings::SiteMeeting> =
|
||||
serde_json::from_slice(&find(AggregateKind::Meetings, Some(scope))?.payload)
|
||||
.map_err(|_| AggregateError::TooLarge("decode"))?;
|
||||
let mut reminders: Vec<crate::domain::reminders::Reminder> =
|
||||
serde_json::from_slice(&find(AggregateKind::Reminders, Some(scope))?.payload)
|
||||
.map_err(|_| AggregateError::TooLarge("decode"))?;
|
||||
// Envelope binding: every record must name the envelope's site.
|
||||
for record_site in reports
|
||||
.iter()
|
||||
.map(|r| r.site_id.as_str())
|
||||
.chain(workers.iter().map(|t| t.site_id.as_str()))
|
||||
.chain(tasks.iter().map(|t| t.site_id.as_str()))
|
||||
.chain(procurement.iter().map(|p| p.site_id.as_str()))
|
||||
.chain(meetings.iter().map(|m| m.site_id.as_str()))
|
||||
{
|
||||
if record_site != scope {
|
||||
return Err(AggregateError::CrossSiteRecord);
|
||||
}
|
||||
}
|
||||
for reminder in &reminders {
|
||||
if reminder.site_id.as_deref() != Some(scope) {
|
||||
return Err(AggregateError::CrossSiteRecord);
|
||||
}
|
||||
}
|
||||
all_reports.append(&mut reports);
|
||||
all_workers.append(&mut workers);
|
||||
all_tasks.append(&mut tasks);
|
||||
all_procurement.append(&mut procurement);
|
||||
all_meetings.append(&mut meetings);
|
||||
all_reminders.append(&mut reminders);
|
||||
let directory: Option<crate::domain::meetings::ProjectDirectory> =
|
||||
serde_json::from_slice(&find(AggregateKind::Directory, Some(scope))?.payload)
|
||||
.map_err(|_| AggregateError::TooLarge("decode"))?;
|
||||
if let Some(directory) = directory {
|
||||
if directory.site_id != site.id {
|
||||
return Err(AggregateError::CrossSiteRecord);
|
||||
}
|
||||
all_directories.push(directory);
|
||||
}
|
||||
}
|
||||
let suppliers = serde_json::from_slice(&find(AggregateKind::SuppliersShared, None)?.payload)
|
||||
.map_err(|_| AggregateError::TooLarge("decode"))?;
|
||||
let (selection, threads): (Option<String>, BTreeMap<String, Vec<String>>) =
|
||||
serde_json::from_slice(&find(AggregateKind::PreferencesLocal, None)?.payload)
|
||||
.map_err(|_| AggregateError::TooLarge("decode"))?;
|
||||
Ok(SiteStore {
|
||||
sites,
|
||||
reports: all_reports,
|
||||
workers: all_workers,
|
||||
reminders: all_reminders,
|
||||
tasks: all_tasks,
|
||||
procurement: all_procurement,
|
||||
suppliers,
|
||||
meetings: all_meetings,
|
||||
selected_site_id: selection,
|
||||
chat_threads: threads,
|
||||
directories: all_directories,
|
||||
..SiteStore::default()
|
||||
})
|
||||
}
|
||||
|
||||
/// Every aggregate scope a store owns. Used to initialize and extend the
|
||||
/// ledger without serializing anything.
|
||||
pub fn scopes_for_store(store: &SiteStore) -> Vec<(AggregateKind, Option<SiteId>)> {
|
||||
let mut scopes = vec![
|
||||
(AggregateKind::ProfileSites, None),
|
||||
(AggregateKind::SuppliersShared, None),
|
||||
(AggregateKind::PreferencesLocal, None),
|
||||
];
|
||||
for site in &store.sites {
|
||||
let Ok(id) = SiteId::parse(&site.id) else {
|
||||
continue;
|
||||
};
|
||||
for kind in [
|
||||
AggregateKind::Reports,
|
||||
AggregateKind::Workers,
|
||||
AggregateKind::Tasks,
|
||||
AggregateKind::Procurement,
|
||||
AggregateKind::Meetings,
|
||||
AggregateKind::Reminders,
|
||||
AggregateKind::Directory,
|
||||
] {
|
||||
scopes.push((kind, Some(id.clone())));
|
||||
}
|
||||
}
|
||||
scopes
|
||||
}
|
||||
|
||||
/// Per-aggregate revision ledger with compare-and-swap. Initialized at the
|
||||
/// durable repository revision on open; mutations bump only affected scopes.
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct AggregateLedger {
|
||||
revisions: BTreeMap<(AggregateKind, Option<String>), u64>,
|
||||
}
|
||||
|
||||
impl AggregateLedger {
|
||||
pub fn init(revision: u64, aggregates: &[SiteAggregate]) -> Self {
|
||||
let mut ledger = Self::default();
|
||||
for aggregate in aggregates {
|
||||
ledger.revisions.insert(
|
||||
(
|
||||
aggregate.kind,
|
||||
aggregate.site_id.as_ref().map(|id| id.as_str().to_string()),
|
||||
),
|
||||
revision,
|
||||
);
|
||||
}
|
||||
ledger
|
||||
}
|
||||
|
||||
pub fn get(&self, kind: AggregateKind, site: Option<&SiteId>) -> u64 {
|
||||
self.revisions
|
||||
.get(&(kind, site.map(|id| id.as_str().to_string())))
|
||||
.copied()
|
||||
.unwrap_or(0)
|
||||
}
|
||||
|
||||
/// Atomic bump used after a validated mutation. Overflow is rejected,
|
||||
/// never wrapped.
|
||||
pub fn record_mutation(
|
||||
&mut self,
|
||||
kind: AggregateKind,
|
||||
site: Option<&SiteId>,
|
||||
) -> Result<u64, AggregateError> {
|
||||
let key = (kind, site.map(|id| id.as_str().to_string()));
|
||||
let current = self.revisions.get(&key).copied().unwrap_or(0);
|
||||
let next = current
|
||||
.checked_add(1)
|
||||
.ok_or(AggregateError::StaleRevision)?;
|
||||
self.revisions.insert(key, next);
|
||||
Ok(next)
|
||||
}
|
||||
|
||||
pub fn compare_and_swap(
|
||||
&mut self,
|
||||
kind: AggregateKind,
|
||||
site: Option<&SiteId>,
|
||||
expected: u64,
|
||||
) -> Result<u64, AggregateError> {
|
||||
if self.get(kind, site) != expected {
|
||||
return Err(AggregateError::StaleRevision);
|
||||
}
|
||||
self.record_mutation(kind, site)
|
||||
}
|
||||
|
||||
/// Initialize every scope a store owns at one revision without
|
||||
/// serializing anything. Used on open at the durable revision.
|
||||
pub fn init_scopes(revision: u64, scopes: &[(AggregateKind, Option<SiteId>)]) -> Self {
|
||||
let mut ledger = Self::default();
|
||||
for (kind, site) in scopes {
|
||||
ledger.revisions.insert(
|
||||
(*kind, site.as_ref().map(|id| id.as_str().to_string())),
|
||||
revision,
|
||||
);
|
||||
}
|
||||
ledger
|
||||
}
|
||||
|
||||
/// Register scopes that appeared later (e.g. a newly created site) at
|
||||
/// the current revision. Existing revisions are never reset.
|
||||
pub fn ensure_scopes(&mut self, revision: u64, scopes: &[(AggregateKind, Option<SiteId>)]) {
|
||||
for (kind, site) in scopes {
|
||||
self.revisions
|
||||
.entry((*kind, site.as_ref().map(|id| id.as_str().to_string())))
|
||||
.or_insert(revision);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Resolve an optional UI selection to an explicit context without any
|
||||
/// first-site fallback. Missing, empty, or stale selection is an error.
|
||||
pub fn require_site_context(
|
||||
selected: Option<&str>,
|
||||
known_sites: &[String],
|
||||
base_revision: u64,
|
||||
) -> Result<SiteContext, ContextError> {
|
||||
SiteContext::from_selection(selected, known_sites, base_revision)
|
||||
}
|
||||
|
||||
/// Paginated site-scoped query over one aggregate's records. Page bounds and
|
||||
/// overflow are enforced by [`Page`]; cross-site access is impossible because
|
||||
/// the input slice already comes from the caller's own aggregate.
|
||||
pub fn paginate_records<T: Clone>(records: &[T], page: Page) -> Vec<T> {
|
||||
page.slice(records)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::domain::approvals::ConstructionTask;
|
||||
use crate::domain::daily_report::DailyReport;
|
||||
use crate::domain::site::{Site, SiteNature};
|
||||
|
||||
fn two_site_store() -> (SiteStore, String, String) {
|
||||
let mut store = SiteStore::default();
|
||||
let mut a = Site::new("Alpha", SiteNature::Road, "x");
|
||||
a.id = "site-a".to_string();
|
||||
let mut b = Site::new("Beta", SiteNature::Road, "y");
|
||||
b.id = "site-b".to_string();
|
||||
store.sites.extend([a, b]);
|
||||
store.selected_site_id = Some("site-a".to_string());
|
||||
let mut rep = DailyReport::new(
|
||||
"site-a",
|
||||
chrono::NaiveDate::from_ymd_opt(2026, 9, 7).unwrap(),
|
||||
);
|
||||
rep.entries
|
||||
.push(crate::domain::daily_report::DailyTaskEntry::new(
|
||||
"Worker ID 12345678-A",
|
||||
crate::domain::daily_report::RichText::from_plain("poured slab A"),
|
||||
));
|
||||
store.reports.push(rep);
|
||||
let mut rep_b = DailyReport::new(
|
||||
"site-b",
|
||||
chrono::NaiveDate::from_ymd_opt(2026, 9, 7).unwrap(),
|
||||
);
|
||||
rep_b
|
||||
.entries
|
||||
.push(crate::domain::daily_report::DailyTaskEntry::new(
|
||||
"Worker ID 87654321-B",
|
||||
crate::domain::daily_report::RichText::from_plain("graded road B"),
|
||||
));
|
||||
store.reports.push(rep_b);
|
||||
store
|
||||
.tasks
|
||||
.push(ConstructionTask::new("site-b", "Secret-B-task"));
|
||||
store
|
||||
.chat_threads
|
||||
.insert("room-a".to_string(), vec!["hello-a".to_string()]);
|
||||
(store, "site-a".to_string(), "site-b".to_string())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn one_site_aggregate_excludes_other_sites_pii() {
|
||||
let (store, _, _) = two_site_store();
|
||||
let (aggregates, _) = split_store(&store, 9).unwrap();
|
||||
let reports_a = aggregates
|
||||
.iter()
|
||||
.find(|a| {
|
||||
a.kind == AggregateKind::Reports
|
||||
&& a.site_id.as_ref().is_some_and(|id| id.as_str() == "site-a")
|
||||
})
|
||||
.unwrap();
|
||||
assert!(reports_a.contains("12345678-A"));
|
||||
assert!(!reports_a.contains("87654321-B"));
|
||||
assert!(!reports_a.contains("Secret-B-task"));
|
||||
let tasks_b = aggregates
|
||||
.iter()
|
||||
.find(|a| {
|
||||
a.kind == AggregateKind::Tasks
|
||||
&& a.site_id.as_ref().is_some_and(|id| id.as_str() == "site-b")
|
||||
})
|
||||
.unwrap();
|
||||
assert!(tasks_b.contains("Secret-B-task"));
|
||||
assert!(!tasks_b.contains("12345678-A"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn device_local_state_never_replicates() {
|
||||
let (store, _, _) = two_site_store();
|
||||
let (aggregates, _) = split_store(&store, 9).unwrap();
|
||||
let replicated = replicated_aggregates(&aggregates);
|
||||
assert!(replicated.iter().all(|a| a.kind.is_replicated()));
|
||||
assert!(!replicated.iter().any(|a| a.contains("hello-a")));
|
||||
assert!(!replicated.iter().any(|a| a.contains("room-a")));
|
||||
// …but round-trips locally through the dedicated aggregate.
|
||||
let back = assemble_store(&aggregates).unwrap();
|
||||
assert_eq!(back.selected_site_id.as_deref(), Some("site-a"));
|
||||
assert_eq!(back.thread_lines("room-a"), vec!["hello-a".to_string()]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn shared_suppliers_and_unsited_records_quarantine_loudly() {
|
||||
let (mut store, _, _) = two_site_store();
|
||||
store.push_supplier(crate::domain::procurement::Supplier {
|
||||
id: "sup-1".to_string(),
|
||||
name: "Hardware Ltd".to_string(),
|
||||
kind: crate::domain::procurement::SupplierKind::Hardware,
|
||||
phone: None,
|
||||
email: None,
|
||||
address: None,
|
||||
});
|
||||
let mut orphan = crate::domain::reminders::Reminder::new(
|
||||
crate::domain::reminders::ReminderKind::InspectionDue,
|
||||
"orphan",
|
||||
chrono::Utc::now(),
|
||||
);
|
||||
orphan.site_id = None;
|
||||
store.reminders.push(orphan);
|
||||
let (_, quarantine) = split_store(&store, 1).unwrap();
|
||||
assert!(quarantine
|
||||
.unassigned_suppliers
|
||||
.contains(&"sup-1".to_string()));
|
||||
assert_eq!(quarantine.unsited_reminders.len(), 1);
|
||||
assert!(quarantine.dangling_records.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn assemble_rejects_cross_site_unknown_and_duplicate() {
|
||||
let (store, _, _) = two_site_store();
|
||||
let (mut aggregates, _) = split_store(&store, 1).unwrap();
|
||||
// Cross-site: move a site-b report payload into site-a's envelope.
|
||||
let payload_b = aggregates
|
||||
.iter()
|
||||
.find(|a| {
|
||||
a.kind == AggregateKind::Reports
|
||||
&& a.site_id.as_ref().is_some_and(|id| id.as_str() == "site-b")
|
||||
})
|
||||
.unwrap()
|
||||
.payload
|
||||
.clone();
|
||||
let envelope_a = aggregates
|
||||
.iter_mut()
|
||||
.find(|a| {
|
||||
a.kind == AggregateKind::Reports
|
||||
&& a.site_id.as_ref().is_some_and(|id| id.as_str() == "site-a")
|
||||
})
|
||||
.unwrap();
|
||||
envelope_a.payload = payload_b;
|
||||
assert!(matches!(
|
||||
assemble_store(&aggregates),
|
||||
Err(AggregateError::CrossSiteRecord)
|
||||
));
|
||||
|
||||
let (mut aggregates, _) = split_store(&store, 1).unwrap();
|
||||
aggregates.push(aggregates[0].clone());
|
||||
assert!(matches!(
|
||||
assemble_store(&aggregates),
|
||||
Err(AggregateError::DuplicateAggregate)
|
||||
));
|
||||
|
||||
let (mut aggregates, _) = split_store(&store, 1).unwrap();
|
||||
aggregates[0].version = 99;
|
||||
assert!(matches!(
|
||||
assemble_store(&aggregates),
|
||||
Err(AggregateError::UnsupportedVersion)
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ledger_cas_rejects_stale_writes_and_never_wraps() {
|
||||
let (store, _, _) = two_site_store();
|
||||
let (aggregates, _) = split_store(&store, 4).unwrap();
|
||||
let mut ledger = AggregateLedger::init(4, &aggregates);
|
||||
let site_a = SiteId::parse("site-a").unwrap();
|
||||
assert_eq!(ledger.get(AggregateKind::Reports, Some(&site_a)), 4);
|
||||
assert_eq!(
|
||||
ledger.compare_and_swap(AggregateKind::Reports, Some(&site_a), 3),
|
||||
Err(AggregateError::StaleRevision)
|
||||
);
|
||||
assert_eq!(
|
||||
ledger.compare_and_swap(AggregateKind::Reports, Some(&site_a), 4),
|
||||
Ok(5)
|
||||
);
|
||||
// Other scopes are untouched by the bump.
|
||||
let site_b = SiteId::parse("site-b").unwrap();
|
||||
assert_eq!(ledger.get(AggregateKind::Reports, Some(&site_b)), 4);
|
||||
let mut full = AggregateLedger::default();
|
||||
full.revisions
|
||||
.insert((AggregateKind::Tasks, None), u64::MAX);
|
||||
assert_eq!(
|
||||
full.record_mutation(AggregateKind::Tasks, None),
|
||||
Err(AggregateError::StaleRevision)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn selection_fallback_is_impossible() {
|
||||
let known = vec!["site-a".to_string(), "site-b".to_string()];
|
||||
assert!(require_site_context(None, &known, 1).is_err());
|
||||
assert!(require_site_context(Some(""), &known, 1).is_err());
|
||||
assert!(require_site_context(Some("site-z"), &known, 1).is_err());
|
||||
let ctx = require_site_context(Some("site-b"), &known, 7).unwrap();
|
||||
assert_eq!(ctx.site_id.as_str(), "site-b");
|
||||
assert_eq!(ctx.base_revision, 7);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pagination_bounds_queries() {
|
||||
let items: Vec<u32> = (0..10).collect();
|
||||
let page = Page::new(8, 5).unwrap();
|
||||
assert_eq!(paginate_records(&items, page), vec![8, 9]);
|
||||
assert!(Page::new(0, 0).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn split_round_trips_every_aggregate() {
|
||||
let (store, _, _) = two_site_store();
|
||||
let (aggregates, _) = split_store(&store, 3).unwrap();
|
||||
// 1 profile + 7 per-site × 2 sites + shared + local = 17.
|
||||
assert_eq!(aggregates.len(), 17);
|
||||
let back = assemble_store(&aggregates).unwrap();
|
||||
assert_eq!(back.sites.len(), 2);
|
||||
assert_eq!(back.reports.len(), 2);
|
||||
assert_eq!(back.tasks.len(), 1);
|
||||
}
|
||||
}
|
||||
156
crates/apps/nigig-site/src/ids.rs
Normal file
156
crates/apps/nigig-site/src/ids.rs
Normal file
|
|
@ -0,0 +1,156 @@
|
|||
//! SITE-03 — opaque typed identifiers with central validation.
|
||||
//!
|
||||
//! Legacy code aliases every id to `String`, so cross-site, wrong-type and
|
||||
//! dangling references survive into exports and sync. These newtypes are
|
||||
//! opaque, validated once at parse time, and serialized transparently so the
|
||||
//! existing `NIGIG2` envelope shape is unchanged.
|
||||
//!
|
||||
//! Limits follow §7: ids are ULID/UUID-shaped, bounded length, no PII.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::fmt;
|
||||
use std::str::FromStr;
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub enum IdError {
|
||||
Empty,
|
||||
TooLong,
|
||||
BadCharset,
|
||||
}
|
||||
|
||||
impl fmt::Display for IdError {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
match self {
|
||||
Self::Empty => write!(f, "SITE-ID-EMPTY"),
|
||||
Self::TooLong => write!(f, "SITE-ID-LIMIT"),
|
||||
Self::BadCharset => write!(f, "SITE-ID-CHARSET"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for IdError {}
|
||||
|
||||
fn validate_raw(raw: &str) -> Result<(), IdError> {
|
||||
if raw.is_empty() {
|
||||
return Err(IdError::Empty);
|
||||
}
|
||||
if raw.len() > 64 {
|
||||
return Err(IdError::TooLong);
|
||||
}
|
||||
if !raw
|
||||
.bytes()
|
||||
.all(|b| b.is_ascii_alphanumeric() || b == b'-' || b == b'_')
|
||||
{
|
||||
return Err(IdError::BadCharset);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
macro_rules! opaque_id {
|
||||
($name:ident) => {
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
|
||||
#[serde(transparent)]
|
||||
pub struct $name(String);
|
||||
|
||||
impl $name {
|
||||
pub fn parse(raw: &str) -> Result<Self, IdError> {
|
||||
validate_raw(raw)?;
|
||||
Ok(Self(raw.to_string()))
|
||||
}
|
||||
|
||||
/// Generate a fresh random id (v4 UUID text, 36 chars, valid charset).
|
||||
pub fn generate() -> Self {
|
||||
Self(uuid::Uuid::new_v4().to_string())
|
||||
}
|
||||
|
||||
pub fn as_str(&self) -> &str {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl fmt::Display for $name {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
f.write_str(&self.0)
|
||||
}
|
||||
}
|
||||
|
||||
impl FromStr for $name {
|
||||
type Err = IdError;
|
||||
fn from_str(s: &str) -> Result<Self, Self::Err> {
|
||||
Self::parse(s)
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
opaque_id!(SiteId);
|
||||
opaque_id!(ReportId);
|
||||
// Named `TaskRecordId` (not `TaskId`) because the legacy domain alias
|
||||
// `domain::approvals::TaskId = String` still exists for encrypted-record
|
||||
// compatibility. New code takes this opaque type; the alias is boundary
|
||||
// legacy and must not gain new producers.
|
||||
opaque_id!(TaskRecordId);
|
||||
opaque_id!(WorkerId);
|
||||
opaque_id!(ScanId);
|
||||
opaque_id!(SupplierId);
|
||||
opaque_id!(MeetingId);
|
||||
opaque_id!(AssetId);
|
||||
opaque_id!(ReminderId);
|
||||
opaque_id!(UserId);
|
||||
opaque_id!(DeviceId);
|
||||
opaque_id!(OperationId);
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn rejects_empty_long_and_wrong_charset() {
|
||||
assert_eq!(SiteId::parse(""), Err(IdError::Empty));
|
||||
assert_eq!(SiteId::parse(&"a".repeat(65)), Err(IdError::TooLong));
|
||||
assert_eq!(SiteId::parse("site A!"), Err(IdError::BadCharset));
|
||||
assert_eq!(SiteId::parse("site/a"), Err(IdError::BadCharset));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn accepts_uuid_and_ulid_shapes_and_round_trips() {
|
||||
for raw in [
|
||||
"01J9ZQ6K7MNPQRS",
|
||||
"550e8400-e29b-41d4-a716-446655440000",
|
||||
"abc_123-XYZ",
|
||||
] {
|
||||
let id = SiteId::parse(raw).unwrap();
|
||||
assert_eq!(id.as_str(), raw);
|
||||
let json = serde_json::to_string(&id).unwrap();
|
||||
let back: SiteId = serde_json::from_str(&json).unwrap();
|
||||
assert_eq!(back, id);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn generated_ids_are_valid_and_unique() {
|
||||
let a = SiteId::generate();
|
||||
let b = SiteId::generate();
|
||||
assert_ne!(a, b);
|
||||
assert!(SiteId::parse(a.as_str()).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn id_types_do_not_confuse_at_type_level() {
|
||||
fn takes_site(_: SiteId) {}
|
||||
let task = TaskRecordId::generate();
|
||||
// The following would not compile if uncommented, proving
|
||||
// wrong-type ids are rejected statically:
|
||||
// takes_site(task);
|
||||
let _ = task;
|
||||
takes_site(SiteId::generate());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cross_site_string_equality_requires_explicit_comparison() {
|
||||
let a = SiteId::parse("site-a").unwrap();
|
||||
let b = SiteId::parse("site-b").unwrap();
|
||||
assert_ne!(a, b);
|
||||
assert_ne!(a.as_str(), b.as_str());
|
||||
}
|
||||
}
|
||||
|
|
@ -7,6 +7,7 @@
|
|||
|
||||
use makepad_widgets::ScriptVm;
|
||||
|
||||
pub mod aggregates;
|
||||
mod ai_refine;
|
||||
pub mod containment;
|
||||
mod crypto;
|
||||
|
|
@ -15,12 +16,14 @@ pub mod doc_export;
|
|||
pub mod domain;
|
||||
#[cfg(all(test, target_os = "linux"))]
|
||||
pub mod gif;
|
||||
pub mod ids;
|
||||
#[cfg(all(test, target_os = "linux"))]
|
||||
pub mod ocr;
|
||||
#[cfg(all(test, target_os = "linux"))]
|
||||
pub mod report_pdf;
|
||||
pub(crate) mod repository;
|
||||
pub mod scheduler;
|
||||
pub mod site_context;
|
||||
pub mod site_frame;
|
||||
pub mod store;
|
||||
#[cfg(all(test, target_os = "linux"))]
|
||||
|
|
|
|||
|
|
@ -52,7 +52,9 @@ pub fn schedule_daily_eod(site_id: &str, date: chrono::NaiveDate, eod_local_hour
|
|||
fire_at,
|
||||
);
|
||||
r.site_id = Some(site_id.to_string());
|
||||
SiteStore::mutate_scoped(site_id, |s| s.push_reminder(r))
|
||||
SiteStore::scoped_context(site_id)
|
||||
.map(|context| SiteStore::mutate_scoped(&context, |s| s.push_reminder(r)))
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
/// Record an encrypted local reminder for a scheduled meeting without
|
||||
|
|
@ -60,7 +62,9 @@ pub fn schedule_daily_eod(site_id: &str, date: chrono::NaiveDate, eod_local_hour
|
|||
pub fn schedule_monthly_before_meeting(site_id: &str, meeting_at: chrono::DateTime<Utc>) -> bool {
|
||||
let mut r = crate::domain::reminders::Reminder::monthly_report_before(meeting_at);
|
||||
r.site_id = Some(site_id.to_string());
|
||||
SiteStore::mutate_scoped(site_id, |s| s.push_reminder(r))
|
||||
SiteStore::scoped_context(site_id)
|
||||
.map(|context| SiteStore::mutate_scoped(&context, |s| s.push_reminder(r)))
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
|
@ -103,15 +107,18 @@ mod tests {
|
|||
..Default::default()
|
||||
};
|
||||
let now = Utc::now();
|
||||
let r = crate::domain::reminders::Reminder::new(
|
||||
let mut r = crate::domain::reminders::Reminder::new(
|
||||
crate::domain::reminders::ReminderKind::Custom("test".into()),
|
||||
"Test reminder",
|
||||
now - chrono::Duration::seconds(1),
|
||||
);
|
||||
r.site_id = Some("site-a".to_string());
|
||||
let context =
|
||||
crate::site_context::SiteContext::new(crate::ids::SiteId::parse("site-a").unwrap(), 0);
|
||||
let id = r.id.clone();
|
||||
s.reminders.push(r);
|
||||
assert_eq!(s.due_reminders(now).len(), 1);
|
||||
assert_eq!(s.due_reminders_for(&context, now).len(), 1);
|
||||
s.mark_fired(&id);
|
||||
assert!(s.due_reminders(now).is_empty());
|
||||
assert!(s.due_reminders_for(&context, now).is_empty());
|
||||
}
|
||||
}
|
||||
|
|
|
|||
155
crates/apps/nigig-site/src/site_context.rs
Normal file
155
crates/apps/nigig-site/src/site_context.rs
Normal file
|
|
@ -0,0 +1,155 @@
|
|||
//! SITE-03 — explicit site context, pagination, and revision compare-and-swap.
|
||||
//!
|
||||
//! No command, query, export, asset access, reminder, or sync event may run
|
||||
//! without a typed [`SiteContext`]. Missing or stale selection is
|
||||
//! [`ContextError::NoSiteSelected`]; it never falls back to the first site.
|
||||
|
||||
use crate::ids::SiteId;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub enum ContextError {
|
||||
NoSiteSelected,
|
||||
StaleSelection,
|
||||
UnknownSite,
|
||||
}
|
||||
|
||||
impl std::fmt::Display for ContextError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
Self::NoSiteSelected => write!(f, "SITE-CONTEXT-NONE"),
|
||||
Self::StaleSelection => write!(f, "SITE-CONTEXT-STALE"),
|
||||
Self::UnknownSite => write!(f, "SITE-CONTEXT-UNKNOWN"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for ContextError {}
|
||||
|
||||
/// Explicit authorization scope for one site at one revision.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct SiteContext {
|
||||
pub site_id: SiteId,
|
||||
/// Last revision the caller observed; commands fail on mismatch (CAS).
|
||||
pub base_revision: u64,
|
||||
}
|
||||
|
||||
impl SiteContext {
|
||||
pub fn new(site_id: SiteId, base_revision: u64) -> Self {
|
||||
Self {
|
||||
site_id,
|
||||
base_revision,
|
||||
}
|
||||
}
|
||||
|
||||
/// Build from an optional selection without any first-site fallback.
|
||||
pub fn from_selection(
|
||||
selected: Option<&str>,
|
||||
known_sites: &[String],
|
||||
base_revision: u64,
|
||||
) -> Result<Self, ContextError> {
|
||||
let raw = selected.ok_or(ContextError::NoSiteSelected)?;
|
||||
if raw.is_empty() {
|
||||
return Err(ContextError::NoSiteSelected);
|
||||
}
|
||||
let site_id = SiteId::parse(raw).map_err(|_| ContextError::NoSiteSelected)?;
|
||||
if !known_sites.iter().any(|s| s == site_id.as_str()) {
|
||||
return Err(ContextError::StaleSelection);
|
||||
}
|
||||
Ok(Self::new(site_id, base_revision))
|
||||
}
|
||||
}
|
||||
|
||||
/// Bounded page request for repository queries (§7: pagination mandatory).
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub struct Page {
|
||||
pub offset: usize,
|
||||
pub limit: usize,
|
||||
}
|
||||
|
||||
impl Page {
|
||||
pub const MAX_LIMIT: usize = 1_000;
|
||||
|
||||
pub fn new(offset: usize, limit: usize) -> Result<Self, &'static str> {
|
||||
if limit == 0 || limit > Self::MAX_LIMIT {
|
||||
return Err("SITE-PAGE-LIMIT");
|
||||
}
|
||||
let end = offset.checked_add(limit).ok_or("SITE-PAGE-OVERFLOW")?;
|
||||
let _ = end;
|
||||
Ok(Self { offset, limit })
|
||||
}
|
||||
|
||||
pub fn slice<T: Clone>(&self, items: &[T]) -> Vec<T> {
|
||||
items
|
||||
.iter()
|
||||
.skip(self.offset)
|
||||
.take(self.limit)
|
||||
.cloned()
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
|
||||
/// Compare-and-swap on an aggregate revision.
|
||||
pub fn check_revision(expected: u64, current: u64) -> Result<(), ContextError> {
|
||||
if expected == current {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(ContextError::StaleSelection)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn missing_selection_never_falls_back_to_first_site() {
|
||||
let known = vec!["site-a".to_string(), "site-b".to_string()];
|
||||
assert_eq!(
|
||||
SiteContext::from_selection(None, &known, 1),
|
||||
Err(ContextError::NoSiteSelected)
|
||||
);
|
||||
assert_eq!(
|
||||
SiteContext::from_selection(Some(""), &known, 1),
|
||||
Err(ContextError::NoSiteSelected)
|
||||
);
|
||||
// Even with sites present, None must not resolve to site-a.
|
||||
assert!(SiteContext::from_selection(None, &known, 1).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stale_selection_is_rejected_not_substituted() {
|
||||
let known = vec!["site-a".to_string()];
|
||||
assert_eq!(
|
||||
SiteContext::from_selection(Some("site-b"), &known, 1),
|
||||
Err(ContextError::StaleSelection)
|
||||
);
|
||||
assert_eq!(
|
||||
SiteContext::from_selection(Some("site!!"), &known, 1),
|
||||
Err(ContextError::NoSiteSelected)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn valid_selection_yields_context() {
|
||||
let known = vec!["site-a".to_string()];
|
||||
let ctx = SiteContext::from_selection(Some("site-a"), &known, 7).unwrap();
|
||||
assert_eq!(ctx.site_id.as_str(), "site-a");
|
||||
assert_eq!(ctx.base_revision, 7);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pagination_enforces_bounds_and_overflow() {
|
||||
assert!(Page::new(0, 0).is_err());
|
||||
assert!(Page::new(0, 1_001).is_err());
|
||||
assert!(Page::new(usize::MAX, 1).is_err());
|
||||
let page = Page::new(1, 2).unwrap();
|
||||
assert_eq!(page.slice(&[1, 2, 3, 4]), vec![2, 3]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn revision_cas_rejects_stale_writes() {
|
||||
assert!(check_revision(3, 3).is_ok());
|
||||
assert_eq!(check_revision(2, 3), Err(ContextError::StaleSelection));
|
||||
}
|
||||
}
|
||||
|
|
@ -352,8 +352,13 @@ impl Widget for ApprovalsPage {
|
|||
notes: None,
|
||||
});
|
||||
}
|
||||
let accepted =
|
||||
crate::store::SiteStore::mutate_scoped(&site_id, |store| store.push_task(task));
|
||||
let accepted = crate::store::SiteStore::scoped_context(&site_id)
|
||||
.map(|context| {
|
||||
crate::store::SiteStore::mutate_scoped(&context, |store| {
|
||||
store.push_task(task)
|
||||
})
|
||||
})
|
||||
.unwrap_or(false);
|
||||
if !accepted {
|
||||
let message = crate::store::SiteStore::mutation_status(false);
|
||||
self.view
|
||||
|
|
@ -397,9 +402,13 @@ impl Widget for ApprovalsPage {
|
|||
self.view.redraw(cx);
|
||||
return;
|
||||
};
|
||||
let accepted = crate::store::SiteStore::mutate_scoped(&site_id, |store| {
|
||||
store.tasks.retain(|task| task.id != id);
|
||||
});
|
||||
let accepted = crate::store::SiteStore::scoped_context(&site_id)
|
||||
.map(|context| {
|
||||
crate::store::SiteStore::mutate_scoped(&context, |store| {
|
||||
store.tasks.retain(|task| task.id != id);
|
||||
})
|
||||
})
|
||||
.unwrap_or(false);
|
||||
if !accepted {
|
||||
let message = crate::store::SiteStore::mutation_status(false);
|
||||
self.view
|
||||
|
|
@ -469,9 +478,13 @@ impl Widget for ApprovalsPage {
|
|||
self.view.redraw(cx);
|
||||
return;
|
||||
};
|
||||
let accepted = crate::store::SiteStore::mutate_scoped(&site_id, |store| {
|
||||
store.generate_template_for_site(&site_id);
|
||||
});
|
||||
let accepted = crate::store::SiteStore::scoped_context(&site_id)
|
||||
.map(|context| {
|
||||
crate::store::SiteStore::mutate_scoped(&context, |store| {
|
||||
store.generate_template_for_site(&site_id);
|
||||
})
|
||||
})
|
||||
.unwrap_or(false);
|
||||
if !accepted {
|
||||
let message = crate::store::SiteStore::mutation_status(false);
|
||||
self.view
|
||||
|
|
@ -564,11 +577,15 @@ impl ApprovalsPage {
|
|||
let Some(site_id) = crate::store::SiteStore::read().selected_site_id() else {
|
||||
return;
|
||||
};
|
||||
let accepted = crate::store::SiteStore::mutate_scoped(&site_id, |store| {
|
||||
if let Some(task) = store.tasks.iter_mut().find(|task| task.id == id) {
|
||||
task.status = status.clone();
|
||||
}
|
||||
});
|
||||
let accepted = crate::store::SiteStore::scoped_context(&site_id)
|
||||
.map(|context| {
|
||||
crate::store::SiteStore::mutate_scoped(&context, |store| {
|
||||
if let Some(task) = store.tasks.iter_mut().find(|task| task.id == id) {
|
||||
task.status = status.clone();
|
||||
}
|
||||
})
|
||||
})
|
||||
.unwrap_or(false);
|
||||
if !accepted {
|
||||
let message = crate::store::SiteStore::mutation_status(false);
|
||||
self.view
|
||||
|
|
@ -590,23 +607,27 @@ impl ApprovalsPage {
|
|||
};
|
||||
let mut found = false;
|
||||
let mut pending = false;
|
||||
let accepted = crate::store::SiteStore::mutate_scoped(&site_id, |store| {
|
||||
if let Some(task) = store.tasks.iter_mut().find(|task| task.id == id) {
|
||||
found = true;
|
||||
if let Some(inspection) = task
|
||||
.inspections
|
||||
.iter_mut()
|
||||
.find(|inspection| inspection.passed.is_none())
|
||||
{
|
||||
pending = true;
|
||||
inspection.passed = Some(passed);
|
||||
inspection.inspected_at = Some(chrono::Utc::now());
|
||||
if passed {
|
||||
task.status = crate::domain::approvals::TaskStatus::Inspected;
|
||||
let accepted = crate::store::SiteStore::scoped_context(&site_id)
|
||||
.map(|context| {
|
||||
crate::store::SiteStore::mutate_scoped(&context, |store| {
|
||||
if let Some(task) = store.tasks.iter_mut().find(|task| task.id == id) {
|
||||
found = true;
|
||||
if let Some(inspection) = task
|
||||
.inspections
|
||||
.iter_mut()
|
||||
.find(|inspection| inspection.passed.is_none())
|
||||
{
|
||||
pending = true;
|
||||
inspection.passed = Some(passed);
|
||||
inspection.inspected_at = Some(chrono::Utc::now());
|
||||
if passed {
|
||||
task.status = crate::domain::approvals::TaskStatus::Inspected;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
})
|
||||
})
|
||||
.unwrap_or(false);
|
||||
if !accepted {
|
||||
let message = crate::store::SiteStore::mutation_status(false);
|
||||
self.view
|
||||
|
|
@ -630,41 +651,47 @@ impl ApprovalsPage {
|
|||
self.rows.clear();
|
||||
return;
|
||||
};
|
||||
self.rows = store
|
||||
.tasks_for_site(&site_id)
|
||||
.into_iter()
|
||||
.map(|t| {
|
||||
let dates = match (t.start, t.end) {
|
||||
(Some(s), Some(e)) => format!("{s} → {e}"),
|
||||
(Some(s), None) => format!("from {s}"),
|
||||
(None, Some(e)) => format!("until {e}"),
|
||||
(None, None) => "no dates".to_string(),
|
||||
};
|
||||
let qty = match (t.quantity, t.quantity_unit.clone()) {
|
||||
(Some(q), Some(u)) => format!("{q} {u}"),
|
||||
(Some(q), None) => format!("{q}"),
|
||||
_ => "no qty".to_string(),
|
||||
};
|
||||
let insp = if t.inspections.is_empty() {
|
||||
"no inspection".to_string()
|
||||
} else {
|
||||
t.inspections
|
||||
.iter()
|
||||
.map(|i| i.kind.label())
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ")
|
||||
};
|
||||
TaskRowData {
|
||||
id: t.id.clone(),
|
||||
title: format!(
|
||||
"• {} ({})",
|
||||
t.title,
|
||||
t.phase.clone().unwrap_or_else(|| "no phase".into())
|
||||
),
|
||||
sub: format!("{dates} • {qty} • {insp} • {:?}", t.status),
|
||||
}
|
||||
let context = crate::store::SiteStore::scoped_context(&site_id).ok();
|
||||
self.rows = context
|
||||
.as_ref()
|
||||
.map(|context| {
|
||||
store
|
||||
.tasks_for_site(context)
|
||||
.into_iter()
|
||||
.map(|t| {
|
||||
let dates = match (t.start, t.end) {
|
||||
(Some(s), Some(e)) => format!("{s} → {e}"),
|
||||
(Some(s), None) => format!("from {s}"),
|
||||
(None, Some(e)) => format!("until {e}"),
|
||||
(None, None) => "no dates".to_string(),
|
||||
};
|
||||
let qty = match (t.quantity, t.quantity_unit.clone()) {
|
||||
(Some(q), Some(u)) => format!("{q} {u}"),
|
||||
(Some(q), None) => format!("{q}"),
|
||||
_ => "no qty".to_string(),
|
||||
};
|
||||
let insp = if t.inspections.is_empty() {
|
||||
"no inspection".to_string()
|
||||
} else {
|
||||
t.inspections
|
||||
.iter()
|
||||
.map(|i| i.kind.label())
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ")
|
||||
};
|
||||
TaskRowData {
|
||||
id: t.id.clone(),
|
||||
title: format!(
|
||||
"• {} ({})",
|
||||
t.title,
|
||||
t.phase.clone().unwrap_or_else(|| "no phase".into())
|
||||
),
|
||||
sub: format!("{dates} • {qty} • {insp} • {:?}", t.status),
|
||||
}
|
||||
})
|
||||
.collect::<Vec<_>>()
|
||||
})
|
||||
.collect();
|
||||
.unwrap_or_default();
|
||||
}
|
||||
|
||||
fn update_task_list(&mut self, cx: &mut Cx) {
|
||||
|
|
|
|||
|
|
@ -272,14 +272,19 @@ impl Widget for MeetingsPage {
|
|||
return;
|
||||
};
|
||||
let mut found: Option<(String, chrono::DateTime<chrono::Utc>)> = None;
|
||||
let accepted = crate::store::SiteStore::mutate_scoped(&site_id, |store| {
|
||||
if let Some(meeting) = store.meetings.iter_mut().rev().find(|meeting| {
|
||||
meeting.site_id == site_id && meeting.title.eq_ignore_ascii_case(&title)
|
||||
}) {
|
||||
meeting.reschedule(new_at);
|
||||
found = Some((meeting.site_id.clone(), meeting.scheduled_at));
|
||||
}
|
||||
});
|
||||
let accepted = crate::store::SiteStore::scoped_context(&site_id)
|
||||
.map(|context| {
|
||||
crate::store::SiteStore::mutate_scoped(&context, |store| {
|
||||
if let Some(meeting) = store.meetings.iter_mut().rev().find(|meeting| {
|
||||
meeting.site_id == site_id
|
||||
&& meeting.title.eq_ignore_ascii_case(&title)
|
||||
}) {
|
||||
meeting.reschedule(new_at);
|
||||
found = Some((meeting.site_id.clone(), meeting.scheduled_at));
|
||||
}
|
||||
})
|
||||
})
|
||||
.unwrap_or(false);
|
||||
if !accepted {
|
||||
let message = crate::store::SiteStore::mutation_status(false);
|
||||
self.view
|
||||
|
|
@ -336,16 +341,20 @@ impl Widget for MeetingsPage {
|
|||
self.view.redraw(cx);
|
||||
return;
|
||||
};
|
||||
let accepted = crate::store::SiteStore::mutate_scoped(&site_id, |store| {
|
||||
store.push_contact(
|
||||
&site_id,
|
||||
crate::domain::meetings::MeetingAttendee {
|
||||
user_id: uuid::Uuid::new_v4().to_string(),
|
||||
display_name: name,
|
||||
email: None,
|
||||
},
|
||||
);
|
||||
});
|
||||
let accepted = crate::store::SiteStore::scoped_context(&site_id)
|
||||
.map(|context| {
|
||||
crate::store::SiteStore::mutate_scoped(&context, |store| {
|
||||
store.push_contact(
|
||||
&site_id,
|
||||
crate::domain::meetings::MeetingAttendee {
|
||||
user_id: uuid::Uuid::new_v4().to_string(),
|
||||
display_name: name,
|
||||
email: None,
|
||||
},
|
||||
);
|
||||
})
|
||||
})
|
||||
.unwrap_or(false);
|
||||
if !accepted {
|
||||
let message = crate::store::SiteStore::mutation_status(false);
|
||||
self.view
|
||||
|
|
@ -422,9 +431,13 @@ impl Widget for MeetingsPage {
|
|||
});
|
||||
}
|
||||
let meeting_site_id = m.site_id.clone();
|
||||
let accepted = crate::store::SiteStore::mutate_scoped(&meeting_site_id, |store| {
|
||||
store.push_meeting(m.clone())
|
||||
});
|
||||
let accepted = crate::store::SiteStore::scoped_context(&meeting_site_id)
|
||||
.map(|context| {
|
||||
crate::store::SiteStore::mutate_scoped(&context, |store| {
|
||||
store.push_meeting(m.clone())
|
||||
})
|
||||
})
|
||||
.unwrap_or(false);
|
||||
if !accepted {
|
||||
let message = crate::store::SiteStore::mutation_status(false);
|
||||
self.view
|
||||
|
|
@ -538,28 +551,29 @@ impl Widget for MeetingsPage {
|
|||
impl MeetingsPage {
|
||||
fn update_meetings_list(&mut self, cx: &mut Cx) {
|
||||
let store = crate::store::SiteStore::read();
|
||||
let Some(site_id) = store.selected_site_id() else {
|
||||
self.rows.clear();
|
||||
self.update_directory(cx);
|
||||
return;
|
||||
};
|
||||
self.rows = store
|
||||
.meetings_for(&site_id)
|
||||
.into_iter()
|
||||
.map(|m| MeetingRowData {
|
||||
title: m.title.clone(),
|
||||
sub: format!(
|
||||
"{} [{:?}] — {} attendees{}",
|
||||
m.scheduled_at.format("%Y-%m-%d %H:%M"),
|
||||
m.status,
|
||||
m.attendees.len(),
|
||||
m.location
|
||||
.clone()
|
||||
.map(|l| format!(" @ {l}"))
|
||||
.unwrap_or_default()
|
||||
),
|
||||
.selected_site_id()
|
||||
.and_then(|id| crate::store::SiteStore::scoped_context(&id).ok())
|
||||
.map(|context| {
|
||||
store
|
||||
.meetings_for(&context)
|
||||
.into_iter()
|
||||
.map(|m| MeetingRowData {
|
||||
title: m.title.clone(),
|
||||
sub: format!(
|
||||
"{} [{:?}] — {} attendees{}",
|
||||
m.scheduled_at.format("%Y-%m-%d %H:%M"),
|
||||
m.status,
|
||||
m.attendees.len(),
|
||||
m.location
|
||||
.clone()
|
||||
.map(|l| format!(" @ {l}"))
|
||||
.unwrap_or_default()
|
||||
),
|
||||
})
|
||||
.collect::<Vec<_>>()
|
||||
})
|
||||
.collect();
|
||||
.unwrap_or_default();
|
||||
self.update_directory(cx);
|
||||
}
|
||||
|
||||
|
|
@ -567,7 +581,11 @@ impl MeetingsPage {
|
|||
let store = crate::store::SiteStore::read();
|
||||
let names = store
|
||||
.selected_site_id()
|
||||
.map(|site_id| store.directory_names(&site_id))
|
||||
.and_then(|id| {
|
||||
crate::store::SiteStore::scoped_context(&id)
|
||||
.ok()
|
||||
.map(|context| store.directory_names(&context))
|
||||
})
|
||||
.unwrap_or_default();
|
||||
let dir_text = if names.is_empty() {
|
||||
"No contacts yet.".to_string()
|
||||
|
|
|
|||
|
|
@ -256,9 +256,13 @@ impl Widget for ProcurementPage {
|
|||
self.view.redraw(cx);
|
||||
return;
|
||||
};
|
||||
let mut sched = store.procurement_for(&site_id).cloned().unwrap_or_else(|| {
|
||||
crate::domain::procurement::ProcurementSchedule::new(site_id.clone())
|
||||
});
|
||||
let context = crate::store::SiteStore::scoped_context(&site_id).ok();
|
||||
let mut sched = context
|
||||
.as_ref()
|
||||
.and_then(|context| store.procurement_for(context).cloned())
|
||||
.unwrap_or_else(|| {
|
||||
crate::domain::procurement::ProcurementSchedule::new(site_id.clone())
|
||||
});
|
||||
let mut line = crate::domain::procurement::MaterialLine::new(
|
||||
site_id.clone(),
|
||||
name,
|
||||
|
|
@ -267,9 +271,14 @@ impl Widget for ProcurementPage {
|
|||
);
|
||||
line.supplier_id = supplier_id;
|
||||
sched.lines.push(line);
|
||||
let accepted = crate::store::SiteStore::mutate_scoped(&site_id, |store| {
|
||||
store.push_procurement(sched)
|
||||
});
|
||||
let accepted = context
|
||||
.as_ref()
|
||||
.map(|context| {
|
||||
crate::store::SiteStore::mutate_scoped(context, |store| {
|
||||
store.push_procurement(sched)
|
||||
})
|
||||
})
|
||||
.unwrap_or(false);
|
||||
if !accepted {
|
||||
let message = crate::store::SiteStore::mutation_status(false);
|
||||
self.view
|
||||
|
|
@ -387,16 +396,20 @@ impl Widget for ProcurementPage {
|
|||
self.view.redraw(cx);
|
||||
return;
|
||||
};
|
||||
let accepted = crate::store::SiteStore::mutate_scoped(&site_id, |store| {
|
||||
store.push_supplier(crate::domain::procurement::Supplier {
|
||||
id: uuid::Uuid::new_v4().to_string(),
|
||||
name,
|
||||
kind,
|
||||
phone: None,
|
||||
email: None,
|
||||
address: None,
|
||||
});
|
||||
});
|
||||
let accepted = crate::store::SiteStore::scoped_context(&site_id)
|
||||
.map(|context| {
|
||||
crate::store::SiteStore::mutate_scoped(&context, |store| {
|
||||
store.push_supplier(crate::domain::procurement::Supplier {
|
||||
id: uuid::Uuid::new_v4().to_string(),
|
||||
name,
|
||||
kind,
|
||||
phone: None,
|
||||
email: None,
|
||||
address: None,
|
||||
});
|
||||
})
|
||||
})
|
||||
.unwrap_or(false);
|
||||
if !accepted {
|
||||
let message = crate::store::SiteStore::mutation_status(false);
|
||||
self.view
|
||||
|
|
@ -470,7 +483,11 @@ impl ProcurementPage {
|
|||
.collect();
|
||||
let schedule = store
|
||||
.selected_site_id()
|
||||
.and_then(|site_id| store.procurement_for(&site_id))
|
||||
.and_then(|site_id| {
|
||||
crate::store::SiteStore::scoped_context(&site_id)
|
||||
.ok()
|
||||
.and_then(|context| store.procurement_for(&context))
|
||||
})
|
||||
.map(|sched| {
|
||||
sched
|
||||
.lines
|
||||
|
|
|
|||
|
|
@ -283,9 +283,13 @@ impl Widget for ReportEditorPage {
|
|||
let date = chrono::Local::now().date_naive();
|
||||
let mut entry = crate::domain::daily_report::DailyTaskEntry::new(title, rich);
|
||||
entry.work_station = if ws.trim().is_empty() { None } else { Some(ws) };
|
||||
let accepted = crate::store::SiteStore::mutate_scoped(&site_id, |store| {
|
||||
store.push_task_entry(&site_id, date, entry)
|
||||
});
|
||||
let accepted = crate::store::SiteStore::scoped_context(&site_id)
|
||||
.map(|context| {
|
||||
crate::store::SiteStore::mutate_scoped(&context, |store| {
|
||||
store.push_task_entry(&site_id, date, entry)
|
||||
})
|
||||
})
|
||||
.unwrap_or(false);
|
||||
let message = crate::store::SiteStore::mutation_status(accepted);
|
||||
self.view
|
||||
.label(cx, ids!(refined_preview.refined_text))
|
||||
|
|
|
|||
|
|
@ -230,11 +230,16 @@ impl SiteReportsPage {
|
|||
let store = crate::store::SiteStore::read();
|
||||
let today = chrono::Local::now().date_naive();
|
||||
let selected = store.selected_site();
|
||||
let (daily, monthly) = if let Some(site) = selected {
|
||||
let site_reports = store.reports_for_site(&site.id);
|
||||
let today_report = store.report_for_day(&site.id, today);
|
||||
let selected_context = selected.as_ref().and_then(|site| {
|
||||
crate::store::SiteStore::scoped_context(&site.id)
|
||||
.ok()
|
||||
.map(|context| (site, context))
|
||||
});
|
||||
let (daily, monthly) = if let Some((site, context)) = selected_context {
|
||||
let site_reports = store.reports_for_site(&context);
|
||||
let today_report = store.report_for_day(&context, today);
|
||||
let today_entries = today_report.map_or(0, |report| report.entries.len());
|
||||
let month_reports = store.reports_for_month(&site.id, today.year(), today.month());
|
||||
let month_reports = store.reports_for_month(&context, today.year(), today.month());
|
||||
let month_entries: usize = month_reports
|
||||
.iter()
|
||||
.map(|report| report.entries.len())
|
||||
|
|
@ -264,7 +269,8 @@ impl SiteReportsPage {
|
|||
)
|
||||
};
|
||||
|
||||
let today_reports = store.reports_for_date(today);
|
||||
let scope_ids: Vec<String> = store.sites.iter().map(|site| site.id.clone()).collect();
|
||||
let today_reports = store.reports_for_date(&scope_ids, today);
|
||||
let today_entries: usize = today_reports
|
||||
.iter()
|
||||
.map(|report| report.entries.len())
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ use chrono::NaiveDate;
|
|||
use serde::{Deserialize, Serialize};
|
||||
use zeroize::Zeroizing;
|
||||
|
||||
use crate::aggregates::{scopes_for_store, AggregateKind, AggregateLedger};
|
||||
use crate::domain::approvals::ConstructionTask;
|
||||
use crate::domain::daily_report::DailyReport;
|
||||
use crate::domain::meetings::{ProjectDirectory, SiteMeeting};
|
||||
|
|
@ -22,6 +23,7 @@ use crate::domain::workers::WorkersDailyTable;
|
|||
use crate::repository::{
|
||||
RepositoryFailure, RepositoryOpen, RepositoryWriter, SiteRepository, WriterHealth,
|
||||
};
|
||||
use crate::site_context::SiteContext;
|
||||
|
||||
const STORE_VERSION: u32 = 2;
|
||||
#[cfg(test)]
|
||||
|
|
@ -289,46 +291,57 @@ struct RuntimeStore {
|
|||
value: SiteStore,
|
||||
access: StoreAccess,
|
||||
writer: Option<RepositoryWriter<SiteStore>>,
|
||||
/// Per-aggregate revisions, initialized at the durable revision on open.
|
||||
/// Scoped mutations bump only the mutated site's aggregates.
|
||||
ledger: AggregateLedger,
|
||||
}
|
||||
|
||||
impl RuntimeStore {
|
||||
fn load() -> Self {
|
||||
let empty = |access: StoreAccess| Self {
|
||||
value: SiteStore::default(),
|
||||
access,
|
||||
writer: None,
|
||||
ledger: AggregateLedger::init_scopes(0, &scopes_for_store(&SiteStore::default())),
|
||||
};
|
||||
let path = match SiteStore::file_path() {
|
||||
Ok(path) => path,
|
||||
Err(failure) => {
|
||||
return Self {
|
||||
value: SiteStore::default(),
|
||||
access: StoreAccess::RecoveryRequired(failure),
|
||||
writer: None,
|
||||
};
|
||||
return empty(StoreAccess::RecoveryRequired(failure));
|
||||
}
|
||||
};
|
||||
let repository = SiteRepository::runtime(path);
|
||||
match repository.open_versioned_json::<SiteStore>(STORE_VERSION) {
|
||||
Ok(RepositoryOpen::Absent) => Self {
|
||||
value: SiteStore::default(),
|
||||
access: StoreAccess::SetupRequired,
|
||||
writer: None,
|
||||
},
|
||||
Ok(RepositoryOpen::Absent) => empty(StoreAccess::SetupRequired),
|
||||
Ok(RepositoryOpen::Open(document)) => {
|
||||
match RepositoryWriter::start(repository, document.metadata) {
|
||||
Ok(writer) => Self {
|
||||
value: document.value,
|
||||
access: StoreAccess::ReadyEncrypted,
|
||||
writer: Some(writer),
|
||||
},
|
||||
Err(failure) => Self {
|
||||
value: document.value,
|
||||
access: StoreAccess::ConfidentialWritesDisabled(map_repository(failure)),
|
||||
writer: None,
|
||||
},
|
||||
Ok(writer) => {
|
||||
let revision = writer.health().accepted_revision;
|
||||
let value = document.value;
|
||||
let ledger =
|
||||
AggregateLedger::init_scopes(revision, &scopes_for_store(&value));
|
||||
Self {
|
||||
value,
|
||||
access: StoreAccess::ReadyEncrypted,
|
||||
writer: Some(writer),
|
||||
ledger,
|
||||
}
|
||||
}
|
||||
Err(failure) => {
|
||||
let value = document.value;
|
||||
let ledger = AggregateLedger::init_scopes(0, &scopes_for_store(&value));
|
||||
Self {
|
||||
value,
|
||||
access: StoreAccess::ConfidentialWritesDisabled(map_repository(
|
||||
failure,
|
||||
)),
|
||||
writer: None,
|
||||
ledger,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(failure) => Self {
|
||||
value: SiteStore::default(),
|
||||
access: StoreAccess::RecoveryRequired(map_repository(failure)),
|
||||
writer: None,
|
||||
},
|
||||
Err(failure) => empty(StoreAccess::RecoveryRequired(map_repository(failure))),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -425,11 +438,35 @@ impl SiteStore {
|
|||
Self::mutate_result(None, change).is_ok()
|
||||
}
|
||||
|
||||
/// Site-scoped mutation rejects absent, stale or implicit context before
|
||||
/// invoking the closure, then rejects/discards any out-of-scope change.
|
||||
/// Site-scoped mutation. The typed `SiteContext` is mandatory: there is
|
||||
/// no `&str` overload, so no call site can compile a site mutation
|
||||
/// without explicit context. Absent, stale or implicit context is
|
||||
/// rejected before the closure runs, as is a stale base revision;
|
||||
/// out-of-scope changes are then rejected/discarded by the fence.
|
||||
/// It is crate-private so every call site is auditable inside this package.
|
||||
pub(crate) fn mutate_scoped(site_id: &str, change: impl FnOnce(&mut SiteStore)) -> bool {
|
||||
Self::mutate_result(Some(site_id), change).is_ok()
|
||||
pub(crate) fn mutate_scoped(
|
||||
context: &SiteContext,
|
||||
change: impl FnOnce(&mut SiteStore),
|
||||
) -> bool {
|
||||
Self::mutate_result(Some(context), change).is_ok()
|
||||
}
|
||||
|
||||
/// Bind a raw site id to an explicit context at the current durable
|
||||
/// revision. Screens call this before `mutate_scoped` and before site
|
||||
/// queries. Missing, empty, or stale selection is `InvalidScope` —
|
||||
/// never substitution with another site.
|
||||
pub fn scoped_context(site_id: &str) -> Result<SiteContext, StoreFailure> {
|
||||
let runtime = runtime_cell()
|
||||
.lock()
|
||||
.unwrap_or_else(|error| error.into_inner());
|
||||
let sites: Vec<String> = runtime.value.sites.iter().map(|s| s.id.clone()).collect();
|
||||
let revision = runtime
|
||||
.writer
|
||||
.as_ref()
|
||||
.map(|writer| writer.health().accepted_revision)
|
||||
.unwrap_or(0);
|
||||
SiteContext::from_selection(Some(site_id), &sites, revision)
|
||||
.map_err(|_| StoreFailure::InvalidScope)
|
||||
}
|
||||
|
||||
fn profile_mutation_fence(&self) -> Result<Zeroizing<Vec<u8>>, StoreFailure> {
|
||||
|
|
@ -506,7 +543,7 @@ impl SiteStore {
|
|||
}
|
||||
|
||||
fn mutate_result(
|
||||
scope: Option<&str>,
|
||||
scope: Option<&SiteContext>,
|
||||
change: impl FnOnce(&mut SiteStore),
|
||||
) -> Result<u64, StoreFailure> {
|
||||
let mut runtime = runtime_cell()
|
||||
|
|
@ -521,21 +558,34 @@ impl SiteStore {
|
|||
StoreAccess::ReadyEncrypted => unreachable!(),
|
||||
});
|
||||
}
|
||||
if let Some(site_id) = scope {
|
||||
if let Some(context) = scope {
|
||||
let site_id = context.site_id.as_str();
|
||||
if runtime.value.selected_site_id.as_deref() != Some(site_id)
|
||||
|| runtime.value.site(site_id).is_none()
|
||||
{
|
||||
return Err(StoreFailure::InvalidScope);
|
||||
}
|
||||
// Revision compare-and-swap: the caller must have observed the
|
||||
// current durable revision. Stale contexts fail, never overwrite.
|
||||
let accepted = runtime
|
||||
.writer
|
||||
.as_ref()
|
||||
.map(|writer| writer.health().accepted_revision)
|
||||
.unwrap_or(0);
|
||||
if context.base_revision != accepted {
|
||||
return Err(StoreFailure::RevisionConflict);
|
||||
}
|
||||
}
|
||||
let fence_before = match scope {
|
||||
Some(site_id) => runtime.value.site_mutation_fence(site_id)?,
|
||||
Some(context) => runtime
|
||||
.value
|
||||
.site_mutation_fence(context.site_id.as_str())?,
|
||||
None => runtime.value.profile_mutation_fence()?,
|
||||
};
|
||||
let mut candidate = runtime.value.clone();
|
||||
change(&mut candidate);
|
||||
let fence_after = match scope {
|
||||
Some(site_id) => candidate.site_mutation_fence(site_id)?,
|
||||
Some(context) => candidate.site_mutation_fence(context.site_id.as_str())?,
|
||||
None => candidate.profile_mutation_fence()?,
|
||||
};
|
||||
if fence_before != fence_after {
|
||||
|
|
@ -549,6 +599,38 @@ impl SiteStore {
|
|||
.map_err(map_repository);
|
||||
match submit {
|
||||
Ok(revision) => {
|
||||
match scope {
|
||||
Some(context) => {
|
||||
let site = Some(context.site_id.clone());
|
||||
// Only the mutated site's aggregates advance; every
|
||||
// other site keeps its revision (SITE-03 exit).
|
||||
for kind in [
|
||||
AggregateKind::Reports,
|
||||
AggregateKind::Workers,
|
||||
AggregateKind::Tasks,
|
||||
AggregateKind::Procurement,
|
||||
AggregateKind::Meetings,
|
||||
AggregateKind::Reminders,
|
||||
AggregateKind::Directory,
|
||||
] {
|
||||
let _ = runtime.ledger.record_mutation(kind, site.as_ref());
|
||||
}
|
||||
}
|
||||
None => {
|
||||
let _ = runtime
|
||||
.ledger
|
||||
.record_mutation(AggregateKind::ProfileSites, None);
|
||||
let _ = runtime
|
||||
.ledger
|
||||
.record_mutation(AggregateKind::PreferencesLocal, None);
|
||||
// A profile mutation may have created sites; register
|
||||
// their scopes at the new revision without resetting
|
||||
// existing ones.
|
||||
runtime
|
||||
.ledger
|
||||
.ensure_scopes(revision, &scopes_for_store(&candidate));
|
||||
}
|
||||
}
|
||||
runtime.value = candidate;
|
||||
Ok(revision)
|
||||
}
|
||||
|
|
@ -699,22 +781,39 @@ impl SiteStore {
|
|||
self.reports.push(rep);
|
||||
}
|
||||
}
|
||||
pub fn reports_for_site(&self, site_id: &str) -> Vec<&DailyReport> {
|
||||
/// Site-scoped report query. Takes an explicit [`SiteContext`]: without
|
||||
/// one this API does not compile, so no call site can authorize a query
|
||||
/// through an implicit or stale selection.
|
||||
pub fn reports_for_site(&self, context: &SiteContext) -> Vec<&DailyReport> {
|
||||
let site_id = context.site_id.as_str();
|
||||
self.reports
|
||||
.iter()
|
||||
.filter(|r| r.site_id == site_id)
|
||||
.collect()
|
||||
}
|
||||
pub fn report_for_day(&self, site_id: &str, date: NaiveDate) -> Option<&DailyReport> {
|
||||
pub fn report_for_day(&self, context: &SiteContext, date: NaiveDate) -> Option<&DailyReport> {
|
||||
let site_id = context.site_id.as_str();
|
||||
self.reports
|
||||
.iter()
|
||||
.find(|r| r.site_id == site_id && r.date == date)
|
||||
}
|
||||
pub fn reports_for_date(&self, date: NaiveDate) -> Vec<&DailyReport> {
|
||||
self.reports.iter().filter(|r| r.date == date).collect()
|
||||
/// Multi-site compilation helper. The scope is an explicit site list —
|
||||
/// never an implicit "all sites" — so the Overall Supervisor's roll-up
|
||||
/// authorizes exactly the sites it names.
|
||||
pub fn reports_for_date(&self, site_ids: &[String], date: NaiveDate) -> Vec<&DailyReport> {
|
||||
self.reports
|
||||
.iter()
|
||||
.filter(|r| r.date == date && site_ids.contains(&r.site_id))
|
||||
.collect()
|
||||
}
|
||||
pub fn reports_for_month(&self, site_id: &str, year: i32, month: u32) -> Vec<&DailyReport> {
|
||||
pub fn reports_for_month(
|
||||
&self,
|
||||
context: &SiteContext,
|
||||
year: i32,
|
||||
month: u32,
|
||||
) -> Vec<&DailyReport> {
|
||||
use chrono::Datelike;
|
||||
let site_id = context.site_id.as_str();
|
||||
self.reports
|
||||
.iter()
|
||||
.filter(|r| r.site_id == site_id && r.date.year() == year && r.date.month() == month)
|
||||
|
|
@ -754,10 +853,17 @@ impl SiteStore {
|
|||
pub fn push_reminder(&mut self, r: Reminder) {
|
||||
self.reminders.push(r);
|
||||
}
|
||||
pub fn due_reminders(&self, now: chrono::DateTime<chrono::Utc>) -> Vec<&Reminder> {
|
||||
/// Due reminders for one explicit site context. Reminders without a site
|
||||
/// binding are quarantined (see `aggregates`) and never surface here.
|
||||
pub fn due_reminders_for(
|
||||
&self,
|
||||
context: &SiteContext,
|
||||
now: chrono::DateTime<chrono::Utc>,
|
||||
) -> Vec<&Reminder> {
|
||||
let site_id = context.site_id.as_str();
|
||||
self.reminders
|
||||
.iter()
|
||||
.filter(|r| !r.fired && r.fire_at <= now)
|
||||
.filter(|r| !r.fired && r.fire_at <= now && r.site_id.as_deref() == Some(site_id))
|
||||
.collect()
|
||||
}
|
||||
/// Mark a candidate revision only; persistence is exclusively owned by
|
||||
|
|
@ -768,11 +874,11 @@ impl SiteStore {
|
|||
}
|
||||
}
|
||||
|
||||
// ---- Compiled view helpers (item 5) ----
|
||||
pub fn compiled_daily(&self, date: NaiveDate) -> Vec<DailyReport> {
|
||||
// ---- Compiled view helpers (explicit multi-site scope) ----
|
||||
pub fn compiled_daily(&self, site_ids: &[String], date: NaiveDate) -> Vec<DailyReport> {
|
||||
self.reports
|
||||
.iter()
|
||||
.filter(|r| r.date == date)
|
||||
.filter(|r| r.date == date && site_ids.contains(&r.site_id))
|
||||
.cloned()
|
||||
.collect()
|
||||
}
|
||||
|
|
@ -786,7 +892,8 @@ impl SiteStore {
|
|||
self.tasks.push(task);
|
||||
}
|
||||
}
|
||||
pub fn tasks_for_site(&self, site_id: &str) -> Vec<&ConstructionTask> {
|
||||
pub fn tasks_for_site(&self, context: &SiteContext) -> Vec<&ConstructionTask> {
|
||||
let site_id = context.site_id.as_str();
|
||||
self.tasks.iter().filter(|t| t.site_id == site_id).collect()
|
||||
}
|
||||
/// Remove from a candidate revision only. The caller must use
|
||||
|
|
@ -828,7 +935,8 @@ impl SiteStore {
|
|||
self.procurement.push(sched);
|
||||
}
|
||||
}
|
||||
pub fn procurement_for(&self, site_id: &str) -> Option<&ProcurementSchedule> {
|
||||
pub fn procurement_for(&self, context: &SiteContext) -> Option<&ProcurementSchedule> {
|
||||
let site_id = context.site_id.as_str();
|
||||
self.procurement.iter().find(|p| p.site_id == site_id)
|
||||
}
|
||||
|
||||
|
|
@ -853,13 +961,15 @@ impl SiteStore {
|
|||
self.meetings.push(m);
|
||||
}
|
||||
}
|
||||
pub fn meetings_for(&self, site_id: &str) -> Vec<&SiteMeeting> {
|
||||
pub fn meetings_for(&self, context: &SiteContext) -> Vec<&SiteMeeting> {
|
||||
let site_id = context.site_id.as_str();
|
||||
self.meetings
|
||||
.iter()
|
||||
.filter(|m| m.site_id == site_id)
|
||||
.collect()
|
||||
}
|
||||
pub fn directory_for(&self, site_id: &str) -> Option<&ProjectDirectory> {
|
||||
pub fn directory_for(&self, context: &SiteContext) -> Option<&ProjectDirectory> {
|
||||
let site_id = context.site_id.as_str();
|
||||
self.directories.iter().find(|d| d.site_id == site_id)
|
||||
}
|
||||
/// Append without persisting — for use inside the scoped mutation boundary.
|
||||
|
|
@ -884,12 +994,12 @@ impl SiteStore {
|
|||
}
|
||||
}
|
||||
/// Directory view: stored contacts plus every attendee ever scheduled.
|
||||
pub fn directory_names(&self, site_id: &str) -> Vec<String> {
|
||||
pub fn directory_names(&self, context: &SiteContext) -> Vec<String> {
|
||||
let mut names: Vec<String> = self
|
||||
.directory_for(site_id)
|
||||
.directory_for(context)
|
||||
.map(|d| d.contacts.iter().map(|c| c.display_name.clone()).collect())
|
||||
.unwrap_or_default();
|
||||
for m in self.meetings_for(site_id) {
|
||||
for m in self.meetings_for(context) {
|
||||
for a in &m.attendees {
|
||||
if !names
|
||||
.iter()
|
||||
|
|
@ -912,6 +1022,12 @@ pub(crate) static TEST_ENV_LOCK: std::sync::OnceLock<std::sync::Mutex<()>> =
|
|||
mod tests {
|
||||
use super::*;
|
||||
use crate::crypto::{EnvelopeHeader, KeyId, OsRandom, StoreId};
|
||||
use crate::ids::SiteId;
|
||||
use crate::site_context::SiteContext;
|
||||
|
||||
fn context_for(site_id: &str, base_revision: u64) -> SiteContext {
|
||||
SiteContext::new(SiteId::parse(site_id).unwrap(), base_revision)
|
||||
}
|
||||
|
||||
struct TestEnvironment {
|
||||
path: PathBuf,
|
||||
|
|
@ -1142,9 +1258,12 @@ mod tests {
|
|||
let (store, site_id) = selected_store();
|
||||
environment.write_store(&store, 1);
|
||||
environment.reopen();
|
||||
assert!(SiteStore::mutate_scoped(&site_id, |candidate| {
|
||||
candidate.push_task(ConstructionTask::new(&site_id, "Worker ID 12345678"));
|
||||
}));
|
||||
assert!(SiteStore::mutate_scoped(
|
||||
&context_for(&site_id, 1),
|
||||
|candidate| {
|
||||
candidate.push_task(ConstructionTask::new(&site_id, "Worker ID 12345678"));
|
||||
}
|
||||
));
|
||||
let accepted = SiteStore::persistence_health().accepted_revision;
|
||||
assert_eq!(accepted, 2);
|
||||
assert!(SiteStore::flush_writer_queue(10_000));
|
||||
|
|
@ -1167,9 +1286,12 @@ mod tests {
|
|||
environment.write_store(&store, 1);
|
||||
environment.reopen();
|
||||
let called = AtomicBool::new(false);
|
||||
assert!(!SiteStore::mutate_scoped("wrong-site", |_| {
|
||||
called.store(true, Ordering::SeqCst);
|
||||
}));
|
||||
assert!(!SiteStore::mutate_scoped(
|
||||
&context_for("wrong-site", 1),
|
||||
|_| {
|
||||
called.store(true, Ordering::SeqCst);
|
||||
}
|
||||
));
|
||||
assert!(!called.load(Ordering::SeqCst));
|
||||
assert_eq!(SiteStore::persistence_health().accepted_revision, 1);
|
||||
}
|
||||
|
|
@ -1184,9 +1306,12 @@ mod tests {
|
|||
environment.write_store(&store, 1);
|
||||
environment.reopen();
|
||||
|
||||
assert!(!SiteStore::mutate_scoped(&selected_id, |candidate| {
|
||||
candidate.push_task(ConstructionTask::new(&other_id, "wrong site"));
|
||||
}));
|
||||
assert!(!SiteStore::mutate_scoped(
|
||||
&context_for(&selected_id, 1),
|
||||
|candidate| {
|
||||
candidate.push_task(ConstructionTask::new(&other_id, "wrong site"));
|
||||
}
|
||||
));
|
||||
assert!(!SiteStore::mutate_profile(|candidate| {
|
||||
candidate.push_task(ConstructionTask::new(&selected_id, "not profile data"));
|
||||
}));
|
||||
|
|
@ -1206,9 +1331,12 @@ mod tests {
|
|||
let _ = std::fs::remove_dir_all(&preserved);
|
||||
std::fs::rename(&parent, &preserved).unwrap();
|
||||
|
||||
assert!(SiteStore::mutate_scoped(&site_id, |candidate| {
|
||||
candidate.push_task(ConstructionTask::new(&site_id, "unsaved"));
|
||||
}));
|
||||
assert!(SiteStore::mutate_scoped(
|
||||
&context_for(&site_id, 1),
|
||||
|candidate| {
|
||||
candidate.push_task(ConstructionTask::new(&site_id, "unsaved"));
|
||||
}
|
||||
));
|
||||
assert!(!SiteStore::flush_writer_queue(10_000));
|
||||
let health = SiteStore::persistence_health();
|
||||
assert!(health.has_unsaved_changes());
|
||||
|
|
@ -1236,7 +1364,8 @@ mod tests {
|
|||
environment.write_store(&store, 4);
|
||||
environment.reopen();
|
||||
for index in 0..30 {
|
||||
assert!(SiteStore::mutate_scoped(&site_id, |candidate| {
|
||||
let context = SiteStore::scoped_context(&site_id).expect("live explicit context");
|
||||
assert!(SiteStore::mutate_scoped(&context, |candidate| {
|
||||
candidate.push_task(ConstructionTask::new(&site_id, format!("task-{index:02}")));
|
||||
}));
|
||||
assert!(SiteStore::persistence_health().pending_depth <= 1);
|
||||
|
|
@ -1266,6 +1395,36 @@ mod tests {
|
|||
let august = NaiveDate::from_ymd_opt(2026, 8, 7).unwrap();
|
||||
store.reports.push(DailyReport::new(&second_id, september));
|
||||
store.reports.push(DailyReport::new(&second_id, august));
|
||||
assert_eq!(store.reports_for_month(&second_id, 2026, 9).len(), 1);
|
||||
assert_eq!(
|
||||
store
|
||||
.reports_for_month(&context_for(&second_id, 0), 2026, 9)
|
||||
.len(),
|
||||
1
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stale_revision_and_missing_selection_fail_without_invoking_mutation() {
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
|
||||
let environment = TestEnvironment::new("stale-rev");
|
||||
let (store, site_id) = selected_store();
|
||||
environment.write_store(&store, 1);
|
||||
environment.reopen();
|
||||
// A stale base revision fails compare-and-swap before the closure.
|
||||
let stale = context_for(&site_id, 0);
|
||||
let called = AtomicBool::new(false);
|
||||
assert!(!SiteStore::mutate_scoped(&stale, |_| {
|
||||
called.store(true, Ordering::SeqCst);
|
||||
}));
|
||||
assert!(!called.load(Ordering::SeqCst));
|
||||
assert_eq!(SiteStore::persistence_health().accepted_revision, 1);
|
||||
// The live helper binds the current revision; unknown ids fail.
|
||||
let live = SiteStore::scoped_context(&site_id).expect("live context");
|
||||
assert_eq!(live.base_revision, 1);
|
||||
assert!(SiteStore::scoped_context("ghost-site").is_err());
|
||||
assert!(SiteStore::mutate_scoped(&live, |candidate| {
|
||||
candidate.push_task(ConstructionTask::new(&site_id, "live"));
|
||||
}));
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -3,12 +3,35 @@ name = "nigig_doc_scanner"
|
|||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
|
||||
# Feature layout (see SITE-14 in `nigig-site/EXECUTION_PLAN.md`):
|
||||
# - `app`: full scanner application — Makepad UI, nigig-core/uikit services,
|
||||
# device location. Needed only to run the scanner binary itself.
|
||||
# - `ocr`: on-device text recognition — pure Rust plus `image` buffers only.
|
||||
# No UI, no location, no network, no model download. Downstream device
|
||||
# crates (e.g. `nigig-site`) enable `ocr` with `default-features = false`
|
||||
# so recognition never drags the app shell into their production graph.
|
||||
# - `default = ["app", "ocr"]`: building/running the scanner app itself.
|
||||
[features]
|
||||
default = ["app", "ocr"]
|
||||
app = [
|
||||
"dep:makepad-widgets",
|
||||
"dep:nigig-core",
|
||||
"dep:nigig-uikit",
|
||||
"dep:robius-location",
|
||||
]
|
||||
ocr = []
|
||||
|
||||
[[bin]]
|
||||
name = "nigig_doc_scanner"
|
||||
path = "src/main.rs"
|
||||
required-features = ["app"]
|
||||
|
||||
[dependencies]
|
||||
makepad-widgets = { workspace = true }
|
||||
nigig-core = { path = "../../nigig-core" }
|
||||
nigig-uikit = { path = "../../nigig-uikit" }
|
||||
makepad-widgets = { workspace = true, optional = true }
|
||||
nigig-core = { path = "../../nigig-core", optional = true }
|
||||
nigig-uikit = { path = "../../nigig-uikit", optional = true }
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
chrono = { version = "0.4", features = ["serde"] }
|
||||
robius-location = { git = "https://github.com/project-robius/robius", rev = "b766e62b0600f5d2ee21cc6995648346fc277bd8" }
|
||||
robius-location = { git = "https://github.com/project-robius/robius", rev = "b766e62b0600f5d2ee21cc6995648346fc277bd8", optional = true }
|
||||
image = { version = "0.25", default-features = false, features = ["png", "jpeg"] }
|
||||
|
|
|
|||
|
|
@ -1,8 +1,13 @@
|
|||
#[cfg(feature = "app")]
|
||||
use makepad_widgets::ScriptVm;
|
||||
|
||||
#[cfg(feature = "app")]
|
||||
pub mod scanner_frame;
|
||||
pub mod scanner_core;
|
||||
#[cfg(feature = "ocr")]
|
||||
pub mod ocr;
|
||||
|
||||
#[cfg(feature = "app")]
|
||||
pub fn script_mod(vm: &mut ScriptVm) {
|
||||
nigig_uikit::script_mod(vm);
|
||||
scanner_frame::script_mod(vm);
|
||||
|
|
@ -10,8 +15,13 @@ pub fn script_mod(vm: &mut ScriptVm) {
|
|||
|
||||
|
||||
// Compatibility shims for source moved out of pageflipnav during staged migration.
|
||||
// These belong to the app shell and stay behind the `app` feature so the `ocr`
|
||||
// feature alone (used by downstream device crates) never links UI/location.
|
||||
#[cfg(feature = "app")]
|
||||
pub mod dir { pub use nigig_core::dir::*; }
|
||||
#[cfg(feature = "app")]
|
||||
pub mod shared { pub use nigig_uikit::shared::*; }
|
||||
#[cfg(feature = "app")]
|
||||
pub mod persistence {
|
||||
pub use nigig_core::persistence::*;
|
||||
pub mod offline_store { pub use nigig_core::persistence::offline_store::*; }
|
||||
|
|
@ -19,13 +29,15 @@ pub mod persistence {
|
|||
#[cfg(not(target_arch = "wasm32"))]
|
||||
pub mod matrix_state { pub use nigig_core::persistence::matrix_state::*; }
|
||||
}
|
||||
#[cfg(feature = "app")]
|
||||
pub mod features {
|
||||
pub mod action_page_navigation { pub use nigig_uikit::action_page_navigation::*; }
|
||||
}
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
#[cfg(all(feature = "app", not(target_arch = "wasm32")))]
|
||||
pub mod tile_service { pub use nigig_core::tile_service::*; }
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
#[cfg(all(feature = "app", not(target_arch = "wasm32")))]
|
||||
pub mod location { pub use nigig_core::location::*; }
|
||||
#[cfg(feature = "app")]
|
||||
pub mod home {
|
||||
pub mod navigation_tab_bar {
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
|
|
|
|||
881
crates/apps/nigig_doc_scanner/src/ocr.rs
Normal file
881
crates/apps/nigig_doc_scanner/src/ocr.rs
Normal file
|
|
@ -0,0 +1,881 @@
|
|||
//! Functional on-device text recognition owned by the scanner crate.
|
||||
//!
|
||||
//! Scope contract (SITE-14, `nigig-site/EXECUTION_PLAN.md`):
|
||||
//!
|
||||
//! - This is the ONLY recognition engine `nigig-site` may call, via this
|
||||
//! module behind the `ocr` cargo feature. It is pure Rust plus `image`
|
||||
//! buffers: no UI, no location, no network, no model download, no new
|
||||
//! third-party dependency. Downstream crates enable
|
||||
//! `nigig_doc_scanner/ocr` with `default-features = false`.
|
||||
//! - `nigig-pdf-document`'s `ocr` module is an engine-free PDF text-layer
|
||||
//! seam and must never be pulled in for recognition.
|
||||
//! - Coverage is exactly the embedded glyph set: ASCII `0-9` and uppercase
|
||||
//! Latin `A-Z` (ID serials and block-capitals names). Anything else lowers
|
||||
//! confidence and yields `None`, never invented text.
|
||||
//! - Every suggestion requires user confirmation in the calling crate; the
|
||||
//! engine never approves identity.
|
||||
//!
|
||||
//! Pipeline: raw 8-bit grayscale → budget preflight (checked arithmetic,
|
||||
//! BEFORE allocation) → Otsu binarization → budgeted connected-component
|
||||
//! labeling → line/word grouping → 5x7 normalization → embedded-template
|
||||
//! match with confidence floor and runner-up margin.
|
||||
|
||||
use image::{DynamicImage, GrayImage};
|
||||
|
||||
/// Why recognition failed. `Ok` with an empty layer means "no text found",
|
||||
/// which callers must render as manual entry, never as an error.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub enum OcrError {
|
||||
/// No engine configured (callers without the `ocr` feature path).
|
||||
NoEngine,
|
||||
/// Dimensions are zero, the buffer length disagrees, or pixels are blank.
|
||||
InvalidImage(String),
|
||||
/// A §7 budget stopped work before/while allocating.
|
||||
BudgetExceeded(String),
|
||||
/// The engine refused the request deterministically.
|
||||
Engine(String),
|
||||
}
|
||||
|
||||
impl std::fmt::Display for OcrError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
OcrError::NoEngine => write!(f, "no OCR engine configured"),
|
||||
OcrError::InvalidImage(s) => write!(f, "invalid image for OCR: {s}"),
|
||||
OcrError::BudgetExceeded(s) => write!(f, "OCR budget exceeded: {s}"),
|
||||
OcrError::Engine(s) => write!(f, "OCR engine error: {s}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for OcrError {}
|
||||
|
||||
/// One recognized word with its box normalized to 0..1 in image space
|
||||
/// (the same convention as the PDF text-layer seam).
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct OcrWord {
|
||||
pub text: String,
|
||||
/// `[x0, y0, x1, y1]` normalized, upper-left origin in image space.
|
||||
pub rect: [f64; 4],
|
||||
pub confidence: f32,
|
||||
}
|
||||
|
||||
/// All recognition results for one image.
|
||||
#[derive(Clone, Debug, Default, PartialEq)]
|
||||
pub struct OcrLayer {
|
||||
pub words: Vec<OcrWord>,
|
||||
}
|
||||
|
||||
impl OcrLayer {
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.words.is_empty()
|
||||
}
|
||||
|
||||
pub fn len(&self) -> usize {
|
||||
self.words.len()
|
||||
}
|
||||
|
||||
pub fn text(&self) -> String {
|
||||
self.words
|
||||
.iter()
|
||||
.map(|w| w.text.as_str())
|
||||
.collect::<Vec<_>>()
|
||||
.join(" ")
|
||||
}
|
||||
|
||||
pub fn mean_confidence(&self) -> f32 {
|
||||
if self.words.is_empty() {
|
||||
return 0.0;
|
||||
}
|
||||
self.words.iter().map(|w| w.confidence).sum::<f32>() / self.words.len() as f32
|
||||
}
|
||||
}
|
||||
|
||||
/// Recognition engine seam. Implementations take raw 8-bit grayscale bytes so
|
||||
/// callers without an `image` dependency (e.g. `nigig-site` production) can
|
||||
/// use them.
|
||||
pub trait OcrEngine: Send + Sync {
|
||||
fn recognize(
|
||||
&self,
|
||||
image: &[u8],
|
||||
width: u32,
|
||||
height: u32,
|
||||
) -> Result<Vec<OcrWord>, OcrError>;
|
||||
}
|
||||
|
||||
/// An engine that always refuses, for hosts without OCR.
|
||||
#[derive(Clone, Copy, Debug, Default)]
|
||||
pub struct NoopOcr;
|
||||
|
||||
impl OcrEngine for NoopOcr {
|
||||
fn recognize(&self, _image: &[u8], _width: u32, _height: u32) -> Result<Vec<OcrWord>, OcrError> {
|
||||
Err(OcrError::NoEngine)
|
||||
}
|
||||
}
|
||||
|
||||
/// Hard budgets. Defaults mirror §7 (20 MiB encoded / 8,192 edge / 40 MP).
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
pub struct OcrLimits {
|
||||
pub max_pixels: u64,
|
||||
pub max_edge: u32,
|
||||
pub max_components: usize,
|
||||
pub max_words: usize,
|
||||
/// Suggestions below this confidence are dropped (`None` downstream).
|
||||
pub min_confidence: f32,
|
||||
}
|
||||
|
||||
impl OcrLimits {
|
||||
pub fn id() -> Self {
|
||||
Self {
|
||||
max_pixels: 40_000_000,
|
||||
max_edge: 8_192,
|
||||
max_components: 100_000,
|
||||
max_words: 1_024,
|
||||
min_confidence: 0.80,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn preflight(len: usize, width: u32, height: u32, limits: &OcrLimits) -> Result<usize, OcrError> {
|
||||
if width == 0 || height == 0 {
|
||||
return Err(OcrError::InvalidImage("zero dimension".to_string()));
|
||||
}
|
||||
if width > limits.max_edge || height > limits.max_edge {
|
||||
return Err(OcrError::BudgetExceeded("edge".to_string()));
|
||||
}
|
||||
let pixels = (u64::from(width))
|
||||
.checked_mul(u64::from(height))
|
||||
.ok_or_else(|| OcrError::BudgetExceeded("pixel-overflow".to_string()))?;
|
||||
if pixels > limits.max_pixels {
|
||||
return Err(OcrError::BudgetExceeded("pixels".to_string()));
|
||||
}
|
||||
let pixels_usize = usize::try_from(pixels)
|
||||
.map_err(|_| OcrError::BudgetExceeded("pixel-address".to_string()))?;
|
||||
if len != pixels_usize {
|
||||
return Err(OcrError::InvalidImage("length-mismatch".to_string()));
|
||||
}
|
||||
Ok(pixels_usize)
|
||||
}
|
||||
|
||||
fn otsu_threshold(hist: &[u64; 256], total: u64) -> u8 {
|
||||
let mut sum_all: u64 = 0;
|
||||
for (i, count) in hist.iter().enumerate() {
|
||||
sum_all += i as u64 * count;
|
||||
}
|
||||
let mut sum_bg: u64 = 0;
|
||||
let mut weight_bg: u64 = 0;
|
||||
let mut best = 0u8;
|
||||
let mut best_var: u128 = 0;
|
||||
for t in 0..256 {
|
||||
weight_bg += hist[t];
|
||||
if weight_bg == 0 {
|
||||
continue;
|
||||
}
|
||||
let weight_fg = total - weight_bg;
|
||||
if weight_fg == 0 {
|
||||
break;
|
||||
}
|
||||
sum_bg += t as u64 * hist[t];
|
||||
let mean_bg = sum_bg as f64 / weight_bg as f64;
|
||||
let mean_fg = (sum_all - sum_bg) as f64 / weight_fg as f64;
|
||||
let var = weight_bg as f64 * weight_fg as f64 * (mean_bg - mean_fg).powi(2);
|
||||
if (var as u128) > best_var {
|
||||
best_var = var as u128;
|
||||
best = t as u8;
|
||||
}
|
||||
}
|
||||
best
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
struct Component {
|
||||
min_x: u32,
|
||||
min_y: u32,
|
||||
max_x: u32,
|
||||
max_y: u32,
|
||||
}
|
||||
|
||||
impl Component {
|
||||
fn width(&self) -> u32 {
|
||||
self.max_x - self.min_x + 1
|
||||
}
|
||||
|
||||
fn height(&self) -> u32 {
|
||||
self.max_y - self.min_y + 1
|
||||
}
|
||||
|
||||
fn center_y(&self) -> f64 {
|
||||
(f64::from(self.min_y) + f64::from(self.max_y)) / 2.0
|
||||
}
|
||||
}
|
||||
|
||||
/// Budgeted connected-component labeling over dark (text) pixels.
|
||||
fn label_components(
|
||||
dark: &[bool],
|
||||
width: u32,
|
||||
height: u32,
|
||||
limits: &OcrLimits,
|
||||
) -> Result<Vec<Component>, OcrError> {
|
||||
let pixels = dark.len();
|
||||
let mut visited = vec![false; pixels];
|
||||
let mut components = Vec::new();
|
||||
let mut stack = Vec::new();
|
||||
let w = width as usize;
|
||||
|
||||
for start in 0..pixels {
|
||||
if !dark[start] || visited[start] {
|
||||
continue;
|
||||
}
|
||||
if components.len() >= limits.max_components {
|
||||
return Err(OcrError::BudgetExceeded("components".to_string()));
|
||||
}
|
||||
let mut min_x = u32::MAX;
|
||||
let mut min_y = u32::MAX;
|
||||
let mut max_x = 0u32;
|
||||
let mut max_y = 0u32;
|
||||
let mut area: u64 = 0;
|
||||
stack.clear();
|
||||
stack.push(start);
|
||||
visited[start] = true;
|
||||
while let Some(idx) = stack.pop() {
|
||||
let x = (idx % w) as u32;
|
||||
let y = (idx / w) as u32;
|
||||
if x < min_x {
|
||||
min_x = x;
|
||||
}
|
||||
if y < min_y {
|
||||
min_y = y;
|
||||
}
|
||||
if x > max_x {
|
||||
max_x = x;
|
||||
}
|
||||
if y > max_y {
|
||||
max_y = y;
|
||||
}
|
||||
area += 1;
|
||||
// 8-neighbourhood: glyph strokes join diagonally (the slash in
|
||||
// "0"/"Q", the apices of "XVWKMN"). The inter-glyph quiet gap is
|
||||
// wider than one pixel in every direction, so neighbours never
|
||||
// merge through it.
|
||||
for oy in [-1i32, 0, 1] {
|
||||
for ox in [-1i32, 0, 1] {
|
||||
if ox == 0 && oy == 0 {
|
||||
continue;
|
||||
}
|
||||
let nx = x as i32 + ox;
|
||||
let ny = y as i32 + oy;
|
||||
if nx < 0 || ny < 0 || nx >= width as i32 || ny >= height as i32 {
|
||||
continue;
|
||||
}
|
||||
let n = ny as usize * w + nx as usize;
|
||||
if dark[n] && !visited[n] {
|
||||
visited[n] = true;
|
||||
stack.push(n);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
let candidate = Component {
|
||||
min_x,
|
||||
min_y,
|
||||
max_x,
|
||||
max_y,
|
||||
};
|
||||
// Noise and frame rejection: keep plausible glyph blobs only. A blob
|
||||
// covering more than a quarter of the frame is a border, not a glyph.
|
||||
let bbox_area = u64::from(candidate.width()) * u64::from(candidate.height());
|
||||
if area >= 12
|
||||
&& candidate.width() >= 2
|
||||
&& candidate.height() >= 2
|
||||
&& bbox_area <= pixels as u64 / 4
|
||||
{
|
||||
components.push(candidate);
|
||||
}
|
||||
}
|
||||
Ok(components)
|
||||
}
|
||||
|
||||
// --- Embedded 5x7 glyph set: 0-9, A-Z. `1` = ink. --------------------------
|
||||
|
||||
const GLYPH_W: usize = 5;
|
||||
const GLYPH_H: usize = 7;
|
||||
|
||||
#[allow(clippy::too_many_lines)]
|
||||
fn glyph_rows(ch: char) -> Option<[&'static str; 7]> {
|
||||
match ch {
|
||||
'0' => Some(["01110", "10001", "10011", "10101", "11001", "10001", "01110"]),
|
||||
'1' => Some(["00100", "01100", "00100", "00100", "00100", "00100", "01110"]),
|
||||
'2' => Some(["01110", "10001", "00001", "00010", "00100", "01000", "11111"]),
|
||||
'3' => Some(["11111", "00010", "00100", "00010", "00001", "10001", "01110"]),
|
||||
'4' => Some(["00010", "00110", "01010", "10010", "11111", "00010", "00010"]),
|
||||
'5' => Some(["11111", "10000", "11110", "00001", "00001", "10001", "01110"]),
|
||||
'6' => Some(["00110", "01000", "10000", "11110", "10001", "10001", "01110"]),
|
||||
'7' => Some(["11111", "00001", "00010", "00100", "01000", "01000", "01000"]),
|
||||
'8' => Some(["01110", "10001", "10001", "01110", "10001", "10001", "01110"]),
|
||||
'9' => Some(["01110", "10001", "10001", "01111", "00001", "00010", "01100"]),
|
||||
'A' => Some(["01110", "10001", "10001", "11111", "10001", "10001", "10001"]),
|
||||
'B' => Some(["11110", "10001", "10001", "11110", "10001", "10001", "11110"]),
|
||||
'C' => Some(["01110", "10001", "10000", "10000", "10000", "10001", "01110"]),
|
||||
'D' => Some(["11110", "10001", "10001", "10001", "10001", "10001", "11110"]),
|
||||
'E' => Some(["11111", "10000", "10000", "11110", "10000", "10000", "11111"]),
|
||||
'F' => Some(["11111", "10000", "10000", "11110", "10000", "10000", "10000"]),
|
||||
'G' => Some(["01110", "10001", "10000", "10111", "10001", "10001", "01111"]),
|
||||
'H' => Some(["10001", "10001", "10001", "11111", "10001", "10001", "10001"]),
|
||||
'I' => Some(["01110", "00100", "00100", "00100", "00100", "00100", "01110"]),
|
||||
'J' => Some(["00111", "00010", "00010", "00010", "00010", "10010", "01100"]),
|
||||
'K' => Some(["10001", "10010", "10100", "11000", "10100", "10010", "10001"]),
|
||||
'L' => Some(["10000", "10000", "10000", "10000", "10000", "10000", "11111"]),
|
||||
'M' => Some(["10001", "11011", "10101", "10101", "10001", "10001", "10001"]),
|
||||
'N' => Some(["10001", "11001", "11001", "10101", "10011", "10011", "10001"]),
|
||||
'O' => Some(["01110", "10001", "10001", "10001", "10001", "10001", "01110"]),
|
||||
'P' => Some(["11110", "10001", "10001", "11110", "10000", "10000", "10000"]),
|
||||
'Q' => Some(["01110", "10001", "10001", "10001", "10101", "10010", "01101"]),
|
||||
'R' => Some(["11110", "10001", "10001", "11110", "10100", "10010", "10001"]),
|
||||
'S' => Some(["01111", "10000", "10000", "01110", "00001", "00001", "11110"]),
|
||||
'T' => Some(["11111", "00100", "00100", "00100", "00100", "00100", "00100"]),
|
||||
'U' => Some(["10001", "10001", "10001", "10001", "10001", "10001", "01110"]),
|
||||
'V' => Some(["10001", "10001", "10001", "10001", "10001", "01010", "00100"]),
|
||||
'W' => Some(["10001", "10001", "10001", "10101", "10101", "11011", "10001"]),
|
||||
'X' => Some(["10001", "10001", "01010", "00100", "01010", "10001", "10001"]),
|
||||
'Y' => Some(["10001", "10001", "01010", "00100", "00100", "00100", "00100"]),
|
||||
'Z' => Some(["11111", "00001", "00010", "00100", "01000", "10000", "11111"]),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn glyph_bits(ch: char) -> Option<[bool; 35]> {
|
||||
let rows = glyph_rows(ch)?;
|
||||
let mut bits = [false; 35];
|
||||
for (r, row) in rows.iter().enumerate() {
|
||||
for (c, b) in row.bytes().enumerate().take(GLYPH_W) {
|
||||
bits[r * GLYPH_W + c] = b == b'1';
|
||||
}
|
||||
}
|
||||
Some(bits)
|
||||
}
|
||||
|
||||
/// Normalize one component to 5x7 by aspect-preserving area sampling, then
|
||||
/// template-match. The ink bbox is padded symmetrically to the exact 5:7
|
||||
/// cell aspect first, so narrow glyphs ("1", "I") keep their proportions
|
||||
/// instead of stretching; every source pixel contributes to exactly one
|
||||
/// cell by pixel-center mapping. Returns `(char, confidence)` or `None`
|
||||
/// below the floor/margin.
|
||||
fn match_component(
|
||||
dark: &[bool],
|
||||
width: u32,
|
||||
height: u32,
|
||||
component: &Component,
|
||||
min_confidence: f32,
|
||||
) -> Option<(char, f32)> {
|
||||
let bw = component.width();
|
||||
let bh = component.height();
|
||||
// Pad to 5:7 aspect with integer arithmetic, clamped to the frame.
|
||||
// `total_pad_*` is split into a floor-half and the remainder so the
|
||||
// region stays centered; clamping only kicks in for glyphs touching
|
||||
// the frame edge (degenerate input, documented).
|
||||
let target_w = (u64::from(bh) * 5 + 6) / 7;
|
||||
let target_h = (u64::from(bw) * 7 + 4) / 5;
|
||||
let total_pad_x = target_w.saturating_sub(u64::from(bw));
|
||||
let total_pad_y = target_h.saturating_sub(u64::from(bh));
|
||||
let left_pad = total_pad_x / 2;
|
||||
let right_pad = total_pad_x - left_pad;
|
||||
let top_pad = total_pad_y / 2;
|
||||
let bottom_pad = total_pad_y - top_pad;
|
||||
let x0 = component
|
||||
.min_x
|
||||
.saturating_sub(left_pad.min(u64::from(component.min_x)) as u32);
|
||||
let y0 = component
|
||||
.min_y
|
||||
.saturating_sub(top_pad.min(u64::from(component.min_y)) as u32);
|
||||
let x1 = component.max_x.saturating_add(
|
||||
right_pad
|
||||
.min(u64::from(width.saturating_sub(1).saturating_sub(component.max_x)))
|
||||
as u32,
|
||||
);
|
||||
let y1 = component.max_y.saturating_add(
|
||||
bottom_pad
|
||||
.min(u64::from(height.saturating_sub(1).saturating_sub(component.max_y)))
|
||||
as u32,
|
||||
);
|
||||
if x1 <= x0 || y1 <= y0 {
|
||||
return None;
|
||||
}
|
||||
let region_w = (x1 - x0 + 1) as f64;
|
||||
let region_h = (y1 - y0 + 1) as f64;
|
||||
|
||||
let mut ink = [0u64; 35];
|
||||
let mut total = [0u64; 35];
|
||||
for y in y0..=y1 {
|
||||
for x in x0..=x1 {
|
||||
let cx = ((((x - x0) as f64 + 0.5) * GLYPH_W as f64) / region_w) as usize;
|
||||
let cy = ((((y - y0) as f64 + 0.5) * GLYPH_H as f64) / region_h) as usize;
|
||||
let (cx, cy) = (cx.min(GLYPH_W - 1), cy.min(GLYPH_H - 1));
|
||||
let i = cy * GLYPH_W + cx;
|
||||
total[i] += 1;
|
||||
if dark[(y * width + x) as usize] {
|
||||
ink[i] += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
let mut sample = [false; 35];
|
||||
let mut ink_cells = 0usize;
|
||||
for i in 0..35 {
|
||||
let cell_ink = total[i] > 0 && ink[i] * 2 >= total[i];
|
||||
sample[i] = cell_ink;
|
||||
if cell_ink {
|
||||
ink_cells += 1;
|
||||
}
|
||||
}
|
||||
// Degenerate blobs (specks, solid blocks) are not glyphs.
|
||||
if ink_cells < 3 || ink_cells > 32 {
|
||||
return None;
|
||||
}
|
||||
|
||||
let mut best = ('?', 0.0f32);
|
||||
let mut second = 0.0f32;
|
||||
for ch in "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ".chars() {
|
||||
let template = glyph_bits(ch)?;
|
||||
let mut same = 0usize;
|
||||
for i in 0..35 {
|
||||
if sample[i] == template[i] {
|
||||
same += 1;
|
||||
}
|
||||
}
|
||||
let score = same as f32 / 35.0;
|
||||
if score > best.1 {
|
||||
second = best.1;
|
||||
best = (ch, score);
|
||||
} else if score > second {
|
||||
second = score;
|
||||
}
|
||||
}
|
||||
// Floor plus runner-up margin: ambiguous shapes fall back to manual entry.
|
||||
if best.1 >= min_confidence && best.1 - second >= 0.03 {
|
||||
Some(best)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// On-device template-matching reader. See the module docs for the honesty
|
||||
/// contract: fixed glyph set, confidence floor, confirmation required.
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
pub struct TemplateOcr {
|
||||
limits: OcrLimits,
|
||||
}
|
||||
|
||||
impl TemplateOcr {
|
||||
pub fn new(limits: OcrLimits) -> Self {
|
||||
Self { limits }
|
||||
}
|
||||
|
||||
pub fn id_reader() -> Self {
|
||||
Self::new(OcrLimits::id())
|
||||
}
|
||||
|
||||
pub fn recognize_layer(
|
||||
&self,
|
||||
gray: &[u8],
|
||||
width: u32,
|
||||
height: u32,
|
||||
) -> Result<OcrLayer, OcrError> {
|
||||
let pixels = preflight(gray.len(), width, height, &self.limits)?;
|
||||
|
||||
let mut hist = [0u64; 256];
|
||||
for pixel in gray {
|
||||
hist[*pixel as usize] += 1;
|
||||
}
|
||||
// A blank frame has no text; report empty, not an error.
|
||||
let distinct = hist.iter().filter(|count| **count > 0).count();
|
||||
if distinct < 2 {
|
||||
return Ok(OcrLayer::default());
|
||||
}
|
||||
let threshold = otsu_threshold(&hist, pixels as u64);
|
||||
let dark: Vec<bool> = gray.iter().map(|p| *p <= threshold).collect();
|
||||
|
||||
let mut components = label_components(&dark, width, height, &self.limits)?;
|
||||
if components.is_empty() {
|
||||
return Ok(OcrLayer::default());
|
||||
}
|
||||
components.sort_by(|a, b| {
|
||||
a.center_y()
|
||||
.partial_cmp(&b.center_y())
|
||||
.unwrap_or(std::cmp::Ordering::Equal)
|
||||
.then_with(|| a.min_x.cmp(&b.min_x))
|
||||
});
|
||||
|
||||
// Group into lines by vertical-center proximity.
|
||||
let mut lines: Vec<Vec<Component>> = Vec::new();
|
||||
for component in components {
|
||||
let placed = lines.iter_mut().find(|line| {
|
||||
let first = line[0];
|
||||
let tolerance =
|
||||
0.5 * f64::from(first.height().max(component.height()));
|
||||
(component.center_y() - first.center_y()).abs() <= tolerance
|
||||
});
|
||||
match placed {
|
||||
Some(line) => line.push(component),
|
||||
None => lines.push(vec![component]),
|
||||
}
|
||||
}
|
||||
|
||||
let mut words = Vec::new();
|
||||
for line in &mut lines {
|
||||
line.sort_by_key(|c| c.min_x);
|
||||
let line_height = line.iter().map(Component::height).max().unwrap_or(1);
|
||||
// A gap wider than 60% of the line height starts a new word.
|
||||
let gap_break = (f64::from(line_height) * 0.6).max(2.0);
|
||||
let mut current: Vec<(Component, char, f32)> = Vec::new();
|
||||
let mut previous_right: Option<u32> = None;
|
||||
// Recognize first so word splitting uses only confident glyphs;
|
||||
// stray marks never glue or split words.
|
||||
let mut glyphs: Vec<Option<(Component, char, f32)>> = Vec::new();
|
||||
for component in line.iter() {
|
||||
glyphs.push(
|
||||
match_component(&dark, width, height, component, self.limits.min_confidence)
|
||||
.map(|(ch, conf)| (*component, ch, conf)),
|
||||
);
|
||||
}
|
||||
for item in glyphs.into_iter().flatten() {
|
||||
if let Some(right) = previous_right {
|
||||
let gap = item.0.min_x.saturating_sub(right) as f64;
|
||||
if gap > gap_break && !current.is_empty() {
|
||||
push_word(
|
||||
&mut words,
|
||||
std::mem::take(&mut current),
|
||||
width,
|
||||
height,
|
||||
self.limits.max_words,
|
||||
)?;
|
||||
}
|
||||
}
|
||||
previous_right = Some(item.0.max_x);
|
||||
current.push(item);
|
||||
}
|
||||
if !current.is_empty() {
|
||||
push_word(
|
||||
&mut words,
|
||||
current,
|
||||
width,
|
||||
height,
|
||||
self.limits.max_words,
|
||||
)?;
|
||||
}
|
||||
}
|
||||
Ok(OcrLayer { words })
|
||||
}
|
||||
}
|
||||
|
||||
fn push_word(
|
||||
out: &mut Vec<OcrWord>,
|
||||
glyphs: Vec<(Component, char, f32)>,
|
||||
width: u32,
|
||||
height: u32,
|
||||
max_words: usize,
|
||||
) -> Result<(), OcrError> {
|
||||
if out.len() >= max_words {
|
||||
return Err(OcrError::BudgetExceeded("words".to_string()));
|
||||
}
|
||||
if glyphs.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
let text: String = glyphs.iter().map(|(_, ch, _)| *ch).collect();
|
||||
let confidence = glyphs.iter().map(|(_, _, c)| *c).sum::<f32>() / glyphs.len() as f32;
|
||||
let min_x = glyphs.iter().map(|(c, _, _)| c.min_x).min().unwrap_or(0);
|
||||
let min_y = glyphs.iter().map(|(c, _, _)| c.min_y).min().unwrap_or(0);
|
||||
let max_x = glyphs.iter().map(|(c, _, _)| c.max_x).max().unwrap_or(0);
|
||||
let max_y = glyphs.iter().map(|(c, _, _)| c.max_y).max().unwrap_or(0);
|
||||
out.push(OcrWord {
|
||||
text,
|
||||
rect: [
|
||||
f64::from(min_x) / f64::from(width),
|
||||
f64::from(min_y) / f64::from(height),
|
||||
f64::from(max_x + 1) / f64::from(width),
|
||||
f64::from(max_y + 1) / f64::from(height),
|
||||
],
|
||||
confidence,
|
||||
});
|
||||
Ok(())
|
||||
}
|
||||
|
||||
impl OcrEngine for TemplateOcr {
|
||||
fn recognize(
|
||||
&self,
|
||||
image: &[u8],
|
||||
width: u32,
|
||||
height: u32,
|
||||
) -> Result<Vec<OcrWord>, OcrError> {
|
||||
Ok(self.recognize_layer(image, width, height)?.words)
|
||||
}
|
||||
}
|
||||
|
||||
/// Corner detection over a raw grayscale buffer (glue over `scanner_core`,
|
||||
/// which works on `DynamicImage`, so downstream crates need no `image` dep).
|
||||
pub fn detect_corners_grayscale(
|
||||
gray: &[u8],
|
||||
width: u32,
|
||||
height: u32,
|
||||
) -> Option<[(f32, f32); 4]> {
|
||||
let pixels = (width as usize).checked_mul(height as usize)?;
|
||||
if gray.len() != pixels {
|
||||
return None;
|
||||
}
|
||||
let buffer = GrayImage::from_raw(width, height, gray.to_vec())?;
|
||||
let corners = crate::scanner_core::detect_document_corners(&DynamicImage::ImageLuma8(buffer))?;
|
||||
Some([
|
||||
(corners[0].x, corners[0].y),
|
||||
(corners[1].x, corners[1].y),
|
||||
(corners[2].x, corners[2].y),
|
||||
(corners[3].x, corners[3].y),
|
||||
])
|
||||
}
|
||||
|
||||
/// Field suggestions from one recognized layer. A digit run (length >= 4) is
|
||||
/// the ID-number candidate; the best same-line letter run is the name
|
||||
/// candidate. Either is `None` below confidence/charset/length rules — the
|
||||
/// caller must then force manual entry.
|
||||
pub fn suggest_id_fields(layer: &OcrLayer) -> (Option<String>, Option<String>) {
|
||||
let mut best_id: Option<(&OcrWord, usize)> = None;
|
||||
for word in &layer.words {
|
||||
if word.confidence < OcrLimits::id().min_confidence {
|
||||
continue;
|
||||
}
|
||||
if word.text.len() >= 4 && word.text.bytes().all(|b| b.is_ascii_digit()) {
|
||||
let better = match &best_id {
|
||||
None => true,
|
||||
Some((current, _)) => {
|
||||
word.text.len() > current.text.len()
|
||||
|| (word.text.len() == current.text.len()
|
||||
&& word.confidence > current.confidence)
|
||||
}
|
||||
};
|
||||
if better {
|
||||
best_id = Some((word, word.text.len()));
|
||||
}
|
||||
}
|
||||
}
|
||||
// Names may span words on one visual line ("AMINA DIALLO"): join the
|
||||
// highest-confidence line whose words are all uppercase letters.
|
||||
let mut best_line: Option<(f32, String)> = None;
|
||||
let mut index = 0usize;
|
||||
while index < layer.words.len() {
|
||||
let word = &layer.words[index];
|
||||
let line_y = (word.rect[1] + word.rect[3]) / 2.0;
|
||||
let mut line_words: Vec<&OcrWord> = vec![word];
|
||||
let mut next = index + 1;
|
||||
while next < layer.words.len() {
|
||||
let candidate = &layer.words[next];
|
||||
let candidate_y = (candidate.rect[1] + candidate.rect[3]) / 2.0;
|
||||
if (candidate_y - line_y).abs() > 0.05 {
|
||||
break;
|
||||
}
|
||||
line_words.push(candidate);
|
||||
next += 1;
|
||||
}
|
||||
let letters_only = line_words.iter().all(|w| {
|
||||
w.confidence >= OcrLimits::id().min_confidence
|
||||
&& !w.text.is_empty()
|
||||
&& w.text.bytes().all(|b| b.is_ascii_uppercase())
|
||||
});
|
||||
if letters_only {
|
||||
let letters: usize = line_words.iter().map(|w| w.text.len()).sum();
|
||||
if letters >= 2 {
|
||||
let text = line_words
|
||||
.iter()
|
||||
.map(|w| w.text.as_str())
|
||||
.collect::<Vec<_>>()
|
||||
.join(" ");
|
||||
if text.len() <= 256 {
|
||||
let confidence = line_words
|
||||
.iter()
|
||||
.map(|w| w.confidence)
|
||||
.sum::<f32>()
|
||||
/ line_words.len() as f32;
|
||||
let better = match &best_line {
|
||||
None => true,
|
||||
Some((current_conf, current_text)) => {
|
||||
(confidence - *current_conf) > f32::EPSILON
|
||||
&& text.len() >= current_text.len()
|
||||
|| text.len() > current_text.len()
|
||||
}
|
||||
};
|
||||
if better {
|
||||
best_line = Some((confidence, text));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
index = next;
|
||||
}
|
||||
let id = best_id
|
||||
.map(|(word, _)| word.text.clone())
|
||||
.filter(|s| s.len() <= 64);
|
||||
let name = best_line.map(|(_, text)| text);
|
||||
(id, name)
|
||||
}
|
||||
|
||||
/// Deterministic fixture renderer: draws `text` (supported charset: space,
|
||||
/// `0-9`, `A-Z`, anything else returns `None`) with the embedded glyph set at
|
||||
/// integer `scale` (1..=8), black on white with a quiet margin. Integration
|
||||
/// tests in this crate and in `nigig-site` share these vectors, so engine
|
||||
/// behavior is pinned across both crates.
|
||||
pub fn render_line_image(text: &str, scale: u32) -> Option<(Vec<u8>, u32, u32)> {
|
||||
if scale == 0 || scale > 8 || text.is_empty() || text.len() > 256 {
|
||||
return None;
|
||||
}
|
||||
for ch in text.chars() {
|
||||
if ch != ' ' && glyph_rows(ch.to_ascii_uppercase()).is_none() {
|
||||
return None;
|
||||
}
|
||||
}
|
||||
let advance = (GLYPH_W as u32 + 1) * scale;
|
||||
let margin = 2 * scale;
|
||||
let text_w = text.chars().count() as u32 * advance;
|
||||
let width = text_w + 2 * margin;
|
||||
let height = (GLYPH_H as u32) * scale + 2 * margin;
|
||||
let pixels = (width as usize).checked_mul(height as usize)?;
|
||||
if pixels > OcrLimits::id().max_pixels as usize {
|
||||
return None;
|
||||
}
|
||||
let mut gray = vec![255u8; pixels];
|
||||
for (i, ch) in text.chars().enumerate() {
|
||||
if ch == ' ' {
|
||||
continue;
|
||||
}
|
||||
let rows = glyph_rows(ch.to_ascii_uppercase())?;
|
||||
let gx = margin + i as u32 * advance;
|
||||
for (r, row) in rows.iter().enumerate() {
|
||||
for (c, b) in row.bytes().enumerate() {
|
||||
if b != b'1' {
|
||||
continue;
|
||||
}
|
||||
for dy in 0..scale {
|
||||
for dx in 0..scale {
|
||||
let x = gx + c as u32 * scale + dx;
|
||||
let y = margin + r as u32 * scale + dy;
|
||||
gray[(y * width + x) as usize] = 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Some((gray, width, height))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn reader() -> TemplateOcr {
|
||||
TemplateOcr::id_reader()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn synthetic_id_line_reads_exactly() {
|
||||
let (gray, w, h) = render_line_image("0701234567", 3).expect("fixture");
|
||||
let layer = reader()
|
||||
.recognize_layer(&gray, w, h)
|
||||
.expect("synthetic digits");
|
||||
assert_eq!(layer.text(), "0701234567");
|
||||
assert!(layer.mean_confidence() >= 0.80);
|
||||
let (id, name) = suggest_id_fields(&layer);
|
||||
assert_eq!(id.as_deref(), Some("0701234567"));
|
||||
assert_eq!(name, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn synthetic_name_and_id_read_exactly_at_two_scales() {
|
||||
for scale in [2, 4] {
|
||||
let (gray, w, h) = render_line_image("AMINA DIALLO", scale).expect("fixture");
|
||||
let layer = reader()
|
||||
.recognize_layer(&gray, w, h)
|
||||
.expect("synthetic name");
|
||||
assert_eq!(layer.text(), "AMINA DIALLO", "scale {scale}");
|
||||
let (id, name) = suggest_id_fields(&layer);
|
||||
assert_eq!(name.as_deref(), Some("AMINA DIALLO"), "scale {scale}");
|
||||
assert_eq!(id, None);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn full_charset_precision_and_recall_are_exact_on_synthetic() {
|
||||
let line = "0123456789 ABCDEFGHIJKLMNOPQRSTUVWXYZ";
|
||||
let (gray, w, h) = render_line_image(line, 2).expect("fixture");
|
||||
let layer = reader().recognize_layer(&gray, w, h).expect("charset");
|
||||
// Recall: every emitted word matches the source; precision: the full
|
||||
// source round-trips with no extras or substitutions.
|
||||
assert_eq!(layer.text(), line);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn blank_and_solid_frames_yield_empty_never_text() {
|
||||
let blank = vec![255u8; 64 * 32];
|
||||
let layer = reader().recognize_layer(&blank, 64, 32).expect("blank");
|
||||
assert!(layer.is_empty());
|
||||
assert_eq!(layer.text(), "");
|
||||
let (id, name) = suggest_id_fields(&layer);
|
||||
assert_eq!(id, None);
|
||||
assert_eq!(name, None);
|
||||
// A solid ink block is a blob, not glyphs.
|
||||
let solid = vec![0u8; 64 * 32];
|
||||
let layer = reader().recognize_layer(&solid, 64, 32).expect("solid");
|
||||
assert!(layer.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn budgets_fail_before_allocation() {
|
||||
let tight = OcrLimits {
|
||||
max_pixels: 100,
|
||||
..OcrLimits::id()
|
||||
};
|
||||
let engine = TemplateOcr::new(tight);
|
||||
let big = vec![255u8; 64 * 32];
|
||||
assert_eq!(
|
||||
engine.recognize_layer(&big, 64, 32),
|
||||
Err(OcrError::BudgetExceeded("pixels".to_string()))
|
||||
);
|
||||
// Length mismatch and zero dimensions are invalid images, not budgets.
|
||||
assert!(reader().recognize_layer(&[0u8; 10], 4, 4).is_err());
|
||||
assert!(reader().recognize_layer(&[], 0, 0).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unsupported_charset_never_renders_or_suggests() {
|
||||
assert_eq!(render_line_image("héllo", 2), None);
|
||||
assert_eq!(render_line_image("", 2), None);
|
||||
assert_eq!(render_line_image("AB", 0), None);
|
||||
assert_eq!(render_line_image("AB", 9), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn noop_engine_always_refuses() {
|
||||
assert_eq!(
|
||||
NoopOcr.recognize(&[0u8; 100], 10, 10),
|
||||
Err(OcrError::NoEngine)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn corners_glue_detects_a_bordered_document() {
|
||||
// White page, thick black border: real edges for `scanner_core`.
|
||||
let (w, h) = (120u32, 120u32);
|
||||
let mut gray = vec![255u8; (w * h) as usize];
|
||||
for y in 10..110 {
|
||||
for x in 10..110 {
|
||||
if x < 14 || x >= 106 || y < 14 || y >= 106 {
|
||||
gray[(y * w + x) as usize] = 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
let corners = detect_corners_grayscale(&gray, w, h);
|
||||
assert!(corners.is_some());
|
||||
assert_eq!(detect_corners_grayscale(&gray[..100], w, h), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn short_digit_runs_are_not_id_suggestions() {
|
||||
let (gray, w, h) = render_line_image("AB 12", 3).expect("fixture");
|
||||
let layer = reader().recognize_layer(&gray, w, h).expect("short");
|
||||
let (id, _) = suggest_id_fields(&layer);
|
||||
assert_eq!(id, None);
|
||||
}
|
||||
}
|
||||
|
|
@ -2,7 +2,7 @@
|
|||
//! This module replaces the placeholder with actual image processing
|
||||
//! that can be used as a crate from `nigig-site` for worker ID OCR.
|
||||
|
||||
use image::{DynamicImage, GenericImageView, GrayImage, Luma};
|
||||
use image::{DynamicImage, GrayImage, Luma};
|
||||
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
pub struct Point { pub x: f32, pub y: f32 }
|
||||
|
|
@ -90,6 +90,7 @@ pub fn enhance_for_ocr(cropped: &DynamicImage) -> DynamicImage {
|
|||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use image::{GenericImage, GenericImageView};
|
||||
#[test]
|
||||
fn detects_corners_on_synthetic_doc() {
|
||||
let mut img = DynamicImage::new_rgb8(200, 200);
|
||||
|
|
|
|||
|
|
@ -5,6 +5,16 @@ edition = "2021"
|
|||
license = "MIT OR Apache-2.0"
|
||||
description = "PDF document model: page tree, annotations, AcroForm, destinations. No UI dependencies."
|
||||
|
||||
[features]
|
||||
# The OCR seam (`src/ocr.rs`: engine trait + hidden-text-layer types only, no
|
||||
# engine, no new dependencies) is opt-out rather than opt-in so existing
|
||||
# consumers keep building. Consumers that only need text recognition must NOT
|
||||
# depend on this crate for it — the owned on-device engine lives in
|
||||
# `nigig_doc_scanner` behind its `ocr` feature. Gating here documents that
|
||||
# boundary and lets minimal builds skip even the seam.
|
||||
default = ["ocr"]
|
||||
ocr = []
|
||||
|
||||
[dependencies]
|
||||
nigig-pdf-cos = { path = "../pdf-cos" }
|
||||
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ pub mod document;
|
|||
pub mod flatten;
|
||||
pub mod form;
|
||||
pub mod form_actions;
|
||||
#[cfg(feature = "ocr")]
|
||||
pub mod ocr;
|
||||
pub mod page;
|
||||
pub mod page_ops;
|
||||
|
|
@ -44,6 +45,7 @@ pub use form_actions::{
|
|||
};
|
||||
pub use page::{CMapData, ExtGStateResource, FontEncoding, FontResource, PdfPage, XObjectResource};
|
||||
pub use save::{save_annotation_edits, save_form_edits, SaveReport};
|
||||
#[cfg(feature = "ocr")]
|
||||
pub use ocr::{NoopOcr, OcrEngine, OcrError, OcrLayer, OcrWord, StubOcr};
|
||||
pub use pdf_a::{check_pdf_a, PdfAFindings, PdfAIssue, PdfAProfile};
|
||||
pub use signature::{
|
||||
|
|
|
|||
|
|
@ -11,6 +11,12 @@
|
|||
//! text as a hidden text layer, which is what makes a scanned page
|
||||
//! searchable and selectable — the same mechanism dart-pdf's `ocr_layer.dart`
|
||||
//! uses, minus the engine choice.
|
||||
//!
|
||||
//! The owned on-device recognition engine lives in `nigig_doc_scanner`
|
||||
//! (`ocr` cargo feature), NOT here: device crates must depend on
|
||||
//! `nigig_doc_scanner/ocr` for recognition and must not pull this document
|
||||
//! crate (and its signing stack) just to read text. This module is gated
|
||||
//! behind the `ocr` cargo feature for the same reason.
|
||||
|
||||
use std::fmt;
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue