# Map Visual Test Application This is a Makepad-based visual regression test application for the `nigig-map` widget. ## Overview This application renders map tiles in various scenarios and captures screenshots for visual regression testing. It can run in both interactive mode (for manual inspection) and automated mode (for CI/CD). ## Test Scenarios The application tests 13 different scenarios: 1. **Empty Map** - Map with no tiles loaded 2. **Single Tile** - Map with one tile (Amsterdam center) 3. **Multiple Tiles** - 3x3 grid of tiles 4. **Zoom Level 10** - Low zoom overview 5. **Zoom Level 12** - Medium zoom 6. **Zoom Level 14** - Standard city zoom 7. **Zoom Level 16** - High zoom with details 8. **Dark Theme** - Dark theme rendering 9. **POIs Visible** - Point of interest icons 10. **Labels Visible** - Text labels 11. **Roads Render** - Road geometry 12. **Water Features** - Water/canal rendering 13. **Buildings Render** - Building footprints ## Running Tests ### Prerequisites - Rust toolchain (stable) - Makepad dependencies (graphics libraries) - Golden images directory: `golden_images/` ### Run All Visual Tests ```bash cd crates/apps/map/tests/makepad_test_app cargo test --test visual_regression ``` ### Run Specific Test ```bash cargo test --test visual_regression test_single_tile ``` ### Update Golden Images To regenerate all golden images (use with caution): ```bash UPDATE_GOLDEN=1 cargo test --test visual_regression ``` ### Interactive Mode To run the application interactively (for manual inspection): ```bash cargo run --bin map_visual_test ``` ## Test Configuration ### Screenshot Size Tests render at 800x600 pixels by default. To change the size, modify the test code: ```rust cx.set_window_size(1024, 768); ``` ### Comparison Threshold Tests allow 1% pixel difference by default. To adjust the threshold: ```rust let threshold = 0.01; // 1% difference allowed // Change to: let threshold = 0.02; // 2% difference allowed ``` ## Understanding Test Failures When a visual regression is detected, the test will: 1. Show the percentage difference 2. Save a diff image to `diff/_diff.png` 3. Fail with a descriptive error message ### Example Failure Output ``` Visual regression detected for 'single_tile_amsterdam'. Diff: 2.34% (threshold: 1.00%). Check diff image: diff/single_tile_amsterdam_diff.png ``` ### Investigating Failures 1. **Open the diff image** - Shows the current render 2. **Compare with golden** - Side-by-side comparison 3. **Check recent changes** - What code changes could affect rendering? 4. **Update if intentional** - Run with `UPDATE_GOLDEN=1` ## Directory Structure ``` makepad_test_app/ ├── Cargo.toml # Test app manifest ├── src/ │ └── main.rs # Test application ├── tests/ │ └── visual_regression.rs # Visual regression tests ├── golden_images/ # Reference screenshots │ ├── empty_map.png │ ├── single_tile_amsterdam.png │ ├── multiple_tiles_grid.png │ ├── zoom_level_10.png │ ├── zoom_level_12.png │ ├── zoom_level_14.png │ ├── zoom_level_16.png │ ├── dark_theme.png │ ├── pois_visible.png │ ├── labels_visible.png │ ├── roads_render.png │ ├── water_features.png │ └── buildings_render.png └── diff/ # Difference images (generated on failure) ``` ## Integration with CI Add to your CI pipeline: ```yaml - name: Run Makepad Visual Tests run: | cd crates/apps/map/tests/makepad_test_app cargo test --test visual_regression ``` ### CI Artifacts Upload diff images on failure: ```yaml - name: Upload Diff Images if: failure() uses: actions/upload-artifact@v2 with: name: visual-test-diffs path: crates/apps/map/tests/makepad_test_app/diff/ ``` ## Headless Mode For CI environments without a display, use headless mode: ```bash cargo test --test visual_regression --features headless ``` This requires the `headless` feature to be enabled in Cargo.toml. ## Troubleshooting ### Issue: "Failed to create window" **Cause:** No display available (headless environment) **Solution:** Use headless mode or set up virtual display (Xvfb) ### Issue: "Failed to load golden image" **Cause:** Golden image doesn't exist yet **Solution:** Run with `UPDATE_GOLDEN=1` to create it ### Issue: "Visual regression detected" **Cause:** Rendered output differs from golden image **Solution:** - If intentional: Update golden with `UPDATE_GOLDEN=1` - If bug: Fix the rendering issue ### Issue: "Tests are slow" **Cause:** Rendering is computationally expensive **Solution:** - Run tests in parallel: `cargo test --test visual_regression -- --test-threads=4` - Reduce screenshot size - Skip slow tests in CI ## Best Practices 1. **Review golden images** - Before committing, visually inspect new golden images 2. **Small changes** - Make rendering changes incrementally 3. **Document changes** - Update golden images with descriptive commit messages 4. **Test locally first** - Run visual tests before pushing 5. **Keep golden images** - Don't delete golden images, update them ## Advanced Usage ### Custom Test Scenarios Add new test scenarios by: 1. Adding a variant to `TestScenario` enum in `main.rs` 2. Implementing the scenario setup in `after_apply` 3. Creating a test function in `visual_regression.rs` 4. Running with `UPDATE_GOLDEN=1` to create the golden image ### Parameterized Tests Test multiple cities: ```rust #[test] fn test_multiple_cities() { let cities = vec![ ("amsterdam", 4.9041, 52.3676), ("london", -0.1276, 51.5074), ("paris", 2.3522, 48.8566), ]; for (name, lon, lat) in cities { let mut app = MapVisualTestApp::new(); // ... setup ... assert_visual_match(&format!("city_{}", name), &screenshot); } } ``` ### Performance Testing Measure render time: ```rust #[test] fn test_render_performance() { let start = std::time::Instant::now(); // ... render ... let duration = start.elapsed(); // Should render in < 16ms (60fps) assert!(duration.as_millis() < 16); } ``` ## Resources - [Makepad Documentation](https://makepad.dev) - [Makepad Widgets Repository](https://github.com/makepad/makepad) - [Visual Regression Testing Best Practices](https://martinfowler.com/articles/visual-testing.html) ## Support For issues with visual tests: 1. Check this README first 2. Review the [Makepad Testing Guide](../MAKEPAD_TESTING_GUIDE.md) 3. Search existing issues in the repository 4. Create a new issue with: - Test name - Failure output - Diff image - Steps to reproduce