nigig-org/crates/apps/map/tests/makepad_test_app/README.md

Map Visual Test Application

This is a Makepad-based visual regression test application for the nigig-map widget.

Overview

This application renders map tiles in various scenarios and captures screenshots for visual regression testing. It can run in both interactive mode (for manual inspection) and automated mode (for CI/CD).

Test Scenarios

The application tests 13 different scenarios:

1. Empty Map - Map with no tiles loaded
2. Single Tile - Map with one tile (Amsterdam center)
3. Multiple Tiles - 3x3 grid of tiles
4. Zoom Level 10 - Low zoom overview
5. Zoom Level 12 - Medium zoom
6. Zoom Level 14 - Standard city zoom
7. Zoom Level 16 - High zoom with details
8. Dark Theme - Dark theme rendering
9. POIs Visible - Point of interest icons
10. Labels Visible - Text labels
11. Roads Render - Road geometry
12. Water Features - Water/canal rendering
13. Buildings Render - Building footprints

Running Tests

Prerequisites

• Rust toolchain (stable)
• Makepad dependencies (graphics libraries)
• Golden images directory: golden_images/

Run All Visual Tests

cd crates/apps/map/tests/makepad_test_app
cargo test --test visual_regression

Run Specific Test

cargo test --test visual_regression test_single_tile

Update Golden Images

To regenerate all golden images (use with caution):

UPDATE_GOLDEN=1 cargo test --test visual_regression

Interactive Mode

To run the application interactively (for manual inspection):

cargo run --bin map_visual_test

Test Configuration

Screenshot Size

Tests render at 800x600 pixels by default. To change the size, modify the test code:

cx.set_window_size(1024, 768);

Comparison Threshold

Tests allow 1% pixel difference by default. To adjust the threshold:

let threshold = 0.01; // 1% difference allowed
// Change to:
let threshold = 0.02; // 2% difference allowed

Understanding Test Failures

When a visual regression is detected, the test will:

1. Show the percentage difference
2. Save a diff image to diff/<test_name>_diff.png
3. Fail with a descriptive error message

Example Failure Output

Visual regression detected for 'single_tile_amsterdam'. 
Diff: 2.34% (threshold: 1.00%). 
Check diff image: diff/single_tile_amsterdam_diff.png

Investigating Failures

1. Open the diff image - Shows the current render
2. Compare with golden - Side-by-side comparison
3. Check recent changes - What code changes could affect rendering?
4. Update if intentional - Run with UPDATE_GOLDEN=1

Directory Structure

makepad_test_app/
├── Cargo.toml                    # Test app manifest
├── src/
│   └── main.rs                   # Test application
├── tests/
│   └── visual_regression.rs      # Visual regression tests
├── golden_images/                # Reference screenshots
│   ├── empty_map.png
│   ├── single_tile_amsterdam.png
│   ├── multiple_tiles_grid.png
│   ├── zoom_level_10.png
│   ├── zoom_level_12.png
│   ├── zoom_level_14.png
│   ├── zoom_level_16.png
│   ├── dark_theme.png
│   ├── pois_visible.png
│   ├── labels_visible.png
│   ├── roads_render.png
│   ├── water_features.png
│   └── buildings_render.png
└── diff/                         # Difference images (generated on failure)

Integration with CI

Add to your CI pipeline:

- name: Run Makepad Visual Tests
  run: |
    cd crates/apps/map/tests/makepad_test_app
    cargo test --test visual_regression

CI Artifacts

Upload diff images on failure:

- name: Upload Diff Images
  if: failure()
  uses: actions/upload-artifact@v2
  with:
    name: visual-test-diffs
    path: crates/apps/map/tests/makepad_test_app/diff/

Headless Mode

For CI environments without a display, use headless mode:

cargo test --test visual_regression --features headless

This requires the headless feature to be enabled in Cargo.toml.

Troubleshooting

Issue: "Failed to create window"

Cause: No display available (headless environment)

Solution: Use headless mode or set up virtual display (Xvfb)

Issue: "Failed to load golden image"

Cause: Golden image doesn't exist yet

Solution: Run with UPDATE_GOLDEN=1 to create it

Issue: "Visual regression detected"

Cause: Rendered output differs from golden image

Solution:

• If intentional: Update golden with UPDATE_GOLDEN=1
• If bug: Fix the rendering issue

Issue: "Tests are slow"

Cause: Rendering is computationally expensive

Solution:

• Run tests in parallel: cargo test --test visual_regression -- --test-threads=4
• Reduce screenshot size
• Skip slow tests in CI

Best Practices

1. Review golden images - Before committing, visually inspect new golden images
2. Small changes - Make rendering changes incrementally
3. Document changes - Update golden images with descriptive commit messages
4. Test locally first - Run visual tests before pushing
5. Keep golden images - Don't delete golden images, update them

Advanced Usage

Custom Test Scenarios

Add new test scenarios by:

1. Adding a variant to TestScenario enum in main.rs
2. Implementing the scenario setup in after_apply
3. Creating a test function in visual_regression.rs
4. Running with UPDATE_GOLDEN=1 to create the golden image

Parameterized Tests

Test multiple cities:

#[test]
fn test_multiple_cities() {
    let cities = vec![
        ("amsterdam", 4.9041, 52.3676),
        ("london", -0.1276, 51.5074),
        ("paris", 2.3522, 48.8566),
    ];
    
    for (name, lon, lat) in cities {
        let mut app = MapVisualTestApp::new();
        // ... setup ...
        assert_visual_match(&format!("city_{}", name), &screenshot);
    }
}

Performance Testing

Measure render time:

#[test]
fn test_render_performance() {
    let start = std::time::Instant::now();
    // ... render ...
    let duration = start.elapsed();
    
    // Should render in < 16ms (60fps)
    assert!(duration.as_millis() < 16);
}

Resources

• Makepad Documentation
• Makepad Widgets Repository
• Visual Regression Testing Best Practices

Support

For issues with visual tests:

1. Check this README first
2. Review the Makepad Testing Guide
3. Search existing issues in the repository
4. Create a new issue with:
   • Test name
   • Failure output
   • Diff image
   • Steps to reproduce