nigig-org/crates/apps/map/tests/MAKEPAD_TESTING_GUIDE.md

Makepad Visual Testing Guide

This guide explains how to run visual regression tests for the map widget using Makepad's testing framework.

Overview

Makepad provides a visual testing framework that allows you to:

• Render widgets in a headless environment
• Capture screenshots of rendered output
• Compare against golden images
• Detect visual regressions automatically

Prerequisites

The tests require:

• Rust toolchain (stable)
• Makepad test runner (part of makepad-widgets)
• Golden images directory: crates/apps/map/tests/golden_images/

Test Structure

Test File Location

crates/apps/map/tests/
├── makepad_visual_tests.rs    # Visual regression tests
├── golden_images/              # Reference screenshots
│   ├── empty_map.png
│   ├── single_tile_amsterdam.png
│   ├── multiple_tiles_grid.png
│   ├── zoom_level_10.png
│   ├── zoom_level_12.png
│   ├── zoom_level_14.png
│   ├── zoom_level_16.png
│   ├── dark_theme.png
│   ├── pois_visible.png
│   ├── labels_visible.png
│   ├── roads_render.png
│   ├── water_features.png
│   └── buildings_render.png
└── diff/                       # Difference images (generated on failure)

Test Cases

1. test_empty_map_render - Map with no tiles loaded
2. test_single_tile_render - Map with one tile (Amsterdam)
3. test_multiple_tiles_render - 3x3 grid of tiles
4. test_zoom_levels - Zoom levels 10, 12, 14, 16
5. test_dark_theme - Dark theme rendering
6. test_pois_visible - POI icons at high zoom
7. test_labels_visible - Text labels at high zoom
8. test_roads_render - Road geometry rendering
9. test_water_features - Water/canal rendering
10. test_buildings_render - Building footprints

Running Tests

Run All Visual Tests

cd crates/apps/map
cargo test --test makepad_visual_tests

Run Specific Test

cargo test --test makepad_visual_tests test_single_tile_render

Update Golden Images

To regenerate all golden images (use with caution):

UPDATE_GOLDEN=1 cargo test --test makepad_visual_tests

Verbose Output

cargo test --test makepad_visual_tests -- --nocapture

Test Configuration

Screenshot Size

Tests render at 800x600 pixels by default. To change:

// In makepad_visual_tests.rs
let mut cx = Cx::new();
cx.set_window_size(1024, 768);

Comparison Threshold

Tests allow 1% pixel difference by default. To adjust:

let threshold = 0.01; // 1% difference allowed
// Change to:
let threshold = 0.02; // 2% difference allowed

Understanding Test Failures

When a visual regression is detected, the test will:

1. Show the percentage difference
2. Save a diff image to tests/diff/<test_name>_diff.png
3. Fail with a descriptive error message

Example Failure Output

Visual regression detected for 'single_tile_amsterdam'. 
Diff: 2.34% (threshold: 1.00%). 
Check diff image: tests/diff/single_tile_amsterdam_diff.png

Investigating Failures

1. Open the diff image - Shows pixel-level differences highlighted in red
2. Compare golden vs actual - Side-by-side comparison
3. Check recent changes - What code changes could affect rendering?
4. Run with UPDATE_GOLDEN=1 - If changes are intentional, update golden

Common Issues

Issue: "Failed to load golden image"

Cause: Golden image doesn't exist yet Solution: Run with UPDATE_GOLDEN=1 to create it

Issue: "Visual regression detected"

Cause: Rendered output differs from golden image Solution:

• If intentional: Update golden with UPDATE_GOLDEN=1
• If bug: Fix the rendering issue

Issue: "Test hangs"

Cause: Map widget waiting for tile data Solution: Ensure test tiles are pre-loaded or mocked

Integration with CI

Add to your CI pipeline:

- name: Run Makepad Visual Tests
  run: |
    cd crates/apps/map
    cargo test --test makepad_visual_tests

CI Artifacts

Upload diff images on failure:

- name: Upload Diff Images
  if: failure()
  uses: actions/upload-artifact@v2
  with:
    name: visual-test-diffs
    path: crates/apps/map/tests/diff/

Best Practices

1. Review golden images - Before committing, visually inspect new golden images
2. Small changes - Make rendering changes incrementally
3. Document changes - Update golden images with descriptive commit messages
4. Test locally first - Run visual tests before pushing
5. Keep golden images - Don't delete golden images, update them

Advanced Usage

Custom Test Fixtures

Create reusable test setups:

fn setup_amsterdam_map() -> (MapTestApp, Cx) {
    let mut app = MapTestApp::new();
    let mut cx = Cx::new();
    
    app.map_view.set_center(4.9041, 52.3676);
    app.map_view.set_zoom(14.0);
    
    let tile_key = TileKey::from_lon_lat(4.9041, 52.3676, 14);
    app.map_view.load_tile(tile_key);
    
    (app, cx)
}

Parameterized Tests

Test multiple configurations:

#[test]
fn test_multiple_cities() {
    let cities = vec![
        ("amsterdam", 4.9041, 52.3676),
        ("london", -0.1276, 51.5074),
        ("paris", 2.3522, 48.8566),
    ];
    
    for (name, lon, lat) in cities {
        let mut app = MapTestApp::new();
        let mut cx = Cx::new();
        
        app.map_view.set_center(lon, lat);
        app.map_view.set_zoom(14.0);
        
        let tile_key = TileKey::from_lon_lat(lon, lat, 14);
        app.map_view.load_tile(tile_key);
        
        app.ui.draw_walk(&mut cx, &mut Scope::empty(), Walk::default());
        
        let screenshot = cx.screenshot();
        assert_visual_match(&format!("city_{}", name), screenshot);
    }
}

Performance Testing

Measure render time:

#[test]
fn test_render_performance() {
    let (mut app, mut cx) = setup_amsterdam_map();
    
    let start = std::time::Instant::now();
    app.ui.draw_walk(&mut cx, &mut Scope::empty(), Walk::default());
    let duration = start.elapsed();
    
    // Should render in < 16ms (60fps)
    assert!(duration.as_millis() < 16, "Render took {}ms", duration.as_millis());
}

Troubleshooting

Tests pass locally but fail in CI

Possible causes:

• Different graphics drivers
• Different font rendering
• Different anti-aliasing

Solution: Increase threshold or use software rendering in CI

Flaky tests (sometimes pass, sometimes fail)

Possible causes:

• Race conditions in tile loading
• Non-deterministic rendering order
• Floating point precision issues

Solution: Ensure tiles are fully loaded before capturing, sort rendering order

Tests are slow

Possible causes:

• Large golden images
• Many test cases
• Complex rendering

Solution:

• Reduce screenshot size
• Parallelize tests
• Cache tile data

Resources

• Makepad Documentation
• Makepad Widgets Repository
• Visual Regression Testing Best Practices

Support

For issues with Makepad visual tests:

1. Check this guide first
2. Search existing issues in the repository
3. Create a new issue with:
   • Test name
   • Failure output
   • Diff image
   • Steps to reproduce