nigig-org/MAKEPAD_TEST_FRAMEWORK.md

# Makepad Test Framework - Official Guide (2026)

## Executive Summary

**`makepad_test`** is Makepad's **official Rust-native UI regression testing framework**. It allows you to write UI tests that:
- Live next to the package they exercise (in `tests/` directory)
- Run through normal `cargo test`
- Drive the app through the Studio protocol in headless mode
- Support both headless and visible Studio modes
- Provide structured selectors, locators, waits, and assertions

**Key Discovery:** Makepad DOES have a testing framework - it's called `makepad_test` and it's located in `libs/makepad_test/`.

## Quick Start

### 1. Add Dependency

Add to your package's `Cargo.toml`:

```toml
[dev-dependencies]
makepad-test = { path = "../../libs/makepad_test", version = "0.1.0" }
```

### 2. Create Integration Test

Create `tests/ui.rs` in your package:

```rust
use makepad_test::{makepad_test, Selector, TestApp};

#[makepad_test]
fn smoke_test(app: TestApp) {
    // Wait for a widget to be visible
    app.locator(Selector::id("my_button"))
        .wait_visible()
        .click();
    
    // Wait for text to appear
    app.locator(Selector::id("status_label"))
        .wait_text("Button clicked!");
}
```

### 3. Run Tests

```bash
# Headless mode (default)
cargo test -p my-package --test ui -- --test-threads=1

# Visible mode (for debugging)
MAKEPAD_TEST_VISIBLE=1 cargo test -p my-package --test ui -- --test-threads=1
```

## Core Concepts

### Authoring Model

Tests live beside the package they exercise:

```
examples/my_app/
├── Cargo.toml
├── src/main.rs
└── tests/
    └── ui.rs          # UI tests here
```

The `#[makepad_test]` macro:
- Uses `env!("CARGO_MANIFEST_DIR")` for mount root
- Uses `env!("CARGO_PKG_NAME")` for package to run
- Keeps normal Rust workflow intact

### Macro Behavior

`#[makepad_test]` expands to a `#[test]` wrapper that:

1. Starts `StudioHub::start_in_process`
2. Mounts the current package directory
3. Runs the current package headlessly
4. Waits for `BuildStarted` and `AppStarted`
5. Passes a `TestApp` into your test body
6. Captures failure artifacts on errors or panics

**Supported signatures:**

```rust
#[makepad_test]
fn smoke(app: TestApp) {
    // ...
}

#[makepad_test]
fn smoke(app: TestApp) -> Result<(), TestError> {
    // ...
    Ok(())
}
```

**Unsupported:**
- async tests
- methods with `self`
- generic test functions
- macro arguments

### Runtime Defaults

The runtime is synchronous and serial-first:

- **Action timeout:** 10s
- **Poll interval:** 50ms
- **Artifacts:** `target/makepad_test/<package>/<test>/`

The in-process runner serializes app sessions, so UI suites should be invoked with `--test-threads=1`.

## Selectors

Selectors are snapshot-based and match structured widget state.

### Constructors

```rust
Selector::all()                    // Match all widgets
Selector::id("widget_id")          // Match by widget ID
Selector::widget_type("TextInput") // Match by widget type
Selector::raw("text:hello")        // Raw query string
```

### Builder Filters

```rust
Selector::id("button")
    .text_exact("Click Me")        // Exact text match
    .text_contains("Click")        // Partial text match
    .nth(0)                        // Nth match (0-indexed)
    .window("panel_window")        // Specific window
    .window_index(1)               // Window by index
    .any_window()                  // Any window (not just primary)
```

**Note:** Selectors default to the primary window for single-window tests.

## Locators

`Locator` methods require **exactly one visible match** for interaction. This strictness prevents tests from silently clicking the wrong widget.

### Common Actions

```rust
app.locator(Selector::id("input_field"))
    .wait_visible()
    .fill("hello")              // Clear and type
    .wait_value("hello")
    .press_key(KeyCode::Enter);
```

**Available interaction helpers:**
- `click()` - Click the widget
- `type_text("text")` - Type text (appends)
- `fill("text")` - Clear and type
- `clear()` - Clear the widget
- `press_key(KeyCode::Enter)` - Press a key
- `press_key_with_modifiers(KeyCode::A, KeyModifiers::CTRL)` - Key with modifiers
- `scroll(x, y)` - Scroll
- `drag_by(x, y)` - Drag

### Waits and Assertions

```rust
app.locator(Selector::id("label"))
    .wait_visible()             // Wait until visible
    .wait_hidden()              // Wait until hidden
    .wait_count(5)              // Wait for count
    .wait_text("Hello")         // Wait for text
    .assert_text("Hello")       // Assert text (immediate)
    .wait_value("value")        // Wait for value
    .assert_value("value")      // Assert value (immediate)
    .wait_checked(true)         // Wait for checked state
    .assert_checked(true)       // Assert checked (immediate)
    .wait_enabled(true)         // Wait for enabled state
    .assert_enabled(true);      // Assert enabled (immediate)
```

### Inspection Helpers

```rust
let snapshot = app.locator(Selector::id("widget")).snapshot();
let count = app.locator(Selector::all()).count();
let widget_state = app.locator(Selector::id("widget")).widget_snapshot();
let tree = app.widget_dump();
let screenshot = app.screenshot();
app.wait_for_log_contains("App started");
```

### Lower-Level Escape Hatch

```rust
app.forward(vec![/* StudioToApp messages */]);
```

## Structured Widget State

Each snapshot record exposes:

- **widget id** - Unique identifier
- **widget type** - Widget class name
- **bounds** - Position and size
- **window id** - Window identifier
- **window index** - Window index
- **visible/enabled state** - Boolean flags
- **widget-specific state** (when available):
  - `text` - Label text
  - `value` - Input value
  - `checked` - Checkbox/toggle state
  - `selected` - Selection state

This covers common widgets (labels, buttons, text inputs, checkboxes, dock tabs, multi-window widgets) without scraping raw dumps.

## Failure Artifacts

Failed tests write to:

```
target/makepad_test/<package>/<test>/
```

**Typical contents:**
- `failure.txt` - Failure message
- `logs.txt` - Application logs
- `widget-snapshot.json` - Structured widget state
- `widget-tree.txt` or `widget-tree-error.txt` - Raw widget tree
- `failure-screenshot.png` or `failure-screenshot-error.txt` - Screenshot

If a capture step fails, the runtime writes a `*-error.txt` file instead of silently dropping the artifact.

## Running Tests

### Package-Local

```bash
cargo test -p makepad-example-text-input --test ui -- --test-threads=1
```

### Curated Repo Suite (macOS)

```bash
tools/run_ui_tests.sh
```

This runner executes:
- `makepad-example-text-input`
- `makepad-example-counter`
- `makepad-example-todo`
- `makepad-example-floating-panel`
- `makepad-example-splash`

and prints the artifact directory for each package.

## Visible Studio Mode

By default, `makepad_test` launches the app headlessly through an in-process hub. For local debugging, you can switch to a visible Studio-backed run:

```bash
MAKEPAD_TEST_VISIBLE=1 cargo test -p makepad-example-counter --test ui -- --test-threads=1
```

### Visible Mode Behavior

- Reuses the same `TestApp` and `Locator` APIs
- Connects to an already running Makepad Studio instance
- Clears older builds for the same package before launching
- Launches through Studio `Run`, so the app is visible in Studio's runview

### Environment Variables

- `MAKEPAD_TEST_VISIBLE=1` - Enable visible mode
- `MAKEPAD_TEST_STUDIO=127.0.0.1:8001` - Override Studio address
- `MAKEPAD_TEST_STUDIO_MOUNT=makepad` - Override mount name
- `MAKEPAD_TEST_STARTUP_DELAY_MS=1000` - Wait after app appears
- `MAKEPAD_TEST_ACTION_DELAY_MS=750` - Wait after each interaction
- `MAKEPAD_TEST_KEEP_OPEN_MS=3000` - Keep app open before shutdown

### Example with Pacing

```bash
MAKEPAD_TEST_VISIBLE=1 \
MAKEPAD_TEST_STARTUP_DELAY_MS=1000 \
MAKEPAD_TEST_ACTION_DELAY_MS=750 \
MAKEPAD_TEST_KEEP_OPEN_MS=3000 \
cargo test -p makepad-example-counter --test ui -- --test-threads=1
```

## Headless Transport

The runtime reuses the Studio protocol rather than inventing a separate automation channel:

- The hub runs in-process
- The app runs headless
- Widget snapshots, screenshots, and logs move through the Studio protocol
- Direct stdio is used for headless control where supported

This keeps the test surface aligned with how Studio itself talks to Makepad apps.

## Troubleshooting

### Test Times Out or Fails to Resolve Widget

1. **Inspect logs:**
   ```bash
   cat target/makepad_test/<package>/<test>/logs.txt
   ```

2. **Inspect widget snapshot:**
   ```bash
   cat target/makepad_test/<package>/<test>/widget-snapshot.json
   ```
   Look for `text`, `value`, `checked`, `selected` state.

3. **Inspect widget tree:**
   ```bash
   cat target/makepad_test/<package>/<test>/widget-tree.txt
   ```

4. **Verify selector is scoped tightly enough**

### Hub-Level Transport Diagnostics

```bash
MAKEPAD_STUDIO_HUB_DEBUG=1 cargo test -p makepad-example-text-input --test ui -- --test-threads=1
```

**Note:** Screenshot capture has a longer timeout than normal widget-state queries because PNG encoding and transport cost more than structured snapshot requests.

## Complete Example: Testing a Map Widget

### 1. Add Dependency

```toml
# crates/apps/map/Cargo.toml
[dev-dependencies]
makepad-test = { path = "../../../libs/makepad_test", version = "0.1.0" }
```

### 2. Create Test File

```rust
// crates/apps/map/tests/ui.rs
use makepad_test::{makepad_test, Selector, TestApp};

#[makepad_test]
fn map_renders(app: TestApp) {
    // Wait for map widget to be visible
    app.locator(Selector::id("map_view"))
        .wait_visible();
    
    // Take a screenshot
    let screenshot = app.screenshot();
    assert!(!screenshot.is_empty(), "Screenshot should not be empty");
}

#[makepad_test]
fn map_zoom_in(app: TestApp) {
    // Wait for map to be visible
    app.locator(Selector::id("map_view"))
        .wait_visible();
    
    // Click zoom in button
    app.locator(Selector::id("zoom_in_button"))
        .wait_visible()
        .click();
    
    // Wait for zoom level to update
    app.locator(Selector::id("zoom_label"))
        .wait_text("Zoom: 15");
}

#[makepad_test]
fn map_pan(app: TestApp) {
    // Wait for map to be visible
    app.locator(Selector::id("map_view"))
        .wait_visible();
    
    // Drag to pan
    app.locator(Selector::id("map_view"))
        .drag_by(100.0, 50.0);
    
    // Wait for coordinates to update
    app.locator(Selector::id("coords_label"))
        .wait_text("Center: 36.82, -1.29");
}

#[makepad_test]
fn map_shows_pois(app: TestApp) {
    // Wait for map to be visible
    app.locator(Selector::id("map_view"))
        .wait_visible();
    
    // Zoom in to show POIs
    app.locator(Selector::id("zoom_in_button"))
        .click();
    
    // Wait for POI markers to appear
    app.locator(Selector::widget_type("PoiMarker"))
        .wait_count(5);
}

#[makepad_test]
fn map_search(app: TestApp) {
    // Wait for search input
    app.locator(Selector::id("search_input"))
        .wait_visible()
        .fill("Nairobi");
    
    // Press Enter to search
    app.locator(Selector::id("search_input"))
        .press_key(KeyCode::Enter);
    
    // Wait for search results
    app.locator(Selector::id("search_results"))
        .wait_visible()
        .wait_count(10);
}
```

### 3. Run Tests

```bash
# Headless mode
cargo test -p nigig-map --test ui -- --test-threads=1

# Visible mode (for debugging)
MAKEPAD_TEST_VISIBLE=1 cargo test -p nigig-map --test ui -- --test-threads=1

# With pacing (for watching)
MAKEPAD_TEST_VISIBLE=1 \
MAKEPAD_TEST_STARTUP_DELAY_MS=1000 \
MAKEPAD_TEST_ACTION_DELAY_MS=750 \
MAKEPAD_TEST_KEEP_OPEN_MS=3000 \
cargo test -p nigig-map --test ui -- --test-threads=1
```

## Current Limitations

- **Current-package execution only** - Cannot test external packages
- **Synchronous API only** - No async test support
- **No visual diffing** - No built-in image comparison
- **No trace viewer** - No timeline visualization
- **Some complex widgets** - Need more structured state over time

**Milestone 1** is intentionally scoped around reliable Rust-local UI regression coverage first, with cross-platform expansion and richer tooling following after the harness stabilizes.

## Best Practices

### 1. Use Specific Selectors

```rust
// Good - specific
app.locator(Selector::id("submit_button"))

// Bad - too broad
app.locator(Selector::widget_type("Button"))
```

### 2. Wait Before Interacting

```rust
// Good - wait for visibility
app.locator(Selector::id("input"))
    .wait_visible()
    .fill("text");

// Bad - might not be ready
app.locator(Selector::id("input"))
    .fill("text");
```

### 3. Use Assertions for Immediate Checks

```rust
// Good - assert immediately
app.locator(Selector::id("label"))
    .assert_text("Hello");

// Bad - wait when you don't need to
app.locator(Selector::id("label"))
    .wait_text("Hello");
```

### 4. Inspect Failures

Always check failure artifacts when tests fail:
```bash
cat target/makepad_test/<package>/<test>/logs.txt
cat target/makepad_test/<package>/<test>/widget-snapshot.json
```

### 5. Use Visible Mode for Debugging

```bash
MAKEPAD_TEST_VISIBLE=1 \
MAKEPAD_TEST_ACTION_DELAY_MS=750 \
cargo test -p my-package --test ui -- --test-threads=1
```

## Resources

- [makepad_test README](https://github.com/makepad/makepad/blob/dev/libs/makepad_test/README.md)
- [makepad_test GUIDE](https://github.com/makepad/makepad/blob/dev/libs/makepad_test/GUIDE.md)
- [Makepad Examples](https://github.com/makepad/makepad/tree/dev/examples)
- [Studio Remote Protocol](https://github.com/makepad/makepad/blob/dev/AGENTS.md)

## Conclusion

`makepad_test` is Makepad's **official UI testing framework** that provides:

- ✅ Rust-native tests (no external tools)
- ✅ Headless and visible modes
- ✅ Structured selectors and locators
- ✅ Waits and assertions
- ✅ Failure artifacts (logs, screenshots, widget dumps)
- ✅ Studio protocol integration
- ✅ Cross-platform support (macOS first, others coming)

This is the **correct and current approach** for testing Makepad applications in 2026.

**Key Points:**
- Add `makepad-test` as a dev-dependency
- Use `#[makepad_test]` macro
- Write tests in `tests/ui.rs`
- Run with `cargo test --test ui -- --test-threads=1`
- Use visible mode for debugging
- Inspect failure artifacts when tests fail

The framework is actively developed and will gain more features (visual diffing, trace viewer, cross-platform support) in future milestones.