fe21c07d84
Nine agent skills for building Makepad/Rust games, ported from majidmanzarpour/threejs-game-skills. Same director-routed workflow and premium bar; runtime rewritten for this fork's game crates. - makepad-game-director entrypoint, routing, continuity, asset probe - makepad-gameplay-systems loop, movers vs rigid bodies, input, camera, netplay - makepad-aaa-graphics-builder lighting, shaders, budget, visual scorecard - makepad-game-ui-designer HUD, menus, touch and XR UI - makepad-debug-profiler defect bisection and profiling - makepad-qa-release verification ladder, evidence, packaging - makepad-3d-generator CC0 model search, casts, procedural geometry - makepad-image-generator texgen textures, palettes, sky, icons - makepad-audio-generator sample bank, mixer, material impacts, 3D audio Written against the real APIs in libs/game/*, libs/sim and apps/arcade: the game.* verb table, GameRenderer adaptive quality, the packed 6-float GameMeshVertex layout, script_mod! splash styling, makepad-test driving, and the BUDGETS.md numbers. No paid generation API is required - the CC0 library plus seeded makepad-game-gen replaces them. Includes install.sh (Codex/Claude), validate-skills.sh and a repo-aware probe_assets.sh; both scripts verified against this checkout.
128 lines
6.5 KiB
Markdown
128 lines
6.5 KiB
Markdown
---
|
||
name: makepad-audio-generator
|
||
description: "Source, synthesize and wire audio for Makepad games: the CC0 sound library, WAV/Ogg Vorbis decoding, sample banks, mixer and voice priority, 3D positional audio, material-based impact sounds, and the audio director. Use for SFX, ambience, UI sounds, music jingles, footsteps, impacts, engine sound and 'it needs sound' requests."
|
||
---
|
||
|
||
# Makepad Audio Generator
|
||
|
||
`makepad-game-audio` is a complete sampled-audio stack: decode, bank, mix, and gameplay-driven emission. No external audio API is needed.
|
||
|
||
## The crate
|
||
|
||
| Piece | Type | Role |
|
||
| --- | --- | --- |
|
||
| Decode | `decode(bytes) -> Result<Pcm, AudioError>`, `wav`, `ogg`, `vorbis` | WAV and Ogg Vorbis to PCM |
|
||
| Bank | `SampleBank`, `SampleId` | loaded samples, looked up by id |
|
||
| Mixer | `Mixer`, `VoiceSpec`, `VoiceHandle`, `Priority` | playback, voice management |
|
||
| Director | `AudioDirector`, `SoundEvent`, `Category`, `Placement` | gameplay events → sounds |
|
||
| Materials | `Material`, `MaterialPair`, `ImpactCurve` | what a collision sounds like |
|
||
| RNG | `LocalRng` | seeded variation |
|
||
|
||
3D positioning lives in `makepad-game-script::audio3d` (`Listener`), with emission helpers in `makepad-game-blocks::audio_emit`. Script verbs: `sfx`, `sfx_at`, `beep`, `jingle`, plus `ToneWave` / `AudioRequest` in the script dispatch layer.
|
||
|
||
The crate has **no dependencies** — decoding, mixing and the Vorbis implementation (`codebook`, `floor`, `mdct`, `residue`) are all in-tree.
|
||
|
||
## Sourcing
|
||
|
||
The CC0 library includes 556 sounds across seven Kenney packs — impact, interface, sci-fi, music jingles, UI, RPG and digital audio.
|
||
|
||
```bash
|
||
./apps/arcade/download_assets.sh
|
||
./apps/arcade/download_assets.sh --transcode # ogg -> wav, needs ffmpeg
|
||
```
|
||
|
||
Search by description through `makepad-game-assets` with `AUDIO_CATEGORIES`, the same index as models. On a score tie the kind the query implies wins — "metal clang" and "win music" are audio requests, "spaceship" is not.
|
||
|
||
Kenney audio ships `.ogg` only; the in-tree Vorbis decoder handles it, and `--transcode` produces WAV where ffmpeg exists.
|
||
|
||
## Synthesis
|
||
|
||
For UI beeps, tones and simple jingles, synthesize rather than ship a file: `beep` and `jingle` verbs with `ToneWave`. Generate PCM directly for procedural engine notes and pitch-swept effects — deterministic, zero download weight, and infinitely variable.
|
||
|
||
## Wiring audio to gameplay
|
||
|
||
**Drive from real events.** `dynamics.take_mover_impacts()` returns `Vec<MoverImpact>` — that is the correct source for impact sounds, decals and particle bursts. Detecting collisions a second time in gameplay code produces double-triggered sounds and drifting decals.
|
||
|
||
```rust
|
||
for impact in dynamics.take_mover_impacts() {
|
||
director.emit(SoundEvent {
|
||
category: Category::Impact,
|
||
placement: Placement::At(impact.pos),
|
||
// material pair + ImpactCurve select the sample and shape the gain
|
||
});
|
||
}
|
||
```
|
||
|
||
`Material` / `MaterialPair` / `ImpactCurve` are the point of this design: wood-on-stone and metal-on-metal should not be the same sample at the same volume, and impact energy should scale gain and brightness. This is what makes a world sound physical rather than annotated.
|
||
|
||
## Voice management
|
||
|
||
`Mixer` with `Priority`. A game emits far more sound events than it can usefully play — a wave of enemies produces dozens of footsteps per second, and playing all of them is both expensive and mud.
|
||
|
||
- **Cap concurrent voices** and let `Priority` evict. Player feedback outranks ambience always.
|
||
- **Cull by distance** before mixing, not after.
|
||
- **Coalesce**: many identical events in one frame become one voice, slightly louder.
|
||
- **Debounce** repeats of the same sample within a few tens of milliseconds — otherwise phase-cancelling copies produce a metallic artefact.
|
||
|
||
Uncapped voices are the standard audio performance and quality failure, in that order.
|
||
|
||
## Variation
|
||
|
||
Repetition is what makes game audio sound cheap. With `LocalRng`:
|
||
|
||
- **Pitch**: ±5–10% random per trigger. The single highest-value line of audio code in most games.
|
||
- **Gain**: ±10%.
|
||
- **Sample choice**: 3–5 variants per common event, round-robin with a shuffle rather than pure random (pure random repeats audibly).
|
||
|
||
Seed it: `LocalRng`, not thread RNG, so networked clients agree.
|
||
|
||
## 3D audio
|
||
|
||
`audio3d::Listener` plus `Placement`. Attach the listener to the camera, not the player — the player hears what the camera sees, and decoupling them is disorienting.
|
||
|
||
Distance attenuation should be roughly inverse-square with a near-field clamp (or sounds at the listener's position become infinitely loud), plus a maximum range beyond which the voice is culled entirely. Pan by the listener's right vector. Keep UI sounds and player feedback **non-positional** — a hit confirmation that pans is a hit confirmation that gets missed.
|
||
|
||
## The mix
|
||
|
||
Rough starting levels, then tune by ear at a realistic volume:
|
||
|
||
| Category | Level |
|
||
| --- | --- |
|
||
| Player feedback (hit, pickup, damage) | loudest |
|
||
| UI | just below |
|
||
| Nearby gameplay | mid |
|
||
| Distant gameplay | low |
|
||
| Ambience | quiet, continuous |
|
||
| Music | under everything, ducking on important events |
|
||
|
||
Duck ambience and music when player feedback fires. Provide master / SFX / music sliders in settings; they are expected, and they are the accessibility fallback.
|
||
|
||
## Feedback stacking
|
||
|
||
A hit must land on several channels in the same frame: HUD change, camera shake, particle burst, **and** sound. Sound is the channel players notice missing first, and the one most often left until last.
|
||
|
||
## Verification
|
||
|
||
`makepad-game-audio` is Cx-free, so it unit-tests headlessly:
|
||
|
||
```bash
|
||
cargo test -p makepad-game-audio
|
||
```
|
||
|
||
Assert: decode produces the expected sample rate, channel count and length; the mixer never exceeds the voice cap; priority evicts the right voice; the material table maps every pair a scene can produce; and no `emit` path can allocate unboundedly.
|
||
|
||
Then **listen**. Play the game for a few minutes: is anything repeating audibly, is anything clipping, does the mix hold when the action peaks, and is there silence where something should have been heard?
|
||
|
||
## Checklist
|
||
|
||
- [ ] Every impact, pickup, and UI action has a sound
|
||
- [ ] Impacts driven by `take_mover_impacts()`, not polling
|
||
- [ ] Material pairs mapped for every surface combination in the scene
|
||
- [ ] Pitch and gain varied per trigger, seeded
|
||
- [ ] 3–5 variants for high-frequency events
|
||
- [ ] Voice cap enforced, priority set, distance culling on
|
||
- [ ] Listener on the camera
|
||
- [ ] UI and player feedback non-positional
|
||
- [ ] Ambience ducks under feedback
|
||
- [ ] Volume sliders present and persisted
|
||
- [ ] Nothing clips at peak action
|