makepad/widgets/families/voice/src/voice_wave.rs
Admin da58218587 draw, widgets: the library is makepad-widgets-core plus a crate per widget family, eighty new widgets and the storybook, relief-lit skeuomorphic surfaces, the feedback panel, keyboard focus and tooltips clear of the cursor
makepad-widgets becomes a front crate over widgets/core and widgets/families (each family a crate of its own, linked by features, so an app compiles only the families it uses); widgets/feedback is the Send feedback panel. The dev PRs (#1241-#1264) are ported to the new layout, as in the merge of dev into work.

Covers work ebf442a94..2b1a41df4 for draw widgets xr code_editor:

- widgets, storybook: the widget catalogue app, some eighty new widgets, a theme store with mixable style sheets, and the rule that a press belongs to
- wm, widgets: the style tween starts with all eight weights, and the new style sheet takes the last number instead of macOS's
- platform: the Android desk swipes at 120 Hz — offscreen render passes and framebuffers cached, SVG meshes kept, font families that complete
- Button: activate from the keyboard when focused (#1241)
- app_main!: only ship the fonts an app declares (#1247)
- platform, widgets, wm: every Vulkan pass renders through a negative-height viewport again, so a Wayland or X11 window stands upright, and no capture
- widgets: the supersampled resolve and the map's shadow mask sample their textures as stored, like every other render texture
- widgets, platform: the last twelve warnings go
- cef, video, platform: what Stage's browser needs from the shared layers
- apps: what the CI box found
- ScrollBar: add `show_handle` for a view that scrolls without a grabbable bar (#1254)
- widgets: children a lookup discovers reach the dump, the snapshot and the flood searches
- tests: the workspace suite, run as a whole for the first time, passes outside the example UI tests
- examples: the UI tests pass, for the reasons they failed
- libs: no warnings in the workspace check on any row
- tests: the last three failures of the CI box's night
- Linux: fix window chrome button hovers and how maximized/fullscreen windows work (#1255)
- Tooltip: position anchored tooltips in the same draw (#1256)
- tests: the three binaries over ten seconds in release come under it
- widgets: let the host install script mods into every Splash isolate
- widgets: a pooled test context forgets the last case's Escape claim
- widgets: the pooled test context's resets replace the globals
- RadioButton: fix its touch hover state and click-off behavior (#1258)
- HtmlLink: fix its hover and pressed states (#1257)
- RadioButton: take key focus on the click, not on the press (#1259)
- feat(text): CoreText outline fallback for hvgl-only fonts (macOS)
- draw text: the CoreText outline fallback is only ever resolved for a face whose outlines live solely in hvgl, and font-family diagnostics are the `f
- platform, widgets, wm: a touch that is taken away is cancelled, never released
- wm on Android: every app is its own process
- widgets: touch lists scroll with Android's physics
- task, cargo-makepad, widgets, audio: the task manager chooses its columns, graphs every process' network and disk traffic and installs itself as a D
- widgets, draw: skeuomorphic surfaces light each other through an optional relief buffer
- widgets, platform: a texture can light the relief buffer, menus take colour chips, and a style reload recompiles changed shader functions
- widgets: relief surfaces can travel like mechanical keys, and dark surfaces take less neighbour light
- platform, widgets, audio_route, ai: window crossfades and whole-frame presents, caption controls that click, themed menus, a stereo-pair audio tap,
- platform, widgets, ai: a restyle recompiles nothing it already has, shows only complete frames, and a window can own its caption
- widgets: relief surfaces can swing their light toward a point
- widgets, builder: the big widget families are features, all on by default, and the Builder takes none of them
- ai: Claude Desktop drives any app with the F10 panel
- Constrain Splash external I/O to the host service bridge (#1243)
- PortalList: stop following the end when scrolling to an earlier item (#1261)
- draw: glass of a view that stopped drawing leaves the screen
- widgets: the Widget lookup methods and with_script_vm_id are compiled once, not once per widget type
- widgets: the library moves to makepad-widgets-core in widgets/core, and makepad-widgets becomes its front crate
- widgets: the Window, widget tree and panel theme reach the tweaker, voice, AI slot and dock through hooks, not through their modules
- widgets: every widget family is a crate of its own under widgets/families, and makepad-widgets links, re-exports and registers the ones its features
- aichat, widgets: the chat reads the design feedback through the tweaker's hook, so linking the chat no longer needs the tweaker
- apps, examples, libs: each crate builds only the widget families it uses, and the design overlay is each app's own default feature
- widgets: the build scripts rerun on their own edits only, and makepad-widgets no longer reads MAKEPAD
- feedback: Send feedback, a caption icon and a small panel that shows exactly what is sent
- feedback: the panel's wording reads right and draws in any font
- feedback: takes makepad-widgets without its default families
- widgets: the empty parts of an app's caption bar drag the window again
- ScrollBar: fade out when idle, like macOS overlay scrollers (#1264)
- platform, draw: web builds start again
- widgets: the keyboard reaches boxes and modals
- widgets: a hover tooltip never sits under the mouse cursor
- feedback: Send clipboard, and the dialog works from the keyboard

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 20:10:50 +02:00

754 lines
25 KiB
Rust

use crate::{
makepad_derive_widget::*,
makepad_draw::*,
view::View,
widget::*,
window_voice_input::{VoiceInjectEvent, WindowVoiceInput},
};
const VOICE_TARGET_SAMPLE_RATE: usize = 16_000;
const VOICE_LOOP_SECONDS: usize = 1;
const WAVE_SAMPLES_PER_TEXEL: usize = 160; // 10ms @ 16k
const WAVE_TEX_WIDTH: usize =
(VOICE_TARGET_SAMPLE_RATE * VOICE_LOOP_SECONDS) / WAVE_SAMPLES_PER_TEXEL;
const DISPLAY_TARGET_PEAK: f32 = 0.72;
const DISPLAY_MIN_REF: f32 = 0.003;
const DISPLAY_SILENCE_PEAK: f32 = 0.005;
const DISPLAY_MAX_GAIN: f32 = 10.0;
const DISPLAY_ATTACK: f32 = 0.20;
const DISPLAY_RELEASE: f32 = 0.06;
const DISPLAY_GAIN_UP: f32 = 0.08;
const DISPLAY_GAIN_DOWN: f32 = 0.22;
const DISPLAY_MU: f32 = 14.0;
const DISPLAY_SILENCE_CARRIER_BASE: f32 = 0.006;
const DISPLAY_SILENCE_CARRIER_SWING: f32 = 0.004;
#[derive(Clone, Debug, Default)]
pub enum VoiceWaveAction {
#[default]
None,
Clicked(KeyModifiers),
RecordVoice(bool),
InjectText(String),
InjectEnter,
/// Speech energy appeared/disappeared on the open mic (RMS gate).
/// Emitted on change; apps use the rising edge for TTS barge-in.
VoiceActivity(bool),
}
script_mod! {
use mod.prelude.widgets_internal.*
set_type_default() do #(DrawVoiceWave::script_shader(vm)){
..mod.draw.DrawQuad
wave_tex: texture_2d(float)
mic_on: 0.0
voice_active: 0.0
submit_flash: 0.0
ring_head: 0.0
pixel: fn() {
let p = self.pos - vec2(0.5, 0.5)
let r = length(p)
let tau = 6.2831855
let angle = atan2(p.y, p.x)
let u = fract(angle / tau + 0.5 + self.ring_head + 0.25)
let sample = self.wave_tex.sample(vec2(u, 0.5))
// Decode two packed u16 values from one BGRA texel.
// high16 -> (a,r), low16 -> (g,b)
let hi_u = sample.w + sample.x / 256.0
let lo_u = sample.y + sample.z / 256.0
let v0 = clamp((hi_u - 0.5) * 2.0, -1.0, 1.0)
let v1 = clamp((lo_u - 0.5) * 2.0, -1.0, 1.0)
let amp = clamp(max(abs(v0), abs(v1)), 0.0, 1.0)
let amp_vis = clamp(amp * 2.0, 0.0, 1.0)
let ring_center = 0.41
let ring_half = max((0.015 + amp_vis * 0.074) * self.mic_on, 0.0001)
let ring_dist = abs(r - ring_center)
let ring_mask = clamp(1.0 - ring_dist / ring_half, 0.0, 1.0)
let halo_dist = abs(r - (ring_center + ring_half * 0.75))
let halo_mask = clamp(1.0 - halo_dist / (0.018 + amp_vis * 0.024), 0.0, 1.0)
* 0.36
* self.mic_on
let button_radius = 0.23
let center_radius = 0.15
let button_mask = clamp(1.0 - (r - button_radius) * 115.0, 0.0, 1.0)
let center_mask = clamp(1.0 - (r - center_radius) * 150.0, 0.0, 1.0)
let active_mix = clamp(self.voice_active + amp_vis * 0.7 * self.mic_on, 0.0, 1.0)
let ring_off = vec3(0.56, 0.59, 0.58)
let ring_on = vec3(1.0, 0.30, 0.34)
let ring_hot = vec3(1.0, 0.50, 0.46)
let ring_submit = vec3(1.0, 0.78, 0.33)
let button_off = vec3(0.28, 0.30, 0.30)
let button_on = vec3(0.90, 0.16, 0.20)
let button_submit = vec3(1.0, 0.56, 0.22)
let mut ring_color = ring_off.mix(ring_on, self.mic_on)
ring_color = ring_color.mix(ring_hot, active_mix * 0.7)
ring_color = ring_color.mix(ring_submit, self.submit_flash)
let mut button_color = button_off.mix(button_on, self.mic_on)
button_color = button_color.mix(button_submit, self.submit_flash)
let center_hot = vec3(1.0, 0.36, 0.38).mix(vec3(1.0, 0.66, 0.36), self.submit_flash)
let center_color = button_color.mix(center_hot, 0.22 + 0.18 * self.voice_active)
let ring_alpha = clamp(ring_mask + halo_mask, 0.0, 1.0)
let ring_fade = clamp(1.0 - (r - 0.50) * 18.0, 0.0, 1.0)
let ring_alpha = ring_alpha * ring_fade
let button_alpha = clamp(button_mask, 0.0, 1.0)
let center_alpha = center_mask * button_alpha
let mut color = vec3(0.0, 0.0, 0.0)
color = color.mix(ring_color, ring_alpha)
color = color.mix(button_color, button_alpha)
color = color.mix(center_color, center_alpha)
let alpha = clamp(ring_alpha + button_alpha, 0.0, 1.0)
return vec4(color, alpha)
}
}
mod.widgets.VoiceWaveBase = #(VoiceWave::register_widget(vm))
mod.widgets.VoiceWave = set_type_default() do mod.widgets.VoiceWaveBase {
width: 24
height: 24
margin: Inset{top: 1 right: 8}
}
}
#[derive(Script, ScriptHook)]
#[repr(C)]
pub struct DrawVoiceWave {
#[deref]
draw_super: DrawQuad,
#[live]
mic_on: f32,
#[live]
voice_active: f32,
#[live]
submit_flash: f32,
#[live]
ring_head: f32,
}
#[derive(Script, ScriptHook, Widget)]
pub struct VoiceWave {
#[uid]
uid: WidgetUid,
#[source]
source: ScriptObjectRef,
#[walk]
walk: Walk,
#[redraw]
#[live]
draw_bg: DrawVoiceWave,
#[visible]
#[live(true)]
visible: bool,
#[new]
wave_texture: Texture,
#[rust]
texture_ready: bool,
#[rust]
append_min: f32,
#[rust]
append_max: f32,
#[rust]
display_env: f32,
#[rust]
display_gain: f32,
#[rust]
silence_phase: f32,
#[rust]
append_count: usize,
#[rust]
write_index: usize,
#[rust]
silence_bins_run: usize,
#[rust]
silence_frozen: bool,
#[rust]
animating: bool,
#[rust]
reset_on_next_append: bool,
#[rust]
voice_input: WindowVoiceInput,
#[rust]
voice_initialized: bool,
#[rust]
last_voice_active: bool,
/// Also accept Escape as a push-to-talk key (opt-in per app — Escape
/// doubles as cancel/dismiss elsewhere, so hosts choose).
#[live(false)]
ptt_use_escape: bool,
#[rust]
ptt_f1_down: bool,
#[rust]
ptt_owns_capture: bool,
}
impl VoiceWave {
fn ensure_texture(&mut self, cx: &mut Cx) {
if !self.texture_ready {
self.wave_texture = Texture::new_with_format(
cx,
TextureFormat::VecBGRAu8_32 {
data: Some(vec![Self::min_max_to_texel(0.0, 0.0); WAVE_TEX_WIDTH]),
width: WAVE_TEX_WIDTH,
height: 1,
updated: TextureUpdated::Full,
},
);
self.texture_ready = true;
}
}
fn update_ring_head(&mut self) {
self.draw_bg.ring_head = self.write_index as f32 / WAVE_TEX_WIDTH as f32;
}
fn signed_to_u16(v: f32) -> u32 {
(((v.clamp(-1.0, 1.0) * 0.5) + 0.5) * 65535.0).round() as u32
}
fn compand_mu(v: f32) -> f32 {
let x = v.clamp(-1.0, 1.0);
let mag = x.abs();
let y = ((1.0 + DISPLAY_MU * mag).ln() / (1.0 + DISPLAY_MU).ln()).clamp(0.0, 1.0);
if x < 0.0 {
-y
} else {
y
}
}
fn gain_from_ref(reference_level: f32) -> f32 {
if reference_level <= DISPLAY_MIN_REF {
1.0
} else {
(DISPLAY_TARGET_PEAK / reference_level).clamp(1.0, DISPLAY_MAX_GAIN)
}
}
fn normalize_min_max(min_v: f32, max_v: f32, gain: f32) -> (f32, f32) {
(
Self::compand_mu(min_v * gain),
Self::compand_mu(max_v * gain),
)
}
fn min_max_to_texel(min_v: f32, max_v: f32) -> u32 {
let lo = Self::signed_to_u16(min_v.min(max_v));
let hi = Self::signed_to_u16(max_v.max(min_v));
(hi << 16) | lo
}
fn push_bin(&mut self, wave_buf: &mut [u32], min_v: f32, max_v: f32) -> bool {
let peak = min_v.abs().max(max_v.abs());
let is_silent_bin = peak < DISPLAY_SILENCE_PEAK;
if is_silent_bin && self.silence_frozen {
return false;
}
if peak < DISPLAY_SILENCE_PEAK {
self.display_env += (DISPLAY_MIN_REF - self.display_env) * 0.08;
self.silence_phase = (self.silence_phase + 0.35) % 6.2831855;
} else {
let coeff = if peak > self.display_env {
DISPLAY_ATTACK
} else {
DISPLAY_RELEASE
};
self.display_env += (peak - self.display_env) * coeff;
}
let target_gain = Self::gain_from_ref(self.display_env);
let gain_coeff = if target_gain < self.display_gain {
DISPLAY_GAIN_DOWN
} else {
DISPLAY_GAIN_UP
};
self.display_gain += (target_gain - self.display_gain) * gain_coeff;
let (mut min_raw, mut max_raw) = Self::normalize_min_max(min_v, max_v, self.display_gain);
if peak < DISPLAY_SILENCE_PEAK {
let carrier = DISPLAY_SILENCE_CARRIER_BASE
+ DISPLAY_SILENCE_CARRIER_SWING * self.silence_phase.sin().abs();
let center = ((min_raw + max_raw) * 0.5).clamp(-0.03, 0.03);
min_raw = (center - carrier).clamp(-0.08, 0.08);
max_raw = (center + carrier).clamp(-0.08, 0.08);
}
wave_buf[self.write_index] = Self::min_max_to_texel(min_raw, max_raw);
self.write_index = (self.write_index + 1) % WAVE_TEX_WIDTH;
self.update_ring_head();
if is_silent_bin {
self.silence_bins_run = (self.silence_bins_run + 1).min(WAVE_TEX_WIDTH);
if self.silence_bins_run >= WAVE_TEX_WIDTH {
self.silence_frozen = true;
self.animating = false;
} else {
self.animating = true;
}
} else {
self.silence_bins_run = 0;
self.silence_frozen = false;
self.animating = true;
}
true
}
pub fn clear_wave(&mut self, cx: &mut Cx) {
self.ensure_texture(cx);
let mut wave_buf = self.wave_texture.take_vec_u32(cx);
wave_buf.fill(Self::min_max_to_texel(0.0, 0.0));
self.wave_texture.put_back_vec_u32(cx, wave_buf, None);
self.append_min = 0.0;
self.append_max = 0.0;
self.display_env = DISPLAY_MIN_REF;
self.display_gain = 1.0;
self.silence_phase = 0.0;
self.append_count = 0;
self.write_index = 0;
self.silence_bins_run = 0;
self.silence_frozen = false;
self.animating = false;
self.update_ring_head();
self.reset_on_next_append = false;
self.draw_bg.redraw(cx);
}
pub fn set_wave_chunk(&mut self, cx: &mut Cx, samples: &[f32]) {
self.ensure_texture(cx);
let mut wave_buf = self.wave_texture.take_vec_u32(cx);
wave_buf.fill(Self::min_max_to_texel(0.0, 0.0));
self.append_count = 0;
self.display_env = DISPLAY_MIN_REF;
self.display_gain = 1.0;
self.silence_phase = 0.0;
self.write_index = 0;
self.silence_bins_run = 0;
self.silence_frozen = false;
self.animating = true;
if !samples.is_empty() {
let keep = (WAVE_TEX_WIDTH * WAVE_SAMPLES_PER_TEXEL).min(samples.len());
let start = samples.len() - keep;
let recent = &samples[start..];
for window in recent.chunks(WAVE_SAMPLES_PER_TEXEL) {
let mut min_v = 1.0f32;
let mut max_v = -1.0f32;
for s in window {
min_v = min_v.min(*s);
max_v = max_v.max(*s);
}
if min_v > max_v {
min_v = 0.0;
max_v = 0.0;
}
let _ = self.push_bin(&mut wave_buf, min_v, max_v);
}
}
self.wave_texture.put_back_vec_u32(cx, wave_buf, None);
self.append_min = 0.0;
self.append_max = 0.0;
self.append_count = 0;
self.reset_on_next_append = false;
self.draw_bg.redraw(cx);
}
pub fn append_samples(&mut self, cx: &mut Cx, samples: &[f32]) {
if samples.is_empty() {
return;
}
self.ensure_texture(cx);
if self.reset_on_next_append {
self.clear_wave(cx);
}
let mut wave_buf = self.wave_texture.take_vec_u32(cx);
let mut wrote_any = false;
for s in samples {
if self.append_count == 0 {
self.append_min = *s;
self.append_max = *s;
} else {
self.append_min = self.append_min.min(*s);
self.append_max = self.append_max.max(*s);
}
self.append_count += 1;
if self.append_count >= WAVE_SAMPLES_PER_TEXEL {
wrote_any |= self.push_bin(&mut wave_buf, self.append_min, self.append_max);
self.append_count = 0;
}
}
self.wave_texture.put_back_vec_u32(cx, wave_buf, None);
if wrote_any {
self.draw_bg.redraw(cx);
}
}
pub fn set_mic_on(&mut self, cx: &mut Cx, on: bool) {
self.draw_bg.mic_on = if on { 1.0 } else { 0.0 };
if !on {
self.clear_wave(cx);
} else {
self.silence_bins_run = 0;
self.silence_frozen = false;
self.animating = true;
self.draw_bg.redraw(cx);
}
}
pub fn set_voice_active(&mut self, cx: &mut Cx, active: bool) {
self.draw_bg.voice_active = if active { 1.0 } else { 0.0 };
self.draw_bg.redraw(cx);
}
pub fn set_submit_flash(&mut self, cx: &mut Cx, flash: bool) {
self.draw_bg.submit_flash = if flash { 1.0 } else { 0.0 };
if flash {
self.reset_on_next_append = true;
}
self.draw_bg.redraw(cx);
}
pub fn is_animating(&self) -> bool {
self.animating
}
fn ensure_voice_initialized(&mut self, cx: &mut Cx) {
if self.voice_initialized {
return;
}
self.voice_initialized = true;
log!("voice: init voice_wave uid={:?}", self.widget_uid());
self.voice_input.ensure_audio_callback(cx, 0);
}
fn sync_mic_state(&mut self, cx: &mut Cx) {
let enabled = self.voice_input.is_enabled();
self.set_mic_on(cx, enabled);
if !enabled {
self.voice_input.clear_pending();
self.set_voice_active(cx, false);
self.set_submit_flash(cx, false);
}
self.drain_and_refresh_voice(cx);
}
fn emit_inject_events(&self, cx: &mut Cx, events: Vec<VoiceInjectEvent>) {
let uid = self.widget_uid();
for event in events {
match event {
VoiceInjectEvent::Text(text) => {
cx.widget_action(uid, VoiceWaveAction::InjectText(text));
}
VoiceInjectEvent::Enter => {
cx.widget_action(uid, VoiceWaveAction::InjectEnter);
}
}
}
}
/// Drain pending wave samples from voice_input into our own waveform display,
/// then refresh visual state (voice_active, submit_flash, next_frame scheduling).
fn drain_and_refresh_voice(&mut self, cx: &mut Cx) {
// Drain pending wave samples
while let Some(chunk) = self.voice_input.take_wave_chunk() {
self.append_samples(cx, &chunk);
}
// Refresh visual state
let state = self.voice_input.visual_state();
if state.voice_active != self.last_voice_active {
self.last_voice_active = state.voice_active;
cx.widget_action(
self.widget_uid(),
VoiceWaveAction::VoiceActivity(state.voice_active),
);
}
self.set_voice_active(cx, state.voice_active);
self.set_submit_flash(cx, state.submit_flash);
if self.animating
|| state.voice_active
|| state.submit_flash
|| state.pending_wave
|| state.pending_inject
{
self.voice_input.request_next_frame(cx);
}
}
fn set_enabled(&mut self, cx: &mut Cx, enabled: bool) {
self.ensure_voice_initialized(cx);
self.voice_input.set_enabled(cx, enabled);
self.sync_mic_state(cx);
}
/// Load the speech models now (worker spawn + whisper preload) without
/// starting capture — eager AI boot without a hot mic.
fn prewarm(&mut self, cx: &mut Cx) {
self.ensure_voice_initialized(cx);
}
/// See [`WindowVoiceInput::set_echo_cancellation`]: arm the ducking
/// voice-processing path only while the app's own audio plays.
fn set_echo_cancellation(&mut self, cx: &mut Cx, on: bool) {
self.voice_input.set_echo_cancellation(cx, on);
}
// NOTE: no ensure_voice_initialized here — initialization registers the
// GLOBAL audio-input callback slot (last registrant wins, platform-side),
// so a passive VoiceWave (e.g. the Window caption one) that inits on
// event traffic steals the mic audio from the active instance and every
// sample is silently discarded. Only set_enabled/prewarm may initialize.
fn handle_voice_event(&mut self, cx: &mut Cx, event: &Event) {
let uid = self.widget_uid();
if let Event::AudioDevices(devices) = event {
self.voice_input.handle_audio_devices(cx, devices);
self.sync_mic_state(cx);
}
if let Event::PermissionResult(result) = event {
if self.voice_input.handle_permission_result(cx, result) {
self.sync_mic_state(cx);
cx.widget_action(
uid,
VoiceWaveAction::RecordVoice(self.voice_input.is_enabled()),
);
}
}
if let Event::Signal = event {
let events = self.voice_input.process_signal_no_wave(cx);
self.drain_and_refresh_voice(cx);
self.emit_inject_events(cx, events);
}
if self.voice_input.is_timer_event(event) {
self.drain_and_refresh_voice(cx);
let events = self.voice_input.drain_ready_inject_events(cx);
self.emit_inject_events(cx, events);
}
if let Event::Shutdown = event {
self.voice_input.shutdown(cx);
}
// F1 push-to-talk (+ Escape when opted in) and Cmd/Ctrl+1 toggle.
// Both keys drive the same logical talk button.
let is_ptt_key = |key_code: KeyCode, use_escape: bool| {
key_code == KeyCode::F1 || (use_escape && key_code == KeyCode::Escape)
};
if let Event::KeyDown(key_event) = event {
if is_ptt_key(key_event.key_code, self.ptt_use_escape) && !key_event.is_repeat {
if !self.ptt_f1_down {
self.ptt_f1_down = true;
self.ptt_owns_capture = !self.voice_input.is_enabled();
if self.ptt_owns_capture {
self.set_enabled(cx, true);
cx.widget_action(uid, VoiceWaveAction::RecordVoice(true));
}
}
}
let is_hotkey = !key_event.is_repeat
&& (key_event.modifiers.logo || key_event.modifiers.control)
&& key_event.key_code == KeyCode::Key1;
if is_hotkey {
let enabled = !self.voice_input.is_enabled();
self.set_enabled(cx, enabled);
cx.widget_action(uid, VoiceWaveAction::RecordVoice(enabled));
}
}
if let Event::KeyUp(key_event) = event {
if is_ptt_key(key_event.key_code, self.ptt_use_escape) && self.ptt_f1_down {
self.ptt_f1_down = false;
if self.ptt_owns_capture {
self.ptt_owns_capture = false;
self.set_enabled(cx, false);
self.voice_input.arm_enter_after_next_transcript();
cx.widget_action(uid, VoiceWaveAction::RecordVoice(false));
}
}
}
}
}
impl Widget for VoiceWave {
fn handle_event(&mut self, cx: &mut Cx, event: &Event, _scope: &mut Scope) {
if !self.visible && !matches!(event, Event::FingerCancel(_)) {
return;
}
self.handle_voice_event(cx, event);
let uid = self.widget_uid();
match event.hits(cx, self.draw_bg.area()) {
Hit::FingerHoverIn(_) => {
cx.set_cursor(MouseCursor::Hand);
}
Hit::FingerUp(fe) => {
if fe.is_over {
let enabled = !self.voice_input.is_enabled();
self.set_enabled(cx, enabled);
cx.widget_action(uid, VoiceWaveAction::RecordVoice(enabled));
cx.widget_action(uid, VoiceWaveAction::Clicked(fe.modifiers));
}
}
_ => (),
}
}
fn draw_walk(&mut self, cx: &mut Cx2d, _scope: &mut Scope, walk: Walk) -> DrawStep {
if !self.visible {
return DrawStep::done();
}
self.ensure_texture(cx);
self.draw_bg.draw_vars.set_texture(0, &self.wave_texture);
self.draw_bg.draw_walk(cx, walk);
DrawStep::done()
}
}
/// Let the window hand its actions to the wave in its caption bar. Run by
/// the family's registration.
pub fn install_voice_hooks(cx: &mut Cx) {
crate::widget_hooks::WidgetHooks::of(cx).voice = Some(crate::widget_hooks::VoiceHooks {
window_actions,
});
}
fn window_actions(cx: &mut Cx, actions: &Actions, view: &mut View, scope: &mut Scope) {
let voice_wave = view.voice_wave(cx, ids!(voice_wave));
voice_wave.handle_actions(cx, actions, view, scope);
}
impl VoiceWaveRef {
pub fn clicked(&self, actions: &Actions) -> bool {
if let Some(item) = actions.find_widget_action(self.widget_uid()) {
matches!(item.cast(), VoiceWaveAction::Clicked(_))
} else {
false
}
}
/// Handle voice actions from the actions queue, injecting text/enter events
/// into the given view. Returns `Some(enabled)` when voice recording state changed.
pub fn handle_actions(
&self,
cx: &mut Cx,
actions: &Actions,
view: &mut View,
scope: &mut Scope,
) -> Option<bool> {
let mut result = None;
for action in actions.filter_widget_actions_cast::<VoiceWaveAction>(self.widget_uid()) {
match action {
VoiceWaveAction::RecordVoice(enabled) => {
result = Some(enabled);
}
VoiceWaveAction::InjectText(text) => {
let text_event = Event::TextInput(TextInputEvent {
input: text,
replace_last: false,
was_paste: false,
..Default::default()
});
view.handle_event(cx, &text_event, scope);
}
VoiceWaveAction::InjectEnter => {
let key = KeyEvent {
key_code: KeyCode::ReturnKey,
is_repeat: false,
modifiers: KeyModifiers::default(),
time: 0.0,
};
view.handle_event(cx, &Event::KeyDown(key), scope);
view.handle_event(cx, &Event::KeyUp(key), scope);
}
_ => {}
}
}
result
}
pub fn prewarm(&self, cx: &mut Cx) {
if let Some(mut inner) = self.borrow_mut() {
inner.prewarm(cx);
}
}
pub fn set_echo_cancellation(&self, cx: &mut Cx, on: bool) {
if let Some(mut inner) = self.borrow_mut() {
inner.set_echo_cancellation(cx, on);
}
}
pub fn set_enabled(&self, cx: &mut Cx, enabled: bool) {
if let Some(mut inner) = self.borrow_mut() {
inner.set_enabled(cx, enabled);
}
}
pub fn is_enabled(&self) -> bool {
if let Some(inner) = self.borrow() {
inner.voice_input.is_enabled()
} else {
false
}
}
pub fn clear_wave(&self, cx: &mut Cx) {
if let Some(mut inner) = self.borrow_mut() {
inner.clear_wave(cx);
}
}
pub fn set_wave_chunk(&self, cx: &mut Cx, samples: &[f32]) {
if let Some(mut inner) = self.borrow_mut() {
inner.set_wave_chunk(cx, samples);
}
}
pub fn append_samples(&self, cx: &mut Cx, samples: &[f32]) {
if let Some(mut inner) = self.borrow_mut() {
inner.append_samples(cx, samples);
}
}
pub fn set_mic_on(&self, cx: &mut Cx, on: bool) {
if let Some(mut inner) = self.borrow_mut() {
inner.set_mic_on(cx, on);
}
}
pub fn set_voice_active(&self, cx: &mut Cx, active: bool) {
if let Some(mut inner) = self.borrow_mut() {
inner.set_voice_active(cx, active);
}
}
pub fn set_submit_flash(&self, cx: &mut Cx, flash: bool) {
if let Some(mut inner) = self.borrow_mut() {
inner.set_submit_flash(cx, flash);
}
}
pub fn is_animating(&self) -> bool {
if let Some(inner) = self.borrow() {
inner.is_animating()
} else {
false
}
}
}