97e9572f42
Apps ask the hub for a recognizer or a voice and get one; where it runs is the hub's decision. AiHub::start_stt / start_tts return poll-driven sessions shaped like the chat session. The Auto ladder is Whisper/Kokoro in this process (weights present, machine election), on the machine node over loopback, on a LAN node, else the OS engine; SpeechReach::Local is the "don't reach out" knob. Audio always comes back as PCM: the app owns the device. Three layers: - makepad-ai-speech is the whole speech model family, engines only. libs/voice (Whisper + Silero VAD) folds in as the `whisper` and `vad` modules next to kokoro and indextts, each a cargo feature; the Apple bridges and the Speaker/VoiceTranscriber selection leave it. - makepad-system-speech (new) is the OS speech services as blocking fns: Apple SpeechAnalyzer/AVSpeechSynthesizer via Swift, Windows.Media.Speech* on the vendored bindings, Android SpeechRecognizer/TextToSpeech through MakepadSpeech.java (API 26 floor), espeak-ng on Linux. It models the two STT shapes honestly: PCM in (Whisper, Apple) versus an engine that owns the microphone (Android, Windows), with capabilities the caller reads. - the hub grows speech sessions, in-process Whisper/Kokoro workers with the residency election, a `whisper` wire backend (stt domain, registry entry pinned to ggerganov/whisper.cpp) so a Mac can serve a Quest, and a `language` field on the generate request. Consumers: the Window voice input runs on an STT session and switches to engine-mic mode when the recognizer owns the microphone; converse's SpeechOutput is a lazily started TTS session plus a pump thread; route drops its private speech copy for converse; vj's lyrics fallback and the alignment bakes call the engines directly. Verified here: speech-roundtrip through the real sessions (Apple voice in, in-process Whisper on Metal out, 4.3% WER); system-speech-test TTS->STT verbatim; hub/converse/system-speech unit tests; msvc, aarch64-android and linux-gnu cross-checks; Java against android-34. Windows, Android and Linux bridges are compile-checked only. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
94 lines
846 B
Text
94 lines
846 B
Text
0.001670
|
|
0.006884
|
|
0.008911
|
|
0.007857
|
|
0.005907
|
|
0.005961
|
|
0.005976
|
|
0.024519
|
|
0.027974
|
|
0.189549
|
|
0.989981
|
|
0.988843
|
|
0.993698
|
|
0.979928
|
|
0.965923
|
|
0.999283
|
|
0.999337
|
|
0.999410
|
|
0.998439
|
|
0.999663
|
|
0.999359
|
|
0.999934
|
|
0.999934
|
|
0.999953
|
|
0.999978
|
|
0.999982
|
|
0.999974
|
|
0.999986
|
|
0.999986
|
|
0.999993
|
|
0.999980
|
|
0.999936
|
|
0.999793
|
|
0.999920
|
|
0.999796
|
|
0.997782
|
|
0.510518
|
|
0.067032
|
|
0.020053
|
|
0.022056
|
|
0.197112
|
|
0.982401
|
|
0.998537
|
|
0.999205
|
|
0.999432
|
|
0.999953
|
|
0.999132
|
|
0.999887
|
|
0.999988
|
|
0.999993
|
|
0.999998
|
|
0.999996
|
|
0.999994
|
|
0.999996
|
|
0.999998
|
|
0.999997
|
|
0.999976
|
|
0.999993
|
|
0.999989
|
|
0.999995
|
|
0.999998
|
|
0.999978
|
|
0.999981
|
|
0.999998
|
|
0.999997
|
|
0.999999
|
|
0.999996
|
|
0.999994
|
|
0.999997
|
|
0.999999
|
|
0.999998
|
|
0.999997
|
|
0.999997
|
|
0.999996
|
|
0.999989
|
|
0.999937
|
|
0.999964
|
|
0.999891
|
|
0.999835
|
|
0.999562
|
|
0.991714
|
|
0.257646
|
|
0.025675
|
|
0.010938
|
|
0.008862
|
|
0.008489
|
|
0.007878
|
|
0.007458
|
|
0.007176
|
|
0.006921
|
|
0.006759
|
|
0.006662
|
|
0.006630
|
|
0.006659
|