97e9572f42
Apps ask the hub for a recognizer or a voice and get one; where it runs is the hub's decision. AiHub::start_stt / start_tts return poll-driven sessions shaped like the chat session. The Auto ladder is Whisper/Kokoro in this process (weights present, machine election), on the machine node over loopback, on a LAN node, else the OS engine; SpeechReach::Local is the "don't reach out" knob. Audio always comes back as PCM: the app owns the device. Three layers: - makepad-ai-speech is the whole speech model family, engines only. libs/voice (Whisper + Silero VAD) folds in as the `whisper` and `vad` modules next to kokoro and indextts, each a cargo feature; the Apple bridges and the Speaker/VoiceTranscriber selection leave it. - makepad-system-speech (new) is the OS speech services as blocking fns: Apple SpeechAnalyzer/AVSpeechSynthesizer via Swift, Windows.Media.Speech* on the vendored bindings, Android SpeechRecognizer/TextToSpeech through MakepadSpeech.java (API 26 floor), espeak-ng on Linux. It models the two STT shapes honestly: PCM in (Whisper, Apple) versus an engine that owns the microphone (Android, Windows), with capabilities the caller reads. - the hub grows speech sessions, in-process Whisper/Kokoro workers with the residency election, a `whisper` wire backend (stt domain, registry entry pinned to ggerganov/whisper.cpp) so a Mac can serve a Quest, and a `language` field on the generate request. Consumers: the Window voice input runs on an STT session and switches to engine-mic mode when the recognizer owns the microphone; converse's SpeechOutput is a lazily started TTS session plus a pump thread; route drops its private speech copy for converse; vj's lyrics fallback and the alignment bakes call the engines directly. Verified here: speech-roundtrip through the real sessions (Apple voice in, in-process Whisper on Metal out, 4.3% WER); system-speech-test TTS->STT verbatim; hub/converse/system-speech unit tests; msvc, aarch64-android and linux-gnu cross-checks; Java against android-34. Windows, Android and Linux bridges are compile-checked only. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
125 lines
1.1 KiB
Text
125 lines
1.1 KiB
Text
0.001670
|
|
0.006884
|
|
0.008911
|
|
0.007857
|
|
0.005907
|
|
0.005961
|
|
0.005853
|
|
0.005640
|
|
0.005431
|
|
0.005199
|
|
0.005048
|
|
0.004903
|
|
0.004736
|
|
0.004585
|
|
0.004447
|
|
0.004328
|
|
0.004227
|
|
0.004146
|
|
0.004075
|
|
0.004013
|
|
0.003958
|
|
0.003909
|
|
0.003866
|
|
0.003827
|
|
0.003792
|
|
0.003761
|
|
0.003733
|
|
0.003707
|
|
0.003684
|
|
0.003662
|
|
0.003643
|
|
0.046922
|
|
0.091335
|
|
0.032943
|
|
0.016851
|
|
0.012011
|
|
0.011462
|
|
0.011553
|
|
0.015711
|
|
0.016051
|
|
0.020546
|
|
0.012831
|
|
0.016623
|
|
0.013786
|
|
0.009262
|
|
0.012462
|
|
0.010707
|
|
0.009211
|
|
0.011519
|
|
0.009395
|
|
0.008641
|
|
0.011720
|
|
0.006910
|
|
0.011767
|
|
0.011173
|
|
0.011786
|
|
0.009410
|
|
0.010207
|
|
0.006543
|
|
0.008935
|
|
0.004527
|
|
0.008444
|
|
0.308113
|
|
0.886465
|
|
0.800355
|
|
0.841162
|
|
0.941019
|
|
0.952068
|
|
0.931752
|
|
0.951532
|
|
0.996996
|
|
0.987273
|
|
0.963912
|
|
0.916398
|
|
0.934681
|
|
0.976599
|
|
0.959740
|
|
0.958482
|
|
0.995248
|
|
0.991659
|
|
0.963664
|
|
0.833650
|
|
0.852119
|
|
0.798673
|
|
0.885277
|
|
0.861838
|
|
0.790138
|
|
0.544370
|
|
0.507743
|
|
0.693386
|
|
0.480111
|
|
0.320396
|
|
0.280049
|
|
0.260582
|
|
0.306071
|
|
0.276967
|
|
0.223136
|
|
0.351467
|
|
0.302804
|
|
0.240960
|
|
0.218990
|
|
0.179366
|
|
0.203898
|
|
0.217385
|
|
0.176540
|
|
0.144584
|
|
0.161964
|
|
0.137464
|
|
0.198509
|
|
0.135460
|
|
0.122961
|
|
0.107259
|
|
0.162894
|
|
0.119223
|
|
0.155010
|
|
0.121629
|
|
0.097150
|
|
0.098875
|
|
0.105571
|
|
0.120456
|
|
0.121552
|
|
0.104072
|
|
0.136501
|
|
0.093362
|
|
0.119742
|