Content tagged with "voice ai"
Taking DualTurn from an Interspeech 2026 paper into a live voice agent: a ~1.5M-parameter model trained with zero manual labels, fastest end-to-end in real-call testing against cloud and open-source turn detectors, the only one in the industry that tells a backchannel from a real interruption, and set up to keep improving for free from every call it handles.
The checklist for making a voice agent "sound human": less interrupting, more backchanneling, an even exchange, mostly targets things that aren't broken. Measured against the right human (task dialogue, not casual chat), almost everything holds up except response speed, and fixing that carelessly breaks the one thing the agent already gets right.
Speech-to-speech models know when to speak but can't reason. Cascaded LLM pipelines can reason but only react to silence. DualTurn pretrains on dual-channel human conversation to bring S2S-level turn-taking into a standard ASR-LLM-TTS stack.
Dual-channel generative pretraining for learning natural turn-taking in spoken dialogue without labeled data. A 0.5B model that outperforms models 6x its size on turn prediction.
A small multimodal LLM that reads paralinguistic signal — emotion, prosody, speaker traits — directly from speech audio instead of through an ASR transcript, built alongside the release of SpeechLLM at Skit.ai.
A multimodal speech LLM that processes audio directly to enhance conversational AI while reducing overhead compared to traditional ASR-LLM-TTS pipelines.
Two techniques for incorporating prosody into end-to-end SLU: prosody-attention and prosody-distillation. Up to 8% intent classification accuracy improvement on SLURP.
Map-Mix: a data augmentation approach using model training dynamics to guide latent mixup sampling, giving ~2% weighted F1 improvement on low-resource dialect classification.
The first public Indian-accented SLU dataset in the banking domain. SSL speech representations beat ASR-based approaches for intent classification.
How deep learning models can isolate independent factors of variation in data through VAEs and Beta-TCVAE, enabling controlled synthesis and better downstream representations.
A semi-supervised framework for speaker profiling that leverages external unlabelled corpora via supervised, unsupervised, and consistency training, achieving RMSE of 6.8 years on age estimation.