Shangeth Rajaa

Voice AI Researcher | Senior ML Scientist at Anyreach AI

Anyreach AI

Biography

Senior ML Scientist at Anyreach AI, working on Turn-Taking in Full-Duplex Spoken Dialogue Systems, Multi-Modal Speech LLMs for understanding and synthesis, and Automatic Speech Translation.

6+ years in Voice AI research across industry and academia: Anyreach AI, Skit.ai, NTU Singapore, IBM Research, INRIA Paris. Have built and led ML and research teams in early-stage startups, taking products from zero to production. 7 peer-reviewed publications at Interspeech, ICASSP, NeurIPS, and PMLR.

Dual Degree in B.E. EEE & M.Sc. Mathematics from BITS Pilani.

Building something in Voice AI? Always happy to talk, consulting, research, or just a good conversation.

Interests

Turn-Taking & Full-Duplex Dialogue
Multi-Modal Speech LLMs
Spoken Language Understanding
Speaker Profiling & Representation
Conversational AI Systems

What I Work On

Turn-Taking & Full-Duplex Dialogue — Building AI that knows when to speak and when to listen. Core research at Anyreach AI: predicting turn-taking signals, agent actions, and word-level boundaries in real-time spoken dialogue.

Multi-Modal Speech LLMs — Integrating speech encoders with large language models for end-to-end spoken dialogue, speech understanding, and automatic speech translation.

Spoken Language Understanding (SLU) — Direct speech-to-intent without ASR cascades, including prosodic attention, knowledge distillation, and Indian-accented speech datasets.

Speaker Profiling & Representation — Semi-supervised and self-supervised learning for age, gender, accent, and emotion estimation from raw speech signals.

Conversational AI Systems — End-to-end voice agents: ASR · TTS · NLU · Dialogue Management · RLHF/DPO alignment for goal-driven spoken dialogue.

All code and experiments on GitHub →

Experience

Senior Machine Learning Scientist

Anyreach AI

Feb 2025 – Present

Turn Taking in Full Duplex Spoken Dialogue Systems.
Multi-Modal LLMs for Speech Understanding and Synthesis.
Automatic Speech Translation with Multi-Modal LLMs.

Senior Machine Learning Researcher

ScoreTravel AI

Aug 2024 – Feb 2025 Bengaluru, India

Multi-Modal LLM (Vision, Speech).
Ranking algorithm with LLM Reasoning.

Machine Learning Researcher

Skit (previously Vernacular.ai)

Jun 2021 – Jun 2024 Bengaluru, India

Multi-Modal LLMs for E2E Dialogue Systems.
Language Models as Dialogue Agents, RLHF/DPO.
Spoken Dialogue Systems (ASR/TTS/NLU/E2E-SLU).
Improving E2E-SLU with Prosodic features.
Modelling and disentangling speaker/prosodic/content representation from speech.
Spoken Language identification / Emotion recognition / Speaker Profiling.

Research Assistant

Speech and Language Laboratory - NTU, Singapore

Aug 2020 – Jun 2021 Singapore

Speech Representation and Speaker Profiling under Prof. Chng Eng Siong.

Research Intern

IBM Research Labs

May 2020 – Aug 2020 Delhi, India

Deep Reinforcement Learning based optimization framework for Data Quality Sequence Workflow. Submitted at SIGMOD-2021.

Skills and Frameworks:

Deep Reinforcement Learning
Deep Learning
Data Quality
Structured Data
PyTorch

ML Facilitator

Google AI

Aug 2019 – Mar 2020 Bangalore, India

Deep learning and Computer vision Content Developer

OpenCV.org

May 2019 – Sep 2019 California, USA

Development of projects and content creation for Deep Learning with PyTorch and Computer Vision courses.

Skills and Frameworks:

Deep Learning
PyTorch
Tensorflow
OpenCV

Research Collaborator

INRIA

Apr 2019 – Jul 2020 Paris, France

Research project on Auto Deep Learning under Dr. Isabelle Guyon in collaboration with LRI, France and Google Zurich.

Baseline Submissions to AutoDL competition for NeurIPS 2019, AutoCV, AutoNLP, AutoSeries competitions.
Implementing PyTorch modules to work with Tensorflow code and dataset.
Creation of Datasets for all competitions of AutoDL.

Skills and Frameworks:

Deep learning Research
Auto Deep Learning
PyTorch
Tensorflow

AI Developer

OpexAI

Sep 2018 – Nov 2018 Bangalore, India

Research and Development of Self driving car models.

Lane Detection using Deep Learning
Steering angle prediction for self driving cars
Object recognition for self driving cars.

Software Developer

KGLLP Fintech

Jul 2018 – Nov 2018 Bangalore, India

Financial Software development in Python.
Data Processing & end to end data pipeline.
Machine Learning models for stock prediction.

Computer Vision Developer

Science and Technology Center

May 2018 – Jul 2018 Chennai, India

Developed a Computer Vision Security System for the campus with Flask server and machine learning models.

Research Collaborators

Prof. Chng Eng Siong — Speech and Language Lab, Nanyang Technological University, Singapore
Prof. Isabelle Guyon — INRIA Paris & Google Brain
Nitin Gupta — IBM Research Labs
Prof. Ashwin Srinivasan — BITS Pilani Goa Campus
Prof. Jajati Keshari Sahoo — BITS Pilani Goa Campus

Featured Publications

Shangeth Rajaa

March 2026 Interspeech 2026 (Submitted)

DualTurn: Learning Turn-Taking from Dual-Channel Generative Speech Pretraining

DualTurn learns natural conversational turn-taking via generative pretraining on dual-channel audio, outperforming larger models on turn prediction tasks.

PDF

Shangeth Rajaa, Abhinav Tushar

January 2024 Model Release

SpeechLLM: Multi-Modal LLM for Speech Understanding

SpeechLLM: A multimodal LLM combining speech encoders with TinyLlama for joint ASR, gender, age, accent, and emotion prediction from audio.

Code GitHub HuggingFace

Shangeth Rajaa, Kriti Anandan, Swaraj Dalmia, Tarun Gupta, Chng Eng Siong

June 2023 ICASSP 2023

Improving spoken language identification with MAP-Mix

MAP-Mix: Training-dynamics-guided data augmentation for spoken language identification, achieving ~2% F1 improvement over random mixup at ICASSP 2023.

PDF arXiv