X2-NativeCursor improves real-time text progress tracking in streaming speech
X2-NativeCursor: Native-Token Text Progress Tracking for Incremental-Text Streaming Codec TTS
Computation and Language
Summary
It’s hard for speech software to keep track of which word is being spoken in real time, especially when the text arrives before the speech is produced. The authors created X2-NativeCursor, a lightweight tool that follows the speech progress by looking at internal speech tokens instead of waiting for audio output. This method tracks spoken words more accurately and faster without changing the main speech generator. It works well for Chinese and adapts to different speech models.
What this means in practice
- •For voice assistant developers: Implement precise real-time text highlighting and interruption handling during streaming speech output to improve user interaction.$Commercial implications: Enables commercial voice assistant products to track speech progress accurately and responsively, improving user experience.
- •For call center software teams: Enhance dialogue systems with better speech progress tracking for synchronized transcription display and smoother turn-taking management.
Authors
Zehan Liu, Carl Chen, Rime Wen, Kaiqi Fu, Altman Lin, Shawn Qin, Lights Shi, Roy Gan, Hao Wang, Qian Wang
Abstract
Incremental-text streaming text-to-speech (TTS) needs online text progress tracking for synchronized highlighting, interruption handling, and dialogue-history updates. Input text arrives before it is spoken, so text arrival alone cannot indicate speech progress. Existing waveform-based alignment requires complete audio or adds acoustic processing during streaming. We propose X2-NativeCursor, a lightweight observer that tracks progress from native speech tokens before waveform decoding without changing the TTS generator. Its normalization plan links spoken labels to their original-text spans. Text and native-token encoders feed a local matcher that estimates the current label position. A separate output rule converts revisable position estimates into a cursor that never moves backward. Mean absolute error against an automatic reference is 0.151 Chinese characters with 80-ms lookahead, versus 1.253 characters with 320-ms lookahead for an online waveform baseline. Alignment real-time factor also decreases from 0.3598 to 0.0180 relative to this baseline. Lower tracking error is retained under a second automatic alignment reference. We evaluate X2-NativeCursor on Qwen3-TTS and validate its adaptation to CosyVoice2 by training a separate observer for each backbone. Code is publicly available at https://github.com/X-Square-Robot/X2Streaming-TTS.