Turnwise improves streaming video decision making with turn level credit
Turns Matter: Advancing Streaming Video RL via Turn-Level Credit
Computer Vision and Pattern RecognitionComputation and LanguageMachine Learning
Summary
Understanding streaming videos requires making lots of decisions about when to respond or keep watching. The authors found that earlier methods gave rewards for entire video turns, which could cause repeated unnecessary answers or discourage useful follow-ups. They propose TurnWise, a new approach that gives feedback for each individual decision during the video, helping the system learn better when to speak or wait. Their method improves learning without needing extra complex tools and works better than previous approaches.
What this means in practice
- •For video streaming developers: Improve decision-making in real-time video analysis systems to respond more accurately at each interaction point.
- •For interactive ai system designers: Enhance policies in AI systems that decide when to act or wait for more input based on turn-wise feedback.
Authors
Shuai Dong, Kanghui Tian, Linghan Chen, Wenqi Shao, Hongrui Wu, Haowen Hou, Yongfu Zhu, Xuehui Wang, Ziyue Wang, Wenpu Liu, Weichu Xie, Yuqi Xu, Congcong Wang, Kaiwen Tuo, Caoyuan Ma, Taiqiang Wu, Ji Zhao, Wenzheng Chang, Dianyi Wang, Siyuan Wang, Zhongyu Wei, Yu Qiao, Jiaqi Wang
Abstract
Streaming video understanding requires models to repeatedly decide whether to respond or continue observing. Recently, Group Relative Policy Optimization (GRPO) has been applied to this setting, with existing approaches emphasizing reward design while retaining a single trajectory-level advantage across turns. Such shared credit can reinforce redundant replies after earlier successes and discourage useful follow-ups after earlier failures. We propose TurnWise, a turn-level GRPO framework that aligns learning with individual streaming decisions. TurnWise addresses the mismatch between trajectory-level feedback and turn-level interaction, promoting informative responses and reducing redundancy. Specifically, turn-level credit assignment uses silence-completed prefix scoring to construct group-relative advantages from each decision's remaining return, separating its learning signal from earlier rewards. Turn-mean policy optimization averages token losses within each turn and then across turns, giving response and waiting decisions equal weight regardless of output length. These complementary designs improve streaming policy learning without a learned critic or additional rollouts. Extensive experiments show our method outperforms prior RL approaches for streaming video understanding.