Papers for

audio equipment designers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Audio visual models struggle to judge timing and event order in videos

AVTrace: Diagnosing Audio-Visual Temporal Reasoning in Omni Models

Abstract: Omni models can describe video content, but can they locate events in time, preserve event order, and judge audio-visual synchronization? We introduce AVTrace (Audio-Visual Temporal Reasoning Assessment and Capability Evaluation), a silver-standard diagnostic suite spanning onset and span grounding, synchronization, next-step prediction, cross-modal localization, chain parsing, and event-conditioned comprehension. It contains 34,114 training examples and category-balanced development and test splits of 3,500 and 7,000 examples. We evaluate five open omni models under their respective input configurations using reference-blind response normalization followed by deterministic scoring. All five off-the-shelf systems score below the test split's majority-label baseline of 0.556 on synchronization verification, and obtain low scores on chain parsing and event-conditioned grounding and comprehension. Development-set perturbations reveal task-dependent sensitivity in Qwen3-Omni-30B to modality removal and changes in visual input processing, without isolating their underlying causes. Parameter-efficient temporal post-training improves Gemma4-E4B-it on several benchmark metrics. On three external image benchmarks, task metrics change modestly, including some degradations, while teacher-forcing perplexity decreases. Together, these findings show that semantic reference-text overlap should not be treated as a proxy for temporal localization, and that AVTrace can identify task-specific weaknesses while providing a testbed for temporal post-training.

Thu 17 SeptComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
Understanding what happens in a video means not just describing scenes but also knowing when events start, how long they last, and if sound and picture match up. The paper presents AVTrace, a set of tests designed to see how well advanced video-audio AI models can handle these timing challenges. The authors found that current top models often fail on tasks like syncing sound and video and following event sequences well. They also show that training models specifically on timing can improve their performance on some of these tasks.
Open 2609.19991v1