Hyperspectral tracker improves video target tracking with spectral memory
HyperDAM: Hyperspectral Distractor-Aware Memory with Amodal Expansion for SAM 3 Tracking
Computer Vision and Pattern Recognition
Summary
Tracking objects in videos can be tricky when different things look very similar. The authors developed a system called HyperDAM that uses special hyperspectral video data to tell apart targets based on their material properties, not just their looks. They also improved how the system decides when to update its memory to avoid confusion from things that look like the target but are different. Their method ranked second in a major competition, showing better accuracy in following targets over time.
What this means in practice
- •For autonomous vehicle engineers: Improve detection and tracking of objects that look alike visually but differ in material by using hyperspectral cues and robust memory updates.
- •For video surveillance teams: Enhance tracking accuracy in complex scenes by rejecting false updates based on spectral inconsistencies and applying amodal corrections for occluded targets.
Authors
Ryoga Yuzawa, Tasuku Takagi
Abstract
Hyperspectral video provides material cues that can disambiguate targets with similar false-color appearance, yet foundation-model trackers update memory primarily from spatial and appearance evidence. We present HyperDAM, a DAM4SAM3-based hyperspectral tracker with three principal contributions. First, HOTC2026-Modal adds human-verified frame-wise modal masks and mask-tight boxes to all 481 organizer-provided HOTC 2026 videos. Second, a frame-zero-calibrated HSI gate rejects spectrally inconsistent updates to the distractor-resolving memory (DRM) without altering the current prediction. Third, a causal spatiotemporal expander adds outward-only amodal corrections from frozen SAM features. Static-scene recovery and empty-mask RTS smoothing address target switches and full occlusion. Model selection prioritizes cross-domain robustness over leaderboard-specific optimization. The final system ranked second in HOTC 2026, achieving 68.0093% AUC and 87.7703% DP@20 in the organizer's private evaluation.