Parameter-Dynamic Adaptive Fusion and Calibration Network for RGBT Tracking
2026-08-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors explain that current RGBT trackers use fixed ways to combine visual and thermal data, which doesn't work well when targets or scenes change. They designed PAFCNet, a system that creates custom settings based on the tracked target to better mix the data and adjust over time. Their method uses a special network (TA-HyperNet) to adapt fusion and calibration parameters dynamically, helping improve tracking accuracy in changing situations. Tests show their approach performs well across different RGBT tracking challenges.
RGBT trackingmultimodal fusionadaptive fusionhypernetworkspatio-temporal calibrationtarget appearance variationtracking benchmarksparameter-dynamic fusiontemporal information propagation
Authors
Zhaoding Ding, Chenglong Li, Jiandong Jin, Kewei Ying, Wentao Wu
Abstract
Existing RGBT trackers typically employ fusion functions with fixed parameters across different targets and scenarios. Although dynamic-architecture methods improve fusion flexibility by selecting among predefined operations, they still cannot adapt the fusion parameters to the evolving target state. To address these issues, we propose a Parameter-Dynamic Adaptive Fusion and Calibration Network (PAFCNet) for RGBT tracking. PAFCNet dynamically generates target-conditioned parameters for multimodal fusion and temporal calibration, enabling the tracking process to adapt to target appearance variations and modality quality fluctuations. Specifically, we introduce a Target-Adaptive Hypernetwork (TA-HyperNet) that leverages template representations, which preserve stable target identity and recent appearance changes with less background interference, to generate target-conditioned parameters for subsequent fusion and calibration. Based on TA-HyperNet, we design a target-aware parameter-dynamic fusion module that uses the generated parameters to modulate the fusion process. This enables the fusion module to adapt to changes in target appearance and complex scene conditions. Furthermore, since spatio-temporal information propagation may accumulate tracking noise, we propose a dynamic spatio-temporal calibration module that employs TA-HyperNet to generate calibration parameters for spatio-temporal tokens. By dynamically calibrating historical information before propagation, the module improves the reliability of temporal representations. Experimental results demonstrate that PAFCNet achieves competitive performance on multiple RGBT tracking benchmarks.