Vision wireless fusion improves locating multiple users in cities
Vision-Wireless Fusion for Multi-User Localization: A Cross-Modal Transformer Approach
Computer Vision and Pattern Recognition
Summary
Finding the positions of many people in busy city areas is hard because radio signals can get mixed up by buildings and noise. The authors created a new method that combines wireless signal data tagged with unique user codes and images of the environment. Their system matches each user's wireless signals with visual clues using a special transformer design, allowing better guessing of where each person is. Tests showed this combined approach works better than using signals or images alone.
What this means in practice
- •For urban network operators: Improve accuracy of locating multiple users simultaneously in dense urban wireless networks by integrating visual data with wireless signals.
- •For smart city system developers: Enhance user localization services in complex city environments by fusing visual information with user-specific wireless channel data.
Authors
Can Zheng, Jiguang He, Guofa Cai, Henk Wymeersch, Merouane Debbah
Abstract
Accurate multi-user localization is challenging in complex urban environments, where wireless measurements can become ambiguous under noise, blockage, and multipath, while visual observations provide complementary spatial context. This paper presents a vision-wireless fusion framework for multi-user localization using pilot-indexed channel state information (CSI). Orthogonal pilot indices preserve the identities of the communicating UEs in the CSI token sequence and localization outputs. The model encodes each pilot-indexed CSI observation as a query token and uses cross-attention to retrieve user-specific information from spatial visual memory. Self-attention among CSI tokens further captures inter-user interactions, while the resulting multimodal representations are used for user-wise localization. Experiments on different datasets show consistent improvements over model-based, CSI-only, and multimodal-fusion baselines. Further experiments evaluate the model under different wireless and visual conditions.