Wireless foundation models improve with diverse learning objectives and multi-branch design
Scaling of Wireless Foundation Models via Representation Diversity and Multi-Branch Architectures
Artificial Intelligence
Summary
Wireless foundation models try to learn about radio signals without labels so they can be used for many different tasks. The authors found that just making these models bigger doesn't always help when there isn't much data to train on. Instead, they combined different ways of learning about the signals, each focusing on different features, and found this creates better overall understanding. They built a design that mixes these diverse learning methods into one model, which works well for various wireless tasks like communication and location sensing. This approach keeps benefits while keeping the model size manageable.
What this means in practice
- •For wireless system engineers: Build wireless models that improve performance on communication, sensing, and positioning tasks by combining multiple self-supervised objectives.
- •For signal processing teams: Design efficient single-encoder models using multi-branch architectures that preserve diverse signal representations within a fixed parameter budget.
Authors
Ahmed Mohamed, Ahmed Aboulfotouh, Hatem Abou-Zeid
Abstract
Wireless foundation models learn representations from unlabeled radio signals for reuse across downstream tasks. Scaling model capacity is a common strategy for learning richer representations and improving downstream performance. However, its gains are less consistent in wireless self-supervised learning when pretraining data are limited. We investigate objective diversity as an alternative scaling axis: different self-supervised objectives emphasize different signal properties, and combining their representations can preserve more information to enable diverse tasks. We develop a fusion framework that combines frozen representations from independent encoders trained through reconstructive, predictive, and contrastive learning. This provides a reference for the benefits of diversity, but requires the maintenance of multiple encoders. To retain these benefits within the parameter budget of a standard single-objective encoder, we introduce a jointly trained multi-branch architecture with a shared trunk and objective-specific branches. We pretrain on a heterogeneous corpus of spectrogram and channel state information data and evaluate six downstream tasks that span communication, sensing, and positioning. At an equal output embedding dimension, fusion outperforms the evaluated single-objective encoders on all six tasks while using roughly one-third of their parameters. The multi-branch architecture retains much of the fusion benefit within a single-encoder parameter budget. Representation analyses indicate complementary contributions across objectives, with much of the added benefit retained in components orthogonal to the reconstruction representation subspace. These findings support objective diversity as an effective strategy for scaling wireless foundation models.