More interactions reduce data sent for better task communication
More Interactions Make Communications More Efficient: A Directed Information Bottleneck Approach to Task-Oriented Communication
Information Theory
Summary
Sending only new and useful information can make communication more efficient for specific tasks. The authors propose a method that focuses on the flow of new, task-related information rather than just overall data similarity. They combine modern tools to identify and send only what the receiver does not already know, which helps perform tasks, like classifying videos, with less data sent. Tests showed this approach beats existing methods in accuracy and efficiency.
What this means in practice
- •For video streaming engineers: Reduce data transmission needed for video analysis by sending only new information relevant to classification tasks.
- •For iot device developers: Enable low-bandwidth sensor networks to communicate task-critical updates efficiently by avoiding redundant data transfer.
Authors
Zhirun Zheng, Yiqin Deng, Phone Lin, Yuguang Fang
Abstract
Task-oriented communication improves communication efficiency by transmitting semantic information required for downstream tasks, but may incur redundancy with previously transmitted information. To address this issue, we propose a variational directed information bottleneck joint source-channel coding (VDIB-JSCC) framework that transmits only previously unacquired information. Specifically, we use directed information to characterize the causal flow of task-relevant incremental information and theoretically demonstrate its advantages over mutual information and conditional mutual information. Based on this metric, we formulate a directed information bottleneck (DIB) objective that balances task-relevant information preservation and redundant information compression. We further derive a variational formulation, termed VDIB, whose tractable upper bound serves as the training loss. The encoder integrates a Vision Transformer for frame-wise semantic extraction and a Mamba-based causal state-space model for capturing temporal dependencies and extracting incremental information, while the decoder uses a Mamba-based state-space model for task inference. Experiments on CIFAR-10 and CIFAR-100 video streams demonstrate that VDIB-JSCC achieves higher classification accuracy with less transmitted information than state-of-the-art semantic communication methods. These results show that leveraging historical knowledge enables task completion with less information, improving communication efficiency.