Cloud edge inference compresses deep neural network guidance into tiny hints
HintKD: Hint Knowledge Distillation for Bandwidth-Constrained Cloud-Edge Inference
Information Theory
Summary
Communicating complex information between cloud servers and edge devices often needs large amounts of data, which can be slow and costly. The authors propose HintKD, a method that shrinks this information into very small discrete hints before sending it, greatly reducing bandwidth needs. This method keeps the accuracy of large neural network predictions while sending much less data. The approach was tested on tasks like 6G beam prediction and image classification, showing it works well across different problems.
What this means in practice
- •For mobile network operators: Reduce communication load in cloud-edge systems for 6G beam prediction by sending compact hints instead of large feature data.
- •For mobile app developers: Enable efficient deployment of large AI models on devices with limited bandwidth by using tiny discrete hints to guide edge inference.
Authors
Wanling Luo, Jiayi Zhang, Bokai Xu, Ziheng Liu
Abstract
Collaborative inference between the edge cloud and user equipment (UE) is a promising paradigm for deploying large deep neural networks (DNNs) in 6G networks. However, existing cloud-edge inference and distillation schemes often require the real-time transmission of high-dimensional intermediate features or soft probability vectors, which imposes a substantial communication burden on bandwidth-limited wireless links. To address this challenge, we propose HintKD, a bandwidth-constrained distillation framework for cloud-edge inference. The core idea is to compress the teacher's guidance into compact discrete hints rather than transmit raw features directly. Specifically, a cloud-side teacher is first trained with a maximum conditional mutual information (MCMI) objective to preserve informative intra-class variations. Its latent representation is then mapped to a compact codebook through differentiable vector quantization. During deployment, the cloud transmits only a hint index, and the UE uses a lightweight FiLM-based adapter to modulate student features according to the received codeword. Experiments on the DeepSense 6G beam prediction task show that HintKD preserves competitive accuracy while reducing the inference-side communication payload from O(D_t) for feature transmission or O(C) for logit transmission to O(1) for discrete hint transmission. In addition, experiments on the MNIST dataset further verify the generality of the proposed distillation method across different datasets and tasks. These results show that HintKD achieves a favorable accuracy-bandwidth trade-off by replacing high-dimensional feature or logit transmission with a compact discrete hint index.