Compact vision language model cuts memory use for edge devices
UniCompress: From Multi-LoRA Experts to One Compact Model for Edge Intelligence
Artificial Intelligence
Summary
Vision-language models help computers understand both images and text, but using many task-specific add-ons makes them large and hard to run on small devices. The authors created UniCompress, a way to squash multiple skills into one small, efficient model without extra adapters. It uses a teacher model to guide the shrinking process while keeping accuracy high. This approach made the model about 76% smaller and still good at multiple tasks like recognizing postures and household safety.
What this means in practice
- •For edge device developers: Deploy smaller, adapter-free vision-language models on memory-limited devices performing multiple tasks efficiently.
- •For mobile app developers: Integrate compact multi-skill vision-language models enabling better on-device multimodal interactions with reduced storage needs.
Authors
Mingkang Liu, Peipei Jiao, Weiming Meng, Tianyuan Liu, Lichen Pang, Junyan Li
Abstract
Embodied and edge systems increasingly rely on vision-language models (VLMs) to support heterogeneous skills, yet deploying a separate task adapter for each skill while retaining a high-precision backbone creates substantial memory and management costs. We present UniCompress, a deployment-oriented framework that converts multi-task expertise into one compact, adapter-free model. Starting from a consolidated BF16 Qwen3-VL-4B teacher, UniCompress constructs an asymmetric INT2 student using GPTQ initialization and teacher-guided quantization recovery. The approach jointly quantizes and recovers the projection weights in the vision encoder and language backbone, enabling aggressive compression of the complete vision-language model while preserving its multi-task behavior. We evaluate UniCompress on four heterogeneous embodied tasks: posture recognition, household-safety understanding, multimodal dialogue routing, and mobile-GUI action prediction. UniCompress reduces the approximate weight storage from 8.9 GB in BF16 to 2.1 GB in INT2, corresponding to a reduction of approximately 76%. It improves the INT2 macro task score by 34.3 percentage points over GPTQ-INT2 while remaining within 0.9 points of the consolidated BF16 teacher. These results demonstrate the potential of teacher-guided joint vision-language INT2 recovery for deploying a single multi-skill VLM on resource-constrained embodied platforms.