Micro-inverted-scaling speeds up vision-language models on edge devices
MiX: Micro-Inverted-Scaling for End-to-End Low-Bit Vision-Language Model Acceleration
Hardware ArchitectureComputer Vision and Pattern Recognition
Summary
Vision-language models, which combine images and text understanding, struggle to run quickly on small devices because they require a lot of memory and precise calculations. The authors found that normal ways of shrinking data for these models can cause errors that ruin their results. They created a new method called MiX that changes how numbers are stored to avoid these errors and make computation faster and more energy efficient. This method works well on several models and leads to hardware that is smaller, faster, and uses less power.
What this means in practice
- •For mobile hardware engineers: Design edge device chips that run compact, fast vision-language models using MiX format to reduce memory and power demands.
- •For embedded system developers: Build efficient on-device vision-language applications by integrating MiX-based quantization for better speed and energy use.
Authors
Yuan Liao, Jae-sun Seo
Abstract
The deployment of Vision-Language Models (VLMs) on edge devices is severely bottlenecked by memory bandwidth, necessitating aggressive sub-8-bit quantization. Since edge accelerators are strictly constrained by area and power, they require end-to-end quantized models. However, the extreme dynamic range gap between multi-modal tokens causes standard block formats to suffer "microscaling collapse," where a single massive outlier hijacks the shared exponent, underflowing surrounding elements and destroying attention maps. To break this bottleneck, we propose Micro-Inverted-Scaling (MiX), a novel format that mathematically inverts the microscaling paradigm: rather than grouping multiple mantissas under one shared exponent, MiX groups private, per-element exponents under a single shared mantissa. To handle asymmetric VLM outlier topologies, we introduce an adaptive dual-format (MiX-MX) inference framework. By algebraically factoring out the shared MiX mantissa, this framework maps to a custom accelerator, replacing multipliers with efficient shifters. Evaluated end-to-end on multiple VLMs, our 4.5-bit MiX formulation exhibits equivalent or superior accuracy on multi-modal benchmarks compared to NVFP4. Simultaneously, the MiX accelerator delivers a 25% improvement in area efficiency over the NVFP4 baseline and a 2.3-4.5x speedup with 1.4-2.9x energy reduction across models compared to the state-of-the-art accelerator Focus, proving the inverted-scaling datapath is physically superior for efficient VLM deployment.