Elastic neural networks cut edge AI costs while adapting runtime
Elastoformer: Enabling Dynamic Adaptivity via Elastic Model Transformation
Computer Vision and Pattern RecognitionArtificial IntelligencePerformance
Summary
Edge devices running AI often face changing limits on speed, power, and memory, but regular neural networks can’t easily adjust to those changes. The authors present Elastoformer, which transforms standard neural networks into flexible ones that adapt their operation dynamically without needing multiple separate models. This approach saves a lot of computing work, reduces delays, and cuts memory use while working with different AI architectures. Their tests show significant improvements without extra overhead from managing many models.
What this means in practice
- •For mobile app developers: Deploy vision AI models on edge devices with variable power and memory budgets without maintaining multiple model versions.
- •For embedded systems engineers: Build adaptive neural network applications that dynamically scale computation to meet latency and power constraints in real-time.
Authors
Sudaksh Kalra, Dolly Sapra
Abstract
EdgeAI systems are increasingly employing computer vision applications to enable intelligent, on-device decision-making in real-time. However, these deployments face highly dynamic operational conditions, with fluctuating constraints on latency, power availability, and memory resources. Deep Neural Networks (DNN), which follow fixed computational execution flows, lack the flexibility to adapt to such variability, resulting in inefficient and suboptimal performance in edge scenarios. This underscores the need for architectures that are not only efficient but also dynamically scalable at runtime. In this paper, we propose Elastoformer: A framework that transforms conventional neural networks (NN) into Elastic NN capable of real-time elastic inference. Unlike the conventional bag-of-models approach, which requires maintaining multiple independent models for different operating conditions, Elastoformer offers a single, modular solution that dynamically switches between multiple modes of operation at runtime, adapting efficiently to the changing computational budgets of edge devices without the overhead of managing separate models. Experiments reveal that our framework achieves up to 85% reduction in computation FLOPs, 50% reduction in latency and 76% reduction in memory overhead, while showcasing the architecture agnostic nature of the framework across both Vision Transformers and CNNs. Our code is available at https://github.com/sudaksh14/Elastoformer.