EdgeCoInfer: Hierarchical Collaborative Inference for On-Device Multimodal Large Models
Distributed, Parallel, and Cluster Computing
Summary
The authors address the problem of running many large language models at the same time on mobile edge devices, which is hard due to limited resources and strict requirements. They created a system called EdgeCoInfer that cleverly shares parts of different models and splits individual models into smaller pieces to run more efficiently. To figure out the best way to do this, they combined genetic algorithms with reinforcement learning to handle discrete and continuous decisions. Their method ensures all tasks finish even when many run together, while significantly lowering computing costs and saving memory compared to other methods.
Authors
Lin Tan, David K. Y. Yau, Songtao Guo
Abstract
Modern mobile applications predominantly execute concurrent Multimodal Large Language Models (MLLMs) to provide ubiquitous intelligence. However, satisfying this demand within edge environments faces significant challenges due to multi-task concurrency and strictly coupled hard constraints. To address these issues, we propose EdgeCoInfer, a framework enabling granularity-adaptive deployment by co-optimizing inter-model functional module sharing and \textbf{intra-model fine-grained partitioning}. We solve the underlying Mixed-Integer Non-Linear Programming (MINLP) problem via a Hybrid Evolutionary Hierarchical Reinforcement Learning (HE-HRL) paradigm, which synchronizes a Genetic Algorithm (GA) for discrete model placement with a Soft Actor-Critic (SAC) agent for continuous resource allocation. To navigate the sparse feasible region, we introduce a feasibility-guided constructive execution mechanism, integrating a constructive cut-step decoder with pre-act pruning and a two-phase curriculum strategy for stable adaptation. Experimental results demonstrate that EdgeCoInfer ensures a 100\% task completion rate in high-concurrency scenarios, achieving a 76\% reduction in system cost and 71.88\% memory savings compared to state-of-the-art baselines.