Edge computing system predicts future tasks for better model inference
Predictive Rolling-Horizon Optimization for Commitment-Aware Model-Parallel Inference under Spatio-Temporal Edge Dynamics
Distributed, Parallel, and Cluster Computing
Summary
Running complex AI models across many edge devices is tricky because devices move, resources change, and tasks come randomly. The authors designed PROMISE, a planning system that looks ahead to predict upcoming tasks and device conditions to schedule AI model parts more reliably and efficiently. PROMISE also decides when tasks should finish to keep promises while balancing workload. The team tested their design in simulations and on small computers like Raspberry Pi to show it works well in real conditions.
What this means in practice
- •For edge computing engineers: Schedule AI model execution across distributed edge nodes by predicting workloads and communication changes to meet service deadlines.
- •For iot system integrators: Optimize deployment of AI tasks over mobile IoT devices balancing privacy, network conditions, and compute load over time.
Authors
Minghui Liwang, Chenxi Xu, Wei Gong, Li Li, Wenbo Zhu, Xinlei Yi, Yuhan Su, Xianbin Wang
Abstract
Model-parallel inference over dynamic edge systems requires scheduling decisions that account for not only instantaneous resources but also future resource contention and reliable service commitments. Existing edge-inference designs, however, predominantly optimize performance metrics based on current or short-term system states, without explicitly coupling current assignments with future commitment fulfillment. To address this issue, we propose PROMISE, a predictive rolling-horizon framework for commitment-aware model-parallel inference under spatio-temporal edge dynamics. PROMISE jointly models stochastic task generation, mobility-induced communication variations, privacy-aware model partitioning, and load-dependent edge computing capability. We introduce committed completion time (CCT) as an endogenous service decision and formulate joint SD--ES mapping and CCT determination to balance commitment fulfillment and service reward. To address cross-timeslot coupling, PROMISE estimates future task arrivals and computational workloads over an adaptive horizon and embeds them into certainty-equivalent ES-state rollout. A progress-aware surrogate then evaluates feasible current-stage mappings, while the corresponding CCTs are analytically recovered from predicted completion times. Only the first-stage decisions are committed, and the optimization is repeated with newly observed states in a receding-horizon manner. Numerical experiments demonstrate robust scheduling performance under diverse system scales and workload dynamics, while Raspberry-Pi-based experiments validate the practical feasibility and key system characteristics of model-parallel edge inference.