Deep reinforcement learning improves ioT task offloading with class-aware rewards

SLA-Aware Task Offloading in the IoT Edge-Cloud Continuum: Class-Weighted Deep Reinforcement Learning Against an Exact Optimum

Artificial IntelligenceMachine LearningNetworking and Internet Architecture

Summary

Deciding where to run Internet of Things (IoT) tasks—on a nearby device or faraway cloud—needs to consider different speed requirements to avoid delays. The authors studied how deep reinforcement learning can better manage these decisions by giving smarter penalties to tasks that miss their deadlines. They show that using class-specific penalties helps these learning agents nearly match the best possible decisions in small setups. However, for bigger systems, the agents still make mistakes and don’t outperform simpler methods even with extended training.

What this means in practice

  • For network schedulers: Use class-weighted penalty frameworks to improve task offloading decisions for latency-sensitive IoT traffic in edge-cloud environments.
  • For cloud infrastructure managers: Benchmark reinforcement learning agents against exact optimal policies for efficient resource allocation across edge and cloud computing layers.

Authors

Shilpa Golla, Tharun Kumar Korine Palli, Urmila Bandaru

Abstract

Deep reinforcement learning (RL) is often used to decide where Internet of Things (IoT) tasks with different latency limits run in the edge-cloud continuum, yet some methods give every missed task the same penalty and few compare their agents with an optimal policy. This benchmark study builds four offloading systems small enough to solve exactly by dynamic programming and measures reward and learner against the optimum. A bonus for a met limit makes each miss cost its penalty plus the bonus, which shifts the class weights of published per-class rewards. A class-blind penalty stays 65% to 118% above the optimal cost even with a perfect learner, while a corrected per-class penalty comes within 0.13% of the optimum. With a per-class reward, tuned proximal policy optimization (PPO) agents come within 9% to 11% of the optimum in the smallest system, and the corrected one beats a tuned reservation rule in 9 of 10 seeds. In larger systems the agents stay 41% to 76% above the optimum, and a tuned rule beats their mean. Training ten times longer does not close this gap, and a PPO-Lagrangian agent with reachable targets does no better than the per-class reward.