Energy cost of agentic ai favors local inference over network text exchange
Where Should Agents Live? Energy-Memory Characterization of Agentic AI for the Edge-Cloud Continuum
Artificial IntelligenceMachine LearningMultiagent Systems
Summary
Figuring out where AI agents should run—from nearby edge devices to distant cloud servers—depends on their energy and memory use. The authors studied how energy is spent when AI agents work together and share text across networks. They found that sending text between agents over networks uses very little energy compared to the extra work triggered by that communication on the AI models themselves. This suggests that the main energy cost is in the processing caused by exchanging information, not the communication itself.
What this means in practice
- •For network operators: Decide where to place AI agents across edge and cloud to optimize energy costs using detailed energy and memory models.
- •For cloud infrastructure teams: Optimize operational costs by focusing on compute and memory impacts of inter-agent communication rather than text transmission energy.
Authors
Carolina Fortuna, Vid Hanžel, Tim Strnad, Blaž Bertalanič
Abstract
As telecommunication networks evolve toward autonomous 5G-Advanced and 6G operations, agentic artificial intelligence (AI) workflows, where large language models (LLMs) execute multi-step reasoning, invoke diagnostic tools, retrieve domain knowledge, and coordinate across agent teams, are increasingly embedded across the edge-cloud continuum. While the biological brain accomplishes complex cognition on an exceptionally modest metabolic power budget of approximately 20W contemporary LLMs are profoundly energy- and memory-intensive, making sustainable lifecycle orchestration a critical operational priority. However, existing AI lifecycle metrics evaluate only isolated, single-model inferences or overlook multi-agent execution graphs entirely. Consequently, network operators lack foundational models to determine whether distributed agent communication incurs meaningful energy costs and where across edge-cloud tiers agent teams should physically reside. To address this gap, we introduce agentic-eCAL, generalizing the Energy Cost of AI Lifecycle (eCAL) metric to directed multi-agent workflows by coupling a closed-form two-rate single-call energy model (compute-bound prefill and memory-bound decode) with 7-layer OSI data transport. Grounded in hundreds of GPU benchmark configurations on NVIDIA A100 and H100, 16 open-weight models and 8 orchestration topologies, we validate components of the metric and study workflow placement implications. Our findings demonstrate that inter-agent text transport incurs 0.25% of workflow energy across 5G RAN, metro, and optical links. Therefore in edge-cloud agent placement the dominant energy cost of distribution is often not the transmission of inter-agent text itself, but the additional inference and context processing induced by that communication.