EchoPath enables efficient replay of GUI tasks with image-based targeting
EchoPath: Execution-Level Replayable Memory for GUI Agents
Artificial Intelligence
Summary
Using graphical user interfaces to repeat tasks on computers is often slow because the software has to figure out what to do every time from scratch. The authors created EchoPath, a system that remembers how tasks were done before by saving images and details about the process. When needed again, EchoPath matches parts of the screen visually to replay the exact steps, making the tasks much faster and cheaper to run. It also only runs steps that it can do reliably, avoiding errors.
What this means in practice
- •For enterprise automation teams: Speed up repeated form filling and report exporting by reusing validated GUI task executions with precise visual matching.
- •For desktop software developers: Reduce costs and improve efficiency of GUI-based software bots by integrating image-based coordinate reaiming to ensure task replay reliability.
Authors
Yao Zhao, Aditya Shanmugham, Swastik Roy, Yanxun Xu
Abstract
Computer-use agents increasingly operate browsers, software, and desktop applications via CLI or API portals, but graphical user interface (GUI) still plays an important role in common industrial production scenarios. GUI agents commonly employ fresh observe-plan-ground-act loops, which is inefficient for enterprise tasks that repeatedly update records, process forms, configure tools, and export reports. We introduce EchoPath, a model-agnostic harness that converts artifact-validated GUI trajectories into standardized, parameter-controlled callable memories, analogous to Model Context Protocol (MCP)-style tool calls rather than unstructured experience records. Each memory stores task-intent keys, application and state preconditions, flexible input parameters, GUI evidence, validation provenance, and lifecycle state, so the host agent invokes a targeted procedure only when it can be deterministically replayed in the current runtime. The core mechanism enabling replay is an image-based target-reaiming algorithm that treats stored coordinates as visual evidence, matches the remembered GUI target against the current screen, and emits corrected operation coordinates before execution. During replay, EchoPath rebinds only declared modifiable inputs and rejects ambiguous or incompatible steps to bounded grounding repair or fresh planning. In experiments with real computer-use tasks, EchoPath reduced median token cost by more than 90% and median execution time by about 60%. These results support a bounded form of enterprise GUI memory: validated execution experience can become a controllable callable asset for recurrent work rather than only context for another reasoning pass.