SkillRefine improves industrial planning with better multi-table coordination
SkillRefine: Cross-Source Skill Induction and Execution Validation for LLM Agents in Refinery Planning Software
Computational Engineering, Finance, and Science
Summary
Big industrial software systems are complex and hard to operate because information is scattered and incomplete. The authors created SkillRefine, a new method that helps AI agents learn the right step-by-step skills by combining software manuals, expert actions, and feedback after running plans. Their method checks and fixes these skills through several tests without needing extra labels. SkillRefine showed better accuracy, especially when many tables had to work together, helping AI perform refinery planning tasks more reliably.
What this means in practice
- •For industrial software engineers: Create AI assistants that can reliably operate complex refinery planning systems by learning coordinated skills from documentation and expert usage data.
- •For automation software developers: Develop improved AI modules for planning systems that adapt during execution by detecting and correcting skill application errors without labeled data.
Authors
Dongxiao Liu, Boren Zhu, Yuwen Ding, Linghui Li, Lei Li, Xiaoyong Li
Abstract
Operating industrial planning software such as AspenTech PIMS (Process Industry Modeling System) requires an LLM agent to combine structural knowledge, procedural knowledge from expert records, and constraints revealed only during execution. These evidence sources are heterogeneous and individually incomplete: documentation describes tables and interfaces but omits task-level coordination, expert CASE records expose multi-table modification patterns without explicit schema grounding, and execution feedback reveals latent constraints only when a plan is executed. We present \textsc{SkillRefine}, a framework that exploits the Documentation--Practice Gap between documented structure and expert practice to extract candidate coordination patterns, ground them against table definitions and COM specifications, and compile them into progressively disclosed skill packages with provenance. The library is then refined through label-free compliance screening, oracle-based match decomposition over table, row, column, and value dimensions, and signal-conditioned trajectory attribution for localized repair. We evaluate \textsc{SkillRefine} on PIMS-Bench, a benchmark built from two AspenTech PIMS demonstration models, using disjoint construction and held-out test tasks. On the held-out test set, \textsc{SkillRefine} achieves absolute component match F1 gains of 14\%--30\% across four LLM backbones, with the largest gains on complex multi-table coordination tasks.