Model learns to use Korean public APIs in multiple steps like bigger versions
Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe
Artificial IntelligenceComputation and Language
Summary
Using government websites through computer programs often means clicking many buttons in order. Smaller open-source language models usually struggle to do this correctly. The researchers created a test called KOPA-Bench with real tasks to see how well models handle these multi-step actions. They also developed a method named EDGE that figures out how different tools can link together by trying real examples, helping generate good practice tasks. By training a smaller model with this method, it did almost as well as a much bigger model and improved on other similar tests too.
open-source language modelsmulti-step tool-callingpublic APIsdata sovereigntybenchmarkdata synthesisexecution-grounded graphfine-tuningKOPA-BenchGRPO
Authors
Dain Kim, Eungi Cho, Kyumin Kim, Shinyeong Noh, Kyuseong Lim
Abstract
Data-sovereignty regulations increasingly require public institutions to deploy open-source, on-premise LLM agents that chain multiple tool-calls across live government APIs. However, open-source models consistently underperform in this multi-step setting, and no existing benchmark measures the gap. We introduce the Korean Open Public API Benchmark (KOPA-Bench), comprising 145 real-world tasks. To close this gap, we present EDGE, an Execution-grounded Dynamic Graph for tool-calling data synthEsis driven by live execution. EDGE builds a graph of how each tool's output can feed another's input, keeps only the links that succeed when actually called against the live APIs, and traverses these verified links to synthesize executable multi-step trajectories. Fine-tuned via GRPO on the resulting dataset, our 9B model nearly matches the untuned 27B model from the same family, improving substantially not only on KOPA-Bench but also on the BFCL benchmark.