Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe
Researchers introduced KOPA-Bench, a benchmark of 145 real-world tasks for multi-step tool-calling over Korean open public APIs. They also presented EDGE, an execution-grounded dynamic graph method that synthesizes executable multi-step trajectories by verifying tool-call links against live APIs. Fine-tuning a 9B model with GRPO on EDGE-generated data nearly matched an untuned 27B model from the same family, improving performance on KOPA-Bench and BFCL.
A new benchmark, KOPA-Bench, measures open-source LLM performance on multi-step tool-calling across 145 real-world Korean government API tasks. The EDGE data-synthesis method builds a graph of verified tool-call links using live execution, then generates executable trajectories. Fine-tuning a 9B model with GRPO on this data nearly matches an untuned 27B model, with gains on both KOPA-Bench and BFCL.
EDGE uses live execution to validate which tool outputs can feed into other tool inputs, creating a graph of executable links. Traversing these verified links produces multi-step trajectories that are guaranteed to be executable, reducing hallucinated tool calls. GRPO fine-tuning on this data enables a 9B model to approach the performance of a 27B model, indicating that data quality and execution grounding can compensate for model scale in tool-calling tasks.
The work addresses a gap for data-sovereignty-constrained public institutions that need on-premise, open-source LLM agents for government APIs. By improving smaller open-source models, it lowers the barrier for deploying compliant multi-step tool-calling agents without relying on large proprietary models. The benchmark and synthesis method could accelerate adoption of open-source agents in regulated sectors.
For public institutions and enterprises under data-sovereignty rules, this approach enables cost-effective deployment of smaller open-source models that can reliably chain tool calls across live APIs. It reduces dependence on large proprietary models and may lower inference costs while maintaining task performance.
Observable next signals include adoption of KOPA-Bench by other research groups, replication of EDGE on non-Korean public APIs, and release of fine-tuned open-source models for government tool-calling. Further gains may come from scaling EDGE to more complex tool graphs or combining it with other RL methods.