CoBRA: Learning Tool-Use Boundaries via Counterfactual Margins
CoBRA is a counterfactual boundary-learning framework for tool-augmented language models. It constructs internal and external experts from the same base model, collects paired trajectories, and estimates the reward margin between answering with and without tools. The margin partitions data into internal-favored, external-favored, and ambiguous cases. CoBRA uses clear-margin samples for Boundary-Aware Cold-Start SFT, followed by MARS-RL with reference-split rollouts and counterfactual marginal advantages to optimize boundary decisions. Experiments with retrieval as the main tool on Qwen3-4B are reported.
CoBRA addresses the problem of deciding when to call a tool in large language models. It learns tool-use boundaries by estimating instance-level marginal benefit via counterfactual reward margins. The framework uses paired trajectories from internal and external experts, partitions data by margin, and applies supervised fine-tuning and reinforcement learning to optimize boundary decisions. Initial experiments use retrieval as the tool on Qwen3-4B.
CoBRA introduces counterfactual marginal advantages in reinforcement learning, using reference-split rollouts to compare tool-use vs. no-tool trajectories. This provides a more direct signal for boundary decisions than absolute confidence or task reward. The approach is model-agnostic and can be applied to any tool-augmented LLM.
Efficient tool-use decisions are critical for reducing latency, cost, and error propagation in deployed LLM systems. CoBRA's focus on marginal benefit could lead to more selective tool invocation, improving operational efficiency for AI services that integrate external tools like retrieval, calculators, or APIs.
Reducing unnecessary tool calls can lower inference costs and latency, while improving answer quality by avoiding retrieval noise. This has direct business value for AI providers offering tool-augmented services, potentially enabling more competitive pricing and better user experience.
If CoBRA demonstrates improved boundary decisions on Qwen3-4B, it may be extended to other tools and larger models. Future work could explore multi-tool scenarios, dynamic tool selection, and integration with production systems. The framework may also influence research on cost-aware LLM inference.