DRACO: Fine-Grained Credit Assignment with Dynamic Rubrics for Long-Horizon Agent Training
DRACO (Distributing Rubric-based Advantage for Credit Optimization) generates rubrics dynamically during training, scores them once per completed trajectory, and redistributes that judgment over steps responsible for annotated rubrics to produce differentiated per-step advantages in GRPO. On AppWorld, DRACO gains 15.9 points over the base model and 5.3 points over GRPO trained with a sparse ground-truth reward, without using verifiers. On out-of-domain Tau-Bench, it gains 5.3 points over the base model even without a frontier judge.
DRACO is a method for fine-grained credit assignment in long-horizon agent training when ground-truth success signals are unavailable. It dynamically generates rubrics during training, scores them once per trajectory, and redistributes the judgment over responsible steps to produce per-step advantages in GRPO. The redistribution is closed-form and requires no trained attribution module. Reported results show DRACO outperforms both base models and GRPO with sparse ground-truth rewards on AppWorld and Tau-Bench.
DRACO addresses the outcome-blind setting by replacing sparse trajectory-level rewards with dynamically generated rubrics and a closed-form redistribution mechanism. This yields differentiated per-step advantages for GRPO without additional learned components. The reported gains on AppWorld and Tau-Bench suggest that dynamic rubric generation and step-level credit assignment can improve long-horizon agent training even without verifiers or frontier judges.
The approach may reduce reliance on programmatic verifiers or expensive frontier judges for training long-horizon agents. If the method generalizes, it could lower the cost and complexity of developing capable agents in domains where success is hard to verify automatically.
DRACO could enable training of more capable long-horizon agents without costly verifier infrastructure, potentially reducing development costs and expanding the set of tasks where reinforcement learning from verifiable rewards is feasible.
Observable next signals include independent replications of DRACO on additional benchmarks, comparisons with other credit assignment methods, and extensions to multi-agent or real-world agent tasks. Adoption in open-source agent training frameworks would indicate practical impact.