ARISE-RL: Agentic Rubric-Grounded Iterative Self-Evolution with Reinforcement Learning
ARISE-RL is a full-cycle self-evolution framework that couples a task/rubric Generator and a reasoning Solver through rubric-mediated co-evolution. The Generator grounds tool-related rubric criteria in real tool observations and is rewarded for producing valid, intermediate-difficulty tasks aligned with the Solver's evolving capability boundary. The Solver learns from fine-grained rubric satisfaction signals through multi-step reasoning and tool use. The framework introduces Reward-Gated Self-Evolution Distillation (RG-SED), which selectively distills a memory-augmented variant of the same policy back into itself only when the memory yields empirical reward improvement.
ARISE-RL addresses challenges in training open-ended agents via reinforcement learning, including lack of verifiable gold answers, scalable rubrics, and brittle rewards near the model's capability boundary. It proposes a co-evolution framework where a Generator creates tasks and rubrics grounded in tool observations, and a Solver learns from rubric satisfaction signals. RG-SED reduces distribution mismatch by distilling only when memory improves reward.
The framework uses rubric-mediated co-evolution to align task difficulty with the Solver's capability boundary, and RG-SED to selectively distill memory-augmented policy variants, mitigating noisy rollout contrast and distribution mismatch in group-based policy learning.
This approach could enable more stable and scalable training of open-ended agentic systems, potentially reducing reliance on manually curated rubrics and improving generalization in tool-use scenarios.
Improved training efficiency and robustness for autonomous agents could lower development costs and accelerate deployment of AI systems capable of complex, tool-mediated tasks.
Next signals include empirical validation on long-horizon agentic benchmarks, comparisons against baseline RL methods, and potential adoption in agent training pipelines.