Environment Evolution for Terminal Agents
A paper proposes environment evolution, which incrementally increases environment difficulty off-policy and schedules evolved environments generation by generation during training to provide continuous learning signals. It derives three evolution directions from the multi-turn learning objective and implements them via a loop-engineered multi-agent harness. Quantitative rollout experiments with Hy4 preview, Claude Opus 5, and GPT-5.6 Sol show environment evolution consistently produces more difficult environments. Effectiveness is validated on Qwen3.6-27B and Qwen3.6-3…
The paper introduces environment evolution for training terminal agents, addressing the limitation of static or co-evolved environments that become less challenging as models improve. It incrementally increases environment difficulty off-policy and schedules evolved environments during training. Three evolution directions are derived from the multi-turn learning objective and implemented through a loop-engineered multi-agent harness. Experiments with Hy4 preview, Claude Opus 5, and GPT-5.6 Sol demonstrate that environment evolution consistently produces more difficult environments, and validation on Qwen3.6-27B and Qwen3.6-3… shows effectiveness.
The method shifts from on-policy co-evolution to off-policy environment evolution, enabling continuous learning signals without dependence on current policy rollouts. The three evolution directions are derived from the multi-turn learning objective, suggesting a principled approach to difficulty scaling. The loop-engineered multi-agent harness likely automates environment generation and difficulty adjustment.
This research addresses a key bottleneck in training terminal agents: the need for scalable, verifiable, and appropriately challenging environments. As frontier models become more capable, static environments provide limited learning signals, so methods like environment evolution could accelerate agent training and improve generalization.
Improved training methods for terminal agents could lead to more capable and reliable AI agents for software development, system administration, and other terminal-based tasks, potentially reducing development costs and time-to-market for agentic products.
Next signals include publication of full experimental results, open-sourcing of the environment evolution harness, and adoption by labs training terminal agents. Further research may explore applying environment evolution to other agent domains and combining it with curriculum learning.