Discriminative World Models for Web Agents
A paper introduces predicted-state matching, a training objective for world models used in web agents. The objective requires predicted representations to distinguish the true resulting state from states reached by alternative actions. Models are trained on a branching web-agent dataset derived from WebArena Go-Browse trajectories. Experiments show the approach outperforms world models trained with supervised next-state prediction on a held-out predicted-state matching benchmark, and improves PRM-style action ranking on WebPRMBench compared with action-only PRMs and PRMs augmented with world models.
The paper proposes a new training objective for world models in web agents, called predicted-state matching. Unlike supervised next-state prediction, which generates fixed representations like HTML or AXTree snapshots, this objective aligns the world model with the downstream ranker by ensuring predicted states are discriminative across candidate actions. The authors use a branching dataset from WebArena Go-Browse trajectories, where each decision point includes multiple alternative actions and their resulting states. Evaluation on a held-out predicted-state matching benchmark shows superior performance over supervised next-state prediction, and the approach also improves PRM-style action ranking on WebPRMBench compared to action-only PRMs and PRMs augmented with world models.
The key technical contribution is replacing supervised next-state prediction with predicted-state matching, which directly optimizes the discriminative power of predicted states for action ranking. This addresses a misalignment between world model training and its use in test-time action selection. The method leverages branching trajectories to provide contrastive examples, enabling the model to learn representations that separate true outcomes from alternatives. The reported improvements on WebPRMBench suggest that integrating this objective with PRMs can enhance action selection in web agents.
This research targets the growing field of web agents, where world models are used for action selection. By improving the alignment between world models and rankers, the approach could lead to more reliable and efficient web automation systems. The use of WebArena Go-Browse trajectories indicates a focus on realistic web environments, which is relevant for commercial applications in browser automation, digital assistants, and enterprise process automation.
Improved world models for web agents can enhance the accuracy and efficiency of automated web interactions, reducing errors and increasing task completion rates. This has direct business value for companies developing web automation tools, RPA solutions, and AI assistants that operate in browser environments. The method could lower the cost of deploying reliable web agents by reducing the need for extensive fine-tuning or manual intervention.
Potential next signals include adoption of predicted-state matching in other world model architectures, extension to non-web agent domains, and integration with larger-scale web agent datasets. Further validation on real-world web tasks and comparison with other ranking methods would strengthen the findings. The approach may also influence the design of PRMs and world models in broader agentic AI systems.