The Dark Souls Learning Environment (DSLE) is a containerized platform providing all 22 boss encounters from Dark Souls: Remastered as Gymnasium-style benchmarks for game-playing agents. It features real-time combat, high-dimensional visual input, and sparse terminal rewards. A subset, DSLE-5, includes five representative bosses. Evaluated methods on DSLE-5: random policy, expert system, evolutionary baseline, PPO, and DQN. The expert system and evolutionary baseline defeated the tutorial boss (Asylum Demon) with 63% and 43% peak win rates, respectively. PPO and DQN showed no measurable learning (≤0.33% win rate on tutorial boss, 0% elsewhere) within tens of wall-clock hours per run. None of the five methods defeated the other four DSLE-5 bosses.
Researchers introduced DSLE, a containerized platform that turns all 22 boss fights from Dark Souls: Remastered into reinforcement learning benchmarks with a Gymnasium interface. They defined a five-boss subset (DSLE-5) for controlled comparisons. On DSLE-5, an expert system and an evolutionary baseline each beat the tutorial boss, but no method defeated the other four bosses. PPO and DQN agents trained from visual input failed to learn, achieving near-zero win rates despite significant compute budgets.
The environment exposes real-time, high-dimensional visual inputs and sparse rewards, making it a challenging testbed for current deep RL algorithms. The failure of PPO and DQN to learn even on the simplest boss suggests that standard model-free approaches struggle with the combination of long time horizons, precise timing, and sparse feedback inherent in action-game combat. The evolutionary baseline's partial success hints that population-based methods may be more robust in this domain, but overall, the results underscore a significant gap in agent capabilities for complex, visually-driven real-time tasks.
This work highlights the limitations of current RL algorithms in handling complex, real-time video game environments, which are often used as proxies for real-world robotics and autonomous systems. The inability of popular methods to make progress on DSLE-5 may shift research focus toward hybrid approaches, better exploration strategies, or model-based RL. For the game AI industry, it reinforces that hand-crafted expert systems still outperform learning agents in many scenarios, but the standardized benchmark could accelerate progress in general game-playing AI.
While not directly commercial, DSLE provides a rigorous benchmark that could drive advances in AI for real-time decision-making, with potential applications in robotics, autonomous vehicles, and industrial automation. Companies developing general-purpose RL platforms or game AI middleware may use DSLE to demonstrate and improve their technology. The environment's containerized design also lowers the barrier for reproducible research, which could accelerate the development of more capable AI systems.
Next signals to watch include: (1) whether the research community adopts DSLE as a standard benchmark and organizes competitions; (2) if new algorithms (e.g., model-based RL, hierarchical RL, or transformer-based policies) can surpass the expert baseline on DSLE-5; (3) extensions of the environment to other Souls-like games or real-time strategy games; and (4) potential transfer of learned policies to robotics tasks requiring real-time visual-motor control.