Voyager: An Open-Ended Embodied Agent with Large Language Models: LLM-Driven Lifelong Learning Agent Autonomously Explores Minecraft
In May 2023, NVIDIA and UT Austin team proposed Voyager, the first LLM-driven lifelong learning agent that continuously explores, acquires skills, and discovers new things in Minecraft without human intervention. Voyager uses GPT-4 and, through an automatic curriculum, skill library, and iterative prompting mechanism, obtains 3.3 times more unique items and unlocks milestones 15.3 times faster.
Voyager demonstrates the potential of LLMs as the core of general-purpose agents, achieving lifelong learning in an open world. It maximizes exploration through an automatic curriculum, stores skills as executable code, and self-improves using environmental feedback. This work provides a reproducible paradigm for general embodied agents (e.g., robots, autonomous driving), proving that the combination of LLM + code library + environmental feedback can continuously accumulate capabilities.
Voyager consists of three components: 1) Automatic Curriculum: generates tasks based on current skills and unexplored areas; 2) Skill Library: represents skills as executable code (e.g., Python functions), supporting composition and reuse; 3) Iterative Prompting Mechanism: injects environmental feedback, execution errors, and self-verification results into GPT-4 prompts to improve programs. Voyager operates entirely through black-box queries to GPT-4, requiring no fine-tuning. Experiments in Minecraft show Voyager significantly outperforms previous SOTA (e.g., AutoGPT) in exploration efficiency, skill acquisition, and generalization. Limitations: relies on GPT-4 API, high cost; skill library may become redundant as scale grows.
The Voyager paradigm can be transferred to robotics, game AI, automated testing, and more. It demonstrates the feasibility of LLMs as a 'brain' controlling embodied systems, potentially spawning a new generation of agent platforms. For the gaming industry, it can develop autonomous NPCs; for manufacturing, it can control robotic arms for complex tasks. However, real-time performance and safety still need to be addressed.
Recommend that robotics companies explore Voyager-like architectures, using LLMs as high-level planners. Game companies can develop AI teammates or opponents based on Voyager. Invest in startups that productize LLM + embodied agents.
Focus on deploying Voyager on real robots, as well as compression and retrieval optimization of the skill library. Will a lightweight version of Voyager (based on open-source LLMs) emerge? Research on skill forgetting and conflict management is needed. Open-source code will drive community reproduction in more environments (e.g., Habitat, Isaac Sim).