Open X-Embodiment: Cross-Robot Generalist Manipulation Policies and Datasets
This paper, a collaboration of 21 institutions, collected data from 22 different robots, covering 527 skills and 160,266 tasks, and trained a high-capacity model called RT-X, demonstrating positive transfer across robot platforms and improving manipulation capabilities of various robots.
This research, by constructing a large-scale, multi-robot, multi-task dataset and training a generalist policy model RT-X, validates that the robotics field can achieve pre-trained model generalization like NLP and CV, providing empirical evidence for the shift from specialized models to generalist foundation models in robot learning.
The paper constructs a cross-robot dataset with standardized data format, including 22 robots, 527 skills, and 160,266 tasks. A high-capacity Transformer-based model RT-X is trained, which, after fine-tuning on multiple robot platforms, exhibits positive transfer, i.e., leveraging experience from other platforms to improve target robot performance. Evaluation involves various robot manipulation tasks, but specific numerical metrics are not provided; the boundary is that the dataset and model primarily target manipulation tasks, and transfer effectiveness depends on data diversity.
This work provides the first large-scale cross-robot dataset and generalist policy baseline for the robotics industry, potentially reducing robot application development costs, promoting standardization and reuse of robot manipulation skills, and accelerating deployment in service robots, industrial robots, and other fields.
It is recommended that robot companies or automation solution providers adopt the RT-X model as a base policy, fine-tune it with their own robot data to shorten skill development cycles; they can also contribute to the dataset to obtain better models.
Future work needs to verify the model's generalization ability on more unseen robots, tasks, and environments, as well as continued performance improvement with larger datasets; specific signals include cross-robot transfer success rate and zero-shot generalization ability.