Better Slots, Better Worlds: Representation Quality & Robustness in Object-Centric World Models
A controlled study of object-centric world models (OCWMs) for visual model-predictive control found that planning success correlates positively with unsupervised slot-quality metrics (FG-ARI, mBO), though gains saturate at high slot quality. With well-bound slots, auxiliary proprioception inputs and masking inductive bias become unnecessary. Under unseen distribution shifts, the OCWM with well-bound slots is more robust than an end-to-end trained scene-centric model.
Researchers conducted a controlled study of object-centric world models (OCWMs) for visual model-predictive control, evaluating representation quality and generalization under distribution shift relative to scene-centric models. They found that planning success correlates positively with unsupervised slot-quality metrics (FG-ARI, mBO), with gains saturating at high slot quality. Well-bound slots eliminate the need for auxiliary proprioception inputs and masking inductive bias used in prior methods. Under unseen distribution shifts, the OCWM with well-bound slots is more robust than the end-to-end trained scene-centric model.
The study isolates the contribution of object-centric representation quality to planning performance. Slot-quality metrics FG-ARI and mBO serve as predictors of downstream control success, but the relationship saturates, suggesting diminishing returns beyond a threshold. The finding that well-bound slots remove the need for proprioception and masking implies that the object-centric inductive bias alone can provide sufficient structure for robust planning, simplifying model architecture.
Object-centric world models could reduce the need for hand-crafted inductive biases and auxiliary inputs in robotic control and autonomous systems. This may lower engineering complexity and improve sample efficiency in real-world applications where distribution shift is common. The robustness advantage under distribution shift is particularly relevant for deployment in unstructured environments.
For robotics and autonomous systems companies, adopting object-centric world models could reduce development time and improve reliability in novel environments. The elimination of auxiliary inputs and masking simplifies the model pipeline, potentially lowering compute and data requirements. Robustness to distribution shift can reduce failure rates in deployed systems, improving safety and customer trust.
Next signals include whether these findings generalize to more complex visual scenes, higher-dimensional action spaces, and real robot platforms. Further work may explore scaling object-centric world models and integrating them with large pre-trained vision backbones. The saturation of slot-quality gains suggests a focus on other bottlenecks such as planning algorithm or dynamics model capacity.