Semigroup-JEPA: Latent Dynamics Consistency for Zero-Shot Physics Generalization
SG-JEPA extends LeWorldModel by action-conditioning the temporal model with the physics parameter and jointly training an encoder and predictor via autoregressive latent rollout. It reduces open-loop prediction error by up to 2x on 2D datasets and increases control success rate up to 2.5x on 3D robotic datasets compared to DINO-WM.
Semigroup-JEPA (SG-JEPA) is a Joint-Embedding Predictive Architecture world model that learns latent dynamics for physics generalization. It extends LeWorldModel by supplying the physics parameter to the temporal model via action-conditioning and jointly training an encoder and predictor through autoregressive latent rollout. Evaluated on dynamical tasks under different gravitational fields, SG-JEPA reduces open-loop prediction error by up to 2 times on two-dimensional datasets and increases control success rate up to 2.5 times for three-dimensional robotic datasets compared to DINO-WM. A linear feature model is developed to explain the advantage.
SG-JEPA incorporates physics parameters into the temporal model via action-conditioning, enabling the latent space to capture law-conditioned dynamics. The autoregressive latent rollout jointly trains encoder and predictor, improving consistency of latent dynamics across varying physical conditions. The linear feature model separates local law-conditioned errors, providing a mechanistic explanation for improved generalization.
This research demonstrates that JEPA-based world models can be adapted for physics-aware prediction and control, potentially benefiting robotics and simulation where physical parameter variation is common. The improved control success rate on 3D robotic tasks suggests practical applicability in robotic manipulation and planning under varying dynamics.
SG-JEPA's improved prediction and control under varying physics could reduce data requirements and improve robustness in robotics and autonomous systems, potentially lowering development costs and enabling deployment in diverse environments.
Future work may extend SG-JEPA to more complex physical systems and real-world robotic platforms. The linear feature model could guide further architectural improvements. Validation on additional benchmarks and integration with policy learning frameworks are likely next steps.