PAC-MAN: Perception-Aware CBF-RL for Whole-Body Safety in Humanoid Dodgeball
Researchers present PAC-MAN, a perception-aware CBF-RL framework for whole-body humanoid dodgeball. The policy uses only segmentation-masked depth from a head-mounted camera. Training-time CBF guidance represents clearance to every body link, with an adversarial motion prior. Evaluated on a controlled any-link contact benchmark with seeded throws in single-throw and deployment-loop regimes. The policy approaches privileged state oracle performance with a fixed onboard camera. Joint-CBF performs best with accurate ball states, degrades under fixed-camera observations when used only as training guidance, and recovers with a ball-tracking gimbal or privileged runtime filter. A lightweight Link-CBF policy is deployed zero-shot on the Unitree G1 robot, tolerating imperfect perception and succeeding on 95% of throws.
PAC-MAN is a perception-aware control-barrier-function reinforcement learning framework that enables a humanoid robot to dodge balls using only onboard camera depth data. The system couples safety constraints with realistic sensing, achieving near-oracle evasion performance. A Link-CBF variant was successfully deployed on a Unitree G1 robot, demonstrating robust real-world whole-body safety under perceptual uncertainty.
The framework integrates control barrier functions with reinforcement learning, using an adversarial motion prior to shape evasive behaviors. It reveals that barrier structure effectiveness depends on perceptual observability: Joint-CBF excels with accurate state estimates but degrades with fixed-camera observations unless augmented by a gimbal or privileged filter. Link-CBF offers a lightweight alternative that tolerates imperfect perception, enabling zero-shot real-world transfer.
This work demonstrates that whole-body safety for dynamic humanoid tasks can be achieved with low-cost onboard sensing, reducing reliance on external motion capture. The successful deployment on a commercially available Unitree G1 robot suggests near-term feasibility for safer humanoid operation in unstructured environments, potentially accelerating adoption in logistics, manufacturing, and entertainment.
Enables humanoid robots to operate safely around humans and dynamic obstacles using only onboard sensors, reducing infrastructure costs and expanding deployment possibilities. The 95% success rate on a real robot indicates readiness for pilot applications in environments requiring reactive whole-body motion, such as warehouse safety zones or interactive entertainment.
Next signals include extending the approach to multi-ball or multi-agent scenarios, integrating predictive perception models to handle faster or erratic projectiles, and testing on other humanoid platforms. Commercialization may involve licensing the safety framework to robotics companies or integrating it into robot operating systems. Further research could explore sim-to-real transfer for more complex whole-body maneuvers.