HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object Interaction

A new white paper introduces HiPHI, a comprehensive large-scale motion capture dataset designed to address the data scarcity currently hindering humanoid robot learning. By utilizing FrameNet, a linguistic framework for human action, researchers have systematically collected data covering a broad spectrum of whole-body motions. The dataset distinguishes itself by incorporating synchronized object trajectories and meshes, which are essential for teaching robots complex real-world tasks such as pushing, pulling, and carrying objects. The paper demonstrates that reinforcement learning policies trained on this data scale effectively, showing significant improvements in performance. Furthermore, the researchers provide evidence of successful sim-to-real transfer, where policies trained on the HiPHI dataset are deployed onto physical humanoid robots. This initiative aims to bridge the gap between internet-scale video data and the high-precision requirements of embodied AI, offering a robust foundation for future developments in humanoid robotics and physical interaction.