[00:00] Introduction to Hover: Neural Whole Body Controller for Humanoids
[00:15] Problem: Current controllers lack versatility across tasks
[00:50] Human motion imitation as a unified control approach
[01:23] Policy distillation: Learning from an oracle policy
[02:01] Command space: Kinematic, joint angle, and root tracking modes
[02:34] Motion retargeting: From human data to robot movements
[03:09] Performance comparison with specialist policies
[03:43] Real-world testing on Unitree H1 robot
[04:15] Comparison with MHC and Masked Mimic approaches
[04:49] Future work and current limitations
[05:18] Reward function design and components
[06:02] D-Agger advantages in policy learning
[06:33] Domain randomization for sim-to-real transfer
[07:06] Conclusions on Hover's contributions
Authors: Tairan He, Wenli Xiao, Toru Lin, Zhengyi Luo, Zhenjia Xu, Zhenyu Jiang, Jan Kautz, Changliu Liu, Guanya Shi, Xiaolong Wang, Linxi Fan, Yuke Zhu
Affiliations: NVIDIA, CMU, UC Berkeley, UT Austin, UC San Diego
Abstract: Humanoid whole-body control requires adapting to diverse tasks such as navigation, loco-manipulation, and tabletop manipulation, each demanding a different mode of control. For example, navigation relies on root velocity tracking, while tabletop manipulation prioritizes upper-body joint angle tracking. Existing approaches typically train individual policies tailored to a specific command space, limiting their transferability across modes. We present the key insight that full-body kinematic motion imitation can serve as a common abstraction for all these tasks and provide general-purpose motor skills for learning multiple modes of whole-body control. Building on this, we propose HOVER (Humanoid Versatile Controller), a multi-mode policy distillation framework that consolidates diverse control modes into a unified policy. HOVER enables seamless transitions between control modes while preserving the distinct advantages of each, offering a robust and scalable solution for humanoid control across a wide range of modes. By eliminating the need for policy retraining for each control mode, our approach improves efficiency and flexibility for future humanoid applications.
Link: https://hover-versatile-humanoid.github.io/
