Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots
An Embodied AI simulation platform for studying collaborative human-robot interaction tasks in home environments.
An Embodied AI simulator for studying human-robot interaction at scale.
Habitat 3.0 enables diverse, efficient and accurate humanoid simulation, human-in-the-loop interaction and collaborative human-robot tasks in home environments.
Accurate Humanoid Simulation
Habitat3.0 models complex deformable humanoid bodies and diversity in appearance and motion, all while ensuring high simulation speed.
Humanoid Avatar Appearance
Habitat 3.0 supports a library of avatars with multiple gender representations, body shapes, and appearances. You can use our set of pre-baked avatars or generate your own.
Diverse Humanoid Motion
Habitat 3.0 uses a motion and behavior generation policy, enabling the programmatic control of avatars for navigation, object interaction, and various other motions. You can also animate avatars with your own motion, via motion capture, or using off-the-shelf text to motion models for motion generation.
Human-in-the-loop Interaction
Habitat3.0 provides an interactive human-in-the-loop interface, facilitating the evaluation of AI agents with real human collaborators.
User Interface
Humans can collaborate with an autonomous robot using mouse and keyboard inputs or a virtual reality (VR) interface.
Benchmark Tasks for Human-Robot Interaction
Aiming at reproducible and standardized benchmarking, we present two collaborative human-robot interaction tasks.
Social Navigation
The robot is tasked with finding and following a humanoid while maintaining a safe distance. Think of making a video call.
Social Rearrangement
A robot and a humanoid work collaboratively to rearrange a set of objects from their initial locations to desired locations, emulating the task of cleaning up a house.
Social Navigation Results
End-to-end RL learns collaborative behaviors such as yielding space to the humanoid, ensuring their unobstructed movement.
Social Rearrangement Results
Learned policies efficiently split the task between the robot and the humanoid, even for unseen collaborators, improving efficiency over the humanoid operating alone.
Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots
Xavi Puig*, Eric Undersander*, Andrew Szot*, Mikael Dallaire Cote*, Ruslan Partsey*, Jimmy Yang*, Ruta Desai*, Alexander William Clegg*, Michal Hlavac, Tiffany Min, Theo Gervet, Vladimír Vondruš, Vincent-Pierre Berges, John Turner, Oleksandr Maksymets, Zsolt Kira, Mrinal Kalakrishnan, Jitendra Malik, Devendra Singh Chaplot, Unnat Jain, Dhruv Batra, Akshara Rai†, Roozbeh Mottaghi†
*, † Equal contribution