Habitat Matterport 3D Semantics Dataset

The Habitat-Matterport 3D Semantics Dataset (HM3D-Semantics v0.1) is the largest-ever dataset of semantically-annotated 3D indoor spaces. HM3D-Semantics v0.1 contains dense semantic annotations for 120 high-resolution 3D scenes from the Habitat-Matterport 3D dataset. The HM3D scenes are annotated with the 1700+ raw object names, which are mapped to 40 Matterport categories. On average, each scene in HM3D-Semantics v0.1 consists of 646 objects from 114 categories. This dataset is the result of 12,000+ hours of human effort for annotation and verification by ~30 annotators.

HM3D-Semantics v0.1 is free and available here for academic, non-commercial research. Researchers can use it with FAIR’s Habitat simulator to train embodied agents, such as home robots and AI assistants, at scale for semantic navigation tasks.

Habitat ObjectNav Challenge 2022

We are announcing the Habitat 2022 ObjectNav challenge based on the HM3D-Semantics v0.1 dataset. We divide the HM3D-Semantics v0.1 scenes into 80 train / 20 val / 20 test scenes. We generate ObjectNav episodes for all scenes following the recommendations from prior work. We include 6 goal categories: bed, chair, plant, sofa, toilet, and tv_monitor (same as prior work). We withhold the test scenes and annotations for the challenge. Researchers can evaluate their models on the test scenes through the Eval AI challenge leaderboard. Below are the distributions of train and val episodes over difficulty levels (i.e., geodesic distance between start and goal positions) and goal categories.

Citing HM3D-Semantics

If you use the HM3D-Semantics dataset in your research, please cite this web-page:

  title={Habitat-Matterport 3D Semantics Dataset},
  howpublished = {\url{https://aihabitat.org/datasets/hm3d-semantics/}},
  year = {2022},