ReplicaCAD Dataset

What is ReplicaCAD?

The ReplicaCAD dataset is an artist recreation of the scanned “FRL apartment” variations from the Replica dataset. This dataset is intended for use in the Habitat simulator for embodied in-home interaction tasks such as object re-arrangement.

All materials are licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) Public License.

Dataset Contents:

We provide two dataset downloads with different properties: one suited to interactive simulation and the other for photorealistic visualization.

Note: Both downloadable datasets contain 84 of the 105 variations described in the paper and shown in the video with the remaining 21 scenes (1 macro variation and associated 20 micro variations) withheld as a test set for challenge evaluation.

ReplicaCAD Interactive:

🤗 Browse ReplicaCAD_dataset on Hugging Face 🤗

Intended for use with a PBR shader. Contains 1 empty scene and 6 re-creations of the scanned “FRL apartment” variations staged with both large furniture and small objects and ready for dynamic simulation in Habitat-sim. Also included are 84 (of 105) artist authored re-arrangements of large furniture (fully static placements except articulations) organized into 5 macro variations (as different tenants may organize the same apartment) each with an additional 20 micro variations (with a few pieces of furniture moved/swapped).

90+ 3D object assets with convex collision geometry and physical properties (mass, friction, restitution) as well as receptacle metadata for use generating object clutter (e.g. for rearrangement tasks).

6 stage (i.e., static background) assets emptied of all but architectural features (1 each for FRL apartment and the 5 macro variations).

6+ URDF assets defining articulated furniture and door properties including receptacle metadata for generating object clutter.

1 SceneDataset configuration file which aggregates all config and asset paths for one-line import in Habitat.

.navmesh files (in navmeshes/ directory) for every scene computed for an agent with 0.3m radius (e.g. appropriate for a Fetch robot base) and additional .navmesh files (in navmeshes_default/ directory) computed with Habitat default agent parameters for optional use.

84 + 6 SceneDataset configuration files defining object metadata and scene layouts for easy use in the Habitat simulator referencing the Fetch tuned NavMeshes.

ReplicaCAD with baked lighting:

🤗 Browse ReplicaCAD_baked_lighting on Hugging Face 🤗

Contains the same 84 (of 105) artist authored re-arrangements of large furniture described in ReplicaCAD Interactive with synthetic global illumination baked into the textures for more photo-realistic visualization. All articulated furniture is included with baked lighting textures, but all other furniture is static.

84 stage (i.e., static background) assets with baked light color textures

6 URDF assets with baked light color textures

1 SceneDataset configuration file which aggregates all config and asset paths for one-line import in Habitat.

.navmesh files (in navmeshes/ directory) for every scene computed for an agent with 0.3m radius (e.g. appropriate for a Fetch robot base) and additional .navmesh files (in navmeshes_default/ directory) computed with Habitat default agent parameters for optional use.

84 Habitat SceneDataset configuration files defining the placement of the articulated objects within the stages. Also includes global receptacle metadata in world space equivalent to local receptacle metadata from ReplicaCAD interactive. References the Fetch tuned NavMeshes.

Quickstart in Habitat-sim:

ReplicaCAD can be easily downloaded and used in Habitat-sim in only a few minutes!

First, clone the habitat-sim repo or install with conda as described here.

Then download the dataset:

Tip: using the datasets_download utility is a fast and easy way to setup the most recent version for immediate use with Habitat. No post-processing necessary!

Note that you can choose the data folder and either dataset may be downloaded and used independently of the other.

# with conda install
python -m habitat_sim.utils.datasets_download --uids replica_cad_dataset replica_cad_baked_lighting --data-path /path/to/data/

# with source
python /path/to/habitat_sim/src_python/habitat_sim/utils/datasets_download.py --uids replica_cad_dataset replica_cad_baked_lighting --data-path /path/to/data/

For display systems, simply run the interactive viewer in either C++ or python. With these applications you can navigate the scene with mouse and WASD, pick up and move objects, and open/close doors, drawers, and cabinets.

#C++ viewer application:
# ./build/viewer if compiling locally
# Loading a ReplicaCAD Interactive scene
habitat-viewer --enable-physics --dataset /path/to/data/replica_cad/replicaCAD.scene_dataset_config.json -- apt_1
# Loading a ReplicaCAD Baked Lighting scene
habitat-viewer --enable-physics --dataset /path/to/data/replica_cad_baked_lighting/replicaCAD_baked.scene_dataset_config.json -- sc1_staging_00

#Python examples/viewer.py
#NOTE: habitat-sim/ directory must be on your `PYTHONPATH`
# Loading a ReplicaCAD Interactive scene
python examples/viewer.py --dataset /path/to/data/replica_cad/replicaCAD.scene_dataset_config.json --scene apt_1
# Loading a ReplicaCAD Baked Lighting scene
python examples/viewer.py --dataset /path/to/data/replica_cad_baked_lighting/replicaCAD_baked.scene_dataset_config.json --scene sc1_staging_00

Alternatively try the quickstart Jupyter notebook (examples/tutorials/notebooks/ReplicaCAD_quickstart.ipynb), also available as a python script for headless installations:

python path/to/habitat-sim/examples/tutorials/nb_python/ReplicaCAD_quickstart.py

Citing ReplicaCAD

Using ReplicaCAD in your research? Please cite the following paper: arxiv

@inproceedings{szot2021habitat,
    title     =     {Habitat 2.0: Training Home Assistants to Rearrange their Habitat},
    author    =     {Andrew Szot and Alex Clegg and Eric Undersander and Erik Wijmans and Yili Zhao and John Turner and Noah Maestre and Mustafa Mukadam and Devendra Chaplot and Oleksandr Maksymets and Aaron Gokaslan and Vladimir Vondrus and Sameer Dharur and Franziska Meier and Wojciech Galuba and Angel Chang and Zsolt Kira and Vladlen Koltun and Jitendra Malik and Manolis Savva and Dhruv Batra},
    booktitle   =     {Advances in Neural Information Processing Systems (NeurIPS)},
    year      =     {2021}
}