Classes

• module habitat
  • module config
    • module default_structured_configs
      • class HabitatConfig The entry point for the configuration of Habitat. It holds the environment, simulator, task and dataset configurations.
      • class DatasetConfig Configuration for the dataset of the task.
      • class TaskConfig The definition of the task in Habitat.
      • class EnvironmentConfig Some habitat environment configurations.
      • class StopActionConfig In Navigation tasks only, the stop action is a discrete action. When called, the Agent will request to stop the navigation task. Note that this action is needed to succeed in a Navigation task since the Success is determined by the Agent calling the stop action within range of the target. Note that this is different from the RearrangeStopActionConfig that works for Rearrangement tasks only instead of the Navigation tasks.
      • class MoveForwardActionConfig In Navigation tasks only, this discrete action will move the robot forward by a fixed amount determined by the SimulatorConfig.forward_step_size amount.
      • class TurnLeftActionConfig In Navigation tasks only, this discrete action will rotate the robot to the left by a fixed amount determined by the SimulatorConfig.turn_angle amount.
      • class TurnLeftActionConfig In Navigation tasks only, this discrete action will rotate the robot to the left by a fixed amount determined by the SimulatorConfig.turn_angle amount.
      • class TurnRightActionConfig In Navigation tasks only, this discrete action will rotate the robot to the right by a fixed amount determined by the SimulatorConfig.turn_angle amount.
      • class LookUpActionConfig In Navigation tasks only, this discrete action will rotate the robot’s camera up by a fixed amount determined by the SimulatorConfig.tilt_angle amount.
      • class LookDownActionConfig In Navigation tasks only, this discrete action will rotate the robot’s camera down by a fixed amount determined by the SimulatorConfig.tilt_angle amount.
      • class NumStepsMeasurementConfig In both Navigation and Rearrangement tasks, counts the number of steps since the start of the episode.
      • class DistanceToGoalMeasurementConfig In Navigation tasks only, measures the geodesic distance to the goal.
      • class SuccessMeasurementConfig For Navigation tasks only, Measures 1.0 if the robot reached a success and 0 otherwise. A success is defined as calling the StopAction when the DistanceToGoal Measure is smaller than success_distance.
      • class SPLMeasurementConfig For Navigation tasks only, Measures the SPL (Success weighted by Path Length) ref: On Evaluation of Embodied Agents - Anderson et. al https://arxiv.org/pdf/1807.06757.pdf Measure is always 0 except at success where it will be the ratio of the optimal distance from start to goal over the total distance traveled by the agent. Maximum value is 1. SPL = success * optimal_distance_to_goal / distance_traveled_so_far
      • class SoftSPLMeasurementConfig For Navigation tasks only, Similar to SPL, but instead of a boolean, success is now calculated as 1 - (ratio of distance covered to target). SoftSPL = max(0, 1 - distance_to_goal / optimal_distance_to_goal) * optimal_distance_to_goal / distance_traveled_so_far
      • class DistanceToGoalRewardMeasurementConfig In Navigation tasks only, measures a reward based on the distance towards the goal. The reward is - (new_distance - previous_distance) i.e. the decrease of distance to the goal.
      • class ObjectGoalSensorConfig For Object Navigation tasks only. Generates a discrete observation containing the id of the goal object for the episode.
      • class InstanceImageGoalSensorConfig Used only by the InstanceImageGoal Navigation task. The observation is a rendered image of the goal object within the scene.
      • class InstanceImageGoalHFOVSensorConfig Used only by the InstanceImageGoal Navigation task. The observation is a single float value corresponding to the Horizontal field of view (HFOV) in degrees of the image provided by the InstanceImageGoalSensor.
      • class CompassSensorConfig For Navigation tasks only. The observation of the EpisodicCompassSensor is a single float value corresponding to the angle difference in radians between the current rotation of the robot and the start rotation of the robot along the vertical axis.
      • class GPSSensorConfig For Navigation tasks only. The observation of the EpisodicGPSSensor are two float values corresponding to the vector difference in the horizontal plane between the current position and the start position of the robot (in meters).
      • class PointGoalWithGPSCompassSensorConfig Indicates the position of the point goal in the frame of reference of the robot.
      • class HumanoidDetectorSensorConfig Check if the human is in frame
      • class ArmDepthBBoxSensorConfig Bounding box sensor to check if the object is in frame
      • class SpotHeadStereoDepthSensorConfig For Spot only. Sensor fusion for inputs of Spot stereo pair depth sensor
      • class EmptyActionConfig In Navigation tasks only, the pass action. The robot will do nothing.
      • class ArmActionConfig In Rearrangement tasks only, the action that will move the robot arm around. The action represents to delta angle (in radians) of each joint.
      • class BaseVelocityActionConfig In Rearrangement only. Corresponds to the base velocity. Contains two continuous actions, the first one controls forward and backward motion, the second the rotation.
      • class HumanoidJointActionConfig In Rearrangement only. Corresponds to actions to change the humanoid joints. Contains the parameter num_joints, indicating the joints that can be modified.
      • class HumanoidPickActionConfig In rearrangement tasks only. Config for humanoid to reach objects using IK. For now only contains the number of joints. May be extended with duration of action
      • class RearrangeStopActionConfig In rearrangement tasks only, if the robot calls this action, the task will end.
      • class OracleNavActionConfig Rearrangement Only, Oracle navigation action. This action takes as input a discrete ID which refers to an object in the PDDL domain. The oracle navigation controller then computes the actions to navigate to that desired object.
      • class SelectBaseOrArmActionConfig In rearrangement tasks only, if the robot calls this action, the task will end.
      • class RelativeRestingPositionSensorConfig Rearrangement only. Sensor for the relative position of the end-effector’s resting position, relative to the end-effector’s current position. The three values correspond to the cartesian coordinates of the resting position in the frame of reference of the end effector. The desired resting position is determined by the habitat.task.desired_resting_position coordinates relative to the robot’s base.
      • class IsHoldingSensorConfig Rearrangement only. A single float sensor with value 1.0 if the robot is grasping any object and 0.0 otherwise.
      • class EEPositionSensorConfig Rearrangement only. the cartesian coordinates (3 floats) of the arm’s end effector in the frame of reference of the robot’s base.
      • class JointSensorConfig Rearrangement only. Returns the joint positions of the robot.
      • class HumanoidJointSensorConfig Rearrangement only. Returns the joint positions of the robot.
      • class TargetStartSensorConfig Rearrangement only. Returns the relative position from end effector to a target object that needs to be picked up.
      • class GoalSensorConfig Rearrangement only. Returns the relative position from end effector to a goal position in which the agent needs to place an object.
      • class TargetStartGpsCompassSensorConfig Rearrangement only. Returns the initial position of every object that needs to be rearranged in composite tasks, in 2D polar coordinates.
      • class InitialGpsCompassSensorConfig Rearrangement only. Returns the relative distance to the initial starting location of the agent in 2D polar coordinates.
      • class TargetGoalGpsCompassSensorConfig Rearrangement only. Returns the desired goal position of every object that needs to be rearranged in composite tasks, in 2D polar coordinates.
      • class EndEffectorToRestDistanceMeasurementConfig Rearrangement only. Distance between current end effector position and the resting position of the end effector. Requires that the RelativeRestingPositionSensor is attached to the agent.
      • class RobotForceMeasurementConfig The amount of force in newton’s applied by the robot. It computes both the instant and accumulated.
      • class DoesWantTerminateMeasurementConfig Rearrangement Only. Measures 1 if the agent has called the stop action and 0 otherwise.
      • class ForceTerminateMeasurementConfig If the force is greater than a certain threshold, this measure will be 1.0 and 0.0 otherwise. Note that if the measure is 1.0, the task will end as a result.
      • class ObjectToGoalDistanceMeasurementConfig In rearrangement only. The distance between the target object and the goal position for the object.
      • class ObjAtGoalMeasurementConfig The measure is a dictionary of target indexes to float. The values are 1 if the object is within succ_thresh of the goal position for that object.
      • class ArtObjAtDesiredStateMeasurementConfig Rearrangement open/close container tasks only. Whether the articulated object (fridge or cabinet door) towards a desired state (open or closed) as defined by the task.
      • class RotDistToGoalMeasurementConfig Rearrangement Navigation task only. The angle between the forward direction of the agent and the direction to the goal location.
      • class PddlStageGoalsMeasurementConfig PDDL Rearrangement only. 1.0 if the agent complete a particular stage defined in stage_goals and 0.0 otherwise. Stage goals are specified in the pddl task description.
      • class NavToPosSuccMeasurementConfig Rearrangement Navigation task only. The value is 1.0 if the robot is within success_distance of the goal position.
      • class SocialNavStatsMeasurementConfig Social nav stats computation
      • class NavSeekSuccessMeasurementConfig Social nav seek success measurement
      • class RearrangePickSuccessMeasurementConfig Rearrangement Only. Requires the end_effector_sensor lab sensor. 1.0 if the robot picked the target object.
      • class RearrangePickRewardMeasurementConfig Rearrangement Only. Requires the end_effector_sensor lab sensor. The reward for the pick task.
      • class PlaceSuccessMeasurementConfig Rearrangement Only. Requires the end_effector_sensor lab sensor. 1.0 if the robot placed the target object on the goal position and has its end effector within ee_resting_success_threshold of its resting position.
      • class PlaceRewardMeasurementConfig Rearrangement Only. Requires the end_effector_sensor lab sensor. The reward for the place task.
      • class ArtObjSuccessMeasurementConfig Rearrangement open/close container tasks only. Requires art_obj_at_desired_state. Is 1.0 if the articulated object is in desired state and the end effector is within rest_dist_threshold of the resting position. If must_call_stop is True, the robot must also call the rearrange_stop action.
      • class ArtObjRewardMeasurementConfig Rearrangement open/close container tasks only. Requires art_obj_at_desired_state.
      • class NavToObjSuccessMeasurementConfig Rearrangement Navigation only. Takes the value 1.0 when the Robot successfully navigated to the target object. Depends on nav_to_pos_succ.
      • class NavToObjRewardMeasurementConfig Rearrangement Navigation task only. The reward for rearrangement navigation.
      • class PddlSuccessMeasurementConfig PDDL rearrangement tasks only (rearrange, set_table, tidy_house). It uses a goal pddl expression to validate the success.
      • class RuntimePerfStatsMeasurementConfig If added to the measurements, this will time various sections of code in the simulator and task logic. If using with a multi-environment trainer (like DD-PPO) it is recommended to only log this stat for one environment since this metric can include many numbers.
  • module core
    • module agent Base implementation of agent inside habitat. To build agents inside habitat the user should subclass habitat.Agent and implement the act() and reset() methods.
      • class Agent Abstract class for defining agents which act inside core.env.Env.
    • module benchmark Habitat Challenge Benchmark class: a generic benchmarking framework for locally or remotely evaluating performance of an Agent. Implements evaluation of habitat.Agent inside habitat.Env. habitat.Benchmark creates a habitat.Env which is specified through the config_env parameter in constructor. The evaluation is task agnostic and implemented through metrics defined for habitat.EmbodiedTask.
      • class Benchmark Generic benchmark class for evaluating agents in environments from config.
    • module env
      • class Env Fundamental environment class for habitat.
      • class RLEnv Reinforcement Learning (RL) environment class which subclasses gym.Env.
    • module embodied_task Implements task and measurements needed for training and benchmarking of habitat.Agent inside habitat.Env.
      • class Action An action that can be performed by an agent solving a task in environment. For example for navigation task action classes will be: MoveForwardAction, TurnLeftAction, TurnRightAction. The action can use Task members to pass a state to another action, as well as keep own state and reset when new episode starts.
      • class EmbodiedTask Base class for embodied task. EmbodiedTask holds definition of a task that agent needs to solve: action space, observation space, measures, simulator usage. EmbodiedTask has reset() and step() methods that are called by Env. EmbodiedTask is the one of main dimensions for the framework extension. Once new embodied task is introduced implementation of EmbodiedTask is a formal definition of the task that opens opportunity for others to propose solutions and include it into benchmark results.
      • class Measure Represents a measure that provides measurement on top of environment and task.
      • class Measurements Represents a set of Measures, with each Measure being identified through a unique id.
      • class Metrics Dictionary containing measurements.
      • class SimulatorTaskAction An EmbodiedTask action that is wrapping simulator action.
    • module dataset Implements dataset functionality to be used habitat.EmbodiedTask. habitat.core.dataset abstracts over a collection of habitat.core.Episode. Each episode consists of a single instantiation of a habitat.Agent inside habitat.Env.
      • class BaseEpisode Base class for episode specification that includes only the episode_id and scene_id. This class allows passing the minimum required episode information to identify the episode (unique key) to the habitat baseline process, thus saving evaluation time.
      • class Dataset Base class for dataset specification.
      • class Episode Base class for episode specification that includes initial position and rotation of agent, scene id, episode.
      • class EpisodeIterator Episode Iterator class that gives options for how a list of episodes should be iterated.
    • module simulator Defines the core Simulator and Sensor class wrapper APIs. The classes here are primarily defining abstract APIs which are implemented further downstream.
      • class ActionSpaceConfiguration Attrs base class wrapper for DictConfig defining the action space for a task.
      • class AgentState Represents the rigid transformation state of an agent as a 3D position and quaternion rotation.
      • class BumpSensor Wrapper for non-visual navmesh collision Sensors. See Sensor parent class for more details.
      • class DepthSensor Wrapper for depth Camera Sensors. See Sensor parent class for more details.
      • class Observations Dictionary containing sensor observations
      • class RGBSensor Wrapper for 3-channel color Camera Sensors. See Sensor parent class for more details.
      • class SemanticSensor Wrapper for integer id Camera Sensors where each integer is mapped to an object instance or semantic class. See Sensor parent class for more details.
      • class Sensor Represents a sensor that provides data from the environment to agent. The user of this class needs to implement the get_observation method and the user is also required to set attributes.
      • class SensorSuite Represents a set of sensors, with each sensor being identified through a unique id.
      • class ShortestPathPoint Wrapper for the information embedded in a single point for a ShortestPath planner object: 3D position, quaternion rotation, and the action which led to the state.
      • class Simulator Abstract simulator class for habitat. New simulators to be added to habitat must derive from this class and implement the abstract methods.
    • module registry Registry is central source of truth in Habitat.
      • class Registry
    • module vector_env
      • class ThreadedVectorEnv Provides same functionality as VectorEnv, the only difference is it runs in a multi-thread setup inside a single process.
      • class VectorEnv Vectorized environment which creates multiple processes where each process runs its own environment. Main class for parallelization of training and evaluation.
      • class _ReadWrapper Convenience wrapper to track if a connection to a worker process should have something to read.
      • class _WriteWrapper Convenience wrapper to track if a connection to a worker process can be written to safely. In other words, checks to make sure the result returned from the last write was read.
    • module batch_rendering
      • module env_batch_renderer
        • class EnvBatchRenderer Wrapper for batch rendering functionality, which renders visual sensors of N environments simultaneously.
      • module env_batch_renderer_constants
    • module challenge
      • class Challenge Extends the Benchmark class to run evaluate the current challenge config and submit results to the remote evaluation server.
    • module environments
      • class GymHabitatEnv A registered environment that wraps a RLTaskEnv with the HabGymWrapper to use the default gym API.
      • class GymRegistryEnv A registered environment that wraps a gym environment to be used with habitat-baselines
      • class RLTaskEnv
    • module logging
      • class HabitatLogger
    • module utils A module providing some disconnected core utilities.
      • class DatasetFloatJSONEncoder JSON Encoder that sets a float precision for a space saving purpose and encodes ndarray and quaternion. The encoder is compatible with JSON version 2.0.9.
      • class DatasetJSONEncoder Extension of base JSONEncoder to handle common Dataset types: numpy array, numpy quaternion, Omegaconf, and dataclass.
      • class Singleton This metatclass creates Singleton objects by ensuring only one instance is created and any call is directed to that instance. The mro() function and following dunders, EXCEPT __call__, are inherited from the the stdlib Python library, which defines the “type” class.
  • module gym
    • module gym_env_episode_count_wrapper
      • class EnvCountEpisodeWrapper
    • module gym_env_obs_dict_wrapper
      • class EnvObsDictWrapper
    • module gym_wrapper
      • class HabGymWrapper
    • module gym_definitions
  • module articulated_agent_controllers
    • class HumanoidBaseController Generic class to replay SMPL-X Motions.
    • class HumanoidRearrangeController Humanoid Controller, converts high level actions such as walk, or reach into joints positions
    • class HumanoidSeqPoseController Humanoid Seq Pose Controller, replays a sequence of humanoid poses.
    • class Pose Represents a single humanoid pose: global root transform + joint state in generalized coordinates (dofs). NOTE: assumes a SMPLX representation of humanoids where all joints are represented as quaternions.
    • class Motion Represents a sequential motion, corresponding to a sequence of Poses.
  • module articulated_agents
    • module humanoids
      • class KinematicHumanoid
    • module robots
      • module fetch_suction
        • class FetchSuctionRobot
      • module spot_robot
        • class SpotRobot
      • class FetchRobot
      • class FetchRobotNoWheels
      • class FrankaRobot
      • class SpotRobot
      • class StretchRobot
    • class ArticulatedAgentInterface Generic robot interface defines standard API functions.
    • class ArticulatedAgentBase Generic interface defining standard API functions for a robot with a controllable base.
    • class Manipulator Generic manipulator interface defines standard API functions. Robot with a controllable arm.
    • class MobileManipulator Robot with a controllable base and arm.
    • class MobileManipulatorParams Data to configure a mobile manipulator.
    • class ArticulatedAgentCameraParams Data to configure a camera placement on the articulated agent.
    • class StaticManipulator Robot with a fixed base and controllable arm.
    • class StaticManipulatorParams Data to configure a static manipulator.
  • module datasets
    • module eqa
      • module mp3d_eqa_dataset
        • class Matterport3dDatasetV1 Class inherited from Dataset that loads Matterport3D Embodied Question Answering dataset.
    • module image_nav
      • module instance_image_nav_dataset
        • class InstanceImageNavDatasetV1 Class that loads an Instance Image Navigation dataset.
    • module object_nav
      • module object_nav_dataset
        • class ObjectNavDatasetV1 Class inherited from PointNavDataset that loads Object Navigation dataset.
    • module pointnav
      • module pointnav_dataset
        • class PointNavDatasetV1 Class inherited from Dataset that loads Point Navigation dataset.
    • module rearrange
      • module navmesh_utils
        • class SimpleVelocityControlEnv A simple environment to control the velocity of the robot. Assumes x-forward in robot local space.
      • module rearrange_dataset
        • class RearrangeDatasetV0 Class inherited from PointNavDataset that loads Rearrangement dataset.
        • class RearrangeEpisode Specifies additional objects, targets, markers, and ArticulatedObject states for a particular instance of an object rearrangement task.
      • module samplers
        • module art_sampler
          • class ArtObjCatStateSampler
          • class ArticulatedObjectStateSampler
          • class CompositeArticulatedObjectStateSampler Samples multiple articulated states simultaneously with rejection of invalid configurations.
        • module object_sampler
          • class ObjectSampler Sample an object from a set and try to place it in the scene on a Receptacles from some Receptacle set.
        • module object_target_sampler
          • class ObjectTargetSampler Base sampler for object targets. Instead of sampling from an object template set, sample from an instance set.
        • module receptacle
          • class AABBReceptacle Defines an AABB Receptacle volume above a surface for sampling object placements within a scene.
          • class AnyObjectReceptacle The AnyObjectReceptacle enables any rigid or articulated object or link to be used as a Receptacle without metadata annotation. It uses the top surface of an object’s global space bounding box as a heuristic for the sampling area. The sample efficiency is likely to be poor (especially for concave objects like L-shaped sofas), TODO: this could be mitigated by the option to pre-compute a discrete set of candidate points via raycast upon initialization. Also, this heuristic will not support use of interior surfaces such as cubby and cabinet shelves since volumetric occupancy is not considered.
          • class OnTopOfReceptacle
          • class Receptacle
          • class ReceptacleSet ReceptacleSet(name: str, included_object_substrings: List[str], excluded_object_substrings: List[str], included_receptacle_substrings: List[str], excluded_receptacle_substrings: List[str], is_on_top_of_sampler: bool = False, comment: str = ”)
          • class ReceptacleTracker
          • class TriangleMeshReceptacle Defines a Receptacle surface as a triangle mesh. TODO: configurable maximum height.
        • module scene_sampler
          • class BalancedSceneSampler Evenly splits generated episodes amongst all scenes in the set. Generates all episodes for each scene contiguously for efficiency.
          • class MultiSceneSampler Uniform sampling from a set of scenes.
          • class SceneSampler Abstract Class Samples a scene for the RearrangeGenerator.
          • class SingleSceneSampler Returns a single provided scene using the sampler API
    • module vln
      • module r2r_vln_dataset
        • class VLNDatasetV1 Class inherited from Dataset that loads a Vision and Language Navigation dataset.
    • module utils Tokenize and vocabulary utils originally authored by @apsdehal and are taken from Pythia.
      • class VocabDict
      • class VocabFromText
    • module registration
  • module sims
    • module habitat_simulator
      • module actions This module provides an extendable Enum singleton manager for mapping simulator action names to integer id values. Be default it provides cylinder agent move and look actions.
        • class HabitatSimActionsSingleton Implements an extendable Enum for the mapping of action names to their integer values.
      • module debug_visualizer The DebugVisualizer (DBV) module provides a singleton class for quickly generating custom debug RGB images from an already instantiated Simulator instance. DebugObservation provides a wrapper class for accessing, saving, showing, and manipulating numpy image matrices with PIL. The module also provides some helper functions for highlighting objects with DebugLineRender and stitching images together into a matrix of images.
        • class DebugObservation Observation wrapper to provide a simple interface for managing debug observations and caching the image.
        • class DebugVisualizer Support class for simple visual debugging of a Simulator instance. Assumes the default agent (0) is a camera (i.e. there exists an RGB sensor coincident with agent 0 transformation).
      • module habitat_simulator
        • class HabitatSim Simulator wrapper over habitat-sim
        • class HabitatSimDepthSensor
        • class HabitatSimEquirectangularDepthSensor
        • class HabitatSimEquirectangularRGBSensor
        • class HabitatSimEquirectangularSemanticSensor
        • class HabitatSimFisheyeDepthSensor
        • class HabitatSimFisheyeRGBSensor
        • class HabitatSimFisheyeSemanticSensor
        • class HabitatSimRGBSensor
        • class HabitatSimSemanticSensor
        • class HabitatSimSensor
      • module kinematic_relationship_manager This module implements a singleton manager class for tracking and applying kinematic relationships between objects in the simulation. It is meant to be instantiated upon Simulator init and then updated and applied as objects are moved, applying relative transformations down a parent->child kinematic tree.
        • class KinematicRelationshipManager Manages the kinematic relationships between objects such that object states can be manipulated while maintaining said relationships.
        • class RelationshipGraph Uses two dictionaries to simulate a bi-directional tree relationship between objects.
      • module object_state_machine This module implements a singleton state-machine architecture for representing and managing non-geometric object states via metadata manipulation. For example, tracking and manipulating state such as “powered on” or “clean vs dirty”. This interface is intended to provide a foundation which can be extended for downstream applications.
        • class BooleanObjectState Abstract ObjectStateSpec base class for boolean type states. Defines some standard handling for boolean states.
        • class ObjectIsClean ObjectIsClean state specifies whether an object is clean or dirty.
        • class ObjectIsPoweredOn State specifies whether an appliance object is powered on or off.
        • class ObjectStateMachine Defines the logic for managing multiple states across all objects in the scene.
        • class ObjectStateSpec Abstract base class for object states specifications. Defines the API for inherited and extended states.
      • module sim_utilities This module provides a diverse set of functional utilities for common operations involving the Simulator and ManagedObjects including: object getters from id and handle, geometric utilities, prepositional logic, region queries, articulated object interactions, and more.
    • module registration
  • module tasks
    • module eqa
      • module eqa
        • class AnswerAccuracy AnswerAccuracy
        • class AnswerAction
        • class CorrectAnswer CorrectAnswer
        • class EQAEpisode Specification of episode that includes initial position and rotation of agent, goal, question specifications and optional shortest paths.
        • class EQATask
        • class EpisodeInfo Episode Info
        • class QuestionData
        • class QuestionSensor
    • module nav
      • module instance_image_nav_task
        • class InstanceImageGoal An instance image goal is an ObjectGoal that also contains a collection of InstanceImageParameters.
        • class InstanceImageGoalHFOVSensor A sensor that returns the horizontal field of view (HFOV) in degrees of the current episode’s instance image goal.
        • class InstanceImageGoalNavEpisode Instance ImageGoal Navigation Episode
        • class InstanceImageGoalSensor A sensor for instance-based image goal specification used by the InstanceImageGoal Navigation task. Image goals are rendered according to camera parameters (resolution, HFOV, extrinsics) specified by the dataset.
        • class InstanceImageNavigationTask A task for navigating to a specific object instance specified by a goal image. Built on top of ObjectNavigationTask. Used to explicitly state a type of the task in config.
        • class InstanceImageParameters
      • module nav
        • class Collisions
        • class DistanceToGoal The measure calculates a distance towards the goal.
        • class DistanceToGoalReward The measure calculates a reward based on the distance towards the goal. The reward is - (new_distance - previous_distance) i.e. the decrease of distance to the goal.
        • class EpisodicCompassSensor The agents heading in the coordinate frame defined by the episode, theta=0 is defined by the agents state at t=0
        • class EpisodicGPSSensor The agents current location in the coordinate frame defined by the episode, i.e. the axis it faces along and the origin is defined by its state at t=0
        • class HeadingSensor Sensor for observing the agent’s heading in the global coordinate frame.
        • class ImageGoalSensor Sensor for ImageGoal observations which are used in ImageGoal Navigation.
        • class IntegratedPointGoalGPSAndCompassSensor Sensor that integrates PointGoals observations (which are used PointGoal Navigation) and GPS+Compass.
        • class LookDownAction
        • class LookUpAction
        • class MoveForwardAction
        • class NavigationEpisode Class for episode specification that includes initial position and rotation of agent, scene name, goal and optional shortest paths. An episode is a description of one task instance for the agent.
        • class NavigationGoal Base class for a goal specification hierarchy.
        • class NavigationMovementAgentAction
        • class NavigationTask
        • class PointGoalSensor Sensor for PointGoal observations which are used in PointGoal Navigation.
        • class ProximitySensor Sensor for observing the distance to the closest obstacle
        • class RoomGoal Room goal that can be specified by room_id or position with radius.
        • class SPL SPL (Success weighted by Path Length)
        • class SoftSPL Soft SPL
        • class StopAction
        • class Success Whether or not the agent succeeded at its task
        • class TeleportAction
        • class TopDownMap Top Down Map measure
        • class TurnLeftAction
        • class TurnRightAction
        • class VelocityAction
      • module object_nav_task
        • class ObjectGoal
        • class ObjectGoalNavEpisode
        • class ObjectGoalSensor
        • class ObjectNavigationTask An Object Navigation Task class for a task specific methods. Used to explicitly state a type of the task in config.
        • class ObjectViewLocation
      • module shortest_path_follower
        • class ShortestPathFollower
    • module rearrange
      • module actions
        • module actions
          • class ArmAbsPosAction The arm motor targets are directly set to the joint configuration specified by the action.
          • class ArmAbsPosKinematicAction The arm is kinematically directly set to the joint configuration specified by the action.
          • class ArmAction An arm control and grip control into one action space.
          • class ArmEEAction Uses inverse kinematics (requires pybullet) to apply end-effector position control for the articulated_agent’s arm.
          • class ArmRelPosAction The arm motor targets are offset by the delta joint values specified by the action
          • class ArmRelPosKinematicAction The arm motor targets are offset by the delta joint values specified by the action
          • class ArmRelPosKinematicReducedActionStretch The arm motor targets are offset by the delta joint values specified by the action and the mask. This function is used for Stretch.
          • class ArmRelPosMaskAction The arm motor targets are offset by the delta joint values specified by the action
          • class BaseVelAction The articulated agent base motion is constrained to the NavMesh and controlled with velocity commands integrated with the VelocityControl interface.
          • class BaseVelNonCylinderAction The articulated agent base motion is constrained to the NavMesh and controlled with velocity commands integrated with the VelocityControl interface.
          • class EmptyAction A No-op action useful for testing and in some controllers where we want to wait before the next operation.
          • class HumanoidJointAction
          • class RearrangeStopAction
          • class SelectBaseOrArmAction This is useful if we do not allow base and arm to move at the same time
        • module articulated_agent_action
          • class ArticulatedAgentAction Handles which articulated_agent instance the action is applied to.
        • module grip_actions
          • class GazeGraspAction
          • class GripSimulatorTaskAction
          • class MagicGraspAction
          • class SuctionGraspAction
        • module humanoid_actions
          • class HumanoidPickAction
          • class HumanoidPickObjIdAction
        • module oracle_nav_action
          • class OracleNavAction An action that will convert the index of an entity (in the sense of PddlEntity) to navigate to and convert this to base/humanoid joint control to move the robot to the closest navigable position to that entity. The entity index is the index into the list of all available entities in the current scene. The config flag motion_type indicates whether the low level action will be a base_velocity or a joint control.
          • class OracleNavCoordinateAction An action to drive the anget to a given coordinate
        • module pddl_actions
          • class PddlApplyAction
      • module articulated_agent_manager
        • class ArticulatedAgentData Data needed to manage an agent instance.
        • class ArticulatedAgentManager Handles creating, updating and managing all agent instances.
      • module marker_info
        • class MarkerInfo A data structure to track information about markers in the scene. These are automatically updated based on the position of the articulated link the marker is pinned to.
      • module multi_agent_sensors
        • class AreAgentsWithinThreshold Returns if the agents are close to each other and about to collide, thus the agents should replan.
        • class DidAgentsCollide Detects if the 2 agents in the scene are colliding with each other at the current step. Only supports 2 agent setups.
        • class MultiAgentGlobalPredicatesSensor Returns the predicates ONLY for the agent this sensor is configured for. This is different from GlobalPredicatesSensor which returns the predicates for all agents.
        • class NumAgentsCollide Cumulative number of steps in the episode the agents are in collision.
        • class OtherAgentGps Returns the GPS coordinates of the other agent.
        • class RearrangeCooperateReward PddlSubgoalReward adapted for 2 agent setups to penalize and potentially end the episode on agent collisions.
      • module multi_task
        • module pddl_action
          • class PddlAction
        • module pddl_domain
          • class PddlDomain Manages the information from the PDDL domain and task definition.
          • class PddlProblem
        • module pddl_logical_expr
          • class LogicalExpr Refers to combinations of PDDL expressions or subexpressions.
        • module pddl_predicate
          • class Predicate
        • module pddl_sensors
          • class GlobalPredicatesSensor
          • class MoveObjectsReward A reward based on L2 distances to object/goal.
          • class PddlStageGoals Adds to the metrics [TASK_NAME]_success: Did the agent complete a particular stage defined in stage_goals at ANY point in the episode.
          • class PddlSubgoalReward Reward that gives a sparse reward on completing a PDDL stage-goal.
          • class PddlSuccess Did satisfy all the goal predicates?
        • module pddl_task
          • class PddlTask Task that sets up PDDL manager.
        • module rearrange_pddl
          • class ExprType Hierarchical type in the PDDL system. The user can define custom types and the types in SimulatorObjectType are automatically defined.
          • class PddlEntity Abstract PDDL entity. This is linked to simulator via PddlSimInfo.
          • class PddlSimInfo Manages the mapping between the abstract PDDL and the underlying simulator entities. This also provides some helper methods for accessing PDDL entity simulator properties like object position (which could vary per entity type).
      • module rearrange_grasp_manager
        • class RearrangeGraspManager Manages the agent grasping onto rigid objects and the links of articulated objects.
      • module rearrange_sensors
        • class AbsGoalSensor
        • class AbsTargetStartSensor Relative position from end effector to target object
        • class ArmDepthBBoxSensor Bounding box sensor to check if the object is in frame
        • class BadCalledTerminate Returns 0 if the agent has called the stop action when the success condition is also met or not called the stop action when the success condition is not met. Returns 1 otherwise.
        • class BaseToObjectDistance Gets the distance between the base and all current target object COMs.
        • class DidViolateHoldConstraintMeasure
        • class DoesWantTerminate Returns 1 if the agent has called the stop action and 0 otherwise.
        • class EEPositionSensor
        • class EndEffectorToGoalDistance
        • class EndEffectorToObjectDistance Gets the distance between the end-effector and all current target object COMs.
        • class EndEffectorToRestDistance Distance between current end effector position and position where end effector rests within the robot body.
        • class ForceTerminate If the accumulated force throughout this episode exceeds the limit.
        • class GfxReplayMeasure
        • class GoalSensor Relative to the end effector
        • class HasFinishedHumanoidPickSensor Returns 1 if the agent has finished the oracle nav action. Returns 0 otherwise.
        • class HasFinishedOracleNavSensor Returns 1 if the agent has finished the oracle nav action. Returns 0 otherwise.
        • class HumanoidJointSensor
        • class IsHoldingSensor Binary if the robot is holding an object or grasped onto an articulated object.
        • class JointSensor
        • class JointVelocitySensor
        • class LocalizationSensor The position and angle of the articulated_agent in world coordinates.
        • class MultiObjSensor Abstract parent class for a sensor that specifies the locations of all targets.
        • class NumStepsMeasure The number of steps elapsed in the current episode.
        • class ObjAtGoal Returns if the target object is at the goal (binary) for each of the target objects in the scene.
        • class ObjectToGoalDistance Euclidean distance from the target object to the goal.
        • class PositionGpsCompassSensor
        • class RearrangeReward An abstract class defining some measures that are always a part of any reward function in the Habitat 2.0 tasks.
        • class RelativeRestingPositionSensor
        • class RestingPositionSensor Desired resting position in the articulated_agent coordinate frame.
        • class ReturnToRestDistance Distance between end-effector and resting position if the articulated agent is holding the object.
        • class RobotCollisions Returns a dictionary with the counts for different types of collisions.
        • class RobotForce The amount of force in newton’s accumulatively applied by the robot.
        • class RuntimePerfStats
        • class TargetCurrentSensor This is the ground truth object position sensor relative to the robot end-effector coordinate frame.
        • class TargetGoalGpsCompassSensor
        • class TargetStartGpsCompassSensor
        • class TargetStartSensor Relative position from end effector to target object
        • class ZeroMeasure The number of steps elapsed in the current episode.
      • module rearrange_sim
        • class RearrangeSim
      • module rearrange_task
        • class RearrangeTask Defines additional logic for valid collisions and gripping shared between all rearrangement tasks.
      • module robot_specific_sensors
        • class SpotHeadStereoDepthSensor For Spot only. Sensor fusion for inputs of Spot stereo pair depth sensor. We want to combine head stereo depth images along with resizing it so that its size is the same as the size of the arm depth image. Spot’s arm depth size: (240, 228, 1) Spot’s head stereo depth size: (212, 120, 1)
      • module social_nav
        • module oracle_social_nav_actions
          • class OracleNavCoordAction An action that comments the agent to navigate to a sequence of random navigation targets (or we call these targets (x,y) coordinates)
          • class OracleNavRandCoordAction Oracle Nav RandCoord Action. Selects a random position in the scene and navigates there until reaching. When the arg is 1, does replanning.
        • module social_nav_sensors
          • class HumanoidDetectorSensor
          • class InitialGpsCompassSensor Get the relative distance to the initial starting location of the robot
          • class SocialNavReward Reward that gives a continuous reward for the social navigation task.
          • class SocialNavSeekSuccess Social nav seek success meassurement
          • class SocialNavStats The measure for social navigation
        • module social_nav_task
          • class PddlSocialNavTask Social nav task based on PddlTask for training low-level policies under multi-agent setting
        • module utils
      • module sub_tasks
        • module articulated_object_sensors
          • class ArtJointSensor Gets the joint state (position and velocity) of the articulated object specified by the use_marker_name property in the task object.
          • class ArtJointSensorNoVel Gets the joint state (just position) of the articulated object specified by the use_marker_name property in the task object.
          • class ArtObjAtDesiredState
          • class ArtObjReward A general reward definition for any tasks involving manipulating articulated objects.
          • class ArtObjState Measures the current joint state of the target articulated object.
          • class ArtObjSuccess Measures if the target articulated object joint state is at the success criteria.
          • class EndEffectorDistToMarker The distance of the end-effector to the target marker on the articulated object.
          • class MarkerRelPosSensor Tracks the relative position of a marker to the robot end-effector specified by use_marker_name in the task. This use_marker_name must exist in the task and refer to the name of a marker in the simulator.
        • module articulated_object_task
          • class RearrangeCloseDrawerTaskV1
          • class RearrangeCloseFridgeTaskV1
          • class RearrangeOpenDrawerTaskV1
          • class RearrangeOpenFridgeTaskV1
          • class SetArticulatedObjectTask Base class for all tasks involving manipulating articulated objects.
        • module nav_to_obj_sensors
          • class DistToGoal
          • class NavGoalPointGoalSensor GPS and compass sensor relative to the starting object position or goal position.
          • class NavToObjReward
          • class NavToObjSuccess
          • class NavToPosSucc
          • class OracleNavigationActionSensor
          • class RotDistToGoal
        • module nav_to_obj_task
          • class DynNavRLEnv
          • class NavToInfo
        • module pick_sensors
          • class DidPickObjectMeasure
          • class RearrangePickReward
          • class RearrangePickSuccess
        • module pick_task
          • class RearrangePickTaskV1
        • module place_sensors
          • class PlaceReward
          • class PlaceSuccess
        • module place_task
          • class RearrangePlaceTaskV1
        • module reach_sensors
          • class AnyReachSuccess
          • class RearrangeReachReward
          • class RearrangeReachSuccess
        • module reach_task
          • class RearrangeReachTaskV1
      • module utils
        • class CacheHelper
        • class CollisionDetails
        • class IkHelper
        • class UsesArticulatedAgentInterface For sensors or actions that are agent specific. Used to split actions and sensors between multiple agents.
    • module registration
    • module utils
    • module vln
      • module vln
        • class InstructionData
        • class InstructionSensor
        • class VLNEpisode Specification of episode that includes initial position and rotation of agent, goal specifications, instruction specifications, reference path, and optional shortest paths.
        • class VLNTask
  • module utils
    • module visualizations
      • module fog_of_war
      • module maps
      • module utils
    • module geometry_utils
    • module common
    • module env_utils
    • module pickle5_multiprocessing
      • class CloudpickleWrapper Wrapper that uses cloudpickle to pickle and unpickle the result.
      • class ConnectionWrapper Proxy class for _multiprocessing.Connection which uses ForkingPickler to serialize objects. Will use the Pickle5 backport if available.
    • module profiling_wrapper Wrappers for habitat_sim profiling_utils functions. The wrappers are no-ops if habitat_sim isn’t installed.
      • class RangeContext Annotate a range for profiling. Use as a function decorator or in a with statement. See habitat_sim profiling_utils.