benchmark module
Habitat Challenge Benchmark class: a generic benchmarking framework for locally or remotely evaluating performance of an Agent.
Implements evaluation of habitat.Agent inside habitat.Env.
habitat.Benchmark creates a habitat.Env which is specified through
the config_env parameter in constructor. The evaluation is task agnostic
and implemented through metrics defined for habitat.EmbodiedTask.
Classes
- class Benchmark
- Generic benchmark class for evaluating agents in environments from config.