Habitat Embodied Agents Workshop 2019

CVPR 2019 Workshop, Long Beach, California

Sunday, 16th June, 09:00 AM to 05:30 PM
Seaside Ballroom B, Long Beach Convention & Entertainment Center

Schedule

9:00 AM – 9:10 AM		Welcome and introduction Dhruv Batra (Georgia Tech, FAIR)
9:10 AM – 9:35 AM		Invited Talk Richard Newcombe (Facebook Reality Labs) Creating Real-World Simulators: A new generation of ego-centric and predictive datasets for Embodied AI
9:35 AM – 10:00 AM		Invited Talk Benjamin Kuipers (University of Michigan) Learning about Space: Mapping and Navigation
10:00 AM – 10:25 AM		Invited Talk Raia Hadsell (DeepMind) Navigating Agents
10:25 AM – 10:45 AM		Morning Break
10:45 AM – 11:10 AM		Invited Talk Jitendra Malik (FAIR, UC Berkeley) Visually Guided Navigation - the role of mapping, landmarks and learning
11:10 AM – 11:30 AM		Intro to Habitat Platform Manolis Savva (FAIR, SFU) Towards the Next Generation of Simulation Platforms for Embodied AI
11:30 AM – 11:40 AM		Habitat Challenge on EvalAI Rishabh Jain (Georgia Tech)
11:40 AM – 12:00 PM		Challenge Results and Analysis Abhishek Kadian (FAIR)
12:00 PM – 12:10 PM		PointGoalNav-RGB Track Winners (oral)
12:10 PM – 12:20 PM		PointGoalNav-RGBD Track Winners (oral)
12:20 PM – 12:30 PM		Habitat Team Presentation Erik Wijmans and Bhavana Jain
12:30 PM – 1:50 PM		Lunch Break
1:50 PM – 2:50 PM		Poster Session Location: Pacific Arena Ballroom Allotted Poster Boards: #191 to #230
2:50 PM – 3:15 PM		Invited Talk Julian Straub (Facebook Reality Labs) The Replica Dataset: A Digital Replica of Indoor Spaces
3:15 PM – 3:40 PM		Invited Talk Vladimir Vondrus (Magnum Engine) Building your Research on Magnum
3:40 PM – 4:10 PM		Coffee Break
4:10 PM – 4:30 PM		Habitat Roadmap Oleksandr Maksymets (FAIR)
4:30 PM – 5:30 PM		Panel Discussion + Closing Remarks

Richard Newcombe is the Director of Research Science at Facebook Reality Labs which is building the future of Augmented Reality and Contextualized AI services and devices at Facebook.He Co-Founded Surreal Vision Ltd. in 2014. He worked at Microsoft Research in Cambridge, gaining early access to the first commodity depth sensor, which resulted in the groundbreaking KinectFusion system. He holds BSc at the University of Essex and PhD at Imperial College.

Benjamin Kuipers joined the University of Michigan in January 2009 as Professor of Computer Science and Engineering. Prior to that, he held an endowed Professorship in Computer Sciences at the University of Texas at Austin. He received his B.A. from Swarthmore College, and his Ph.D. from MIT. He investigates the representation of commonsense and expert knowledge, with particular emphasis on the effective use of incomplete knowledge. He has served as Department Chair at UT Austin, and is a Fellow of AAAI, IEEE, and AAAS.

Raia Hadsell, a senior research scientist at DeepMind, has worked on deep learning and robotics problems for over 10 years. After completing a PhD with Yann LeCun at NYU, her research continued at Carnegie Mellon’s Robotics Institute and SRI International, and in early 2014 she joined DeepMind in London to study artificial general intelligence. Her current research focuses on the challenge of continual learning for AI agents and robotic systems. — [Webpage]

Jitendra Malik is the Arthur J. Chick Professor in the Department of Electrical Engineering and Computer Sciences at UC Berkeley. His research group has worked on many different topics in computer vision, computational modeling of human vision and computer graphics. Several well-known concepts and algorithms arose in this research, such as normalized cuts, high dynamic range imaging and R-CNN. He has mentored more than 50 PhD students and postdoctoral fellows. — [Webpage]

Julian Straub is a research scientist at Facebook Reality Labs (FRL). He obtained his PhD in 2017 from MIT CSAIL where he was advised by John W. Fisher III and John Leonard. His research interests include 3D perception, modeling, Bayesian nonparametrics and inference. His team at FRL, Surreal, is focused on formulating a new generation of end-to-end real-time 3D scene understanding systems. — [Webpage]

Vladimir Vondrus is a software consultant focusing on performance, reliability and usability. He’s the founder and main developer of Magnum Engine, an open-source graphics middleware for games and data visualization. His open source work is being used by a diverse range of engineers, game developers, researchers, students and hobbyists around the world. — [Webpage]

Overview

There has been a recent shift in the computer vision community from tasks focusing on internet images to active settings involving embodied agents that perceive and act within 3D environments. Practical deployment of AI agents in the real world requires study of active perception and coupling of perception with control as in embodied agents.

This workshop has two primary objectives. The first is to establish a unified platform and a set of benchmarks to measure progress in embodied agents by hosting an embodied agents challenge named Habitat Challenge. These benchmarks will evaluate algorithms for navigation and question-answering tasks using the Habitat platform. The benchmarks and unified embodied agents platform will catalyze future work, promoting reproducibility, reusability of code, and consistency in evaluation.

The second objective of the workshop is to bring together researchers from the fields of computer vision, language, graphics, and robotics to share work being done in the area of multi-modal AI, as well as discuss future directions in research on embodied agents connecting several recent threads of research on: visual navigation, natural language instruction following, embodied question answering and language grounding.

Embodied Agents Challenge and Call for Papers

The challenge will be based on the PointGoal Navigation task in which the agent must navigate to a target specified in terms of agent-relative Euclidean coordinates. See here for more details.

The winning team will receive Google Cloud Platform credits worth $10k!

Challenge Timeline

Challenge starts	April 3, 2019
Challenge submission deadline	May 18, 2019

People are invited to submit abstracts up to 2 pages describing work in relevant areas of simulation environments, visual navigation, embodied question answering, simulation-to-real transfer, vision & language, etc.

Accepted submissions will be presented as posters at the workshop. The extended abstract should follow the CVPR formatting guidelines and be emailed as a single PDF to the email id mentioned below.

We encourage submissions of relevant work that has been previously published, or is to be presented at the main conference. The accepted abstracts will not appear in the official CVPR proceedings.

Please email abstracts to akadian@fb.com by May 24, 2019.