Interactive Gibson Benchmark: A Benchmark for Interactive Navigation in Cluttered Environments

We present <italic>Interactive Gibson Benchmark</italic>, the first comprehensive benchmark for training and evaluating <italic>Interactive Navigation</italic> solutions. Interactive Navigation tasks are robot navigation problems where physical interaction with objects (e.g., pushing) is allowed and even encouraged to reach the goal. Our benchmark comprises two novel elements: 1) a new experimental simulated environment, the <italic>Interactive Gibson Environment</italic>, that generate photo-realistic images of indoor scenes and simulates realistic physical interactions of robots and common objects found in these scenes; 2) the <italic>Interactive Navigation Score</italic>, a novel metric to study the interplay between navigation and physical interaction of Interactive Navigation solutions. We present and evaluate multiple learning-based baselines in Interactive Gibson Benchmark, and provide insights into regimes of navigation with different trade-offs between navigation, path efficiency and disturbance of surrounding objects. We make our benchmark publicly available<xref ref-type="fn" rid="fn1"><sup>1</sup></xref><fn id="fn1"><label><sup>1</sup></label><p>[Online]. Available: <uri>https://sites.google.com/view/interactivegibsonenv</uri>.</p></fn> and encourage researchers from related robotics disciplines (e.g., planning, learning, control) to propose, evaluate, and compare their Interactive Navigation solutions in Interactive Gibson Benchmark.

[1]  Jitendra Malik,et al.  Gibson Env: Real-World Perception for Embodied Agents , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[2]  Tucker R. Balch,et al.  Ten Years of the AAAI Mobile Robot Competition and Exhibition , 2002, AI Mag..

[3]  Pieter Abbeel,et al.  DoorGym: A Scalable Door Opening Environment And Baseline Agent , 2019, ArXiv.

[4]  Advait Jain,et al.  Behavior-Based Door Opening with Equilibrium Point Control , 2009 .

[5]  Thomas A. Funkhouser,et al.  MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments , 2017, ArXiv.

[6]  Song-Chun Zhu,et al.  VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learning , 2019, ArXiv.

[7]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[8]  Rahul Sukthankar,et al.  Cognitive Mapping and Planning for Visual Navigation , 2017, International Journal of Computer Vision.

[9]  Mike Stilman,et al.  Hierarchical Decision Theoretic Planning for Navigation Among Movable Obstacles , 2012, WAFR.

[10]  Silvio Savarese,et al.  4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Heinz Wörn,et al.  Opening a door with a humanoid robot using multi-sensory tactile feedback , 2008, 2008 IEEE International Conference on Robotics and Automation.

[12]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[13]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[14]  J. A. Valencia,et al.  Evaluation of navigation of an autonomous mobile robot , 2007 .

[15]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[16]  Harry Shum,et al.  Review of image-based rendering techniques , 2000, Visual Communications and Image Processing.

[17]  Silvio Savarese,et al.  Deep Visual MPC-Policy Learning for Navigation , 2019, IEEE Robotics and Automation Letters.

[18]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[19]  Ali Farhadi,et al.  AI2-THOR: An Interactive 3D Environment for Visual AI , 2017, ArXiv.

[20]  Kenji Suzuki,et al.  Linear-time connected-component labeling based on sequential local operations , 2003, Comput. Vis. Image Underst..

[21]  Lihui Wang,et al.  Minimising Energy Consumption for Robot Arm Movement , 2013 .

[22]  Henry Zhu,et al.  Soft Actor-Critic Algorithms and Applications , 2018, ArXiv.

[23]  Keith Redmill,et al.  Systems for Safety and Autonomous Behavior in Cars: The DARPA Grand Challenge Experience , 2007, Proceedings of the IEEE.

[24]  Pieter Abbeel,et al.  Benchmarking Deep Reinforcement Learning for Continuous Control , 2016, ICML.

[25]  Matthias Nießner,et al.  Matterport3D: Learning from RGB-D Data in Indoor Environments , 2017, 2017 International Conference on 3D Vision (3DV).

[26]  Wolfram Burgard,et al.  An Experimental Protocol for Benchmarking Robotic Indoor Navigation , 2014, ISER.

[27]  Jitendra Malik,et al.  Habitat: A Platform for Embodied AI Research , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[28]  Thomas Bräunl.  Research relevance of mobile robot competitions , 1999, IEEE Robotics Autom. Mag..

[29]  George Drettakis,et al.  Scalable inside-out image-based rendering , 2016, ACM Trans. Graph..

[30]  Silvio Savarese,et al.  Joint 2D-3D-Semantic Data for Indoor Scene Understanding , 2017, ArXiv.

[31]  Michael D. Buhrmester,et al.  Amazon's Mechanical Turk , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[32]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[33]  James J. Kuffner,et al.  Planning Among Movable Obstacles with Artificial Constraints , 2008, Int. J. Robotics Res..

[34]  Silvio Savarese,et al.  HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulators , 2019, CoRL.

[35]  Andrew Y. Ng,et al.  Probabilistic Mobile Manipulation in Dynamic Environments, with Application to Opening Doors , 2007, IJCAI.

[36]  Nelson David Muñoz Ceballos,et al.  Quantitative Performance Metrics for Mobile Robots Navigation , 2010 .

[37]  Dima Damen,et al.  Computer Vision and Pattern Recognition (CVPR) , 2009 .

[38]  Marwan Mattar,et al.  Unity: A General Platform for Intelligent Agents , 2018, ArXiv.

[39]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[40]  Feng Gao,et al.  VRGym: a virtual testbed for physical and interactive AI , 2019, ACM TUR-C.

[41]  Angel P. del Pobil,et al.  Benchmarks in Robotics Research , 2006 .

[42]  Sumetee kesorn.  Visual Navigation for Mobile Robots: a Survey , 2012 .

[43]  Paolo Cignoni,et al.  MeshLab: an Open-Source Mesh Processing Tool , 2008, Eurographics Italian Chapter Conference.

[44]  Martin Buehler,et al.  Editorial for Journal of Field Robotics—Special Issue on the DARPA Grand Challenge , 2006, J. Field Robotics.

[45]  Jitendra Malik,et al.  On Evaluation of Embodied Navigation Agents , 2018, ArXiv.

[46]  Lars Petersson,et al.  High-level control of a mobile manipulator for door opening , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[47]  Matthias Nießner,et al.  Scan2CAD: Learning CAD Model Alignment in RGB-D Scans , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Sergey Levine,et al.  Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flight , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[49]  Mike Stilman,et al.  Navigation among movable obstacles , 2007 .

[50]  Li Wang,et al.  The Robotarium: A remotely accessible swarm robotics research testbed , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[51]  Amit Konar,et al.  Energy efficient trajectory planning by a robot arm using invasive weed optimization technique , 2011, 2011 Third World Congress on Nature and Biologically Inspired Computing.