GONet++: Traversability Estimation via Dynamic Scene View Synthesis

Robots that interact with a dynamic environment, such as social robots and autonomous vehicles must be able to navigate through space safely to avoid injury or damage. Hence, effective identification of traversable and non-traversable spaces is crucial for mobile robot operation. In this paper, we propose to tackle the traversability estimation problem using a dynamic view synthesis based framework. View synthesis technique provides the ability to predict the traversability of a wide area around the robot. Unlike traditional view synthesis methods, our method can model dynamic obstacles such as humans, and predict where they would go in the future, allowing the robot to plan in advance and avoid potential collisions with dynamic obstacles. Our method GONet++ is built upon GONet. GONet is only able to predict the traversability of the space right in front of the robot. However, our method applies GONet on the predicted future frames to estimate the traversability of multiple locations around the robot. We demonstrate that our method both quantitatively and qualitatively outperforms the baseline methods in both view synthesis and traversability estimation tasks.

[1]  Wolfram Burgard,et al.  Traversability analysis for mobile robots in outdoor environments: A semi-supervised learning approach based on 3D-lidar data , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Seydou SOUMARE,et al.  Real-Time Obstacle Avoidance by an Autonomous Mobile Robot using an Active Vision Sensor and a Vertically Emitted Laser Slit , 2002 .

[3]  Thomas Brox,et al.  Learning to generate chairs with convolutional neural networks , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[5]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Alexey Dosovitskiy,et al.  End-to-End Driving Via Conditional Imitation Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Vladlen Koltun,et al.  Learning to Act by Predicting the Future , 2016, ICLR.

[8]  George Drettakis,et al.  Scalable inside-out image-based rendering , 2016, ACM Trans. Graph..

[9]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[10]  Ersin Yumer,et al.  Transformation-Grounded Image Generation Network for Novel 3D View Synthesis , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Wolfram Burgard,et al.  The dynamic window approach to collision avoidance , 1997, IEEE Robotics Autom. Mag..

[12]  Bilge Mutlu,et al.  Robots in organizations: The role of workflow, social, and environmental factors in human-robot interaction , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[13]  Michael I. Jordan,et al.  Forward dynamic models in human motor control: Psychophysical evidence , 1994, NIPS.

[14]  Silvio Savarese,et al.  Deep View Morphing , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Silvio Savarese,et al.  GONet: A Semi-Supervised Deep Learning Approach For Traversability Estimation , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  Nikolai Smolyanskiy,et al.  Toward low-flying autonomous MAV trail navigation using deep neural networks for environmental awareness , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Yoichi Hori,et al.  Integrated Motion Control of a Wheelchair in the Longitudinal, Lateral, and Pitch Directions , 2008, IEEE Transactions on Industrial Electronics.

[18]  Kenta Oono,et al.  Chainer : a Next-Generation Open Source Framework for Deep Learning , 2015 .

[19]  James M. Rehg,et al.  Traversability classification for UGV navigation: a comparison of patch and superpixel representations , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Razvan Pascanu,et al.  Imagination-Augmented Agents for Deep Reinforcement Learning , 2017, NIPS.

[21]  Rob Fergus,et al.  Depth Map Prediction from a Single Image using a Multi-Scale Deep Network , 2014, NIPS.

[22]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[23]  Sergey Levine,et al.  Self-Supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Jitendra Malik,et al.  Zero-Shot Visual Imitation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  Oussama Khatib,et al.  A depth space approach to human-robot collision avoidance , 2012, 2012 IEEE International Conference on Robotics and Automation.

[26]  Sergey Levine,et al.  Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics , 2014, NIPS.

[27]  Jitendra Malik,et al.  View Synthesis by Appearance Flow , 2016, ECCV.

[28]  Thomas Brox,et al.  Multi-view 3D Models from Single Images with a Convolutional Network , 2015, ECCV.

[29]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[30]  Sanjiv Singh,et al.  Obstacle detection using adaptive color segmentation and color stereo homography , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[31]  Illah R. Nourbakhsh,et al.  Appearance-Based Obstacle Detection with Monocular Color Vision , 2000, AAAI/IAAI.

[32]  Rahul Sukthankar,et al.  Cognitive Mapping and Planning for Visual Navigation , 2017, International Journal of Computer Vision.

[33]  Roland Siegwart,et al.  From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[34]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[35]  Ye-Hoon Kim,et al.  End-to-end deep learning for autonomous navigation of mobile robot , 2018, 2018 IEEE International Conference on Consumer Electronics (ICCE).

[36]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[37]  Germán Ros,et al.  CARLA: An Open Urban Driving Simulator , 2017, CoRL.

[38]  Sergey Levine,et al.  Stochastic Variational Video Prediction , 2017, ICLR.

[39]  Cyrill Stachniss,et al.  Efficient traversability analysis for mobile robots using the Kinect sensor , 2013, 2013 European Conference on Mobile Robots.

[40]  Yang Gao,et al.  End-to-End Learning of Driving Models from Large-Scale Video Datasets , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Joshua B. Tenenbaum,et al.  Deep Convolutional Inverse Graphics Network , 2015, NIPS.

[42]  John Flynn,et al.  Deep Stereo: Learning to Predict New Views from the World's Imagery , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Manuela M. Veloso,et al.  Visual sonar: fast obstacle avoidance using monocular vision , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[44]  Stephanie Rosenthal,et al.  CoBots: Robust Symbiotic Autonomous Mobile Service Robots , 2015, IJCAI.

[45]  Jürgen Schmidhuber,et al.  World Models , 2018, ArXiv.

[46]  Charles Richter,et al.  Safe Visual Navigation via Deep Learning and Novelty Detection , 2017, Robotics: Science and Systems.

[47]  Noriaki Hirose,et al.  Personal robot assisting transportation to support active human life — Posture stabilization based on feedback compensation of lateral acceleration , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.