OF-VO: Reliable Navigation among Pedestrians Using Commodity Sensors

We present a novel algorithm for safe navigation of a mobile robot among pedestrians. Our approach uses commodity visual sensors, including RGB-D cameras and a 2D lidar, for explicitly predicting the velocities and positions of surrounding obstacles through optical flow estimation and object detection. Given these partial observations of the environment, we present a modified velocity-obstacle (VO) algorithm to compute collision-free trajectories for the robot. A key aspect of our work is the coupling between the perception (OF: optical flow) and planning (VO) components for reliable navigation. Overall, our OF-VO algorithm is a hybrid combination of learning-based and model-based methods and offers better performance over prior algorithms in terms of navigation time and success rate of collision avoidance. We highlight the realtime performance of OF-VO in simulated and real-world dynamic scenes on a Turtlebot robot navigating among pedestrians with commodity sensors. A demo video is available at \url{this https URL}

[1]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[2]  Wolfram Burgard,et al.  The dynamic window approach to collision avoidance , 1997, IEEE Robotics Autom. Mag..

[3]  Paolo Fiorini,et al.  Motion Planning in Dynamic Environments Using Velocity Obstacles , 1998, Int. J. Robotics Res..

[4]  R. Levine,et al.  The Pace of Life in 31 Countries , 1999 .

[5]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[6]  Yoji Kuroda,et al.  Potential Field Navigation of High Speed Unmanned Ground Vehicles on Uneven Terrain , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[7]  Klaus Janschek,et al.  OPTICAL FLOW NAVIGATION FOR AN OUTDOOR UAV USING A WIDE ANGLE MONO CAMERA AND DEM MATCHING , 2006 .

[8]  Dinesh Manocha,et al.  Generalized velocity obstacles , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Dinesh Manocha,et al.  Reciprocal n-Body Collision Avoidance , 2011, ISRR.

[10]  Roland Siegwart,et al.  MAV navigation through indoor corridors using optical flow , 2010, 2010 IEEE International Conference on Robotics and Automation.

[11]  Dinesh Manocha,et al.  Smooth and collision-free navigation for multiple robots under differential-drive constraints , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Dinesh Manocha,et al.  Reciprocal collision avoidance with acceleration-velocity obstacles , 2011, 2011 IEEE International Conference on Robotics and Automation.

[13]  Dinesh Manocha,et al.  LQG-obstacles: Feedback control with collision avoidance for mobile robots with motion and sensing uncertainty , 2012, 2012 IEEE International Conference on Robotics and Automation.

[14]  Karl Tuyls,et al.  Multi-robot collision avoidance with localization uncertainty , 2012, AAMAS.

[15]  Joel W. Burdick,et al.  Robot Motion Planning in Dynamic, Uncertain Environments , 2012, IEEE Transactions on Robotics.

[16]  Jur P. van den Berg,et al.  Reciprocal collision avoidance for robots with linear dynamics using LQR-Obstacles , 2013, 2013 IEEE International Conference on Robotics and Automation.

[17]  Michal Kelemen,et al.  VFH\astTDT (VFH\ast with Time Dependent Tree): A new laser rangefinder based obstacle avoidance method designed for environment with non-static obstacles , 2014, Robotics Auton. Syst..

[18]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[20]  Dinesh Manocha,et al.  Parameter estimation and comparative evaluation of crowd simulations , 2014, Comput. Graph. Forum.

[21]  Marcello R. Napolitano,et al.  A Survey of Optical Flow Techniques for Robotics Navigation Applications , 2014, J. Intell. Robotic Syst..

[22]  Konrad Schindler,et al.  Continuous Energy Minimization for Multitarget Tracking , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  James McLurkin,et al.  The Extended Velocity Obstacle and applying ORCA in the real world , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  David Hsu,et al.  Intention-aware online POMDP planning for autonomous driving in a crowd , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Thomas Brox,et al.  FlowNet: Learning Optical Flow with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[27]  Thomas Brox,et al.  A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[29]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Alberto Elfes,et al.  Real-time autonomous ground vehicle navigation in heterogeneous environments using a 3D LiDAR , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[31]  Hong Xu,et al.  Overview: Generalizations of Multi-Agent Path Finding to Real-World Scenarios , 2017, ArXiv.

[32]  Wolfram Burgard,et al.  Deep reinforcement learning with successor features for navigation across similar environments , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[33]  Rahul Narain,et al.  Uncertainty Models for TTC-Based Collision-Avoidance , 2017, Robotics: Science and Systems.

[34]  Chonhyon Park,et al.  Efficient probabilistic collision detection for non-convex shapes , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Ali Farhadi,et al.  Target-driven visual navigation in indoor scenes using deep reinforcement learning , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Roland Siegwart,et al.  From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[38]  Dinesh Manocha,et al.  PRVO: Probabilistic Reciprocal Velocity Obstacle for multi robot navigation under uncertainty , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[39]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[40]  Sen Wang,et al.  Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning , 2017, RSS 2017.

[41]  Thomas Brox,et al.  FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Bodo Rosenhahn,et al.  Fusion of Head and Full-Body Detectors for Multi-object Tracking , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[43]  Jia Pan,et al.  Fully Distributed Multi-Robot Collision Avoidance via Deep Reinforcement Learning for Safe and Efficient Navigation in Complex Scenarios , 2018, ArXiv.

[44]  Pan Wei,et al.  LiDAR and Camera Detection Fusion in a Real Time Industrial Multi-Sensor Collision Avoidance System , 2018, Electronics.

[45]  Deok-Jin Lee,et al.  An End-to-End Deep Reinforcement Learning-Based Intelligent Agent Capable of Autonomous Exploration in Unknown Environments , 2018, Sensors.

[46]  Armando Solar-Lezama,et al.  Verifiable Reinforcement Learning via Policy Extraction , 2018, NeurIPS.

[47]  Shuzhi Sam Ge,et al.  Role Playing Learning for Socially Concomitant Mobile Robot Navigation , 2017, CAAI Trans. Intell. Technol..

[48]  Wolfram Burgard,et al.  Socially Compliant Navigation Through Raw Depth Inputs with Generative Adversarial Imitation Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[49]  Jonathan P. How,et al.  Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[50]  Mac Schwager,et al.  Fast Reciprocal Collision Avoidance Under Measurement Uncertainty , 2019, ISRR.

[51]  Teng-Hu Cheng,et al.  Estimation and Tracking of a Moving Target by Unmanned Aerial Vehicles , 2019, 2019 American Control Conference (ACC).

[52]  Dinesh Manocha,et al.  Learning Resilient Behaviors for Navigation Under Uncertainty Environments , 2019, ArXiv.

[53]  Soon Kwon,et al.  LiDAR and Camera Fusion Approach for Object Distance Estimation in Self-Driving Vehicles , 2020, Symmetry.

[54]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[55]  Damien Vivet,et al.  A Review of Visual-LiDAR Fusion based Simultaneous Localization and Mapping , 2020, Sensors.

[56]  Dinesh Manocha,et al.  DenseCAvoid: Real-time Navigation in Dense Crowds using Anticipatory Behaviors , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[57]  Dinesh Manocha,et al.  Efficient Probabilistic Collision Detection for Non-Gaussian Noise Distributions , 2019, IEEE Robotics and Automation Letters.

[58]  Jan Kautz,et al.  Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.