Small obstacle detection using stereo vision for autonomous ground vehicle

Small and medium sized obstacles such as rocks, small boulders, bricks left unattended on the road can pose hazards for autonomous as well as human driving situations. Many times these objects are too small on the road and go unnoticed on depth and point cloud maps obtained from state of the art range sensors such as 3D LIDAR. We propose a novel algorithm that fuses both appearance and 3D cues such as image gradients, curvature potentials and depth variance into a Markov Random Field (MRF) formulation that segments the scene into obstacle and non obstacle regions. Appearance and depth data obtained from a ZED stereo pair mounted on a Husky robot is used for this purpose. While identifying true positive obstacles such as rocks, large stones accurately our algorithm is simultaneously robust to false positive sources such as appearance changes on the road, papers and road markings. High accuracy detection in challenging scenes such as when the foreground obstacle blends with the background road scene vindicates the efficacy of the proposed formulation.

[1]  Jian Sun,et al.  Convolutional feature masking for joint object and stuff segmentation , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Sergiu Nedevschi,et al.  Processing Dense Stereo Data Using Elevation Maps: Road Surface, Traffic Isle, and Obstacle Detection , 2010, IEEE Transactions on Vehicular Technology.

[3]  Baoxin Li,et al.  Robust Ground Plane Detection with Normalized Homography in Monocular Sequences from a Robot Platform , 2006, 2006 International Conference on Image Processing.

[4]  Alberto Broggi,et al.  A full-3D voxel-based dynamic obstacle detection for urban scenario using stereo vision , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[5]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[7]  Sergiu Nedevschi,et al.  Polynomial curb detection based on dense stereovision for driving assistance , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[8]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[10]  David W. Jacobs,et al.  Deep hierarchical parsing for semantic segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Uwe Franke,et al.  Towards a Global Optimal Multi-Layer Stixel Representation of Dense 3D Data , 2011, BMVC.

[12]  Sebastian Ramos,et al.  Detecting unexpected obstacles for self-driving cars: Fusing deep learning and geometric modeling , 2016, 2017 IEEE Intelligent Vehicles Symposium (IV).

[13]  K. Madhava Krishna,et al.  Markov Random Field based small obstacle discovery over images , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Sebastian Ramos,et al.  Lost and Found: detecting small road hazards for self-driving vehicles , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Wolfram Burgard,et al.  AdapNet: Adaptive semantic segmentation in adverse environmental conditions , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Julius Ziegler,et al.  Sparse scene flow segmentation for moving object detection in urban environments , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[19]  Alberto Elfes,et al.  Using occupancy grids for mobile robot perception and navigation , 1989, Computer.

[20]  C. Rabe,et al.  Fast detection of moving objects in complex scenarios , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[21]  Yassine Ruichek,et al.  Building variable resolution occupancy grid map from stereoscopic system — A quadtree based approach , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[22]  Markus H. Gross,et al.  Efficient simplification of point-sampled surfaces , 2002, IEEE Visualization, 2002. VIS 2002..

[23]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Fernando Santos Osório,et al.  Robust curb detection and vehicle localization in urban environments , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[25]  Urs A. Muller,et al.  Learning long-range vision for autonomous off-road driving , 2009 .

[26]  Luc Van Gool,et al.  Object Detection and Tracking for Autonomous Navigation in Dynamic Environments , 2010, Int. J. Robotics Res..

[27]  Darius Burschka,et al.  Efficient occupancy grid computation on the GPU with lidar and radar for road boundary detection , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[28]  David Fernández Llorca,et al.  Curvature-based curb detection method in urban environments using stereo and laser , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[29]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.