Gaussian process regression-based robust free space detection for autonomous vehicle by 3-D point cloud and 2-D appearance information fusion

Free space detection is crucial to autonomous vehicles while existing works are not entirely satisfactory. As cameras have many advantages on environment perception, a stereo vision-based robust free space detection method is proposed which mainly depends on geometry information and Gaussian process regression. In this work, in order to improve the performance by exploiting multiple source information, we apply Bayesian framework and conditional random field inference to fuse the multimodal information including 2-D image and 3-D point geometric information. Particularly, a Bayesian framework is used for multiple feature fusion to provide a normalized and flexible output. Gaussian process regression is used to automatically and incrementally regress the data, resulting enhanced performance. Finally, conditional random field with color and geometry constrains is applied to make the result more robust. In order to evaluate the proposed method, quantitative experiments on popular KITTI-road data set and qualitative experiments on our own campus data set are tested. The results show satisfactory and inspiring performance compared to the outstanding works and even are competitive to some relevant Lidar-based methods.

[1]  Di Guo,et al.  Structured Output-Associated Dictionary Learning for Haptic Understanding , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[2]  Christian Laugier,et al.  An efficient formulation of the Bayesian occupation filter for target tracking in dynamic environments , 2008 .

[3]  Denis Fernando Wolf,et al.  Road estimation with sparse 3D points from stereo data , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[4]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[5]  Mark J. Schervish,et al.  Nonstationary Covariance Functions for Gaussian Process Regression , 2003, NIPS.

[6]  Vincent Frémont,et al.  Exploiting fully convolutional neural networks for fast road detection , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[8]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Dawei Zhao,et al.  Monocular Road Detection Using Structured Random Forest , 2016 .

[10]  Vincent Frémont,et al.  Color-based road detection and its evaluation on the KITTI road benchmark , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[11]  Di Guo,et al.  Extreme Kernel Sparse Learning for Tactile Object Recognition , 2017, IEEE Transactions on Cybernetics.

[12]  Uwe Franke,et al.  The Stixel World - A Compact Medium Level Representation of the 3D-World , 2009, DAGM-Symposium.

[13]  Jean-Philippe Tarel,et al.  Real time obstacle detection in stereovision on non flat road geometry through "v-disparity" representation , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[14]  Di Guo,et al.  Object Recognition Using Tactile Measurements: Kernel Sparse Coding Methods , 2016, IEEE Transactions on Instrumentation and Measurement.

[15]  Paulo Santos,et al.  Cross-country obstacle detection: Space-variant resolution and outliers removal , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Sergiu Nedevschi,et al.  Polynomial curb detection based on dense stereovision for driving assistance , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[17]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[18]  Stefan K. Gehrig,et al.  Exploiting the Power of Stereo Confidences , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Denis Fernando Wolf,et al.  Road terrain detection: Avoiding common obstacle detection assumptions using sensor fusion , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[20]  Fuchun Sun,et al.  Robotic Room-Level Localization Using Multiple Sets of Sonar Measurements , 2017, IEEE Transactions on Instrumentation and Measurement.

[21]  Sergiu Nedevschi,et al.  Processing Dense Stereo Data Using Elevation Maps: Road Surface, Traffic Isle, and Obstacle Detection , 2010, IEEE Transactions on Vehicular Technology.

[22]  Guosheng Lin,et al.  Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Jannik Fritsch,et al.  A new performance measure and evaluation benchmark for road detection algorithms , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[24]  Roberto Manduchi,et al.  Obstacle Detection and Terrain Classification for Autonomous Off-Road Navigation , 2005, Auton. Robots.

[25]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[26]  Wolfgang Förstner,et al.  Curb reconstruction using Conditional Random Fields , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[27]  Christian Laugier,et al.  Bayesian Occupancy Filtering for Multitarget Tracking: An Automotive Application , 2006, Int. J. Robotics Res..

[28]  Liang Xiao,et al.  Hybrid conditional random field based camera-LIDAR fusion for road detection , 2017, Inf. Sci..

[29]  Bin Dai,et al.  Velodyne-based curb detection up to 50 meters away , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[30]  Bin Dai,et al.  Gaussian-Process-Based Real-Time Ground Segmentation for Autonomous Land Vehicles , 2013, Journal of Intelligent & Robotic Systems.

[31]  Daniel Cremers,et al.  B-Spline Modeling of Road Surfaces With an Application to Free-Space Estimation , 2009, IEEE Transactions on Intelligent Transportation Systems.

[32]  Franz Kummert,et al.  Spatial ray features for real-time ego-lane extraction , 2012, 2012 15th International IEEE Conference on Intelligent Transportation Systems.

[33]  Christian Jutten,et al.  Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects , 2015, Proceedings of the IEEE.

[34]  Ulrich Hofmann,et al.  Multi-cue, Model-Based Detection and Mapping of Road Curb Features Using Stereo Vision , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[35]  Wolfgang Förstner,et al.  A temporal filter approach for detection and reconstruction of curbs and road surfaces based on Conditional Random Fields , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[36]  S. Nedevschi,et al.  Curb Detection Based on Elevation Maps from Dense Stereo , 2007, 2007 IEEE International Conference on Intelligent Computer Communication and Processing.

[37]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[38]  Paolo Grisleri,et al.  Off-Road Path and Obstacle Detection Using Decision Networks and Stereo Vision , 2007, IEEE Transactions on Intelligent Transportation Systems.

[39]  Rudolf Mester,et al.  Free Space Computation Using Stochastic Occupancy Grids and Dynamic Programming , 2008 .

[40]  L. Piegl,et al.  The NURBS Book , 1995, Monographs in Visual Communications.

[41]  Bertrand Douillard,et al.  On the segmentation of 3D LIDAR point clouds , 2011, 2011 IEEE International Conference on Robotics and Automation.

[42]  Alessandro Corrêa Victorino,et al.  Comprehensive performance analysis of road detection algorithms using the common urban Kitti-road benchmark , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[43]  Andrew McCallum,et al.  An Introduction to Conditional Random Fields , 2010, Found. Trends Mach. Learn..

[44]  Liang Xiao,et al.  CRF based road detection with multi-sensor fusion , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[45]  Raquel Urtasun,et al.  Efficient Joint Segmentation, Occlusion Labeling, Stereo and Flow Estimation , 2014, ECCV.

[46]  Fuchun Sun,et al.  Visual–Tactile Fusion for Object Recognition , 2017, IEEE Transactions on Automation Science and Engineering.

[47]  Sergiu Nedevschi,et al.  Road Surface and Obstacle Detection Based on Elevation Maps from Dense Stereo , 2007, 2007 IEEE Intelligent Transportation Systems Conference.