Curb Detection for Road and Sidewalk Detection

Curb detection, a significant area of autonomous driving, plays an important role in road detection and obstacle avoidance, etc. However, curb detection is challenging due to the problems like occlusions, shadows and the small size of the target. In this paper, we propose a curb detection paradigm for road and sidewalk detection for mobile robots using stereo vision in the urban residential region. First, the flat area is estimated based on the disparity and v-disparity maps generated from stereo matching. In the estimated flat area, to distinguish curbs from the road and obstacles, we propose an efficient 16-dimensional descriptor based on the appearance, geometry, and disparity characteristics of curbs. The curb points can be extracted by an SVM classifier with the obtained descriptors. Second, the curb points are exploited by a vanishing point constrained Dijkstra road model to find the road region, where two shortest paths, generated from the vanishing point to the last row of the cost map, are searched and regarded as the road borders. Following the curb and road region detection results, the sidewalk area is detected by a region-growing-like method based on the geometry characteristic of the sidewalk area and the layout of road scenes. The proposed method has been tested for road and sidewalk detection tasks on the KITTI dataset. The experimental results of the curb, road, and sidewalk detection demonstrate the accuracy and robustness of this method.

[1]  Wolfgang Förstner,et al.  Curb reconstruction using Conditional Random Fields , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[2]  Sergiu Nedevschi,et al.  Curb detection for driving assistance systems: A cubic spline-based approach , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[3]  Massimo Bertozzi,et al.  Vision-based intelligent vehicles: State of the art and perspectives , 2000, Robotics Auton. Syst..

[4]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Li Bai,et al.  Robust Road Modeling and Tracking Using Condensation , 2008, IEEE Transactions on Intelligent Transportation Systems.

[6]  T. Kanade,et al.  Fast and accurate computation of surface normals from range images , 2011, 2011 IEEE International Conference on Robotics and Automation.

[7]  Markus Enzweiler,et al.  Towards multi-cue urban curb recognition , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[8]  Bin Dai,et al.  Velodyne-based curb detection up to 50 meters away , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[9]  Raúl Rojas,et al.  Camera based detection and classification of soft shoulders, curbs and guardrails , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[10]  David Fernández Llorca,et al.  Road curb and lanes detection for autonomous driving on urban scenarios , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[11]  Uwe Franke,et al.  Low-level fusion of color, texture and depth for robust road scene understanding , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[12]  Sheng Xu,et al.  Road Curb Extraction From Mobile LiDAR Point Clouds , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Christoph H. Gierull,et al.  A Compound-Plus-Noise Model for Improved Vessel Detection in Non-Gaussian SAR Imagery , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Jun Tan,et al.  Robust Curb Detection with Fusion of 3D-Lidar and Camera Data , 2014, Sensors.

[15]  Tao Wu,et al.  Roadside curb detection based on fusing stereo vision and mono vision , 2012, Other Conferences.

[16]  Fernando Santos Osório,et al.  Robust curb detection and vehicle localization in urban environments , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[17]  Markus Flierl,et al.  Bayesian estimation of Dirichlet mixture model with variational inference , 2014, Pattern Recognit..

[18]  Jian Li,et al.  Robust Unstructured Road Detection: The Importance of Contextual Information , 2013 .

[19]  David Fernández Llorca,et al.  Curvature-based curb detection method in urban environments using stereo and laser , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[20]  Celestino Ordóñez,et al.  An approach to detect and delineate street curbs from MLS 3D point cloud data , 2015 .

[21]  Ulrich Hofmann,et al.  Multi-cue, Model-Based Detection and Mapping of Road Curb Features Using Stereo Vision , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[22]  Hui Kong,et al.  Generalizing Laplacian of Gaussian Filters for Vanishing-Point Detection , 2013, IEEE Transactions on Intelligent Transportation Systems.

[23]  Wolfgang Förstner,et al.  A temporal filter approach for detection and reconstruction of curbs and road surfaces based on Conditional Random Fields , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[24]  K. Madhava Krishna,et al.  CRF based method for curb detection using semantic cues and stereo depth , 2016, ICVGIP '16.

[25]  Ulrich Hofmann,et al.  Road curb detection based on different elevation mapping techniques , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[26]  Andreas Geiger,et al.  Efficient Large-Scale Stereo Matching , 2010, ACCV.

[27]  Joon Woong Lee,et al.  A lane-departure identification based on LBPE, Hough transform, and linear regression , 2005, Comput. Vis. Image Underst..

[28]  William A. Barrett,et al.  Interactive Segmentation with Intelligent Scissors , 1998, Graph. Model. Image Process..

[29]  Xinbo Gao,et al.  Image super-resolution using non-local Gaussian process regression , 2016, Neurocomputing.

[30]  Xiaonian Wang,et al.  A real-time curb detection and tracking method for UGVs by using a 3D-LIDAR sensor , 2015, 2015 IEEE Conference on Control Applications (CCA).

[31]  Markus Flierl,et al.  Probabilistic Multiview Depth Image Enhancement Using Variational Inference , 2015, IEEE Journal of Selected Topics in Signal Processing.

[32]  Cláudio Rosito Jung,et al.  Lane following and lane departure using a linear-parabolic model , 2005, Image Vis. Comput..

[33]  Jean-Philippe Tarel,et al.  Real time obstacle detection in stereovision on non flat road geometry through "v-disparity" representation , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[34]  Kangru Wang,et al.  Non-flat Road Detection Based on A Local Descriptor , 2016, ArXiv.

[35]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling , 2015, CVPR 2015.

[36]  Martin Kellner,et al.  Laserscanner based road curb feature detection and efficient mapping using local curb descriptions , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[37]  Véronique Prinet,et al.  3D road curb extraction from image sequence for automobile parking assist system , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[38]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[39]  Mohan M. Trivedi,et al.  Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation , 2006, IEEE Transactions on Intelligent Transportation Systems.