Road Detection through CRF based LiDAR-Camera Fusion

In this paper, we propose a road detection method with LiDAR-camera fusion in a novel conditional random field (CRF) framework to exploit both range and color information. In the LiDAR based part, a fast height-difference based scanning strategy is applied in the 2D LiDAR range-image domain and a dense road detection result in camera image domain can be obtained through geometric upsampling given the LiDAR-camera calibration parameters. In the camera based part, a fully convolutional network is applied in the camera image domain. Finally, we fuse the dense and binary road detection results from both LiDAR and camera in a single CRF framework. Experiments show that using a single thread of CPU, the proposed LiDAR based part can operate at a frequency of over 250Hz with sparse output in range image and 40Hz with dense result in camera image for the 64-beam Velodyne scanner. Our CRF fusion method achieves very promising road detection performance on the KITTI-Road dataset.

[1]  Hongdong Li,et al.  Semisupervised and Weakly Supervised Road Detection Based on Generative Adversarial Networks , 2018, IEEE Signal Processing Letters.

[2]  Jian Yang,et al.  Lidar-histogram for fast road and obstacle detection , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Zhe Chen,et al.  RBNet: A Deep Neural Network for Unified Road and Road Boundary Detection , 2017, ICONIP.

[4]  Fernando Santos Osório,et al.  Robust curb detection and vehicle localization in urban environments , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[5]  Jian Yang,et al.  Lidar-based urban road detection by histograms of normalized inverse depths and line scanning , 2017, 2017 European Conference on Mobile Robots (ECMR).

[6]  Renaud Dubé,et al.  SegMatch: Segment based loop-closure for 3D point clouds , 2016, ArXiv.

[7]  Jian Yang,et al.  3-D LiDAR + Monocular Camera: An Inverse-Depth-Induced Fusion Framework for Urban Road Detection , 2018, IEEE Transactions on Intelligent Vehicles.

[8]  Denis Fernando Wolf,et al.  Road terrain detection: Avoiding common obstacle detection assumptions using sensor fusion , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[9]  Ethan Fetaya,et al.  Real-Time Category-Based and General Obstacle Detection for Autonomous Driving , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[10]  Cristiano Premebida,et al.  High-resolution LIDAR-based depth mapping using bilateral filter , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[11]  Cyrill Stachniss,et al.  Fast range image-based segmentation of sparse 3D laser scans for online operation , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[12]  Liang Xiao,et al.  CRF based road detection with multi-sensor fusion , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[13]  Jannik Fritsch,et al.  A new performance measure and evaluation benchmark for road detection algorithms , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[14]  Huan Wang,et al.  Road detection based on the fusion of Lidar and image data , 2017 .

[15]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[16]  Steven Lake Waslander,et al.  In Defense of Classical Image Processing: Fast Depth Completion on the CPU , 2018, 2018 15th Conference on Computer and Robot Vision (CRV).

[17]  Long Chen,et al.  A novel way to organize 3D LiDAR point cloud as 2D depth map height map and surface normal map , 2015, 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[18]  D. T. Lee,et al.  Two algorithms for constructing a Delaunay triangulation , 1980, International Journal of Computer & Information Sciences.

[19]  Liang Xiao,et al.  Hybrid conditional random field based camera-LIDAR fusion for road detection , 2017, Inf. Sci..

[20]  P. Peixoto,et al.  Road Detection Using High Resolution LIDAR , 2014, 2014 IEEE Vehicle Power and Propulsion Conference (VPPC).

[21]  Masatoshi Okutomi,et al.  Depth map upsampling by self-guided residual interpolation , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[22]  Martin Lauer,et al.  Guided depth upsampling for precise mapping of urban environments , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[23]  Edwin Olson,et al.  Fast and robust 3D feature extraction from sparse point clouds , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[24]  Wende Zhang,et al.  LIDAR-based road and road-edge detection , 2010, 2010 IEEE Intelligent Vehicles Symposium.