Integrating Dense LiDAR-Camera Road Detection Maps by a Multi-Modal CRF Model

Road detection is an important task in autonomous navigation systems. In this paper, we propose a road detection method via a LiDAR-camera fusion strategy to exploit both the range and color information. The whole system consists of three parts. In the LiDAR based part, we transform the discrete 3D LiDAR point clouds to continuous 2D LiDAR range images and propose a distance-aware height-difference based scanning approach to get the road estimations quickly. In the camera based part, we apply a light-weight transfer learning based road segmentation network. In the LiDAR-camera fusion part, we transform the detection results from LiDAR and camera to dense and binary ones to solve the data imbalance problem and fuse them in a multi-modal conditional random field (MM-CRF) framework. Experiments show that the proposed MM-CRF fusion method can operate in real-time and achieve competitive performance compared with the state-of-the-art road detection algorithms on the KITTI-Road benchmark.

[1]  Liang Xiao,et al.  CRF based road detection with multi-sensor fusion , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[2]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Roberto Cipolla,et al.  MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving , 2016, 2018 IEEE Intelligent Vehicles Symposium (IV).

[4]  Sertac Karaman,et al.  Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Xueying Qin,et al.  Deep Depth Super-Resolution: Learning Depth Super-Resolution Using Deep Convolutional Neural Network , 2016, ACCV.

[6]  Hui Kong,et al.  Road Detection through CRF based LiDAR-Camera Fusion , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[7]  Liang Xiao,et al.  Hybrid conditional random field based camera-LIDAR fusion for road detection , 2017, Inf. Sci..

[8]  Michael Felsberg,et al.  Propagating Confidences through CNNs for Sparse Data Regression , 2018, BMVC.

[9]  Jannik Fritsch,et al.  A new performance measure and evaluation benchmark for road detection algorithms , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[10]  Fernando Santos Osório,et al.  Robust curb detection and vehicle localization in urban environments , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[11]  Wende Zhang,et al.  LIDAR-based road and road-edge detection , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[12]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[13]  Sertac Karaman,et al.  Self-Supervised Sparse-to-Dense: Self-Supervised Depth Completion from LiDAR and Monocular Camera , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[14]  Thomas Brox,et al.  Sparsity Invariant CNNs , 2017, 2017 International Conference on 3D Vision (3DV).

[15]  Jian Yang,et al.  Lidar-based urban road detection by histograms of normalized inverse depths and line scanning , 2017, 2017 European Conference on Mobile Robots (ECMR).

[16]  P. Peixoto,et al.  Road Detection Using High Resolution LIDAR , 2014, 2014 IEEE Vehicle Power and Propulsion Conference (VPPC).

[17]  Wei Liu,et al.  ParseNet: Looking Wider to See Better , 2015, ArXiv.

[18]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Shichao Yang,et al.  Semantic 3D occupancy mapping through efficient high order CRFs , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[20]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Ethan Fetaya,et al.  Real-Time Category-Based and General Obstacle Detection for Autonomous Driving , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[22]  Jian Yang,et al.  Lidar-histogram for fast road and obstacle detection , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Zhe Chen,et al.  RBNet: A Deep Neural Network for Unified Road and Road Boundary Detection , 2017, ICONIP.

[24]  Masatoshi Okutomi,et al.  Depth map upsampling by self-guided residual interpolation , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[25]  Denis Fernando Wolf,et al.  Road terrain detection: Avoiding common obstacle detection assumptions using sensor fusion , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[26]  Hongdong Li,et al.  Semisupervised and Weakly Supervised Road Detection Based on Generative Adversarial Networks , 2018, IEEE Signal Processing Letters.

[27]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Cristiano Premebida,et al.  High-resolution LIDAR-based depth mapping using bilateral filter , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[29]  Jian Yang,et al.  3-D LiDAR + Monocular Camera: An Inverse-Depth-Induced Fusion Framework for Urban Road Detection , 2018, IEEE Transactions on Intelligent Vehicles.

[30]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[31]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[32]  Lennart Svensson,et al.  LIDAR-Camera Fusion for Road Detection Using Fully Convolutional Neural Networks , 2018, Robotics Auton. Syst..

[33]  Sven Behnke,et al.  Learning depth-sensitive conditional random fields for semantic segmentation of RGB-D images , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[34]  Steven Lake Waslander,et al.  In Defense of Classical Image Processing: Fast Depth Completion on the CPU , 2018, 2018 15th Conference on Computer and Robot Vision (CRV).

[35]  Xiaoou Tang,et al.  Depth Map Super-Resolution by Deep Multi-Scale Guidance , 2016, ECCV.

[36]  Long Chen,et al.  A novel way to organize 3D LiDAR point cloud as 2D depth map height map and surface normal map , 2015, 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[37]  D. T. Lee,et al.  Two algorithms for constructing a Delaunay triangulation , 1980, International Journal of Computer & Information Sciences.

[38]  Wolfgang Förstner,et al.  A temporal filter approach for detection and reconstruction of curbs and road surfaces based on Conditional Random Fields , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[39]  Christoph Stiller,et al.  Joint self-localization and tracking of generic objects in 3D range data , 2013, 2013 IEEE International Conference on Robotics and Automation.

[40]  Martin Lauer,et al.  Guided depth upsampling for precise mapping of urban environments , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).