Progressive LiDAR adaptation for road detection

Despite rapid developments in visual image-based road detection, robustly identifying road areas in visual images remains challenging due to issues like illumination changes and blurry images. To this end, LiDAR sensor data can be incorporated to improve the visual image-based road detection, because LiDAR data is less susceptible to visual noises. However, the main difficulty in introducing LiDAR information into visual image-based road detection is that LiDAR data and its extracted features do not share the same space with the visual data and visual features. Such gaps in spaces may limit the benefits of LiDAR information for road detection. To overcome this issue, we introduce a novel Progressive LiDAR adaptation-aided road detection ( PLARD ) approach to adapt LiDAR information into visual image-based road detection and improve detection performance. In PLARD, progressive LiDAR adaptation consists of two subsequent modules: 1 ) data space adaptation, which transforms the LiDAR data to the visual data space to align with the perspective view by applying altitude difference-based transformation; and 2 ) feature space adaptation, which adapts LiDAR features to visual features through a cascaded fusion structure. Comprehensive empirical studies on the well-known KITTI road detection benchmark demonstrate that PLARD takes advantage of both the visual and LiDAR information, achieving much more robust road detection even in challenging urban scenes. In particular, PLARD outperforms other state-of-the-art road detection models and is currently top of the publicly accessible benchmark leader-board.

[1]  Vincent Frémont,et al.  Vision-Based Road Detection using Contextual Blocks , 2015, ArXiv.

[2]  Guangming Xiong,et al.  Road detection using support vector machine based on online learning and evaluation , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[3]  Thomas A. Funkhouser,et al.  Dilated Residual Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Nick Barnes,et al.  Learning appearance models for road detection , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[5]  Peter Kontschieder,et al.  The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Roberto Cipolla,et al.  MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving , 2016, 2018 IEEE Intelligent Vehicles Symposium (IV).

[7]  Long Chen,et al.  Advances in Vision-Based Lane Detection: Algorithms, Integration, Assessment, and Perspectives on ACP-Based Parallel Vision , 2018, IEEE/CAA Journal of Automatica Sinica.

[8]  Xinge You,et al.  Dynamically Modulated Mask Sparse Tracking , 2017, IEEE Transactions on Cybernetics.

[9]  Franz Kummert,et al.  Spatial ray features for real-time ego-lane extraction , 2012, 2012 15th International IEEE Conference on Intelligent Transportation Systems.

[10]  Rahul Mohan,et al.  Deep Deconvolutional Networks for Scene Parsing , 2014, ArXiv.

[11]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[12]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[13]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Zhe Chen,et al.  Context Refinement for Object Detection , 2018, ECCV.

[15]  Hongdong Li,et al.  Semisupervised and Weakly Supervised Road Detection Based on Generative Adversarial Networks , 2018, IEEE Signal Processing Letters.

[16]  Vincent Frémont,et al.  Exploiting fully convolutional neural networks for fast road detection , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Liang Xiao,et al.  Hybrid conditional random field based camera-LIDAR fusion for road detection , 2017, Inf. Sci..

[19]  Dawei Zhao,et al.  Monocular Road Detection Using Structured Random Forest , 2016 .

[20]  Zhe Chen,et al.  Generic Pixel Level Object Tracker Using Bi-Channel Fully Convolutional Network , 2017, ICONIP.

[21]  Jannik Fritsch,et al.  A new performance measure and evaluation benchmark for road detection algorithms , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[22]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[23]  Lennart Svensson,et al.  Fast LIDAR-based road detection using fully convolutional neural networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[24]  Ian D. Reid,et al.  RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Ankit Laddha,et al.  Map-supervised road detection , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[26]  Jian Yang,et al.  Lidar-histogram for fast road and obstacle detection , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Zhe Chen,et al.  RBNet: A Deep Neural Network for Unified Road and Road Boundary Detection , 2017, ICONIP.

[28]  Ignacio Parra,et al.  Deep fully convolutional networks with random data augmentation for enhanced generalization in road detection , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[29]  Liang Qi,et al.  A dynamic road incident information delivery strategy to reduce urban traffic congestion , 2018, IEEE/CAA Journal of Automatica Sinica.

[30]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[31]  Liang Xiao,et al.  CRF based road detection with multi-sensor fusion , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[32]  Mohamed Aly,et al.  Real time detection of lane markers in urban streets , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[33]  Martial Hebert,et al.  Stacked Hierarchical Labeling , 2010, ECCV.

[34]  Jean Ponce,et al.  Vanishing point detection for road detection , 2009, CVPR.

[35]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Ethan Fetaya,et al.  StixelNet: A Deep Convolutional Network for Obstacle Detection and Road Segmentation , 2015, BMVC.

[37]  Hong Wang,et al.  Parallel planning: a new motion planning framework for autonomous driving , 2019, IEEE/CAA Journal of Automatica Sinica.

[38]  Garrison W. Cottrell,et al.  Understanding Convolution for Semantic Segmentation , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[39]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Ethan Fetaya,et al.  Real-Time Category-Based and General Obstacle Detection for Autonomous Driving , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[41]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Lennart Svensson,et al.  LIDAR-Camera Fusion for Road Detection Using Fully Convolutional Neural Networks , 2018, Robotics Auton. Syst..

[43]  Denis Fernando Wolf,et al.  Road terrain detection: Avoiding common obstacle detection assumptions using sensor fusion , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.