Reverse and Boundary Attention Network for Road Segmentation

Road segmentation is an essential task to perceive the driving environment in autonomous driving and advanced driver assistance systems. With the development of deep learning, road segmentation has achieved great progress in recent years. However, there still remain some problems including the inaccurate road boundary and the illumination variations such as shadows and over-exposure regions. To solve these problems, we propose a residual learning-based network architecture with residual refinement module composed of the reverse attention and boundary attention units for road segmentation. The network first predicts a coarse road region from deeper-level feature maps and gradually refines the prediction by learning the residual in a top-down approach. The reverse and boundary attention units in residual refinement module guide the network to focus on the features in the previously missing region and the region near the road boundary. In addition, we introduce the boundary-aware weighted loss to reduce the false prediction. Experimental results demonstrate that the proposed approach outperforms the state-of-the-art methods in terms of the segmentation accuracy in various benchmark datasets for traffic scene understanding.

[1]  Vincent Frémont,et al.  Vision-Based Road Detection using Contextual Blocks , 2015, ArXiv.

[2]  Roberto Cipolla,et al.  MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving , 2016, 2018 IEEE Intelligent Vehicles Symposium (IV).

[3]  Luciano da Fontoura Costa,et al.  2D Euclidean distance transform algorithms: A comparative survey , 2008, CSUR.

[4]  Philip H. S. Torr,et al.  Combining Appearance and Structure from Motion Features for Road Scene Understanding , 2009, BMVC.

[5]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[6]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[7]  Xiang Bai,et al.  DeepSkeleton: Learning Multi-Task Scale-Associated Deep Side Outputs for Object Skeleton Extraction in Natural Images , 2016, IEEE Transactions on Image Processing.

[8]  L. Davis,et al.  Real-time multiple vehicle detection and tracking from a moving vehicle , 2000, Machine Vision and Applications.

[9]  Josef Pauli,et al.  Superpixel-based Road Segmentation for Real-time Systems using CNN , 2018, VISIGRAPP.

[10]  Zhe Chen,et al.  RBNet: A Deep Neural Network for Unified Road and Road Boundary Detection , 2017, ICONIP.

[11]  Ignacio Parra,et al.  Deep fully convolutional networks with random data augmentation for enhanced generalization in road detection , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[12]  Stefano Messelodi,et al.  Switching Models for Vision-based On-Board Road Detection , 2005 .

[13]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[14]  Bingbing Ni,et al.  Geometric Constrained Joint Lane Segmentation and Lane Boundary Detection , 2018, ECCV.

[15]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Antonio M. López,et al.  Road Detection Based on Illuminant Invariance , 2011, IEEE Transactions on Intelligent Transportation Systems.

[17]  Yang Wang,et al.  Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation , 2016, ISVC.

[18]  Ethan Fetaya,et al.  Real-Time Category-Based and General Obstacle Detection for Autonomous Driving , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[19]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[20]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Wolfram Burgard,et al.  Efficient deep models for monocular road segmentation , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  Hongdong Li,et al.  Semisupervised and Weakly Supervised Road Detection Based on Generative Adversarial Networks , 2018, IEEE Signal Processing Letters.

[23]  Simon Malinowski,et al.  Combining convolutional side-outputs for road image segmentation , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[24]  Vincent Frémont,et al.  Exploiting fully convolutional neural networks for fast road detection , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Xiangyu Zhang,et al.  Large Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  John Zelek,et al.  Road Segmentation in Street View Images Using Texture Information , 2016, 2016 13th Conference on Computer and Robot Vision (CRV).

[27]  Marcelo H. Ang,et al.  Perception, Planning, Control, and Coordination for Autonomous Vehicles , 2017 .

[28]  Xinming Huang,et al.  RoadNet-v2: A 10 ms Road Segmentation Using Spatial Sequence Layer , 2018, ArXiv.

[29]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[30]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[31]  Junyu Gao,et al.  Embedding structured contour and location prior in siamesed fully convolutional networks for road detection , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[32]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[33]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Rahul Mohan,et al.  Deep Deconvolutional Networks for Scene Parsing , 2014, ArXiv.

[35]  Roberto Cipolla,et al.  Semantic object classes in video: A high-definition ground truth database , 2009, Pattern Recognit. Lett..

[36]  Nick Barnes,et al.  Learning appearance models for road detection , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[37]  Ben Wang,et al.  Reverse Attention for Salient Object Detection , 2018, ECCV.

[38]  Dariu Gavrila,et al.  Monocular Pedestrian Detection: Survey and Experiments , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[40]  Guoying Zhao,et al.  SRN: Side-Output Residual Network for Object Symmetry Detection in the Wild , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[42]  Sebastian Thrun,et al.  Towards fully autonomous driving: Systems and algorithms , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[43]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[45]  Subhashis Banerjee,et al.  Deep CNN with color lines model for unmarked road segmentation , 2017, 2017 IEEE International Conference on Image Processing (ICIP).