Implementation of a Lightweight Semantic Segmentation Algorithm in Road Obstacle Detection

Due to deep learning’s accurate cognition of the street environment, the convolutional neural network has achieved dramatic development in the application of street scenes. Considering the needs of autonomous driving and assisted driving, in a general way, computer vision technology is used to find obstacles to avoid collisions, which has made semantic segmentation a research priority in recent years. However, semantic segmentation has been constantly facing new challenges for quite a long time. Complex network depth information, large datasets, real-time requirements, etc., are typical problems that need to be solved urgently in the realization of autonomous driving technology. In order to address these problems, we propose an improved lightweight real-time semantic segmentation network, which is based on an efficient image cascading network (ICNet) architecture, using multi-scale branches and a cascaded feature fusion unit to extract rich multi-level features. In this paper, a spatial information network is designed to transmit more prior knowledge of spatial location and edge information. During the course of the training phase, we append an external loss function to enhance the learning process of the deep learning network system as well. This lightweight network can quickly perceive obstacles and detect roads in the drivable area from images to satisfy autonomous driving characteristics. The proposed model shows substantial performance on the Cityscapes dataset. With the premise of ensuring real-time performance, several sets of experimental comparisons illustrate that SP-ICNet enhances the accuracy of road obstacle detection and provides nearly ideal prediction outputs. Compared to the current popular semantic segmentation network, this study also demonstrates the effectiveness of our lightweight network for road obstacle detection in autonomous driving.

[1]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Peter Reinartz,et al.  Aerial LaneNet: Lane-Marking Semantic Segmentation in Aerial Imagery Using Wavelet-Enhanced Cost-Sensitive Symmetric Fully Convolutional Neural Networks , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Eugenio Culurciello,et al.  ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation , 2016, ArXiv.

[4]  Yanfen Gan,et al.  Efficient Outdoor Video Semantic Segmentation Using Feedback-Based Fully Convolution Neural Network , 2020, IEEE Transactions on Industrial Informatics.

[5]  Chang Wang,et al.  A Framework for Turning Behavior Classification at Intersections Using 3D LIDAR , 2019, IEEE Transactions on Vehicular Technology.

[6]  Jitendra Malik,et al.  Simultaneous Detection and Segmentation , 2014, ECCV.

[7]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[8]  Zhe Chen,et al.  Progressive LiDAR adaptation for road detection , 2019, IEEE/CAA Journal of Automatica Sinica.

[9]  Xiaojuan Qi,et al.  ICNet for Real-Time Semantic Segmentation on High-Resolution Images , 2017, ECCV.

[10]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Yuqi Tang,et al.  Spatial Information Inference Net: Road Extraction Using Road-Specific Contextual Information , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[13]  Syed Ali Tariq,et al.  Multi-Feature View-Based Shallow Convolutional Neural Network for Road Segmentation , 2020, IEEE Access.

[14]  Joaquín Martínez-Sánchez,et al.  Road Environment Semantic Segmentation with Deep Learning from MLS Point Cloud Data , 2019, Sensors.

[15]  Jaehoon Jung,et al.  Object Recognition, Segmentation, and Classification of Mobile Laser Scanning Point Clouds: A State of the Art Review , 2019, Sensors.

[16]  Xiexing Feng,et al.  UnetDVH-Linear: Linear Feature Segmentation by Dilated Convolution with Vertical and Horizontal Kernels , 2020, Sensors.

[17]  Eugenio Culurciello,et al.  LinkNet: Exploiting encoder representations for efficient semantic segmentation , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[18]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Huchuan Lu,et al.  Attentive Feedback Network for Boundary-Aware Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Lei Sun,et al.  Real-Time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection for Road-Driving Images , 2020, IEEE Robotics and Automation Letters.

[23]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[24]  Philip H. S. Torr,et al.  Higher Order Conditional Random Fields in Deep Neural Networks , 2015, ECCV.

[25]  Gregory Shakhnarovich,et al.  Feedforward semantic segmentation with zoom-out features , 2014, CVPR.

[26]  Yu Wang,et al.  Lednet: A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[27]  Yang Li,et al.  Implementation of deep-learning algorithm for obstacle detection and collision avoidance for robotic harvester , 2020, Comput. Electron. Agric..

[28]  Steven C. H. Hoi,et al.  Salient Object Detection With Pyramid Attention and Salient Edges , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Wei Liu,et al.  RAPNet: Residual Atrous Pyramid Network for Importance-Aware Street Scene Parsing , 2020, IEEE Transactions on Image Processing.

[30]  Yuan Feng,et al.  Feasibility Analysis and Application of Reinforcement Learning Algorithm Based on Dynamic Parameter Adjustment , 2020, Algorithms.

[31]  Jan Kautz,et al.  Superpixel Sampling Networks , 2018, ECCV.

[32]  Xiang Bai,et al.  Richer Convolutional Features for Edge Detection , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Kaiming He,et al.  PointRend: Image Segmentation As Rendering , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Jian Yang,et al.  Lidar-histogram for fast road and obstacle detection , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Zhe Chen,et al.  RBNet: A Deep Neural Network for Unified Road and Road Boundary Detection , 2017, ICONIP.

[36]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[37]  Renato Fontes Guimarães,et al.  Deep Semantic Segmentation of Center Pivot Irrigation Systems from Remotely Sensed Data , 2020, Remote. Sens..