SpinNet: Spinning convolutional network for lane boundary detection

In this paper, we propose a simple but effective framework for lane boundary detection, called SpinNet. Considering that cars or pedestrians often occlude lane boundaries and that the local features of lane boundaries are not distinctive, therefore, analyzing and collecting global context information is crucial for lane boundary detection. To this end, we design a novel spinning convolution layer and a brand-new lane parameterization branch in our network to detect lane boundaries from a global perspective. To extract features in narrow strip-shaped fields, we adopt strip-shaped convolutions with kernels which have 1 × n or n × 1 shape in the spinning convolution layer. To tackle the problem of that straight strip-shaped convolutions are only able to extract features in vertical or horizontal directions, we introduce the concept of feature map rotation to allow the convolutions to be applied in multiple directions so that more information can be collected concerning a whole lane boundary. Moreover, unlike most existing lane boundary detectors, which extract lane boundaries from segmentation masks, our lane boundary parameterization branch predicts a curve expression for the lane boundary for each pixel in the output feature map. And the network utilizes this information to predict the weights of the curve, to better form the final lane boundaries. Our framework is easy to implement and end-to-end trainable. Experiments show that our proposed SpinNet outperforms state-of-the-art methods.

[1]  Florentin Wörgötter,et al.  Combining Statistical Hough Transform and Particle Filter for robust lane detection and tracking , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[2]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[3]  Sheng-Fuu Lin,et al.  Lane detection using color-based segmentation , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[4]  Jiman Kim,et al.  End-To-End Ego Lane Estimation Based on Sequential Transfer Learning for Self-Driving Cars , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Luc Van Gool,et al.  Towards End-to-End Lane Detection: an Instance Segmentation Approach , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[6]  Mohamed Aly,et al.  Real time detection of lane markers in urban streets , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[7]  Sanghoon Sull,et al.  Efficient Lane Detection Based on Spatiotemporal Images , 2016, IEEE Transactions on Intelligent Transportation Systems.

[8]  Xiaogang Wang,et al.  Spatial As Deep: Spatial CNN for Traffic Scene Understanding , 2017, AAAI.

[9]  Bingbing Ni,et al.  Geometric Constrained Joint Lane Segmentation and Lane Boundary Detection , 2018, ECCV.

[10]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[11]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[12]  Dan Levi,et al.  3D-LaneNet: End-to-End 3D Multiple Lane Detection , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[13]  Shi-Min Hu,et al.  LineNet: a Zoomable CNN for Crowdsourced High Definition Maps Modeling in Urban Environments , 2018, ArXiv.

[14]  Fei Su,et al.  Fingerprint Matching With Rotation-Descriptor Texture Features , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[15]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Sander Dieleman,et al.  Rotation-invariant convolutional neural networks for galaxy morphology prediction , 2015, ArXiv.

[17]  Zachary Chase Lipton A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[18]  Kwanghoon Sohn,et al.  Real-time illumination invariant lane detection for lane departure warning system , 2015, Expert Syst. Appl..

[19]  Monson H. Hayes,et al.  A Novel Lane Detection System With Efficient Ground Truth Generation , 2012, IEEE Transactions on Intelligent Transportation Systems.

[20]  Stephan J. Garbin,et al.  Harmonic Networks: Deep Translation and Rotation Equivariance , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Fernando A. Mujica,et al.  An Empirical Evaluation of Deep Learning on Highway Driving , 2015, ArXiv.

[24]  Wenhui Zhang,et al.  End to End Video Segmentation for Driving : Lane Detection For Autonomous Car , 2018, ArXiv.

[25]  Long Chen,et al.  Robust Lane Detection From Continuous Driving Scenes Using Deep Neural Networks , 2019, IEEE Transactions on Vehicular Technology.

[26]  Ronen Lerner,et al.  Recent progress in road and lane detection: a survey , 2012, Machine Vision and Applications.

[27]  Luc Van Gool,et al.  End-to-end Lane Detection through Differentiable Least-Squares Fitting , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[28]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[29]  Christoph Stiller,et al.  Kalman Particle Filter for lane recognition on rural roads , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[30]  Minh-Trien Pham,et al.  Real-time lane marker detection using template matching with RGB-D camera , 2018, 2018 2nd International Conference on Recent Advances in Signal Processing, Telecommunications & Computing (SigTelCom).

[31]  Hsueh-Ming Hang,et al.  Efficient Road Lane Marking Detection with Deep Learning , 2018, 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP).

[32]  Zhu Teng,et al.  Real-time lane detection by using multiple cues , 2010, ICCAS 2010.

[33]  In So Kweon,et al.  VPGNet: Vanishing Point Guided Network for Lane and Road Marking Detection and Recognition , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).