BEV-LaneDet: An Efficient 3D Lane Detection Based on Virtual Camera via Key-Points

3D lane detection which plays a crucial role in vehicle routing, has recently been a rapidly developing topic in autonomous driving. Previous works struggle with practicality due to their complicated spatial transformations and inflexible representations of 3D lanes. Faced with the issues, our work proposes an efficient and robust monocular 3D lane detection called BEV-LaneDet with three main contributions. First, we introduce the Virtual Camera that unifies the in/extrinsic parameters of cameras mounted on different vehicles to guarantee the consistency of the spatial relationship among cameras. It can effectively promote the learning procedure due to the unified visual space. We secondly propose a simple but efficient 3D lane representation called Key-Points Representation. This module is more suitable to represent the complicated and diverse 3D lane structures. At last, we present a light-weight and chip-friendly spatial transformation module named Spatial Transformation Pyramid to transform multiscale front-view features into BEV features. Experimental results demonstrate that our work outperforms the state-of-the-art approaches in terms of F-Score, being 10.6% higher on the OpenLane dataset and 4.0% higher on the Apollo 3D synthetic dataset, with a speed of 185 FPS. Code is released at https://github.com/gigo-team/bev_lane_det.

[1]  Ya Wang,et al.  Reconstruct from Top View: A 3D Lane Detection Approach based on Geometry Structure Prior , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[2]  Jifeng Dai,et al.  BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers , 2022, ECCV.

[3]  Junchi Yan,et al.  PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmark , 2022, ECCV.

[4]  Xin Tan,et al.  Rethinking Efficient Lane Detection via Curve Modeling , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Zejian Yuan,et al.  Learning to Predict 3D Lane Shape and Camera Pose from a Single Image via Geometry Constraints , 2021, AAAI.

[6]  Dalong Du,et al.  BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View , 2021, ArXiv.

[7]  Yujie Jin,et al.  Robust Monocular 3D Lane Detection With Dual Attention , 2021, 2021 IEEE International Conference on Image Processing (ICIP).

[8]  Xiaoming Wei,et al.  Structure Guided Lane Detection , 2021, IJCAI.

[9]  Siyu Zhu,et al.  CondLaneNet: a Top-to-down Lane Detection Framework Based on Conditional Convolution , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[10]  Shaul Oron,et al.  3D-LaneNet+: Anchor Free Lane Detection using a Semi-Local Representation , 2020, ArXiv.

[11]  Thiago Oliveira-Santos,et al.  Keep your Eyes on the Lane: Real-time Attention-guided Lane Detection , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Deng Cai,et al.  RESA: Recurrent Feature-Shift Aggregator for Lane Detection , 2020, AAAI.

[13]  Sanja Fidler,et al.  Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D , 2020, ECCV.

[14]  Lutz Eckstein,et al.  A Sim2Real Deep Learning Approach for the Transformation of Images from Multiple Vehicle-Mounted Cameras to a Semantically Segmented Image in Bird’s Eye View , 2020, 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC).

[15]  Seungwoo Yoo,et al.  End-to-End Lane Marker Detection via Row-wise Classification , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[16]  Huanyu Wang,et al.  Ultra Fast Structure-aware Deep Lane Detection , 2020, ECCV.

[17]  Alberto Ferreira de Souza,et al.  PolyLaneNet: Lane Estimation via Deep Polynomial Regression , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[18]  T. Choe,et al.  Gen-LaneNet: A Generalized and Scalable Approach for 3D Lane Detection , 2020, ECCV.

[19]  Bolei Zhou,et al.  Cross-View Semantic Segmentation for Sensing Surroundings , 2019, IEEE Robotics and Automation Letters.

[20]  Dan Levi,et al.  3D-LaneNet: End-to-End 3D Multiple Lane Detection , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[21]  Min Bai,et al.  Deep Multi-Sensor Lane Detection , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  Luc Van Gool,et al.  Towards End-to-End Lane Detection: an Instance Segmentation Approach , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[23]  Xiaogang Wang,et al.  Spatial As Deep: Spatial CNN for Traffic Scene Understanding , 2017, AAAI.

[24]  Luc Van Gool,et al.  Semantic Instance Segmentation with a Discriminative Loss Function , 2017, ArXiv.

[25]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[26]  Serge J. Belongie,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Max Jaderberg,et al.  Spatial Transformer Networks , 2015, NIPS.

[31]  Yilun Wang,et al.  HDMapNet: A Local Semantic Map Learning and Evaluation Framework , 2021 .