Algorithm and hardware implementation for visual perception system in autonomous vehicle: A survey

This paper briefly surveys the recent progress on visual perception algorithms and their corresponding hardware implementations for the emerging application of autonomous driving. In particular, vehicle and pedestrian detection, lane detection and drivable surface detection are presented as three important applications for visual perception. On the other hand, CPU, GPU, FPGA and ASIC are discussed as the major components to form an efficient hardware platform for real-time operation. Finally, several technical challenges are presented to motivate future research and development in the field.

[1]  Raquel Urtasun,et al.  Estimating Drivable Collision-Free Space from Monocular Video , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[2]  Bertrand Douillard,et al.  On the segmentation of 3D LIDAR point clouds , 2011, 2011 IEEE International Conference on Robotics and Automation.

[3]  ZuWhan Kim,et al.  Robust Lane Detection and Tracking in Challenging Scenarios , 2008, IEEE Transactions on Intelligent Transportation Systems.

[4]  Werner Huber,et al.  Experience, Results and Lessons Learned from Automated Driving on Germany's Highways , 2015, IEEE Intelligent Transportation Systems Magazine.

[5]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[6]  Bernt Schiele,et al.  Filtered channel features for pedestrian detection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  John Zolock,et al.  The Use of Stationary Object Radar Sensor Data from Advanced Driver Assistance Systems (ADAS) in Accident Reconstruction , 2016 .

[8]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  Julius Ziegler,et al.  Making Bertha Drive—An Autonomous Journey on a Historic Route , 2014, IEEE Intelligent Transportation Systems Magazine.

[10]  Jianxiong Xiao,et al.  DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles , 2022 .

[12]  Eduardo Ros,et al.  A Comparison of FPGA and GPU for Real-Time Phase-Based Optical Flow, Stereo, and Local Image Features , 2012, IEEE Transactions on Computers.

[13]  Yann LeCun,et al.  Semantic Road Segmentation via Multi-scale Ensembles of Learned Features , 2012, ECCV Workshops.

[14]  Rama Chellappa,et al.  A Learning Approach Towards Detection and Tracking of Lane Markings , 2012, IEEE Transactions on Intelligent Transportation Systems.

[15]  David A. Patterson,et al.  In-datacenter performance analysis of a tensor processing unit , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[16]  Roland Siegwart,et al.  Toward automated driving in cities using close-to-market sensors: An overview of the V-Charge Project , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[17]  L. Mitchell,et al.  Accelerating Cardiac Bidomain Simulations Using Graphics Processing Units , 2012, IEEE Transactions on Biomedical Engineering.

[18]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[19]  Sebastian Thrun,et al.  Towards fully autonomous driving: Systems and algorithms , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[20]  Karsten Schwan,et al.  Keeneland: Bringing Heterogeneous GPU Computing to the Computational Science Community , 2011, Computing in Science & Engineering.

[21]  Xin Li,et al.  Efficient statistical validation of machine learning systems for autonomous driving , 2016, 2016 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[22]  Luca Gatti,et al.  VIAC: An out of ordinary experiment , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[23]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Akila Gothandaraman,et al.  Comparing Hardware Accelerators in Scientific Applications: A Case Study , 2011, IEEE Transactions on Parallel and Distributed Systems.

[25]  David Blaauw,et al.  A 23-mW Face Recognition Processor with Mostly-Read 5T Memory in 40-nm CMOS , 2017, IEEE Journal of Solid-State Circuits.

[26]  Karin Strauss,et al.  Accelerating Deep Convolutional Neural Networks Using Specialized Hardware , 2015 .

[27]  Sagar Behere,et al.  A functional architecture for autonomous driving , 2015, 2015 First International Workshop on Automotive Software Architecture (WASA).

[28]  R. Mobus,et al.  Multi-target multi-object tracking, sensor fusion of radar and infrared , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[29]  William J. Dally,et al.  The GPU Computing Era , 2010, IEEE Micro.

[30]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[31]  Ragunathan Rajkumar,et al.  Towards a viable autonomous driving research platform , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[32]  Yao Zhang,et al.  Parallel Computing Experiences with CUDA , 2008, IEEE Micro.

[33]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Liang Lin,et al.  Is Faster R-CNN Doing Well for Pedestrian Detection? , 2016, ECCV.

[35]  Thomas Bräunl,et al.  Integration of Drive-by-Wire with Navigation Control for a Driverless Electric Race Car , 2014, IEEE Intelligent Transportation Systems Magazine.

[36]  Myoungho Sunwoo,et al.  Development of Autonomous Car—Part II: A Case Study on the Implementation of an Autonomous Driving System Based on Distributed Architecture , 2015, IEEE Transactions on Industrial Electronics.

[37]  Manu Mathew,et al.  High Performance Front Camera ADAS Applications on TI's TDA3X Platform , 2015, 2015 IEEE 22nd International Conference on High Performance Computing (HiPC).

[38]  John E. Stone,et al.  OpenCL: A Parallel Programming Standard for Heterogeneous Computing Systems , 2010, Computing in Science & Engineering.