Embedding vision-based advanced driver assistance systems: a survey

Automated driving will have a big impact on society, creating new possibilities for mobility and reducing road accidents. Current developments aim to provide driver assistance in the form of conditional and partial automation. Computer vision, either alone or combined with other technologies such as radar or lidar, is one of the key technologies of advanced driver assistance systems (ADAS). The presence of vision technologies inside the vehicles is expected to grow as the automation levels increase. However, embedding a vision-based driver assistance system supposes a big challenge due to the special features of vision algorithms, the existing constrains and the strict requirements that need to be fulfilled. The aim of this study is to show the current progress and future directions in the field of vision-based embedded ADAS, bridging the gap between theory and practice. The different hardware and software options are reviewed, and design, development and testing considerations are discussed. Additionally, some outstanding challenges are also identified.

[1]  Jiun-In Guo,et al.  A forward collision avoidance system adopting multi-feature vehicle detection , 2014, 2014 IEEE International Conference on Consumer Electronics - Taiwan.

[2]  Sebastien Glaser,et al.  PerSEE: A central sensors fusion electronic control unit for the development of perception-based ADAS , 2015, 2015 14th IAPR International Conference on Machine Vision Applications (MVA).

[3]  Nanjian Wu.  High speed CMOS vision chips , 2011, 2011 IEEE 54th International Midwest Symposium on Circuits and Systems (MWSCAS).

[4]  Amnon Shashua,et al.  A Computer Vision System on a Chip: a case study from the automotive domain , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[5]  Huei-Yung Lin,et al.  Lane departure and front collision warning using a single camera , 2012, 2012 International Symposium on Intelligent Signal Processing and Communications Systems.

[6]  Shyan-Ming Yuan,et al.  Real-time pedestrian detection technique for embedded driver assistance systems , 2015, 2015 IEEE International Conference on Consumer Electronics (ICCE).

[7]  Tsung-Ying Sun,et al.  A symmetry-based forward vehicle detection and collision warning system on Android smartphone , 2015, 2015 IEEE International Conference on Consumer Electronics - Taiwan.

[8]  Iyad Rahwan,et al.  Autonomous Vehicles Need Experimental Ethics: Are We Ready for Utilitarian Cars? , 2015, ArXiv.

[9]  Lingjiang Kong,et al.  Vision-based multi-scaled vehicle detection and distance relevant mix tracking for driver assistance system , 2015 .

[10]  Holger Blume,et al.  A comprehensive ASIC/FPGA prototyping environment for exploring embedded processing systems for advanced driver assistance applications , 2014, 2014 International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation (SAMOS XIV).

[11]  Torsten Bertram,et al.  Objective evaluation of software architectures in driver assistance systems , 2011, Computer Science - Research and Development.

[12]  Xiangjing An,et al.  Real-time lane departure warning system based on a single FPGA , 2013, EURASIP J. Image Video Process..

[13]  Alberto Broggi,et al.  Extensive Tests of Autonomous Driving Technologies , 2013, IEEE Transactions on Intelligent Transportation Systems.

[14]  Hermann Winner,et al.  Three Decades of Driver Assistance Systems: Review and Future Perspectives , 2014, IEEE Intelligent Transportation Systems Magazine.

[15]  Arthur Daniel Costea,et al.  Fast Pedestrian Detection for Mobile Devices , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[16]  Philip Koopman,et al.  Challenges in Autonomous Vehicle Testing and Validation , 2016 .

[17]  Wolfgang Rosenstiel,et al.  Safe and Efficient Runtime Resource Management in Heterogeneous Systems for Automated Driving , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[18]  Senem Velipasalar,et al.  Robust and Computationally Lightweight Autonomous Tracking of Vehicle Taillights and Signal Detection by Embedded Smart Cameras , 2015, IEEE Transactions on Industrial Electronics.

[19]  Jean-Claude Laprie,et al.  Dependable computing: concepts, limits, challenges , 1995 .

[20]  Matthias Mielke,et al.  ASIC implementation of a Gaussian Pyramid for use in autonomous mobile robotics , 2011, 2011 IEEE 54th International Midwest Symposium on Circuits and Systems (MWSCAS).

[21]  G. B. Finelli,et al.  The Infeasibility of Quantifying the Reliability of Life-Critical Real-Time Software , 1993, IEEE Trans. Software Eng..

[22]  Mohan M. Trivedi,et al.  Snap-DAS: A vision-based driver assistance system on a SnapdragonTM embedded platform , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[23]  Fridtjof Stein.  The challenge of putting vision algorithms into a car , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[24]  Samarjit Chakraborty,et al.  Robust image processing for an omnidirectional camera-based smart car door , 2009, ESTImedia.

[25]  Oihana Otaegui,et al.  Optimising computer vision based ADAS: vehicle detection case study , 2016 .

[26]  Dominique Lavenier,et al.  A Reconfigurable Disparity Engine for Stereovision in Advanced Driver Assistance Systems , 2010, ARC.

[27]  Sergiu Nedevschi,et al.  A stereovision based approach for detecting and tracking lane and forward obstacles on mobile devices , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[28]  Christophe Bobda,et al.  A hardware/software prototyping system for driving assistance investigations , 2016, Journal of Real-Time Image Processing.

[29]  Duan-Yu Chen,et al.  Nighttime Brake-Light Detection by Nakagami Imaging , 2012, IEEE Transactions on Intelligent Transportation Systems.

[30]  Kamel Besbes,et al.  Efficient algorithm for automatic road sign recognition and its hardware implementation , 2013, Journal of Real-Time Image Processing.

[31]  Yucheng Liu,et al.  A Surround View Camera Solution for Embedded Systems , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[32]  Oihana Otaegui,et al.  A reconfigurable embedded vision system for advanced driver assistance , 2015, Journal of Real-Time Image Processing.

[33]  Nau Ozaki,et al.  Implementation and evaluation of image recognition algorithm for an intelligent vehicle using heterogeneous multi-core SoC , 2015, The 20th Asia and South Pacific Design Automation Conference.

[34]  Shyan-Ming Yuan,et al.  A Vision-Based Driver Nighttime Assistance and Surveillance System Based on Intelligent Image Sensing Techniques and a Heterogamous Dual-Core Embedded System Architecture , 2012, Sensors.

[35]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Werner Huber,et al.  Experience, Results and Lessons Learned from Automated Driving on Germany's Highways , 2015, IEEE Intelligent Transportation Systems Magazine.

[37]  Bing-Fei Wu,et al.  Embedded Driver-Assistance System Using Multiple Sensors for Safe Overtaking Maneuver , 2014, IEEE Systems Journal.

[38]  Yunju Baek,et al.  Design and Implementation of Real-Time Vehicular Camera for Driver Assistance and Traffic Congestion Estimation , 2015, Sensors.

[39]  S. Chandrakala,et al.  Android OpenCV based effective driver fatigue and distraction monitoring system , 2015, 2015 International Conference on Computing and Communications Technologies (ICCCT).

[40]  Benoit Vanholme,et al.  Free Space Detection: A Corner Stone of Automated Driving , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[41]  Frank Forster.  Heterogeneous Processors for Advanced Driver Assistance Systems , 2014 .

[42]  Tong Boon Tang,et al.  Vehicle Detection Techniques for Collision Avoidance Systems: A Review , 2015, IEEE Transactions on Intelligent Transportation Systems.

[43]  Roger Reynaud,et al.  Optimal Performance Prediction of ADAS Algorithms on Embedded Parallel Architectures , 2015, 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, and 2015 IEEE 12th International Conference on Embedded Software and Systems.

[44]  Margaret Martonosi,et al.  Leveraging Smartphone Cameras for Collaborative Road Advisories , 2012, IEEE Transactions on Mobile Computing.

[45]  Yike Guo,et al.  Fast Traffic Sign Recognition with a Rotation Invariant Binary Pattern Based Feature , 2015, Sensors.

[46]  Thomas A Henzinger,et al.  Two challenges in embedded systems design: predictability and robustness , 2008, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[47]  Brigitte d'Andréa-Novel,et al.  From virtual to reality, how to prototype, test and evaluate new ADAS: Application to automatic car parking , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[48]  Roger Reynaud,et al.  Towards an Automatic Prediction of Image Processing Algorithms Performances on Embedded Heterogeneous Architectures , 2015, 2015 44th International Conference on Parallel Processing Workshops.

[49]  Guangzhi Qu,et al.  Deploying and Scheduling Vision Based Advanced Driver Assistance Systems (ADAS) on Heterogeneous Multicore Embedded Platform , 2015, 2015 Ninth International Conference on Frontier of Computer Science and Technology.

[50]  Roger Reynaud,et al.  The embeddability of lane detection algorithms on heterogeneous architectures , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[51]  Marc Pollefeys,et al.  Real-time and low latency embedded computer vision hardware based on a combination of FPGA and mobile CPU , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[52]  Michael Hübner,et al.  FPGA based traffic sign detection for automotive camera systems , 2015, 2015 10th International Symposium on Reconfigurable Communication-centric Systems-on-Chip (ReCoSoC).

[53]  Bing-Fei Wu,et al.  Reasoning-Based Framework for Driving Safety Monitoring Using Driving Event Recognition , 2013, IEEE Transactions on Intelligent Transportation Systems.

[54]  Benoit Vanholme,et al.  Fast prototyping of a Highly Autonomous Cooperative Driving System for public roads , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[55]  Jürgen Teich,et al.  Hardware/Software Codesign: The Past, the Present, and Predicting the Future , 2012, Proceedings of the IEEE.

[56]  Jinhui Tang,et al.  Vision-based two-step brake detection method for vehicle collision avoidance , 2016, Neurocomputing.

[57]  Steven E. Shladover,et al.  Potential Cyberattacks on Automated Vehicles , 2015, IEEE Transactions on Intelligent Transportation Systems.