Intelligent corner synthesis via cycle-consistent generative adversarial networks for efficient validation of autonomous driving systems

Today's automotive vehicles are often equipped with powerful data processing systems for driver assistance and/or autonomous driving. To meet the rigorous safety standard, one critical task is to ensure extremely small failure rate over all possible operation conditions. Such a validation task requires a large amount of on-road testing data to cover all possible corners. In this paper, we describe a novel general-purpose methodology to synthetically and efficiently generate a broad spectrum of corner cases for validation purpose. Our proposed method is based upon cycle-consistent generative adversarial networks (CycleGANs) trained by a small set of image samples to mathematically map a nominal case to other corner cases. By taking STOP sign detection as an example, our numerical experiments demonstrate that the proposed approach is able to reduce the validation error by up to 100× given a limited data set for corner cases.

[1]  Julius Ziegler,et al.  Making Bertha Drive—An Autonomous Journey on a Historic Route , 2014, IEEE Intelligent Transportation Systems Magazine.

[2]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[3]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[4]  Wolfgang Rosenstiel,et al.  Simulation of falling rain for robustness testing of video-based surround sensing systems , 2016, 2016 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[5]  Johannes Stallkamp,et al.  Detection of traffic signs in real-world images: The German traffic sign detection benchmark , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[6]  Xuan Zeng,et al.  Impact of circuit-level non-idealities on vision-based autonomous driving systems , 2017, 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[7]  Meng-Yin Fu,et al.  A survey of traffic sign recognition , 2010, 2010 International Conference on Wavelet Analysis and Pattern Recognition.

[8]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[9]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[10]  Zhiyi Yu,et al.  An FPGA-Based Hardware Accelerator for Traffic Sign Detection , 2017, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[11]  Johannes Stallkamp,et al.  Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition , 2012, Neural Networks.

[12]  Motoaki Kawanabe,et al.  Machine Learning in Non-Stationary Environments - Introduction to Covariate Shift Adaptation , 2012, Adaptive computation and machine learning.

[13]  Nidhi Kalra,et al.  Driving to Safety , 2016 .

[14]  Xin Li,et al.  Efficient statistical validation of machine learning systems for autonomous driving , 2016, 2016 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[15]  Ralph Roskies,et al.  Bridges: a uniquely flexible HPC resource for new communities and data analytics , 2015, XSEDE.

[16]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[17]  Eric Dumont,et al.  Nighttime Visibility Analysis and Estimation Method in the Presence of Dense Fog , 2015, IEEE Transactions on Intelligent Transportation Systems.

[18]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[19]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[20]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Anders Lindgren,et al.  State of the Art Analysis: An Overview of Advanced Driver Assistance Systems (ADAS) and Possible Human Factors Issues , 2006 .

[23]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Jürgen Schmidhuber,et al.  Multi-column deep neural network for traffic sign classification , 2012, Neural Networks.

[25]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[26]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[27]  Werner Huber,et al.  Experience, Results and Lessons Learned from Automated Driving on Germany's Highways , 2015, IEEE Intelligent Transportation Systems Magazine.

[28]  Marc Stamminger,et al.  Concerning the applicability of computer graphics for the evaluation of image processing algorithms , 2012, 2012 IEEE International Conference on Vehicular Electronics and Safety (ICVES 2012).

[29]  Luc Van Gool,et al.  Traffic sign recognition — How far are we from the solution? , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[30]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Marc Stamminger,et al.  Hardware-in-the-loop testing of computer vision based driver assistance systems , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[32]  Nancy Wilkins-Diehr,et al.  XSEDE: Accelerating Scientific Discovery , 2014, Computing in Science & Engineering.