Robustness and performance of Deep Reinforcement Learning

Abstract Deep Reinforcement Learning (DRL) has recently obtained considerable attentions. It empowers Reinforcement Learning (RL) with Deep Learning (DL) techniques to address various difficult tasks. In this paper, a novel approach called the Genetic Algorithm of Neuron Coverage (GANC) is proposed. It is motivated for improving the robustness and performance of a DRL network. The GANC uses Genetic Algorithm (GA) to maximise the Neuron Coverage (NC) of a DRL network by producing augmented inputs. We apply this method in the self-driving car applications, where it is crucial to accurately provide a correct decision for different road tracking views. We evaluate our method on the SYNTHIA-SEQS-05 databases in four different driving environments. Our outcomes are very promising – the best driving accuracy reached 97.75% – and are superior to the state-of-the-art results.

[1]  Wai Lok Woo,et al.  A New Approach to Predicting Physical Biometrics from Behavioural Biometrics , 2014 .

[2]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[3]  K. Sasaki,et al.  Learning to simplify , 2016, ACM Trans. Graph..

[4]  Victor S. Lempitsky,et al.  Neural Codes for Image Retrieval , 2014, ECCV.

[5]  Raid Rafi Omar Al-Nima.  Signal processing and machine learning techniques for human verification based on finger textures , 2017 .

[6]  Taolue Chen,et al.  Deep finger texture learning for verifying people , 2018, IET Biom..

[7]  Lei Ma,et al.  DeepGauge: Comprehensive and Multi-Granularity Testing Criteria for Gauging the Robustness of Deep Learning Systems , 2018, ArXiv.

[8]  Marc Peter Deisenroth,et al.  Deep Reinforcement Learning: A Brief Survey , 2017, IEEE Signal Processing Magazine.

[9]  Dawid Połap,et al.  An adaptive genetic algorithm as a supporting mechanism for microscopy image analysis in a cascade of convolution neural networks , 2020, Appl. Soft Comput..

[10]  Antonio M. López,et al.  The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Taolue Chen,et al.  Road Tracking Using Deep Reinforcement Learning for Self-driving Car Applications , 2019, CORES.

[12]  Antonello Rizzi,et al.  Optimization strategies for Microgrid energy management systems by Genetic Algorithms , 2020, Appl. Soft Comput..

[13]  D. M. Knemeijer.  STRIPAI: Determining the Suitability of Implementing Deep Reinforcement Learning Principles into new domains , 2019 .

[14]  Suman Jana,et al.  DeepTest: Automated Testing of Deep-Neural-Network-Driven Autonomous Cars , 2017, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[15]  Andrew Lewis,et al.  The Whale Optimization Algorithm , 2016, Adv. Eng. Softw..

[16]  Junfeng Yang,et al.  Towards Practical Verification of Machine Learning: The Case of Computer Vision Systems , 2017, ArXiv.

[17]  Y. Kiran Kumar,et al.  COMPARISON OF FUSION TECHNIQUES APPLIED TO PRECLINICAL IMAGES: FAST DISCRETE CURVELET TRANSFORM USING WRAPPING TECHNIQUE & WAVELET TRANSFORM , 2009 .

[18]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[19]  Marcin Woźniak,et al.  Red fox optimization algorithm , 2021, Expert Syst. Appl..

[20]  Canyong Wang,et al.  Research and Application of Traffic Sign Detection and Recognition Based on Deep Learning , 2018, 2018 International Conference on Robots & Intelligent System (ICRIS).

[21]  Junfeng Yang,et al.  DeepXplore: Automated Whitebox Testing of Deep Learning Systems , 2017, SOSP.

[22]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[23]  Manoj Kumar,et al.  Genetic Algorithm: Review and Application , 2010 .

[24]  Luis Perez,et al.  The Effectiveness of Data Augmentation in Image Classification using Deep Learning , 2017, ArXiv.

[25]  Maneesh Kumar Singh,et al.  Lipschitz Properties for Deep Convolutional Networks , 2017, ArXiv.

[26]  Necla Özkaya,et al.  An intelligent face features generation system from fingerprints , 2009 .

[27]  Inderjit S. Dhillon,et al.  The Limitations of Adversarial Training and the Blind-Spot Attack , 2019, ICLR.

[28]  Andreas Holzinger,et al.  Augmentor: An Image Augmentation Library for Machine Learning , 2017, J. Open Source Softw..

[29]  Marcin Wozniak,et al.  Polar Bear Optimization Algorithm: Meta-Heuristic with Fast Population Movement and Dynamic Birth and Death Mechanism , 2017, Symmetry.

[30]  Daniel Kroening,et al.  Concolic Testing for Deep Neural Networks , 2018, 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[31]  Yuzhen Lu,et al.  Food Image Recognition by Using Convolutional Neural Networks (CNNs) , 2016, ArXiv.

[32]  Daniel Kroening,et al.  Testing Deep Neural Networks , 2018, ArXiv.

[33]  N A Andriyanov,et al.  The using of data augmentation in machine learning in image processing tasks in the face of data scarcity , 2020 .

[34]  Keiron O'Shea,et al.  An Introduction to Convolutional Neural Networks , 2015, ArXiv.

[35]  Melanie Mitchell,et al.  An introduction to genetic algorithms , 1996 .

[36]  Lei Ma,et al.  DeepMutation: Mutation Testing of Deep Learning Systems , 2018, 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE).

[37]  Geoff Nitschke,et al.  Improving Deep Learning using Generic Data Augmentation , 2017 .

[38]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[39]  Timo Fabritius,et al.  Application of a genetic algorithm based model selection algorithm for identification of carbide-based hot metal desulfurization , 2020, Applied Soft Computing.

[40]  Laurene V. Fausett,et al.  Fundamentals Of Neural Networks , 1994 .

[41]  Jingjing Liu,et al.  Adversarial Examples Construction Towards White-Box Q Table Variation in DQN Pathfinding Training , 2018, 2018 IEEE Third International Conference on Data Science in Cyberspace (DSC).

[42]  J Hayhurst Kelly,et al.  A Practical Tutorial on Modified Condition/Decision Coverage , 2001 .