DELFASE: A Deep Learning Method for Fault Space Exploration

Cyber-Physical Systems (CPSs) are increasingly used in various safety-critical domains; assuring the safety of these systems is of paramount importance. Fault Injection is known as an effective testing method for analyzing the safety of CPSs. However, the total number of faults to be injected in a CPS to explore the entire fault space is normally large and the limited budget for testing forces testers to limit the number of faults injected by e.g., random sampling of the space. In this paper, we propose DELFASE as an automated solution for fault space exploration that relies on Generative Adversarial Networks (GANs) for optimizing the identification of critical faults, and can run in two modes: active and passive. In the active mode, an active learning technique called ranked batch-mode sampling is used to select faults for training the GAN model with, while in the passive mode those faults are selected randomly. The results of our experiments on an adaptive cruise control system show that compared to random sampling, DELFASE is significantly more effective in revealing system weaknesses. In fact, we observed that compared to random sampling that resulted in a fault coverage of around 10%, when using the active and passive modes, the fault coverage of DELFASE could be as high as 89% and 81%, respectively.

[1]  K. Pattabiraman,et al.  An Empirical Study of the Impact of Single and Multiple Bit-Flip Errors in Programs , 2022, IEEE Transactions on Dependable and Secure Computing.

[2]  Behrooz Sangchoolie,et al.  SUFI: A Simulation-based Fault Injection Tool for Safety Evaluation of Advanced Driver Assistance Systems Modelled in SUMO , 2021, 2021 17th European Dependable Computing Conference (EDCC).

[3]  Mahshid Helali Moghadam,et al.  Automated Performance Testing Based on Active Deep Learning , 2021, 2021 IEEE/ACM International Conference on Automation of Software Test (AST).

[4]  Klaus Janschek,et al.  Model-based Fault Injection Experiments for the Safety Analysis of Exoskeleton System , 2021, Proceedings of the 30th European Safety and Reliability Conference and 15th Probabilistic Safety Assessment and Management Conference.

[5]  Joachim Denil,et al.  Machine Learning-assisted Fault Injection , 2020 .

[6]  Klaus Janschek,et al.  Exploring Fault Parameter Space Using Reinforcement Learning-based Fault Injection , 2020, 2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W).

[7]  Ravishankar K. Iyer,et al.  ML-Based Fault Injection for Autonomous Vehicles: A Case for Bayesian Fault Injection , 2019, 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[8]  Lejla Batina,et al.  Optimizing Electromagnetic Fault Injection with Genetic Algorithms , 2019, Automated Methods in Cryptographic Fault Analysis.

[9]  Peter Folkesson,et al.  A Study of the Interplay Between Safety and Security Using Model-Implemented Fault Injection , 2018, 2018 14th European Dependable Computing Conference (EDCC).

[10]  Kai Yang,et al.  Active Learning for Wireless IoT Intrusion Detection , 2018, IEEE Wireless Communications.

[11]  Péter Horváth,et al.  modAL: A modular active learning framework for Python , 2018, ArXiv.

[12]  Nan Zhang,et al.  Optimization methods for the fault injection tool SEInjector , 2018, 2018 International Conference on Information and Computer Technologies (ICICT).

[13]  Philip S. Yu,et al.  An Introduction to Image Synthesis with Generative Adversarial Nets , 2018, ArXiv.

[14]  Klaus Janschek,et al.  ErrorSim: A Tool for Error Propagation Analysis of Simulink Models , 2017, SAFECOMP.

[15]  Wolfgang Rosenstiel,et al.  Verification of Component Fault Trees Using Error Effect Simulations , 2017, IMBSA.

[16]  Nicu Sebe,et al.  Abnormal event detection in videos using generative adversarial nets , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[17]  Anima Anandkumar,et al.  Deep Active Learning for Named Entity Recognition , 2017, Rep4NLP@ACL.

[18]  Marcos André Gonçalves,et al.  Ranked batch-mode active learning , 2017, Inf. Sci..

[19]  Roger Johansson,et al.  Light-Weight Techniques for Improving the Controllability and Efficiency of ISA-Level Fault Injection Tools , 2017, 2017 IEEE 22nd Pacific Rim International Symposium on Dependable Computing (PRDC).

[20]  Joachim Fröhlich,et al.  Testing Safety Properties of Cyber-Physical Systems with Non-Intrusive Fault Injection - An Industrial Case Study , 2016, SAFECOMP Workshops.

[21]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[22]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[23]  Magnus Jonsson,et al.  Combining Fault-Injection with Property-Based Testing , 2013, ES4CPS@DATE.

[24]  Andrew L. Maas.  Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[25]  Henrik Eriksson,et al.  Model-Implemented Fault Injection for Hardware Fault Simulation , 2010, 2010 Workshop on Model-Driven Engineering, Verification, and Validation.

[26]  Henrik Eriksson,et al.  MODIFI: A MODel-Implemented Fault Injection Tool , 2010, SAFECOMP.

[27]  G Coley,et al.  Driver reaction times to familiar but unexpected events , 2010 .

[28]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[29]  Edward A. Lee.  Cyber Physical Systems: Design Challenges , 2008, 2008 11th IEEE International Symposium on Object and Component-Oriented Real-Time Distributed Computing (ISORC).

[30]  Shie Mannor,et al.  A Tutorial on the Cross-Entropy Method , 2005, Ann. Oper. Res..

[31]  Alfredo Benso,et al.  Fault Injection Techniques and Tools for Embedded Systems , 2003 .

[32]  A. Lesiak,et al.  Software Implemented Fault Inserters , 2003 .

[33]  Régis Leveugle,et al.  Using run-time reconfiguration for fault injection in hardware prototypes , 2000, 17th IEEE International Symposium on Defect and Fault Tolerance in VLSI Systems, 2002. DFT 2002. Proceedings..

[34]  Kang G. Shin,et al.  Fault Injection Techniques and Tools , 1997, Computer.

[35]  Claudio Moraga,et al.  The Influence of the Sigmoid Function Parameters on the Speed of Backpropagation Learning , 1995, IWANN.

[36]  Jean Arlat,et al.  Fault Injection for Dependability Validation: A Methodology and Some Applications , 1990, IEEE Trans. Software Eng..