Bayesian Model for Trustworthiness Analysis of Deep Learning Classifiers

In the near future, Artificial Intelligence methods will inevitably enter safety-critical areas. Deep Learning software, deployed on standard computing hardware, is prone to random hardware faults such as bit flips that can result in silent data corruption. We have performed fault injection experiments on three Convolution Neural Network (CNN) image classifiers, including VGG16 and VGG19. Besides the fact that the bit flips indeed drop the classification accuracy, we have observed that these faults result not in random misclassification but tend to particular erroneous sets of classes. This fact shall be taken into account to design a reliable and safe system. For example, we might consider re-running the classifier if it yields a class for such an erroneous set. This paper discusses the results of our fault injection experiments and introduces a new Bayesian Network (BN) model that aggregates these results and enables numerical evaluation of the performance of the CNNs under the influence of random hardware faults. We demonstrate the application of the developed BN model for the trustworthiness analysis. In particular, we show how to evaluate the misclassification probabilities for each resulting class, for the varying probability of random bit-flips.

[1]  Daniel Kroening,et al.  A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability , 2018, Comput. Sci. Rev..

[2]  Qiang Xu,et al.  Fault injection attack on deep neural network , 2017, 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[3]  Jacob Schreiber,et al.  Pomegranate: fast and flexible probabilistic modeling in python , 2017, J. Mach. Learn. Res..

[4]  Fabiano Pereira Libano Reliability analysis of neural networks in FPGAs , 2018 .

[5]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[6]  Alberto Bosio,et al.  A Reliability Analysis of a Deep Neural Network , 2019, 2019 IEEE Latin American Test Symposium (LATS).

[7]  Klaus Janschek,et al.  Quantification of the Impact of Random Hardware Faults on Safety-Critical AI Applications: CNN-Based Traffic Sign Recognition Case Study , 2019, 2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW).

[8]  Nathan DeBardeleben,et al.  TensorFI: A Configurable Fault Injector for TensorFlow Applications , 2018, 2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW).

[9]  Deliang Fan,et al.  Bit-Flip Attack: Crushing Neural Network With Progressive Bit Search , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).