Concept Enforcement and Modularization as Methods for the ISO 26262 Safety Argumentation of Neural Networks

Neural networks (NN) are prone to systematic faults which are hard to detect using the methods recommended by the ISO 26262 automotive functional safety standard. In this paper we propose a unified approach to two methods for NN safety argumentation: Assignment of human interpretable concepts to the internal representation of NNs to enable modularization and formal verification. Feasibility of the required concept embedding analysis is demonstrated in a minimal example and important aspects for generalization are investigated. The contribution of the methods is derived from a proposed generic argumentation structure for a NN model safety case.

[1]  Bolei Zhou,et al.  Network Dissection: Quantifying Interpretability of Deep Visual Representations , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  C. W. Johnson The Increasing Risks of Risk Assessment : On the Rise of Artificial Intelligence and NonDeterminism in Safety-Critical Systems , 2017 .

[3]  Andrea Vedaldi,et al.  Neural Stethoscopes: Unifying Analytic, Auxiliary and Adversarial Network Probing , 2018, ArXiv.

[4]  Klaus-Robert Müller,et al.  Learning how to explain neural networks: PatternNet and PatternAttribution , 2017, ICLR.

[5]  Markus Maurer,et al.  Ontology based Scene Creation for the Development of Automated Vehicles , 2017, 2018 IEEE Intelligent Vehicles Symposium (IV).

[6]  Trevor Darrell,et al.  Textual Explanations for Self-Driving Vehicles , 2018, ECCV.

[7]  Tim Kelly,et al.  Establishing Safety Criteria for Artificial Neural Networks , 2003, KES.

[8]  Marco Gori,et al.  Image Classification Using Deep Learning and Prior Knowledge , 2018, AAAI Workshops.

[9]  Johannes Stallkamp,et al.  The German Traffic Sign Recognition Benchmark: A multi-class classification competition , 2011, The 2011 International Joint Conference on Neural Networks.

[10]  Alberto L. Sangiovanni-Vincentelli,et al.  Counterexample-Guided Data Augmentation , 2018, IJCAI.

[11]  Mireille Blay-Fornarino,et al.  Support of Justification Elicitation: Two Industrial Reports , 2018, CAiSE.

[12]  T. Kathirvalavakumar,et al.  Rule extraction from neural networks — A comparative study , 2012, International Conference on Pattern Recognition, Informatics and Medical Engineering (PRIME-2012).

[13]  Bin Yu,et al.  Interpreting Convolutional Neural Networks Through Compression , 2017, ArXiv.

[14]  Junfeng Yang,et al.  Formal Security Analysis of Neural Networks using Symbolic Intervals , 2018, USENIX Security Symposium.

[15]  Deborah Silver,et al.  Feature Visualization , 1994, Scientific Visualization.

[16]  Hu Wang,et al.  ReNN: Rule-embedded Neural Networks , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[17]  Andrea Vedaldi,et al.  Net2Vec: Quantifying and Explaining How Concepts are Encoded by Filters in Deep Neural Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Huan Liu,et al.  Symbolic Representation of Neural Networks , 1996, Computer.

[19]  Pushmeet Kohli,et al.  A Unified View of Piecewise Linear Neural Network Verification , 2017, NeurIPS.

[20]  Dan Klein,et al.  Learning with Latent Language , 2017, NAACL.

[21]  Michael Siebers,et al.  Explaining Black-Box Classifiers with ILP - Empowering LIME with Aleph to Approximate Non-linear Decisions with Relational Rules , 2018, ILP.

[22]  Jaime F. Fisac,et al.  Reachability-based safe learning with Gaussian processes , 2014, 53rd IEEE Conference on Decision and Control.

[23]  Quanshi Zhang,et al.  Interpreting CNN knowledge via an Explanatory Graph , 2017, AAAI.

[24]  Simon Burton,et al.  Structuring Validation Targets of a Machine Learning Function Applied to Automated Driving , 2018, SAFECOMP.