An Analysis of ISO 26262: Using Machine Learning Safely in Automotive Software

Machine learning (ML) plays an ever-increasing role in advanced automotive functionality for driver assistance and autonomous operation; however, its adequacy from the perspective of safety certification remains controversial. In this paper, we analyze the impacts that the use of ML as an implementation approach has on ISO 26262 safety lifecycle and ask what could be done to address them. We then provide a set of recommendations on how to adapt the standard to accommodate ML.

[1]  Gerald E. Peterson.  Foundation for neural network verification and validation , 1993, Defense, Security, and Sensing.

[2]  Trevor Darrell,et al.  Generating Visual Explanations , 2016, ECCV.

[3]  David M. Rodvold.  A software development process model for artificial neural networks in critical applications , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[4]  Amnon Shashua,et al.  On the Sample Complexity of End-to-end Training vs. Semantic Abstraction Training , 2016, ArXiv.

[5]  Daniel Watzenig,et al.  Functional Safety of Automated Driving Systems: Does ISO 26262 Meet the Challenges? , 2017 .

[6]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[7]  Anuj K. Pradhan,et al.  Literature Review of Behavioral Adaptations to Advanced Driver Assistance Systems , 2016 .

[8]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[9]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[10]  Jim Austin,et al.  Developing artificial neural networks for safety critical systems , 2006, Neural Computing and Applications.

[11]  Dick de Waard,et al.  Behavioural impacts of Advanced Driver Assistance Systems - an overview. , 2001 .

[12]  Joachim Diederich,et al.  The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks , 1998, IEEE Trans. Neural Networks.

[13]  Yan Liu,et al.  Application of Neural Networks in High Assurance Systems: A Survey , 2010, Applications of Neural Networks in High Assurance Systems.

[14]  Mykel J. Kochenderfer,et al.  Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks , 2017, CAV.

[15]  Eric Horvitz,et al.  On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems , 2016, AAAI.

[16]  Willem F. G. Haselager,et al.  Do Robot Performance and Behavioral Style a ↵ ect Human Trust ? A Multi-Method Approach , 2014 .

[17]  Sriram K. Rajamani,et al.  Debugging Machine Learning Tasks , 2016, ArXiv.

[18]  Bernd Spanfelner,et al.  Challenges in applying the ISO 26262 for driver assistance systems , 2012 .

[19]  S. Bhattacharyya,et al.  Certification considerations for adaptive systems , 2015, 2015 International Conference on Unmanned Aircraft Systems (ICUAS).

[20]  Itsuo Takanami,et al.  A fault-value injection approach for multiple-weight-fault tolerance of MNNs , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[21]  Kush R. Varshney,et al.  Engineering safety in machine learning , 2016, 2016 Information Theory and Applications Workshop (ITA).

[22]  Raja Parasuraman,et al.  Humans and Automation: Use, Misuse, Disuse, Abuse , 1997, Hum. Factors.

[23]  John Schulman,et al.  Concrete Problems in AI Safety , 2016, ArXiv.

[24]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[25]  Philip Koopman,et al.  Challenges in Autonomous Vehicle Testing and Validation , 2016 .

[26]  W. Marsden.  I and J , 2012 .

[27]  Andrew G. Barto,et al.  Lyapunov Design for Safe Reinforcement Learning , 2003, J. Mach. Learn. Res..

[28]  Min Wu,et al.  Safety Verification of Deep Neural Networks , 2016, CAV.

[29]  Laura L. Pullum,et al.  Guidance for the Verification and Validation of Neural Networks: Pullum/Guidance for the Verification and Validation of Neural Networks , 2015 .

[30]  R. Ratcliff,et al.  CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE Comparing Exemplar- and Rule- Based Theories of Categorization , 2022 .

[31]  Hermann Winner,et al.  Herausforderungen in der Absicherung von Fahrerassistenzsystemen bei der Benutzung maschinell gelernter und lernender Algorithmen , 2017 .