Developing artificial neural networks for safety critical systems

There are many performance based techniques that aim to improve the safety of neural networks for safety critical applications. However, many of these approaches provide inadequate forms of safety assurance required for certification. As a result, neural networks are typically restricted to advisory roles in safety-related applications. Neural networks have the ability to operate in unpredictable and changing environments. It is therefore desirable to certify them for highly-dependable roles in safety critical systems. This paper outlines the safety criteria which are safety requirements for the behaviour of neural networks. If enforced, the criteria can contribute to justifying the safety of ANN functional properties. Characteristics of potential neural network models are also outlined and are based upon representing knowledge in interpretable and understandable forms. The paper also presents a safety lifecycle for artificial neural networks. This lifecycle focuses on managing behaviour represented by neural networks and contributes to providing acceptable forms of safety assurance.

[1]  N. E. Sharkey,et al.  Diversity , Neural Nets and Safety Critical Applications , 1995 .

[2]  R. Bell,et al.  IEC 61508: functional safety of electrical/electronic/ programme electronic safety-related systems: overview , 1999 .

[3]  David M. Rodvold A software development process model for artificial neural networks in critical applications , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[4]  Neil R. Storey,et al.  Safety-critical computer systems , 1996 .

[5]  Tim Kelly,et al.  Arguing Safety - A Systematic Approach to Managing Safety Cases , 1998 .

[6]  Zeshan Kurd,et al.  Artificial Neural Networks in Safety-critical Applications , 2002 .

[7]  Tim Kelly,et al.  Establishing Safety Criteria for Artificial Neural Networks , 2003, KES.

[8]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..

[9]  Tim Kelly,et al.  Safety Lifecycle for Developing Safety Critical Artificial Neural Networks , 2003, SAFECOMP.

[10]  Joydeep Ghosh,et al.  Hybrid intelligent architecture and its application to water reservoir control , 1997 .

[11]  Hilary Buxton,et al.  A hybrid approach to breast cancer diagnosis , 2001 .

[12]  Noel E. Sharkey,et al.  Neural Nets and Diversity , 1995, SAFECOMP.

[13]  Frank Klawonn,et al.  Combining Neural Networks and Fuzzy Controllers , 1993, FLAI.

[14]  Paulo J,et al.  Industrial use of safety-related artificial neural networks , 2001 .

[15]  Dietmar Reinert,et al.  Risk and system integrity concepts for safety-related control systems , 1992 .

[16]  John A. McDermid,et al.  Software Safety Arguments: Towards a Systematic Categorisation of Evidence , 2002 .

[17]  Jim Austin,et al.  Safety critical neural networks , 1995 .

[18]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[19]  Jude Shavlik,et al.  A Framework for Combining Symbolic and Neural Learning , 1992 .

[20]  Wen Wu,et al.  Towards Developing Verifiable Neural Network Controller , 1996 .

[21]  Ian T. Nabney,et al.  Practical Assessment of Neural Network Applications , 1997, SAFECOMP.

[22]  Eric N. Johnson,et al.  Adaptive guidance and control for autonomous launch vehicles , 2001, 2001 IEEE Aerospace Conference Proceedings (Cat. No.01TH8542).

[23]  Casimir C. Klimasauskas Neural nets tell why , 1991 .

[24]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..