A discussion of practices for enhancing diversity in software designs

This report discusses the practices which have been used or recommended for increasing the degree of diversity between redundant implementations of software or software-based systems. Its purpose is to give useful indications for designers, project managers and safety/reliability assessors in deciding about how great an advantage should be expected from the use of these practices, in absolute and in comparative terms. Existing knowledge does not allow one to state any strong general recommendations, but it is possible to improve on the intuitive justifications usually given for these various practices. This report clarifies the ways the various practices are conjectured to aid system reliability, the factors that should affect their efficacy, and thus, for a practitioner, the aspects of a specific project situation that need to be considered to inform decisions. Thus this report is meant to improve on the many recommendations available in the literature by a more rigorous analysis of the support available for individual recommendations and for decision between them, on the basis of existing known evidence about diversity, of general experience in software engineering and of the result of our reliability modelling work. An executive summary gives the highlights of the report and a guide to the topics treated. The other sections are an introduction giving the scope and background of the work; a general analysis of the factors affecting the achievement of useful diversity and the criteria for choosing among possible "diversity-seeking decisions" to this end, and a summary table of the considerations applying to each category of "diversity-seeking decisions", with explanations of detail in an appendix. This report updates and supersedes the previous DISPO project report AT_DI-D-01-v1.7, "A list of intuitive diversity enhancing measures/practices", 20 February 1998, which was produced at the beginning of the DISPO project, to reflect our changed understanding at the end of the project. Parts of the old report have been eliminated as the corresponding topics are now covered by separate DISPO documents. The information contained in this report has been produced for BEG(UK) Ltd on behalf of the Industry Management Committee (IMC). thus is the joint property of British Energy Generation Ltd, British Energy Generation (UK) Ltd, British Nuclear Fuels Limited and British Nuclear Fuels Magnox Generation Ltd, and their successor companies. Any intellectual property rights arising from or contained in the report are the joint property of British Energy Generation Ltd, British Energy Generation (UK) Ltd, British Nuclear Fuels Limited and British Nuclear Fuels Magnox Generation Ltd, and their successor companies.

[1]  David F. McAllister,et al.  An Experimental Evaluation of Software Redundancy as a Strategy For Improving Reliability , 1991, IEEE Trans. Software Eng..

[2]  Manuel Blum,et al.  Self-testing/correcting with applications to numerical problems , 1990, STOC '90.

[3]  Dave E. Eckhardt,et al.  A Theoretical Basis for the Analysis of Multiversion Software Subject to Coincident Errors , 1985, IEEE Transactions on Software Engineering.

[4]  Bev Littlewood,et al.  A note on reliability estimation of functionally diverse systems , 1999 .

[5]  Paul Ammann,et al.  Data Diversity: An Approach to Software Fault Tolerance , 1988, IEEE Trans. Computers.

[6]  Michael R. Lyu,et al.  Improving the N-version programming process through the evolution of a design paradigm , 1993 .

[7]  Manuel Blum,et al.  Program result-checking: a theory of testing meets a test of theory , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[8]  Lorenzo Strigini,et al.  Software Fault-Tolerance and Design Diversity: Past Experience and Future Evolution , 1985 .

[9]  Ravishankar K. Iyer,et al.  Software Dependability in the Tandem GUARDIAN System , 1995, IEEE Trans. Software Eng..

[10]  G. Robert J. Hockey,et al.  Cognitive Diversity: A Structured Approach to Trapping Human Error , 1995, SAFECOMP.

[11]  Peter G. Bishop,et al.  PODS revisited-a study of software failure behaviour , 1988, [1988] The Eighteenth International Symposium on Fault-Tolerant Computing. Digest of Papers.

[12]  Algirdas Avizienis,et al.  Software Fault Tolerance , 1989, IFIP Congress.

[13]  Peter G. Neumann,et al.  Computer-related risks , 1994 .

[14]  Lorenzo Strigini,et al.  Adjudicators for diverse-redundant components , 1990, Proceedings Ninth Symposium on Reliable Distributed Systems.

[15]  A. Avizienis,et al.  Microprocessor entomology: a taxonomy of design faults in COTS microprocessors , 1999, Dependable Computing for Critical Applications 7.

[16]  Bev Littlewood,et al.  Modeling the Effects of Combining Diverse Software Fault Detection Techniques , 2000, IEEE Trans. Software Eng..

[17]  Bev Littlewood,et al.  Modelling the effects of combining diverse software fault removal techniques , 1999 .

[18]  Andy Roberts,et al.  How Accurate Is Scientific Software? , 1994, IEEE Trans. Software Eng..

[19]  Manuel Blum,et al.  Designing programs that check their work , 1989, STOC '89.

[20]  Bev Littlewood,et al.  Modeling Software Design Diversit y-AR eview , 2001 .

[21]  Jean Arlat,et al.  Definition and analysis of hardware- and software-fault-tolerant architectures , 1990, Computer.

[22]  Bev Littlewood,et al.  Conceptual Modeling of Coincident Failures in Multiversion Software , 1989, IEEE Trans. Software Eng..

[23]  Ying C. Yeh Design considerations in Boeing 777 fly-by-wire computers , 1998, Proceedings Third IEEE International High-Assurance Systems Engineering Symposium (Cat. No.98EX231).

[24]  Arndt Lindner ANSI-C in Safety Critical Applications - Lessons-Learned from Software Evaluation , 1998, SAFECOMP.

[25]  Nancy G. Leveson,et al.  Analysis of Faults in an N-Version Software Experiment , 1990, IEEE Trans. Software Eng..

[26]  Bev Littlewood The Use of Proof in Diversity Arguments , 2000, IEEE Trans. Software Eng..

[27]  Thomas I. McVittie,et al.  An Empirical Investigation of the Effect of Formal Specifications on Program Diversity , 1992 .

[28]  R. P. Hughes,et al.  A new approach to common cause failure , 1987 .

[29]  Michael R. Lyu,et al.  In search of effective diversity: a six-language study of fault-tolerant flight control software , 1988, [1988] The Eighteenth International Symposium on Fault-Tolerant Computing. Digest of Papers.

[30]  Peter G. Bishop,et al.  PODS — A project on diverse software , 1986, IEEE Transactions on Software Engineering.

[31]  Ram Chillarege,et al.  Orthogonal defect classification , 1996 .

[32]  Yennun Huang,et al.  Software rejuvenation: analysis, module and applications , 1995, Twenty-Fifth International Symposium on Fault-Tolerant Computing. Digest of Papers.

[33]  Francesca Saglietti A Classification of Software Diversity Degrees Induced by an Analysis of Fault Types to be Tolerated , 1991, Fault-Tolerant Computing Systems.

[34]  Bev Littlewood The impact of diversity upon common mode failures , 1996 .

[35]  Nancy G. Leveson,et al.  An Empirical Comparison of Software Fault Tolerance and Fault Elimination , 1991, IEEE Trans. Software Eng..

[36]  Peter G. Bishop,et al.  Error Masking: A Source of Failure Dependency in Multi-Version Programs , 1991 .

[37]  Peter G. Bishop The PODS Diversity Experiment , 1988 .

[38]  Algirdas Avizienis,et al.  The N-Version Approach to Fault-Tolerant Software , 1985, IEEE Transactions on Software Engineering.

[39]  James Reason,et al.  Human Error , 1990 .

[40]  G. E. Migneault The cost of software fault tolerance , 1982 .