Reducing Safety-Critical Software Statistical Testing Cost Based on Importance Sampling Technique

Safety-critical software often includes some rarely executed critical operations that are often inadequately tested in statistical testing based reliability estimation. However, it is necessary to assess the reliability of safety-critical software to a high degree of confidence before it is deployed in the field. This paper discusses how to reduce safety-critical software statistical testing cost based on importance sampling technique. When both the critical operations and the entire software are adequately tested, the method can still compute the unbiased software reliability from the test results with much less test cases. Thus, the statistical testing cost of safety-critical software can be reduced effectively The simulated annealing algorithm for calculating optimum transition probabilities of the Markov chain usage model for reducing software statistical testing cost is also presented.

[1]  Walter J. Gutjahr,et al.  Software dependability evaluation based on Markov usage models , 2000, Perform. Evaluation.

[2]  Jesse H. Poore,et al.  Statistical testing of software based on a usage model , 1995, Softw. Pract. Exp..

[3]  Jeffrey M. Voas,et al.  Estimating the Probability of Failure When Testing Reveals No Failures , 1992, IEEE Trans. Software Eng..

[4]  Louwrence D. Erasmus,et al.  Organisational impact of implementing IEC 61850 standard for communication networks and systems in substations , 2013, 2013 Proceedings of PICMET '13: Technology Management in the IT-Driven Services (PICMET).

[5]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[6]  Walter J. Gutjahr Optimal Test Distributions for Software Failure Cost Estimation , 1995, IEEE Trans. Software Eng..

[7]  G. B. Finelli,et al.  The Infeasibility of Quantifying the Reliability of Life-Critical Real-Time Software , 1993, IEEE Trans. Software Eng..

[8]  Kyle Siegrist,et al.  Reliability of Systems with Markov Transfer of Control , 1988, IEEE Trans. Software Eng..

[9]  Michel C. Jeruchim,et al.  Techniques for Estimating the Bit Error Rate in the Simulation of Digital Communication Systems , 1984, IEEE J. Sel. Areas Commun..

[10]  Richard G. Hamlet,et al.  Partition Testing Does Not Inspire Confidence , 1990, IEEE Trans. Software Eng..

[11]  John D. Musa,et al.  Operational profiles in software-reliability engineering , 1993, IEEE Software.

[12]  Mansoor Shafi,et al.  Quick Simulation: A Review of Importance Sampling Techniques in Communications Systems , 1997, IEEE J. Sel. Areas Commun..