Software dependability evaluation based on Markov usage models

Abstract A general technique for computing optimal state transition probabilities for software tests, based on a Markov usage model, is developed. The optimization criterion is maximum precision of unbiased dependability estimates derived from the test results. Three different dependability measures are considered: (i) risk, (ii) safety, and (iii) reliability. As input, pre-information on failure probabilities and losses in case of failure related with single operations is used. The optimization itself is done by means of a numerical procedure which is fast because of the convexity of the underlying stochastic optimization problem. The procedure can be improved by the construction of a distribution with a common lower bound on state transition probabilities; this distribution may also be used in the more general context of structural statistical testing of software.

[1]  Elaine J. Weyuker,et al.  Theories of Program Testing and the Application of Revealing Subdomains , 1980, IEEE Transactions on Software Engineering.

[2]  Richard G. Hamlet,et al.  Partition Testing Does Not Inspire Confidence , 1990, IEEE Trans. Software Eng..

[3]  Jesse H. Poore,et al.  Markov analysis of software specifications , 1993, TSEM.

[4]  R. Taylor,et al.  Partition testing does not inspire confidence , 1988, [1988] Proceedings. Second Workshop on Software Testing, Verification, and Analysis.

[5]  Reuven Y. Rubinstein,et al.  Simulation and the Monte Carlo method , 1981, Wiley series in probability and mathematical statistics.

[6]  James A. Whittaker.  Markov Chain Techniques for Software Testing and Reliability Analysis, PhD Dissertation , 1992 .

[7]  Susan A. Sherer,et al.  Software Failure Risk , 1992, Applications of Modern Technology in Business.

[8]  Walter J. Gutjahr,et al.  Importance Sampling of Test Cases in Markovian Software Usage Models , 1997, Probability in the Engineering and Informational Sciences.

[9]  D MusaJohn.  Operational Profiles in Software-Reliability Engineering , 1993 .

[10]  Walter J. Gutjahr.  Optimal Test Distributions for Software Failure Cost Estimation , 1995, IEEE Trans. Software Eng..

[11]  HeidelbergerPhilip.  Fast simulation of rare events in queueing and reliability models , 1995 .

[12]  W. D. Ehrenberger.  COMBINING PROBABILISTIC AND DETERMINISTIC VERIFICATION EFFORTS , 1992 .

[13]  Philip Heidelberger,et al.  Fast simulation of rare events in queueing and reliability models , 1993, TOMC.

[14]  Denise M. Woit.  A framework for reliability estimation , 1994, Proceedings of 1994 IEEE International Symposium on Software Reliability Engineering.

[15]  Yves Crouzet,et al.  Software Statistical Testing , 1995 .

[16]  Kishor S. Trivedi,et al.  Composite Performance and Dependability Analysis , 1992, Perform. Evaluation.

[17]  G. Pflug,et al.  Stochastic approximation and optimization of random systems , 1992 .

[18]  Jean-Claude Laprie,et al.  Dependability — Its Attributes, Impairments and Means , 1995 .

[19]  Dave E. Eckhardt,et al.  A Theoretical Basis for the Analysis of Multiversion Software Subject to Coincident Errors , 1985, IEEE Transactions on Software Engineering.

[20]  Elaine J. Weyuker.  Using failure cost information for testing and reliability assessment , 1996, TSEM.

[21]  Susan A. Sherer.  Statistical software testing using economic exposure assessments , 1996, Softw. Eng. J..

[22]  Donald L. Iglehart,et al.  Importance sampling for stochastic simulations , 1989 .

[23]  Denise M. Woit.  Operational profile specification, test case generation, and reliability estimation for modules , 1994 .

[24]  P. Thevenod-Fosse,et al.  From random testing of hardware to statistical testing of software , 1991, [1991] Proceedings, Advanced Computer Technology, Reliable Systems and Applications.

[25]  John D. Musa,et al.  Operational profiles in software-reliability engineering , 1993, IEEE Software.

[26]  Perwez Shahabuddin,et al.  Fast Transient Simulation of Markovian Models of Highly Dependable Systems , 1994, Perform. Evaluation.

[27]  Walter J. Gutjahr.  Failure Risk Estimation via Markov Software Usage Models , 1996, SAFECOMP.

[28]  Elaine J. Weyuker,et al.  The Automatic Generation of Load Test Suites and the Assessment of the Resulting Software , 1995, IEEE Trans. Software Eng..

[29]  Lorenzo Strigini,et al.  Acceptance Criteria for Critical Software Based on Testability Estimates and Test Results , 1996, SAFECOMP.

[30]  Jesse H. Poore,et al.  Statistical testing of software based on a usage model , 1995, Softw. Pract. Exp..