Systems testing and statistical test data coverage

Coverage is often used to determine when unit modules have been sufficiently tested. It is difficult to apply the same approach to whole systems. An additional problem is that coverage has no obvious reliability interpretation. A form of coverage called statistical coverage is described that solves some of the problems of systems level coverage and which also has a reliability interpretation. Statistical coverage is based on ideas from statistical testing. Statistical testing is reviewed and new results are introduced that are needed for statistical coverage, and which can also be applied to statistical testing.

[1]  William E. Howden Keynote Address: Auditing Informal Software Testing and Analysis Processes , 1996, APSEC.

[2]  Amrit L. Goel,et al.  Time-Dependent Error-Detection Rate Model for Software Reliability and Other Performance Measures , 1979, IEEE Transactions on Reliability.

[3]  David Lorge Parnas,et al.  Evaluation of safety-critical software , 1990, CACM.

[4]  Simeon C. Ntafos,et al.  An Evaluation of Random Testing , 1984, IEEE Transactions on Software Engineering.

[5]  Jean-Claude Laprie,et al.  The KAT (Knowledge-Action-Transformation) Approach to the Modeling and Evaluation of Reliability and Availability Growth , 1991, IEEE Trans. Software Eng..

[6]  Denise M. Woit Estimating software reliability with hypothesis testing , 1993 .

[7]  Elaine J. Weyuker,et al.  Reliability Testing of Rule-Based Systems , 1996, IEEE Softw..

[8]  William E. Howden,et al.  Functional program testing and analysis , 1986 .

[9]  James A. Whittaker,et al.  A Markov Chain Model for Statistical Software Testing , 1994, IEEE Trans. Software Eng..

[10]  Simeon C. Ntafos,et al.  On some reliability estimation problems in random and partition testing , 1991, Proceedings. 1991 International Symposium on Software Reliability Engineering.

[11]  Jeffrey M. Voas,et al.  Improving the software development process using testability research , 1992, [1992] Proceedings Third International Symposium on Software Reliability Engineering.

[12]  Elaine J. Weyuker,et al.  An Applicable Family of Data Flow Testing Criteria , 1988, IEEE Trans. Software Eng..

[13]  Richard G. Hamlet,et al.  Partition Testing Does Not Inspire Confidence , 1990, IEEE Trans. Software Eng..

[14]  Jesse H. Poore,et al.  Planning and certifying software system reliability , 1993, IEEE Software.