STATEMATE applied to statistical software testing

This paper is concerned with the use of statistical testing as a verification technique for complex software. Statistical testing involves exercising a program with random inputs, the test profile and the number of generated inputs being determined according to criteria based on program structure or software functionality. In case of complex programs, the probabilistic generation must be based on a black box analysis, the adopted criteria being defined from behavior models deduced from the specification. The proposed approach refers to a hierarchical specification produced in the STATEMATE environment. Its feasiblity is exemplified on a safety-critical module from the nuclear field, and the efficiency in revealing actual faults is investigated through experiments involving two versions of the module.

[1]  Hélène Waeselynck,et al.  An investigation of statistical software testing , 1991, Softw. Test. Verification Reliab..

[2]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[3]  Robert F. Roggio,et al.  An automated oracle for software testing , 1992 .

[4]  Elaine J. Weyuker,et al.  Selecting Software Test Data Using Data Flow Information , 1985, IEEE Transactions on Software Engineering.

[5]  Richard G. Hamlet.  Theoretical comparison of testing methods , 1989 .

[6]  Pascale Thévenod-Fosse.  Software Validation by Means of Statistical Testing: Retrospect and Future Direction , 1991 .

[7]  Hélène Waeselynck.  Vérification de logiciels critiques par le test statistique , 1993 .

[8]  Elaine J. Weyuker,et al.  Analyzing Partition Testing Strategies , 1991, IEEE Trans. Software Eng..

[9]  R. Taylor,et al.  Partition testing does not inspire confidence , 1988, [1988] Proceedings. Second Workshop on Software Testing, Verification, and Analysis.

[10]  Bruno Marre,et al.  Toward Automatic Test Data Set Selection Using Algebraic Specifications and Logic Programming , 1991, ICLP.

[11]  William E. Howden,et al.  Functional program testing and analysis , 1986 .

[12]  Simeon C. Ntafos,et al.  An Evaluation of Random Testing , 1984, IEEE Transactions on Software Engineering.

[13]  Amir Pnueli,et al.  What is in a Step: On the Semantics of Statecharts , 1991, TACS.

[14]  Boris Beizer,et al.  Software testing techniques (2. ed.) , 1990 .

[15]  Elaine J. Weyuker,et al.  Assessing the fault-detecting ability of testing methods , 1991, SIGSOFT '91.

[16]  Richard G. Hamlet,et al.  Theoretical comparison of testing methods , 1989, TAV3.

[17]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[18]  Amnon Naamad,et al.  Statemate: a working environment for the development of complex reactive systems , 1988, ICSE '88.

[19]  Simeon C. Ntafos,et al.  A Comparison of Some Structural Testing Strategies , 1988, IEEE Trans. Software Eng..

[20]  Boris Beizer,et al.  Software Testing Techniques , 1983 .

[21]  David Harel,et al.  Statecharts: A Visual Formalism for Complex Systems , 1987, Sci. Comput. Program..

[22]  Hélène Waeselynck,et al.  On Functional Statistical Testing Designed from Software Behavior Models , 1993 .

[23]  Yves Crouzet,et al.  An experimental study on software structural testing: deterministic versus random input generation , 1991, [1991] Digest of Papers. Fault-Tolerant Computing: The Twenty-First International Symposium.

[24]  Elaine J. Weyuker,et al.  On Testing Non-Testable Programs , 1982, Comput. J..