On Statistical Structural Testing of Synchronous Data Flow Programs

The paper addresses the issue of testing programs written in the synchronous data flow language LUSTRE. We define a mixed strategy which combines statistical testing and deterministic extremal values testing. Statistical testing involves exercising a program with random inputs, the input distribution and the number of test data being determined according to test criteria. Three criteria based on the finite state automaton produced by the LUSTRE compiler are studied, the feasibility of the method for designing test sets according to them being exemplified on a real-case study. Then, mutation analysis (here, specific to LUSTRE) is used to assess the efficiency of the test sets. The results allow us (i) to define the most cost-effective criterion for designing efficient statistical test sets of reasonable size, and (ii) to show the high fault revealing power of the corresponding mixed strategy, killing the whole set of 310 mutants involved in the experiments.

[1]  Nicolas Halbwachs,et al.  LUSTRE: a declarative language for real-time programming , 1987, POPL '87.

[2]  Yves Crouzet,et al.  An experimental study on software structural testing: deterministic versus random input generation , 1991, [1991] Digest of Papers. Fault-Tolerant Computing: The Twenty-First International Symposium.

[3]  Frédéric Boussinot,et al.  The ESTEREL language , 1991, Proc. IEEE.

[4]  Ferhat Khendek,et al.  Test Selection Based on Finite State Models , 1991, IEEE Trans. Software Eng..

[5]  Nicolas Halbwachs,et al.  Programming and Verifying Real-Time Systems by Means of the Synchronous Data-Flow Language LUSTRE , 1992, IEEE Trans. Software Eng..

[6]  Nicolas Halbwachs,et al.  LUSTRE: A declarative language for programming synchronous systems* , 1987 .

[7]  Simeon C. Ntafos,et al.  A Comparison of Some Structural Testing Strategies , 1988, IEEE Trans. Software Eng..

[8]  Boris Beizer,et al.  Software Testing Techniques , 1983 .

[9]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[10]  Simeon C. Ntafos,et al.  An Evaluation of Random Testing , 1984, IEEE Transactions on Software Engineering.

[11]  Elaine J. Weyuker,et al.  Selecting Software Test Data Using Data Flow Information , 1985, IEEE Transactions on Software Engineering.

[12]  Albert Benveniste,et al.  The synchronous approach to reactive and real-time systems , 1991 .

[13]  Hélène Waeselynck.  Vérification de logiciels critiques par le test statistique , 1993 .

[14]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[15]  Hélène Waeselynck,et al.  On Functional Statistical Testing Designed from Software Behavior Models , 1993 .

[16]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[17]  Richard G. Hamlet.  Theoretical comparison of testing methods , 1989 .

[18]  Pascal Raymond,et al.  The synchronous data flow programming language LUSTRE , 1991, Proc. IEEE.

[19]  Richard A DeMillo.  Mutation Analysis as a Tool for Software Quality Assurance. , 1980 .

[20]  Simeon C. Ntafos,et al.  ON TESTING WITH REQUIRED ELEMENTS. , 1981 .

[21]  Edward A. Feigenbaum,et al.  Switching and Finite Automata Theory: Computer Science Series , 1990 .

[22]  Pascale Thévenod-Fosse.  Software Validation by Means of Statistical Testing: Retrospect and Future Direction , 1991 .

[23]  Hélène Waeselynck,et al.  STATEMATE applied to statistical software testing , 1993, ISSTA '93.

[24]  Hasan Ural,et al.  Formal methods for test sequence generation , 1992, Comput. Commun..