Packaging experiences for improving testing technique selection

One of the major problems within the software testing area is how to get a suitable set of cases to test a software system. A good set of test cases should assure maximum effectiveness with as few cases as possible. There are now numerous testing techniques available for generating test cases. However, many are never used, while just a few are used over and over again. Testers use little (if any) information about the available techniques, their usefulness and, generally, how suited they are to the project at hand, upon which to base their decision on which testing techniques to use. Using a characterisation schema is one solution for improving testing techniques selection. The schema helps to choose the best-suited techniques for a given project based on relevant information for the purpose of selection, assuring that testers’ selections are systematic. However, a characterisation schema is only part of the solution. We have found that a critical aspect for making a good selection is the availability of the necessary information and the sources of information that have to be consulted to access this information. Any organisation wishing to use characterisation schemas to select SE techniques needs to first address the issue of packaging the information that the schema contains.

[1]  Mary Jean Harrold,et al.  Testing: a roadmap , 2000, ICSE '00.

[2]  A. Jefferson Offutt,et al.  An Empirical Evaluation of Weak Mutation , 1994, IEEE Trans. Software Eng..

[3]  Sira Vegas,et al.  A Characterisation Schema for Software Testing Techniques , 2005, Empirical Software Engineering.

[4]  Thomas J. Ostrand,et al.  Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria , 1994, Proceedings of 16th International Conference on Software Engineering.

[5]  Gregory Tassey,et al.  Prepared for what , 2007 .

[6]  Sigrid Eldh Software Testing Techniques , 2007 .

[7]  W. Eric Wong,et al.  Fault detection effectiveness of mutation and data flow testing , 1995, Software Quality Journal.

[8]  Phyllis G. Frankl,et al.  Further empirical studies of test effectiveness , 1998, SIGSOFT '98/FSE-6.

[9]  James Miller,et al.  Comparing and combining software defect detection techniques: a replicated empirical study , 1997, ESEC '97/FSE-5.

[10]  Phyllis G. Frankl,et al.  All-uses vs mutation testing: An experimental comparison of effectiveness , 1997, J. Syst. Softw..

[11]  Phyllis G. Frankl,et al.  An Experimental Comparison of the Effectiveness of Branch Testing and Data Flow Testing , 1993, IEEE Trans. Software Eng..

[12]  Gregg Rothermel,et al.  An experimental determination of sufficient mutant operators , 1996, TSEM.

[13]  Erik Kamsties,et al.  An Empirical Evaluation of Three Defect-Detection Techniques , 1995, ESEC.

[14]  Will Venters,et al.  Software engineering: theory and practice , 2006 .

[15]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[16]  Natalia Juristo Juzgado,et al.  Identifying relevant information for testing technique selection - an instatiated characterization schema , 2003, International series in software engineering.

[17]  Victor R. Basili,et al.  Comparing the Effectiveness of Software Testing Strategies , 1987, IEEE Transactions on Software Engineering.