A framework of greedy methods for constructing interaction test suites

Greedy algorithms for the construction of software interaction test suites are studied. A framework is developed to evaluate a large class of greedy methods that build suites one test at a time. Within this framework are many instantiations of greedy methods generalizing those in the literature. Greedy algorithms are popular when the time for test suite construction is of paramount concern. We focus on the size of the test suite produced by each instantiation. Experiments are analyzed using statistical techniques to determine the importance of the implementation decisions within the framework. This framework provides a platform for optimizing the accuracy and speed of "one-test-at-a-time" greedy methods.

[1]  Myra B. Cohen,et al.  Covering arrays for efficient fault characterization in complex configuration spaces , 2004, IEEE Transactions on Software Engineering.

[2]  C. Colbourn.  Combinatorial aspects of covering arrays , 2006 .

[3]  A. Hartman.  Software and Hardware Testing Using Combinatorial Covering Suites , 2005 .

[4]  Alan Hartman,et al.  Problems and algorithms for covering arrays , 2004, Discret. Math..

[5]  D. Richard Kuhn,et al.  Software fault interactions and implications for software testing , 2004, IEEE Transactions on Software Engineering.

[6]  Kari J. Nurmela,et al.  Upper bounds for covering arrays by tabu search , 2004, Discret. Appl. Math..

[7]  Myra B. Cohen,et al.  A deterministic density algorithm for pairwise interaction coverage , 2004, IASTED Conf. on Software Engineering.

[8]  Myra B. Cohen,et al.  Designing Test Suites for Software Interactions Testing , 2004 .

[9]  Adrian Dumitrescu,et al.  Efficient Algorithms for Generation of Combinatorial Covering Suites , 2003, ISAAC.

[10]  Christine T. Cheng,et al.  Generating small combinatorial test suites to cover input-output relationships , 2003, Third International Conference on Quality Software, 2003. Proceedings..

[11]  Eric R. Ziegel,et al.  Experimental Design for Combinatorial and High Throughput Materials Development , 2003, Technometrics.

[12]  Myra B. Cohen,et al.  Constructing test suites for interaction testing , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[13]  M. J. Reilly,et al.  An investigation of the applicability of design of experiments to software testing , 2002, 27th Annual NASA Goddard/IEEE Software Engineering Workshop, 2002. Proceedings..

[14]  Tatsuhiro Tsuchiya,et al.  A new method for constructing pair-wise covering designs for software testing , 2002, Information Processing Letters.

[15]  Yu Lei,et al.  A Test Generation Strategy for Pairwise Testing , 2002, IEEE Trans. Software Eng..

[16]  D E Shasha,et al.  Using combinatorial design to study regulation by multiple input signals. A tool for parsimony in the post-genomics era. , 2001, Plant physiology.

[17]  Robert L. Probert,et al.  A measure for component interaction test coverage , 2001, Proceedings ACS/IEEE International Conference on Computer Systems and Applications.

[18]  Alan W. Williams,et al.  Determination of Test Configurations for Pair-Wise Interaction Coverage , 2000, TestCom.

[19]  Yu-Wen Tung,et al.  Automating test case generation for the new generation mission software system , 2000, 2000 IEEE Aerospace Conference. Proceedings (Cat. No.00TH8484).

[20]  Paul Denny,et al.  Case studies and new results in combinatorial enumeration , 2000 .

[21]  K. Burr,et al.  Combinatorial Test Techniques : Table-based Automation , Test Generation and Code Coverage , 1998 .

[22]  Michael L. Fredman,et al.  The AETG System: An Approach to Testing Based on Combinatiorial Design , 1997, IEEE Trans. Software Eng..

[23]  C. L. Mallows,et al.  Applying Design of Experiments to Software Testing , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[24]  C. L. Mallows,et al.  Applying design of experiments to software testing: experience report , 1997, ICSE '97.

[25]  Robert L. Probert,et al.  A practical strategy for testing pair-wise coverage of network interfaces , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[26]  D.M. Cohen,et al.  The Combinatorial Design Approach to Automatic Test Generation , 1996, IEEE Softw..

[27]  Rory A. Fisher,et al.  The Arrangement of Field Experiments , 1992 .

[28]  N. J. A. Sloane,et al.  Lexicographic codes: Error-correcting codes from game theory , 1986, IEEE Trans. Inf. Theory.

[29]  Peter Blundell Gibbons,et al.  Computing techniques for the construction and analysis of block designs. , 1976 .

[30]  Mark B. Wells,et al.  Elements of Combinatorial Computing , 2014 .