Constraints: The Future of Combinatorial Interaction Testing

Combinatorial Interaction Testing (CIT) has gained a lot of attention in the area of software engineering in the last few years. CIT problems have their roots in combinatorics. Mathematicians have been concerned with the NP-complete problem of finding minimal covering arrays (in other words, minimal CIT test suites) since early nineties. With the adoption of these techniques into the area of software testing, an important gap has been identified - namely consideration of real-world constraints. We show that indeed finding an efficient way of handling constraints during search is the key factor in wider applicability of CIT techniques.

[1]  Myra B. Cohen,et al.  Configuration-aware regression testing: an empirical study of sampling and prioritization , 2008, ISSTA '08.

[2]  Hareton K. N. Leung,et al.  A survey of combinatorial testing , 2011, CSUR.

[3]  Myra B. Cohen,et al.  Evaluating improvements to a meta-heuristic search for constrained interaction testing , 2011, Empirical Software Engineering.

[4]  Myra B. Cohen,et al.  Efficiency and early fault detection with lower and higher strength combinatorial interaction testing , 2013, ESEC/FSE 2013.

[5]  Akif Günes Koru,et al.  Prioritizing User-Session-Based Test Cases for Web Applications Testing , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[6]  Katsumi Inoue,et al.  Generating Combinatorial Test Cases by Efficient SAT Encodings Suitable for CDCL SAT Solvers , 2010, LPAR.

[7]  Myra B. Cohen,et al.  Covering arrays for efficient fault characterization in complex configuration spaces , 2004, IEEE Transactions on Software Engineering.

[8]  D. Richard Kuhn,et al.  Pseudo-Exhaustive Testing for Software , 2006, 2006 30th Annual IEEE/NASA Software Engineering Workshop.

[9]  Charles J. Colbourn,et al.  Prioritized interaction testing for pair-wise coverage with seeding and constraints , 2006, Inf. Softw. Technol..

[10]  Michael L. Fredman,et al.  The AETG System: An Approach to Testing Based on Combinatiorial Design , 1997, IEEE Trans. Software Eng..

[11]  Steven David Prestwich,et al.  Constraint Models for the Covering Test Problem , 2006, Constraints.

[12]  Myra B. Cohen,et al.  Constructing Interaction Test Suites for Highly-Configurable Systems in the Presence of Constraints: A Greedy Approach , 2008, IEEE Transactions on Software Engineering.

[13]  Tatsuhiro Tsuchiya,et al.  Using artificial life techniques to generate test cases for combinatorial testing , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[14]  Jeff Yu Lei,et al.  IPOG/IPOG‐D: efficient test generation for multi‐way combinatorial testing , 2008, Softw. Test. Verification Reliab..

[15]  Myra B. Cohen,et al.  Interaction testing of highly-configurable systems in the presence of constraints , 2007, ISSTA '07.

[16]  D. Richard Kuhn,et al.  Software fault interactions and implications for software testing , 2004, IEEE Transactions on Software Engineering.

[17]  Myra B. Cohen,et al.  Constructing test suites for interaction testing , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[18]  Yvan Labiche,et al.  An Orchestrated Survey of Available Algorithms and Tools for Combinatorial Testing , 2014, 2014 IEEE 25th International Symposium on Software Reliability Engineering.

[19]  Myra B. Cohen,et al.  Combinatorial Interaction Regression Testing: A Study of Test Case Generation and Prioritization , 2007, 2007 IEEE International Conference on Software Maintenance.

[20]  Jacques Klein,et al.  Bypassing the Combinatorial Explosion: Using Similarity to Generate and Prioritize T-wise Test Suites for Large Software Product Lines , 2012, ArXiv.