Pinpoint Minimal Failure-Inducing Mode using Itemset Mining under Constraints

A minimal failure-inducing mode (MFM) based on a t-way combinatorial test set and its test results can help programmers identify root causes of failures that are triggered by combination bugs. However, an MFM for systems containing many parameters may be affected by masking effects to result in coincidences correct in practice, which makes pinpointing MFS more difficult. An approach for pinpointing MFM and an iterative framework are proposed. The identifying MFM approach first collects combinatorial test cases and their testing results, then mines the frequent itemset (suspicious MFM) in failed test cases, and finally computes suspiciousness for each MFM belonged to close pattern via contrasting frequency in failed test cases and successful test cases. Through the iterative framework, MFM is pinpointed until a certain stopping criterion is satisfied. Preliminary results of simulation experiments show that this approach is effective.

[1]  Jeff Yu Lei,et al.  Identifying Failure-Inducing Combinations Using Tuple Relationship , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops.

[2]  Hareton K. N. Leung,et al.  The Minimal Failure-Causing Schema of Combinatorial Testing , 2011, TSEM.

[3]  Yu Lei,et al.  Applying combinatorial test data generation to big data applications , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[4]  Xu Bao,et al.  A Software Failure Debugging Method Based on Combinatorial Design Approach for Testing , 2006 .

[5]  Jeff Yu Lei,et al.  Applying Combinatorial Testing to Data Mining Algorithms , 2017, 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[6]  Luc De Raedt,et al.  Itemset mining: A constraint programming perspective , 2011, Artif. Intell..

[7]  R. Suganya,et al.  Data Mining Concepts and Techniques , 2010 .

[8]  Jian Zhang,et al.  Characterizing failure-causing parameter interactions by adaptive testing , 2011, ISSTA '11.

[9]  Raghu Kacker,et al.  Identifying Failure-Inducing Combinations in a Combinatorial Test Set , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[10]  A. Jefferson Offutt,et al.  Test Oracle Strategies for Model-Based Testing , 2017, IEEE Transactions on Software Engineering.

[11]  Jeff Yu Lei,et al.  Evaluating the Effectiveness of BEN in Localizing Different Types of Software Fault , 2016, 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[12]  Jeff Yu Lei,et al.  BEN: A combinatorial testing-based fault localization tool , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[13]  Myra B. Cohen,et al.  Feedback driven adaptive combinatorial testing , 2011, ISSTA '11.