Cost-effective testing based fault localization with distance based test-suite reduction

The aim of testing based fault localization (TBFL) involves improving the efficiency of program debugging by providing developers with a guide of ranked list of suspicious statements. However, collection of testing information of the whole original test-suite is excessively expensive or even infeasible for developers to conduct TBFL. Traditional test-suite reduction (TSR) techniques are utilized to reduce the size of test-suite. However, they entail a time-consuming process of whole testing information collection. In this study, the distance based test-suite reduction (DTSR) technique is proposed. As opposed to the whole testing information, the distances among the test cases are used to guide the process of test-suite reduction in DTSR. Hence, it is only necessary to collect the testing information for a portion of the test cases for TSR and TBFL. The investigation on the Siemens and SIR benchmarks reveals that DTSR can effectively reduce the size of the given test-suite as well as the time cost of TBFL. Additionally, the fault locating effectiveness of DTSR results is close to that when the whole test-suite is used.

[1]  Gregg Rothermel,et al.  Analyzing Regression Test Selection Techniques , 1996, IEEE Trans. Software Eng..

[2]  Mary Jean Harrold,et al.  An empirical study of the effects of test-suite reduction on fault localization , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[3]  John T. Stasko,et al.  Visualization of test information to assist fault localization , 2002, ICSE '02.

[4]  Baowen Xu,et al.  An empirical study on constraint optimization techniques for test generation , 2015, Science China Information Sciences.

[5]  Baowen Xu,et al.  Test Case Prioritization for Web Service Regression Testing , 2010, 2010 Fifth IEEE International Symposium on Service Oriented System Engineering.

[6]  T. H. Tse,et al.  On the adoption of MC/DC and control-flow adequacy for a tight integration of program testing and statistical fault localization , 2013, Inf. Softw. Technol..

[7]  Shujuan Jiang,et al.  Mitigating the Dependence Confounding Effect for Effective Predicate-Based Statistical Fault Localization , 2015, 2015 IEEE 39th Annual Computer Software and Applications Conference.

[8]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[9]  Baowen Xu,et al.  A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization , 2013, TSEM.

[10]  Darko Marinov,et al.  Comparing and combining test-suite reduction and regression test selection , 2015, ESEC/SIGSOFT FSE.

[11]  Tao Xie,et al.  Test input reduction for result inspection to facilitate fault localization , 2010, Automated Software Engineering.

[12]  Kai-Yuan Cai,et al.  Using Partition Information to Prioritize Test Cases for Fault Localization , 2015, 2015 IEEE 39th Annual Computer Software and Applications Conference.

[13]  Shujuan Jiang,et al.  HSFal: Effective fault localization using hybrid spectrum of full slices and execution slices , 2014, J. Syst. Softw..

[14]  Rui Abreu,et al.  A Survey on Software Fault Localization , 2016, IEEE Transactions on Software Engineering.

[15]  T. H. Tse,et al.  Adaptive Random Test Case Prioritization , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[16]  Marc J. Balcer,et al.  The category-partition method for specifying and generating fuctional tests , 1988, CACM.

[17]  Joseph Robert Horgan,et al.  Effect of Test Set Minimization on Fault Detection Effectiveness , 1995, 1995 17th International Conference on Software Engineering.

[18]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[19]  Xiaofang Zhang,et al.  Test Case Prioritization Using Adaptive Random Sequence with Category-Partition-Based Distance , 2016, 2016 IEEE International Conference on Software Quality, Reliability and Security (QRS).

[20]  Tsong Yueh Chen,et al.  On the identification of categories and choices for specification-based test case generation , 2004, Inf. Softw. Technol..

[21]  David Lo,et al.  Diversity maximization speedup for fault localization , 2012, 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.

[22]  Tibor Gyimóthy,et al.  Test suite reduction for fault detection and localization: A combined approach , 2014, 2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering (CSMR-WCRE).

[23]  Gregg Rothermel,et al.  Empirical studies of test‐suite reduction , 2002, Softw. Test. Verification Reliab..

[24]  Nadine Mandran,et al.  Prioritizing test cases with string distances , 2011, Automated Software Engineering.

[25]  Baowen Xu,et al.  Comparing logic coverage criteria on test case prioritization , 2012, Science China Information Sciences.

[26]  Yansheng Lu,et al.  Empirical study of the effects of different profiles on regression test case reduction , 2015, IET Softw..

[27]  Raúl A. Santelices,et al.  Lightweight fault-localization using multiple coverage types , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[28]  Richard W. Hamming,et al.  Error detecting and error correcting codes , 1950 .

[29]  Wes Masri,et al.  Prevalence of coincidental correctness and mitigation of its impact on fault localization , 2014, TSEM.

[30]  Gregg Rothermel,et al.  Empirical Studies of a Safe Regression Test Selection Technique , 1998, IEEE Trans. Software Eng..

[31]  Yves Le Traon,et al.  Improving test suites for efficient fault localization , 2006, ICSE.