Locating Faulty Code Using Failure-Causing Input Combinations in Combinatorial Testing

The high cost of locating faults in programs has motivated the development of techniques that assist in fault localization by automating part of the process of searching for faults. A method to automatically locate faulty code using failure-causing input combination in combinatorial testing is firstly presented in this paper. Firstly, four types of fault locations are figured out by using parameters in the combination to understand the context of the fault. Secondly, based on the four types of fault locations, an algorithm is proposed to isolate candidate faulty code and obtain cause-effect chains of the failure by means of changing values of parameters in the combination. Finally, experiments are carried out on 92 variations of four programs using the known measurement which is improved in search order and search space of code. Results show that the user could avoid searching over 77% of the program in order to localize faults at code-level. Our method has successfully extended the original method of rooting the fault via combinatorial testing from the input-level to the code-level. It could be easily applicable and broadly practicable in combinatorial testing, and greatly narrow down the search focus on code-level faults.

[1]  Thomas W. Reps,et al.  The use of program dependence graphs in software engineering , 1992, International Conference on Software Engineering.

[2]  Myra B. Cohen,et al.  Covering Arrays for Efficient Fault Characterization in Complex Configuration Spaces , 2006, IEEE Trans. Software Eng..

[3]  Xiangyu Zhang,et al.  A study of effectiveness of dynamic slicing in locating real faults , 2006, Empirical Software Engineering.

[4]  Rajiv Gupta,et al.  Fault localization using value replacement , 2008, ISSTA '08.

[5]  Frank Tip,et al.  A survey of program slicing techniques , 1994, J. Program. Lang..

[6]  Vikram S. Adve,et al.  Using likely invariants for automated software fault localization , 2013, ASPLOS '13.

[7]  Lee Naish,et al.  A model for spectra-based software diagnosis , 2011, TSEM.

[8]  Hareton K. N. Leung,et al.  The Minimal Failure-Causing Schema of Combinatorial Testing , 2011, TSEM.

[9]  Jeff Yu Lei,et al.  Isolating Failure-Inducing Combinations in Combinatorial Testing Using Test Augmentation and Classification , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[10]  Raghu Kacker,et al.  Identifying Failure-Inducing Combinations in a Combinatorial Test Set , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[11]  Andreas Zeller,et al.  Locating causes of program failures , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[12]  Steven P. Reiss,et al.  Fault localization with nearest neighbor queries , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[13]  Myra B. Cohen,et al.  Covering arrays for efficient fault characterization in complex configuration spaces , 2004, IEEE Transactions on Software Engineering.

[14]  Auri Marcelo Rizzo Vincenzi,et al.  Toward the determination of sufficient mutant operators for C † , 2001, Softw. Test. Verification Reliab..

[15]  Yuanyuan Zhou,et al.  Triage: diagnosing production run failures at the user's site , 2007, SOSP.

[16]  Andreas Zeller,et al.  Simplifying and Isolating Failure-Inducing Input , 2002, IEEE Trans. Software Eng..

[17]  Lucia Moura,et al.  Locating Errors Using ELAs, Covering Arrays, and Adaptive Testing Algorithms , 2009, SIAM J. Discret. Math..

[18]  Xiangyu Zhang,et al.  Locating faulty code using failure-inducing chops , 2005, ASE.

[19]  A. Zeller.  Isolating cause-effect chains from computer programs , 2002, SIGSOFT '02/FSE-10.