Effective Fault Localization using Code Coverage

Localizing a bug in a program can be a complex and time- consuming process. In this paper we propose a code coverage-based fault localization method to prioritize suspicious code in terms of its likelihood of containing program bugs. Code with a higher risk should be examined before that with a lower risk, as the former is more suspicious (i.e., more likely to contain program bugs) than the latter. We also answer a very important question: how can each additional test case that executes the program successfully help locate program bugs? We propose that with respect to a piece of code, the aid introduced by the first successful test that executes it in computing its likelihood of containing a bug is larger than or equal to that of the second successful test that executes it, which is larger than or equal to that of the third successful test that executes it, etc. A tool, chiDebug, was implemented to automate the computation of the risk of the code and the subsequent prioritization of suspicious code for locating program bugs. A case study using the Siemens suite was also conducted. Data collected from our study support the proposal described above. They also indicate that our method (in particular Heuristics III (c), (d), and (e)) can effectively reduce the search domain for locating program bugs.

[1]  31st Annual International Computer Software and Applications Conference, COMPSAC 2007, Beijing, China, July 24-27, 2007. Volume 1 , 2007, Annual International Computer Software and Applications Conference.

[2]  Andreas Zeller,et al.  Simplifying and Isolating Failure-Inducing Input , 2002, IEEE Trans. Software Eng..

[3]  Yu Qi,et al.  Smart debugging software architectural design in SDL , 2005, J. Syst. Softw..

[4]  A. Zeller Isolating cause-effect chains from computer programs , 2002, SIGSOFT '02/FSE-10.

[5]  Mark Weiser,et al.  Programmers use slices when debugging , 1982, CACM.

[6]  Sudipto Ghosh,et al.  Mining System Tests to Aid Software Maintenance , 1998, Computer.

[7]  Joseph Robert Horgan,et al.  Fault localization using execution slices and dataflow tests , 1995, Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95.

[8]  Yu Qi,et al.  Effective program debugging based on execution slices and inter-block data dependency , 2006, J. Syst. Softw..

[9]  Mary Jean Harrold,et al.  Empirical evaluation of the tarantula automatic fault-localization technique , 2005, ASE.

[10]  James A. Jones Fault localization using visualization of test information , 2004, Proceedings. 26th International Conference on Software Engineering.

[11]  Chao Liu,et al.  Statistical Debugging: A Hypothesis Testing-Based Approach , 2006, IEEE Transactions on Software Engineering.

[12]  Steven P. Reiss,et al.  Fault localization with nearest neighbor queries , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[13]  Thomas J. Ostrand,et al.  Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria , 1994, Proceedings of 16th International Conference on Software Engineering.

[14]  Dieter Hogrefe,et al.  SDL : formal object-oriented language for communicating systems , 1997 .

[15]  Eugene H. Spafford,et al.  Debugging with dynamic slicing and backtracking , 1993, Softw. Pract. Exp..

[16]  H. Cleve,et al.  Locating causes of program failures , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[17]  W. Eric Wong,et al.  An integrated solution for testing and analyzing Java applications in an industrial setting , 2005, 12th Asia-Pacific Software Engineering Conference (APSEC'05).

[18]  Joseph Robert Horgan,et al.  Dynamic program slicing , 1990, PLDI '90.

[19]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.