On the Value of Bug Reports for Retrieval-Based Bug Localization

Software engineering researchers have been applying tools and techniques from information retrieval (IR) to problems such as bug localization to lower the manual effort required to perform maintenance tasks. The central challenge when using an IR-based tool is the formation of a high-quality query. When performing bug localization, one easily accessible source of query words is the bug report. A recent paper investigated the sufficiency of this source by using a genetic algorithm (GA) to build high quality queries. Unfortunately, the GA in essence "cheats" as it makes use of query performance when evolving a good query. This raises the question, is it feasible to attain similar results without "cheating?" One approach to providing cheat-free queries is to employ automatic summarization. The performance of the resulting summaries calls into question the sufficiency of the bug reports as a source of query words. To better understand the situation, Information Need Analysis (INA) is applied to quantify both how well the GA is performing and, perhaps more importantly, how well a bug report captures the vocabulary needed to perform IR-based bug localization. The results find that summarization shows potential to produce high-quality queries, but it requires more training data. Furthermore, while bug reports provide a useful source of query words, they are rather limited and thus query expansion techniques, perhaps in combination with summarization, will likely produce higher-quality queries.

[1]  Alessandro Orso,et al.  Evaluating the usefulness of IR-based fault localization techniques , 2015, ISSTA.

[2]  Bogdan Dit,et al.  Feature location in source code: a taxonomy and survey , 2013, J. Softw. Evol. Process..

[3]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[4]  Christopher D. Manning,et al.  Get To The Point: Summarization with Pointer-Generator Networks , 2017, ACL.

[5]  Sarfraz Khurshid,et al.  An empirical study of long lived bugs , 2014, 2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering (CSMR-WCRE).

[6]  Filip Zivkovic TrumpBot: Seq2Seq with Pointer Sentinel Model , 2017 .

[7]  Dawn J. Lawrie,et al.  The need for software specific natural language techniques , 2017, Empirical Software Engineering.

[8]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[9]  Jian Zhou,et al.  Where should the bugs be fixed? More accurate information retrieval-based bug localization based on bug reports , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[10]  Gabriele Bavota,et al.  Are Bug Reports Enough for Text Retrieval-Based Bug Localization? , 2018, 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME).