Understanding the Contribution of Non-source Documents in Improving Missing Link Recovery: An Empirical Study

Background: Links between issue reports and their fixing commits play an important role in software maintenance. Such link data are often missing in practice and many approaches have been proposed in order to recover them automatically. Most of existing approaches focus on comparing log messages and source code files in commits with issues reports. Besides the two kinds of data in commits, non-source documents (NSDs) such as change logs usually record the fixing activities and sometimes share similar texts as those in issue reports. However, few discussions have been made on the role of NSDs in designing link recovery approaches. Aims: This paper aims at understanding whether and how NSDs affect the performance of link recovery approaches. Method: An empirical study is conducted to evaluate the role of NSDs in link recovery approaches in 18 open source projects with 6370 issues and 22761 commits. Results: With the inclusion of NSDs, link recovery approaches can get an average increase in F-Measure ranging from 2.76% - 25.63%. Further examinations show NSDs contribute to the performance improvement in 15 projects and have exceptions in 3 projects. The performance improvement in the 15 projects is mainly from the filtering of noisy links. On average, 23.59% - 76.30% false links can be excluded by exploiting NSDs in the link recovery approach. We also analyze the 3 projects in which NSDs cannot improve the performance. Our finding shows sophisticated data selection in NSDs is necessary. Conclusions: Our preliminary findings demonstrate that involving NSDs can improve the performance of link recovery approaches in most cases.