Is Learning-to-Rank Cost-Effective in Recommending Relevant Files for Bug Localization?

Software bug localization aiming to determine the locations needed to be fixed for a bug report is one of the most tedious and effort consuming activities in software debugging. Learning-to-rank (LR) is the state-of-the-art approach proposed by Ye et al. to recommending relevant files for bug localization. Ye et al.'s experimental results show that the LR approach significantly outperforms previous bug localization approaches in terms of "precision" and "accuracy". However, this evaluation does not take into account the influence of the size of the recommended files on the efficiency in detecting bugs. In practice, developers will generally spend more code inspection effort to detect bugs if larger files are recommended. In this paper, we use six large-scale open-source Java projects to evaluate the LR approach in the context of effort-aware bug localization. Our results, surprisingly, show that, when taking into account the code inspection effort to detect bugs, the LR approach is similar to or even worse than the standard VSM (Vector Space Model), a naïve IR-based bug localization approach.