A study of effectiveness of deep learning in locating real faults

Abstract Context: The recent progress of deep learning has shown its promising learning ability in making sense of data, and many fields have utilized this learning ability to learn an effective model, successfully solving their problems. Fault localization has explored and used deep learning to server an aid in debugging, showing the promising results on fault localization. However, as far as we know, there is no detailed studies on evaluating the benefits of using deep learning for locating real faults present in programs. Objective: To understand the benefits of deep learning in locating real faults, this paper explores more about deep learning by studying the effectiveness of fault localization using deep learning for a set of real bugs reported in the widely used programs. Method: We use three representative deep learning architectures (i.e. convolutional neural network, recurrent neural network and multi-layer perceptron) for fault localization, and conduct large-scale experiments on 8 real-world programs equipped with all real faults to evaluate their effectiveness on fault localization. Results: We observe that the localization effectiveness varies considerably among three neural networks in the context of real faults. Specifically, convolutional neural network performs the best in locating real faults, showing an average of 38.97% and 26.22% saving over multi-layer perceptron and recurrent neural network respectively; recurrent neural network and multi-layer perceptron yield comparable effectiveness even if the effectiveness of recurrent neural network is marginally higher than multi-layer perceptron. Conclusion: In context of real faults, convolutional neural network is the most effective for fault localization among the investigated architectures, and we suggest potential factors of deep learning for improving fault localization.