A Framework for Evaluating Regression Test Selection Techniques in Industry

Previous research in the area of regression testing has mainly focused on different techniques used to decrease the size of test suites. However, studies that compare the techniques in authentic industrial contexts are few. Aim: The aim of this paper is to introduce an efficient, purposeful framework meant to evaluate regression test selection techniques using only a limited selection of available information. Method: In order to evaluate and compare different regression testing techniques three realistic and important scenarios were recognized and a framework was developed. This was then utilized as a starting point for an evaluation case study which compared regression test selection techniques. Regression test data was collected from a software developing site within Ericsson. Results: The framework evaluation showed that a well-supported decision could be made regarding which regression testing technique a software development organization should use. The comparative case study also showed that, compared to a random selection, a technique based on historical test data improved the fail detection. Conclusions: The contribution of this paper is the framework which can be used as a basis for further research as well as aid practitioners in the analysis and evaluation of regression test selection techniques.