Test Case Prioritization Based on Path Complexity

Software undergoes many modifications after its release. Regression testing is performed to ensure that the modification has not introduced any errors in the software and the software continues to work correctly. Regression testing is an expensive process. Three types of cost reduction techniques are used in regression testing. These techniques i.e., test case selection, test suite minimization and test case prioritization are used to reduce the cost of regression testing and improve the rate of fault detection. The focus of our research is on test case prioritization. Instead of minimizing test suite or selecting fewer test cases, test case prioritization orders test cases in such a way that the test cases detecting more faults are executed earlier. In case of limited resources, only top priority test cases are executed to ensure the reliability of the software. In this research, we have proposed an approach which uses path complexity and branch coverage to prioritize test cases based on assumption that the complex code is more likely to contain faults. Halstead's metric has been used to calculate the path complexity of the test cases. Proposed approach is compared with branch coverage based prioritization technique using some example programs. The results show that proposed approach outperforms existing branch coverage based approach in terms of APFD (Average Percentage of Faults Detected) up to 18% on average.

[1]  Charles J. Colbourn,et al.  Prioritized interaction testing for pair-wise coverage with seeding and constraints , 2006, Inf. Softw. Technol..

[2]  Qi Luo,et al.  Assessing Test Case Prioritization on Real Faults and Mutants , 2018, 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[3]  Adam A. Porter,et al.  A history-based test prioritization technique for regression testing in resource constrained environments , 2002, ICSE '02.

[4]  Hyunsook Do,et al.  Test Case Prioritization Using Requirements-Based Clustering , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[5]  Luciano Baresi,et al.  An Introduction to Software Testing , 2006, FoVMT.

[6]  Gregg Rothermel,et al.  Test Case Prioritization: A Family of Empirical Studies , 2002, IEEE Trans. Software Eng..

[7]  Yves Le Traon,et al.  Comparing White-Box and Black-Box Test Prioritization , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[8]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[9]  Arvinder Kaur,et al.  A GENETIC ALGORITHM FOR REGRESSION TEST CASE PRIORITIZATION USING CODE COVERAGE , 2011 .

[11]  David Clark,et al.  Test Set Diameter: Quantifying the Diversity of Sets of Test Cases , 2015, 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST).

[12]  Mukesh Mann,et al.  Generating and prioritizing optimal paths using ant colony optimization , 2015 .

[13]  Bo Qu,et al.  Test Case Prioritization for Black Box Testing , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[14]  Gregg Rothermel,et al.  Test case prioritization: an empirical study , 1999, Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM'99). 'Software Maintenance for Business Change' (Cat. No.99CB36360).

[15]  R. Uma Maheswari,et al.  Combined Genetic and Simulated Annealing Approach for Test Case Prioritization , 2015 .

[16]  Ilene Burnstein,et al.  Practical Software Testing: A Process-Oriented Approach , 2003 .

[17]  Atif M. Memon,et al.  Test suite prioritization by interaction coverage , 2007, DOSTA '07.

[18]  Neelam Gupta,et al.  Test Case Prioritization Using Relevant Slices , 2006, 30th Annual International Computer Software and Applications Conference (COMPSAC'06).

[19]  Mohd. Ehmer Khan,et al.  A Comparative Study of White Box, Black Box and Grey Box Testing Techniques , 2012 .

[20]  Mary Jean Harrold,et al.  Testing evolving software , 1999, J. Syst. Softw..

[21]  T. Neumann Computers And Intractability A Guide To The Theory Of Np Completeness , 2016 .

[22]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[23]  K. Vimala Devi,et al.  Effectiveness of Testcase Prioritization using APFD Metric: Survey , 2013 .

[24]  Mark Harman,et al.  Regression testing minimization, selection and prioritization: a survey , 2012, Softw. Test. Verification Reliab..

[25]  Ahlam Ansari,et al.  Optimized Regression Test Using Test Case Prioritization , 2016 .

[26]  Joseph Robert Horgan,et al.  A study of effective regression testing in practice , 1997, Proceedings The Eighth International Symposium on Software Reliability Engineering.

[27]  Arvinder Kaur,et al.  Fault coverage-based test suite optimization method for regression testing: learning from mistakes-based approach , 2019, Neural Computing and Applications.