Historical Value-Based Approach for Cost-Cognizant Test Case Prioritization to Improve the Effectiveness of Regression Testing

Regression testing has been used to support software testing activities and assure the acquirement of appropriate quality through several versions of a software program. Regression testing, however, is too expensive because it requires many test case executions, and the number of test cases increases sharply as the software evolves. In this paper, we propose the Historical Value-Based Approach, which is based on the use of historical information, to estimate the current cost and fault severity for cost-cognizant test case prioritization. We also conducted a controlled experiment to validate the proposed approach, the results of which proved the proposed approachpsilas usefulness. As a result of the proposed approach, software testers who perform regression testing are able to prioritize their test cases so that their effectiveness can be improved in terms of average percentage of fault detected per cost.

[1]  Gregg Rothermel,et al.  Understanding and measuring the sources of variation in the prioritization of regression test suites , 2001, Proceedings Seventh International Software Metrics Symposium.

[2]  Hema Srikanth,et al.  On the Economics of Requirements-Based Test Case Prioritization , 2005 .

[3]  Amitabh Srivastava,et al.  Effectively prioritizing tests in development environment , 2002, ISSTA '02.

[4]  John Lane,et al.  IEEE Standard Computer Dictionary: Compilation of IEEE Standard Computer Glossaries , 1991 .

[5]  Adam A. Porter,et al.  A history-based test prioritization technique for regression testing in resource constrained environments , 2002, ICSE '02.

[6]  Gregg Rothermel,et al.  Test Case Prioritization: A Family of Empirical Studies , 2002, IEEE Trans. Software Eng..

[7]  Gregg Rothermel,et al.  A controlled experiment assessing test case prioritization techniques via mutation faults , 2005, 21st IEEE International Conference on Software Maintenance (ICSM'05).

[8]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[9]  Gregg Rothermel,et al.  Incorporating varying test costs and fault severities into test case prioritization , 2001, Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001.

[10]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[11]  Gregg Rothermel,et al.  An empirical study of regression test application frequency , 2005, Softw. Test. Verification Reliab..

[12]  Gregg Rothermel,et al.  Empirical studies of test case prioritization in a JUnit testing environment , 2004, 15th International Symposium on Software Reliability Engineering.

[13]  RothermelGregg,et al.  Supporting Controlled Experimentation with Testing Techniques , 2005 .

[14]  Gregg Rothermel,et al.  Analyzing Regression Test Selection Techniques , 1996, IEEE Trans. Software Eng..

[15]  Gregg Rothermel,et al.  Cost-cognizant Test Case Prioritization , 2006 .

[16]  Gregg Rothermel,et al.  Prioritizing JUnit Test Cases: An Empirical Assessment and Cost-Benefits Analysis , 2006, Empirical Software Engineering.

[17]  Joseph Robert Horgan,et al.  A study of effective regression testing in practice , 1997, Proceedings The Eighth International Symposium on Software Reliability Engineering.

[18]  Gregg Rothermel,et al.  On the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques , 2006, IEEE Transactions on Software Engineering.

[19]  Gregg Rothermel,et al.  On test suite composition and cost-effective regression testing , 2004, TSEM.

[20]  Wei-Tek Tsai,et al.  Regression testing in an industrial environment , 1998, CACM.