Requirement dependencies-based formal approach for test case prioritization in regression testing

Regression testing is the testing activity performed after changes occurred on software. Its aim is to increase confidence that achieved software adjustments have no negative impact on the already functional parts of the software. Test case prioritization is one technique that could be applied in regression testing with the aim to find faults early, resulting in reduced cost and shorten time of testing activities. Thus, prioritizing in the context of regression testing means to re-order test cases such that high priority ones are run first. The current paper addresses the test case prioritization as a consistent part of a larger approach on regression testing, which combines both test case prioritization and test case selection in order to overcome the limitations of each of them. A comprehensive formalization of test case prioritization is provided, incorporating beside the well known ingredients (test case, test requirement, fault, cost) also elements relating to the functional requirements and dependencies between requirements. An evolutionary algorithm is used to construct the re-ordering of test cases, considering as optimization objectives fault detection and cost. A synthetic case study was used to empirically prove our perspective for test case prioritization approach.

[1]  Anjali Choudhary,et al.  A Review on Regression Testing techniques , 2015 .

[2]  Gregory M. Kapfhammer,et al.  History-Based Test Case Prioritization with Software Version Awareness , 2013, 2013 18th International Conference on Engineering of Complex Computer Systems.

[3]  Mohammad Abdollahi Azgomi,et al.  An improved method for test case prioritization by incorporating historical test case data , 2012, Sci. Comput. Program..

[4]  Mark Harman,et al.  Empirical evaluation of pareto efficient multi-objective regression test case prioritisation , 2015, ISSTA.

[5]  Mark Harman,et al.  Search Based Software Engineering: Techniques, Taxonomy, Tutorial , 2010, LASER Summer School.

[6]  Mark Harman,et al.  Faster Fault Finding at Google Using Multi Objective Regression Test Optimisation , 2011 .

[7]  Mauro Pezzè,et al.  Software testing and analysis - process, principles and techniques , 2007 .

[8]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[9]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[10]  R. Kavitha,et al.  Test Case Prioritization for Regression Testing based on Severity of Fault , 2010 .

[11]  Mark Harman,et al.  Pareto efficient multi-objective test case selection , 2007, ISSTA '07.

[12]  Gregg Rothermel,et al.  Test case prioritization: an empirical study , 1999, Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM'99). 'Software Maintenance for Business Change' (Cat. No.99CB36360).

[13]  Chiranjeev Kumar,et al.  Effectiveness of prioritization of test cases based on Faults , 2016, 2016 3rd International Conference on Recent Advances in Information Technology (RAIT).

[14]  Thillaikarasi Muthusamy,et al.  A Test Case Prioritization Method with Weight Factors in Regression Testing Based on Measurement Metrics , 2014 .

[15]  Mark Harman,et al.  Search Algorithms for Regression Test Case Prioritization , 2007, IEEE Transactions on Software Engineering.

[16]  Wan M.N. Wan-Kadir,et al.  A review on prioritization techniques in regression testing , 2016 .

[17]  Gregg Rothermel,et al.  An empirical study of the effects of minimization on the fault detection capabilities of test suites , 1998, Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272).

[18]  Hyunsook Do,et al.  Requirements based test prioritization using risk factors: An industrial study , 2016, Inf. Softw. Technol..

[19]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[20]  Rajib Mall,et al.  An approach to prioritize the regression test cases of object-oriented programs , 2013, CSI Transactions on ICT.

[21]  Fang Yuan,et al.  Epistatic Genetic Algorithm for Test Case Prioritization , 2015, SSBSE.

[22]  Jacques Periaux,et al.  Combinatorial Optimization Using Genetic Algorithms and Game Theory for High Lift Configuration in Aerodynamics , 2003 .

[23]  Mark Harman,et al.  Using hybrid algorithm for Pareto efficient multi-objective test suite minimisation , 2010, J. Syst. Softw..

[24]  Melanie Mitchell,et al.  An introduction to genetic algorithms , 1996 .

[25]  Ladan Tahvildari,et al.  Prioritizing Requirements-Based Regression Test Cases: A Goal-Driven Practice , 2011, 2011 15th European Conference on Software Maintenance and Reengineering.

[26]  Rex Black,et al.  Foundations of Software Testing ISTQB Certification , 2006 .