Regression test suite selection using dependence analysis

Dependence analysis on an Extended Finite State Machine representation of the requirements of a system under test identifies various types of control and data dependencies between transitions caused by a set of modifications on the requirements. These particular types of dependencies capture the effects of the modifications, that is, their direct effects on the changed parts of the system and their side effects on the unchanged parts of the system. Recent work on model‐based regression testing shows that dependencies capturing direct effects and side effects of the changes made on the requirements can be used for regression test suite (RTS) reduction (reducing the size of a given test suite by eliminating redundancies), for RTS prioritization (ordering test cases in a given test suite for early fault detection), or for RTS generation (designing a test suite covering the identified dependencies). This paper proposes an additional use of such dependencies, namely, RTS selection, which is the process of selecting a subset of a given test suite to form an RTS by considering the coverage of dependencies related to the effects of the modifications. The dependencies marked during this process as uncovered provide a basis for augmenting an (incomplete) RTS with test cases covering uncovered dependencies. Copyright © 2012 John Wiley & Sons, Ltd.

[1]  Matteo Fischetti,et al.  Algorithms for the Set Covering Problem , 2000, Ann. Oper. Res..

[2]  Gregg Rothermel,et al.  Test case prioritization: an empirical study , 1999, Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM'99). 'Software Maintenance for Business Change' (Cat. No.99CB36360).

[3]  Bogdan Korel,et al.  Model based regression test reduction using dependence analysis , 2002, International Conference on Software Maintenance, 2002. Proceedings..

[4]  Joseph Robert Horgan,et al.  Effect of Test Set Minimization on Fault Detection Effectiveness , 1995, 1995 17th International Conference on Software Engineering.

[5]  Mats Per Erik Heimdahl,et al.  On the effectiveness of slicing hierarchical state machines: a case study , 1998, Proceedings. 24th EUROMICRO Conference (Cat. No.98EX204).

[6]  Rajiv Gupta,et al.  Program Slicing‐Based Regression Testing Techniques , 1996, Softw. Test. Verification Reliab..

[7]  Neelam Gupta,et al.  Improving Fault Detection Capability by Selectively Retaining Test Cases during Test Suite Reduction , 2007, IEEE Transactions on Software Engineering.

[8]  Alessandro Orso,et al.  Test-Suite Augmentation for Evolving Software , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[9]  Mark Harman,et al.  Model projection: simplifying models in response to restricting the environment , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[10]  Per Runeson,et al.  A systematic review on regression test selection techniques , 2010, Inf. Softw. Technol..

[11]  Bogdan Korel,et al.  Dependence analysis in reduction of requirement based test suites , 2002, ISSTA '02.

[12]  Bogdan Korel,et al.  Application of system models in regression test suite prioritization , 2008, 2008 IEEE International Conference on Software Maintenance.

[13]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[14]  T. Grossman,et al.  Computational Experience with Approximation Algorithms for the Set Covering Problem , 1994 .

[15]  Matthew B. Dwyer,et al.  Evaluating the Effectiveness of Slicing for Model Reduction of Concurrent Object-Oriented Programs , 2006, TACAS.

[16]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[17]  Bogdan Korel,et al.  Slicing of state-based models , 2003, International Conference on Software Maintenance, 2003. ICSM 2003. Proceedings..

[18]  Mark Harman,et al.  Regression testing minimization, selection and prioritization: a survey , 2012, Softw. Test. Verification Reliab..

[19]  Yanping Chen,et al.  Specification-based regression test selection with risk analysis , 2002, CASCON.

[20]  Gregg Rothermel,et al.  Using component metacontent to support the regression testing of component-based software , 2001, Proceedings IEEE International Conference on Software Maintenance. ICSM 2001.

[21]  Insup Lee,et al.  Data flow testing as model checking , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[22]  Mark Harman,et al.  A theoretical and empirical study of EFSM dependence , 2009, 2009 IEEE International Conference on Software Maintenance.

[23]  Gregg Rothermel,et al.  A safe, efficient regression test selection technique , 1997, TSEM.

[24]  Mats Per Erik Heimdahl,et al.  Reduction and slicing of hierarchical state machines , 1997, ESEC '97/FSE-5.

[25]  Yanping Chen,et al.  Regression test suite reduction based on SDL models of system requirements , 2009, J. Softw. Maintenance Res. Pract..

[26]  Mark Harman,et al.  Regression test suite prioritization using system models , 2012, Softw. Test. Verification Reliab..

[27]  Mary Lou Soffa,et al.  Program Slicing‐Based Regression Testing Techniques , 1996 .

[28]  Nikolai Mansurov,et al.  Automatic synthesis of SDL models in use case methodology , 1999, SDL Forum.

[29]  Jean-Pierre Gallois,et al.  Slicing communicating automata specifications: polynomial algorithms for model reduction , 2008, Formal Aspects of Computing.

[30]  Chengying Mao,et al.  Regression testing for component-based software systems by enhancing change information , 2005, 12th Asia-Pacific Software Engineering Conference (APSEC'05).

[31]  Bo Xie.  Requirement based regression test suite reduction using dependence analysis , 2006 .

[32]  Yanping Chen,et al.  Regression test suite reduction using extended dependence analysis , 2007, SOQUA '07.

[33]  Dieter Hogrefe,et al.  The CCITT-Specification and Description Language SDL , 1989, Comput. Networks.

[34]  Yanping Chen,et al.  Model-based regression test suite generation using dependence analysis , 2007, A-MOST '07.

[35]  David Chenho Kung,et al.  A technique for the selective revalidation of OO software , 1997, J. Softw. Maintenance Res. Pract..

[36]  Mark Harman,et al.  Control Dependence for Extended Finite State Machines , 2009, FASE.

[37]  Ji Wang,et al.  Slicing Hierarchical Automata for Model Checking UML Statecharts , 2002, ICFEM.

[38]  Antonia Bertolino,et al.  Using Spanning Sets for Coverage Testing , 2003, IEEE Trans. Software Eng..

[39]  Laurie A. Williams,et al.  System test case prioritization of new and regression test cases , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[40]  David Harel,et al.  Modeling Reactive Systems With Statecharts : The Statemate Approach , 1998 .

[41]  Agata Janowska,et al.  Slicing of Timed Automata with Discrete Data , 2006, Fundam. Informaticae.