On the Effect of Counters in Guard Conditions When State-Based Multi-objective Testing

During test case generation from an extended finite state machine (EFSM), the counter problem is caused by the presence of guard conditions that refer to counter variables. Because such variables are initialized and updated by transitions in the EFSM, every traversal of the state machine graph is not necessarily feasible, i.e., executable. The problem manifests itself by the fact that a transition, a sequence of transitions, or a more complex behavior in the state machine, has to be repeatedly triggered to eventually trigger a specific behavior (another transition). In this paper we define different manifestations of the counter problem and experiment with a new search based solution for that problem. We also investigate how the counter problem affects a multi-objective genetic algorithm that generates test suites from an EFSM. We evaluate our solution and compare it with an existing one, using three different case studies.

[1]  Dan Gusfield,et al.  Algorithms on Strings, Trees, and Sequences - Computer Science and Computational Biology , 1997 .

[2]  Mark Harman,et al.  A multi-objective approach to search-based test data generation , 2007, GECCO '07.

[3]  Yvan Labiche,et al.  Automated state-based online testing real-time embedded software with RTEdge , 2015, 2015 3rd International Conference on Model-Driven Engineering and Software Development (MODELSWARD).

[4]  Lionel C. Briand,et al.  Automated support for deriving test requirements from UML statecharts , 2005, Software & Systems Modeling.

[5]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[6]  Bogdan Korel,et al.  Model based regression test reduction using dependence analysis , 2002, International Conference on Software Maintenance, 2002. Proceedings..

[7]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[8]  Marco Laumanns,et al.  On the Effects of Archiving, Elitism, and Density Based Selection in Evolutionary Multi-objective Optimization , 2001, EMO.

[9]  Joachim Wegener,et al.  Evolutionary test environment for automatic structural testing , 2001, Inf. Softw. Technol..

[10]  Chang Wook Ahn,et al.  On the practical genetic algorithms , 2005, GECCO '05.

[11]  Alexandre Petrenko,et al.  Test Selection Based on Communicating Nondeterministic Finite-State Machines Using a Generalized WP-Method , 1994, IEEE Trans. Software Eng..

[12]  Yvan Labiche,et al.  On the Round Trip Path Testing Strategy , 2010, 2010 IEEE 21st International Symposium on Software Reliability Engineering.

[13]  Mark Harman,et al.  Search Based Software Engineering: Techniques, Taxonomy, Tutorial , 2010, LASER Summer School.

[14]  Yvan Labiche,et al.  Multi-objective Construction of an Entire Adequate Test Suite for an EFSM , 2014, 2014 IEEE 25th International Symposium on Software Reliability Engineering.

[15]  Robert M. Hierons,et al.  Generating Feasible Transition Paths for Testing from an Extended Finite State Machine (EFSM) , 2009, 2009 International Conference on Software Testing Verification and Validation.

[16]  Elaine J. Weyuker,et al.  Comparison of program testing strategies , 1991, TAV4.

[17]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[18]  Phyllis G. Frankl,et al.  An Experimental Comparison of the Effectiveness of Branch Testing and Data Flow Testing , 1993, IEEE Trans. Software Eng..

[19]  Alexandre Petrenko,et al.  Confirming configurations in EFSM testing , 2004, IEEE Transactions on Software Engineering.

[20]  Massimiliano Di Penta,et al.  Assessing, Comparing, and Combining State Machine-Based Testing and Structural Testing: A Series of Experiments , 2011, IEEE Transactions on Software Engineering.

[21]  Gordon Fraser,et al.  Whole Test Suite Generation , 2013, IEEE Transactions on Software Engineering.

[22]  M. Ümit Uyar,et al.  A method enabling feasible conformance test sequence generation for EFSM models , 2004, IEEE Transactions on Computers.

[23]  David E. Goldberg,et al.  A niched Pareto genetic algorithm for multiobjective optimization , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[24]  Robert M. Hierons,et al.  An integrated search-based approach for automatic testing from extended finite state machine (EFSM) models , 2011, Inf. Softw. Technol..

[25]  Lionel C. Briand,et al.  Using simulation to empirically investigate test coverage criteria based on statechart , 2004, Proceedings. 26th International Conference on Software Engineering.

[26]  Phil McMinn,et al.  Search-Based Software Testing: Past, Present and Future , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[27]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[28]  Kim G. Larsen,et al.  Testing real-time embedded software using UPPAAL-TRON: an industrial case study , 2005, EMSOFT.

[29]  Bernhard Peischl,et al.  Test Sequence Generation from Communicating UML State Charts: An Industrial Application of Symbolic Transition Systems , 2010, 2010 10th International Conference on Quality Software.

[30]  Eckart Zitzler,et al.  Indicator-Based Selection in Multiobjective Search , 2004, PPSN.

[31]  John A. Clark,et al.  An automated framework for structural test-data generation , 1998, Proceedings 13th IEEE International Conference on Automated Software Engineering (Cat. No.98EX239).

[32]  Thomas J. Ostrand,et al.  Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria , 1994, Proceedings of 16th International Conference on Software Engineering.

[33]  Ernesto Costa,et al.  Dynamic limits for bloat control in genetic programming and a review of past and current bloat theories , 2009, Genetic Programming and Evolvable Machines.

[34]  Lionel C. Briand,et al.  Achieving scalable model-based testing through test case diversity , 2013, TSEM.

[35]  Gary McGraw,et al.  Generating Software Test Data by Evolution , 2001, IEEE Trans. Software Eng..

[36]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[37]  Mark Harman,et al.  Search-based software engineering , 2001, Inf. Softw. Technol..

[38]  Dan Gusfield,et al.  Algorithms on Strings, Trees, and Sequences - Computer Science and Computational Biology , 1997 .

[39]  Boris Beizer,et al.  Software Testing Techniques , 1983 .