An Adaptive Multi-objective Heuristic Search for Model-Based Testing

Search-based Testing (SBST) consists in the application of metaheuristic search techniques for test case generation. Most studies are focused on the generation of test data for white-box testing, but there are also proposals for model-based test case generation. In previous works, our group developed a method and a tool that supports multi-objective metaheuristic search techniques for test case generation from state models. A problem that practitioners and researchers face when employing search-based test case generation tools is that the algorithms must be properly parameterized for the different problems and even problem instances. However, proper parameter selection is a highly specialized and time-consuming task, which most often is not part of practitioner's skills. In this paper, we introduce an adaptive approach for test case generation in the context of model-based testing. After a literature review, to understand the current approaches for parameter adjustment, we selected some of them to implement. The different algorithms were compared with manual parameter tuning. Preliminary results indicate that the different parameter control algorithms used in this study can be useful and were integrated to the test case generation tool. Their performance may vary according to the model at hand, that is why we think it is better to offer distinct options to the users, besides the already existing manual parameter setting.

[1]  Walter Abrahão dos Santos,et al.  A Multidisciplinary Design Optimization Tool for Spacecraft Equipment Layout Conception , 2014 .

[2]  Mark Harman,et al.  A theoretical and empirical study of EFSM dependence , 2009, 2009 IEEE International Conference on Software Maintenance.

[3]  Irene Moser,et al.  Entropy-based adaptive range parameter control for evolutionary algorithms , 2013, GECCO '13.

[4]  Sanaz Mostaghim,et al.  Adaptive Range Parameter Control , 2012, 2012 IEEE Congress on Evolutionary Computation.

[5]  A. Dias-Neto,et al.  0006/2011 - Threats to Validity in Search-based Software Engineering Empirical Studies , 2011 .

[6]  John A. Clark,et al.  A Principled Evaluation of the Effect of Directed Mutation on Search-Based Statistical Testing , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[7]  Na Zhang,et al.  Path-oriented test cases generation based adaptive genetic algorithm , 2017, PloS one.

[8]  Irene Moser,et al.  A Systematic Literature Review of Adaptive Parameter Control Methods for Evolutionary Algorithms , 2016, ACM Comput. Surv..

[9]  Gordon Fraser,et al.  On Parameter Tuning in Search Based Software Engineering , 2011, SSBSE.

[10]  Stefan Boettcher,et al.  Optimization with extremal dynamics , 2003, Complex..

[11]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[12]  Eliane Martins,et al.  MOST: A Multi-objective Search-Based Testing from EFSM , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[13]  Fabiano Luis de Sousa,et al.  Generalized extremal optimization: An application in heat pipe design , 2004 .

[14]  Zbigniew Michalewicz,et al.  Parameter Control in Evolutionary Algorithms , 2007, Parameter Setting in Evolutionary Algorithms.

[15]  Dirk Thierens,et al.  An Adaptive Pursuit Strategy for Allocating Operator Probabilities , 2005, BNAIC.

[16]  Gordon Fraser,et al.  Parameter tuning or default values? An empirical investigation in search-based software engineering , 2013, Empirical Software Engineering.

[17]  Phil McMinn,et al.  Search-Based Software Testing: Past, Present and Future , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[18]  Roberto Luiz Galski.  Development of improved, hybrid, parallel and multiobjective versions of the generalized extremal optimization method and its application to the design of spatial systems , 2006 .

[19]  Antônio José da Silva Neto,et al.  Multi-objective optimization as a new approach to illumination design of interior spaces , 2011 .

[20]  Eliane Martins,et al.  Generating Feasible Test Paths from an Executable Model Using a Multi-objective Approach , 2010, 2010 Third International Conference on Software Testing, Verification, and Validation Workshops.

[21]  Celso C. Ribeiro,et al.  Reactive GRASP: An Application to a Matrix Decomposition Problem in TDMA Traffic Assignment , 2000, INFORMS J. Comput..

[22]  Robert M. Hierons,et al.  Generating Feasible Transition Paths for Testing from an Extended Finite State Machine (EFSM) , 2009, 2009 International Conference on Software Testing Verification and Validation.

[23]  Mark Hoogendoorn,et al.  Parameter Control in Evolutionary Algorithms: Trends and Challenges , 2015, IEEE Transactions on Evolutionary Computation.

[24]  Gustavo Augusto Lima de Campos,et al.  Automated Test Case Prioritization with Reactive GRASP , 2010, Adv. Softw. Eng..

[25]  Lars Grunske,et al.  Test data generation with a Kalman filter-based adaptive genetic algorithm , 2015, J. Syst. Softw..