An adaptive fitness function based on branch hardness for search based testing

Search based software testing has received great attention as a means of automating the test data generation, and the goal is to improve various criteria. There are different types of coverage criteria. In this paper, we deal with the path coverage. Concretely, we focus on the path that is the most difficult to cover. One major limitation of search based testing is the inefficient and insufficiently informed fitness function. To address this problem, we propose an adaptive fitness function based on branch hardness. The branch hardness is measured by the expected number of visits of each branch in the program, which is modeled by an absorbing discrete time Markov chain. By tuning the parameters of branch hardness heuristically, the search hardness, evaluated by the variation coefficient of the fitness function, of generating test data can be minimized. Therefore, this new fitness function is more flexible than the traditional counterparts. In addition, we point out that the present definition of branch distance and the use of normalizing functions are problematic, and propose some improvements. Finally, the empirical study reveals the promising result of our proposal in this paper.

[1]  Andrea Arcuri,et al.  It really does matter how you normalize the branch distance in search‐based software testing , 2013, Softw. Test. Verification Reliab..

[2]  Paolo Tonella,et al.  Symbolic search-based testing , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[3]  John I. McCool,et al.  Probability and Statistics With Reliability, Queuing and Computer Science Applications , 2003, Technometrics.

[4]  Leonardo Bottaci,et al.  Predicate Expression Cost Functions to Guide Evolutionary Search for Test Data , 2003, GECCO.

[5]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[6]  Gilles Pesant,et al.  Counting Solutions of CSPs: A Structural Approach , 2005, IJCAI.

[7]  Huai Liu,et al.  Adaptive random testing based on distribution metrics , 2009, J. Syst. Softw..

[8]  Gary McGraw,et al.  Generating Software Test Data by Evolution , 2001, IEEE Trans. Software Eng..

[9]  Michel Gendreau,et al.  Hyper-heuristics: a survey of the state of the art , 2013, J. Oper. Res. Soc..

[10]  Toby Walsh,et al.  Handbook of satisfiability , 2009 .

[11]  Lars Grunske,et al.  Analysing the fitness landscape of search-based software testing problems , 2017, Automated Software Engineering.

[12]  Yue Jia Hyperheuristic Search for SBST , 2015, 2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing.

[13]  Mark Harman,et al.  Search-based software engineering , 2001, Inf. Softw. Technol..

[14]  Enrique Alba,et al.  Estimating software testing complexity , 2013, Inf. Softw. Technol..

[15]  Yuanyuan Zhang,et al.  Search Based Software Engineering: A Comprehensive Analysis and Review of Trends Techniques and Applications , 2009 .

[16]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[17]  Bryan F. Jones,et al.  Automatic structural testing using genetic algorithms , 1996, Softw. Eng. J..

[18]  Yann-Gaël Guéhéneuc,et al.  Constraint-Based Fitness Function for Search-Based Software Testing , 2013, CPAIOR.

[19]  John A. Clark,et al.  Using automated search to generate test data for matlab , 2009, GECCO.

[20]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[21]  Michel Verleysen,et al.  K nearest neighbours with mutual information for simultaneous classification and missing data imputation , 2009, Neurocomputing.

[22]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[23]  Lionel C. Briand,et al.  A Systematic Review of the Application and Empirical Investigation of Search-Based Test Case Generation , 2010, IEEE Transactions on Software Engineering.

[24]  Kishor S. Trivedi Probability and Statistics with Reliability, Queuing, and Computer Science Applications , 1984 .

[25]  Joachim Wegener,et al.  Evolutionary test environment for automatic structural testing , 2001, Inf. Softw. Technol..

[26]  G. Myers,et al.  The Art of Software Testing: Myers/Art , 2012 .

[27]  Roy P. Pargas,et al.  Test-Data Generation Using Genetic Algorithms , 1999, Softw. Test. Verification Reliab..