Do Quality Indicators Prefer Particular Multi-objective Search Algorithms in Search-Based Software Engineering?

In Search-Based Software Engineering (SBSE), users typically select a set of Multi-Objective Search Algorithms (MOSAs) for their experiments without any justification, or they simply choose an MOSA because of its popularity (e.g., NSGA-II). On the other hand, users know certain characteristics of solutions they are interested in. Such characteristics are typically measured with Quality Indicators (QIs) that are commonly used to evaluate the quality of solutions produced by an MOSA. Consequently, these QIs are often employed to empirically evaluate a set of MOSAs for a particular search problem to find the best MOSA. Thus, to guide SBSE users in choosing an MOSA that represents the solutions measured by a specific QI they are interested in, we present an empirical evaluation with a set of SBSE problems to study the relationships among commonly used QIs and MOSAs in SBSE. Our aim, by studying such relationships, is to identify whether there are certain characteristics of a QI because of which it prefers a certain MOSA. Such preferences are then used to provide insights and suggestions to SBSE users in selecting an MOSA, given that they know which quality aspects of solutions they are looking for.

[1]  A. Dias-Neto,et al.  0006/2011 - Threats to Validity in Search-based Software Engineering Empirical Studies , 2011 .

[2]  Antonio J. Nebro,et al.  jMetal: A Java framework for multi-objective optimization , 2011, Adv. Eng. Softw..

[3]  Arnaud Gotlieb,et al.  Cost-effective test suite minimization in product lines using search techniques , 2015, J. Syst. Softw..

[4]  Abdel Salam Sayyad,et al.  Pareto-optimal search-based software engineering (POSBSE): A literature survey , 2013, 2013 2nd International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE).

[5]  Marjan Mernik,et al.  A chess rating system for evolutionary algorithms: A new method for the comparison and ranking of evolutionary algorithms , 2014, Inf. Sci..

[6]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[7]  Yan Li,et al.  A Practical Guide to Select Quality Indicators for Assessing Pareto-Based Search Algorithms in Search-Based Software Engineering , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[8]  Marco Laumanns,et al.  SPEA2: Improving the Strength Pareto Evolutionary Algorithm For Multiobjective Optimization , 2002 .

[9]  D. Sheskin Handbook of parametric and nonparametric statistical procedures, 2nd ed. , 2000 .

[10]  Marjan Mernik,et al.  The impact of Quality Indicators on the rating of Multi-objective Evolutionary Algorithms , 2017, Appl. Soft Comput..

[11]  Lionel C. Briand,et al.  A practical guide for using statistical tests to assess randomized algorithms in software engineering , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[12]  David W. Corne,et al.  Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy , 2000, Evolutionary Computation.

[13]  Francisco Luna,et al.  MOCell: A cellular genetic algorithm for multiobjective optimization , 2009 .

[14]  Lionel C. Briand,et al.  A Systematic Review of the Application and Empirical Investigation of Search-Based Test Case Generation , 2010, IEEE Transactions on Software Engineering.

[15]  Enrique Alba,et al.  Solving Three-Objective Optimization Problems Using a New Hybrid Cellular Genetic Algorithm , 2008, PPSN.

[16]  YueTao,et al.  Quality Indicators in Search-based Software Engineering , 2020 .

[17]  Xin Yao,et al.  of Birmingham Quality evaluation of solution sets in multiobjective optimisation , 2019 .

[18]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2012, Springer Berlin Heidelberg.

[19]  Enrique Alba,et al.  SMPSO: A new PSO-based metaheuristic for multi-objective optimization , 2009, 2009 IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making(MCDM).

[20]  Xin Yao,et al.  A Critical Review of "A Practical Guide to Select Quality Indicators for Assessing Pareto-Based Search Algorithms in Search-Based Software Engineering": Essay on Quality Indicator Selection for SBSE , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering: New Ideas and Emerging Technologies Results (ICSE-NIER).

[21]  Ju Ren,et al.  A Survey on End-Edge-Cloud Orchestrated Network Computing Paradigms , 2019, ACM Comput. Surv..