Population Size Specification for Fair Comparison of Multi-objective Evolutionary Algorithms

In general, performance comparison results of optimization algorithms depend on the parameter specifications in each algorithm. For fair comparison, it may be needed to use the best specifications for each algorithm instead of using the same specifications for all algorithms. This is because each algorithm has its best specifications. However, in the evolutionary multi-objective optimization (EMO) field, performance comparison has usually been performed under the same parameter specifications for all algorithms. Especially, the same population size has always been used. In this paper, we discuss this practice from a viewpoint of fair comparison of EMO algorithms. First, we demonstrate that performance comparison results depend on the population size. Next, we explain a new trend of performance comparison where each algorithm is evaluated by selecting a pre-specified number of solutions from the examined solutions (i.e., by selecting a solution subset with a pre-specified size). Then, we discuss the selected subset size specification. Through computational experiments, we show that performance comparison results do not strongly depend on the selected subset size while they depend on the population size.

[1]  Carlos A. Coello Coello,et al.  Approximating Complex Pareto Fronts With Predefined Normal-Boundary Intersection Directions , 2020, IEEE Transactions on Evolutionary Computation.

[2]  Yuren Zhou,et al.  A Many-Objective Evolutionary Algorithm With Pareto-Adaptive Reference Points , 2020, IEEE Transactions on Evolutionary Computation.

[3]  R. Lyndon While,et al.  A review of multiobjective test problems and a scalable test problem toolkit , 2006, IEEE Transactions on Evolutionary Computation.

[4]  Lothar Thiele,et al.  Multiobjective Optimization Using Evolutionary Algorithms - A Comparative Case Study , 1998, PPSN.

[5]  Qingfu Zhang,et al.  An Evolutionary Many-Objective Optimization Algorithm Based on Dominance and Decomposition , 2015, IEEE Transactions on Evolutionary Computation.

[6]  Carlos M. Fonseca,et al.  Hypervolume Subset Selection in Two Dimensions: Formulations and Algorithms , 2016, Evolutionary Computation.

[7]  Tobias Friedrich,et al.  Generic Postprocessing via Subset Selection for Hypervolume and Epsilon-Indicator , 2014, PPSN.

[8]  Thomas Bäck,et al.  Parallel Problem Solving from Nature — PPSN V , 1998, Lecture Notes in Computer Science.

[9]  Xin Yao,et al.  A New Dominance Relation-Based Evolutionary Algorithm for Many-Objective Optimization , 2016, IEEE Transactions on Evolutionary Computation.

[10]  Hisao Ishibuchi,et al.  A New Framework of Evolutionary Multi-Objective Algorithms with an Unbounded External Archive , 2020 .

[11]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[12]  Tapabrata Ray,et al.  Distance-Based Subset Selection for Benchmarking in Evolutionary Multi/Many-Objective Optimization , 2019, IEEE Transactions on Evolutionary Computation.

[13]  Zexuan Zhu,et al.  An Indicator-Based Many-Objective Evolutionary Algorithm With Boundary Protection , 2020, IEEE Transactions on Cybernetics.

[14]  Hisao Ishibuchi,et al.  Performance of Decomposition-Based Many-Objective Algorithms Strongly Depends on Pareto Front Shapes , 2017, IEEE Transactions on Evolutionary Computation.

[15]  Carlos A. Coello Coello,et al.  A Study of the Parallelization of a Coevolutionary Multi-objective Evolutionary Algorithm , 2004, MICAI.

[16]  Hisao Ishibuchi,et al.  How to Specify a Reference Point in Hypervolume Calculation for Fair Performance Comparison , 2018, Evolutionary Computation.

[17]  Luís Paquete,et al.  Implicit enumeration strategies for the hypervolume subset selection problem , 2018, Comput. Oper. Res..

[18]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point Based Nondominated Sorting Approach, Part II: Handling Constraints and Extending to an Adaptive Approach , 2014, IEEE Transactions on Evolutionary Computation.

[19]  Carlos M. Fonseca,et al.  Greedy Hypervolume Subset Selection in Low Dimensions , 2016, Evolutionary Computation.

[20]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[21]  Marco Laumanns,et al.  Scalable Test Problems for Evolutionary Multiobjective Optimization , 2005, Evolutionary Multiobjective Optimization.

[22]  Xiangxiang Zeng,et al.  A Polar-Metric-Based Evolutionary Algorithm , 2020, IEEE Transactions on Cybernetics.

[23]  Qingfu Zhang,et al.  Evolutionary Many-Objective Optimization Based on Adversarial Decomposition , 2017, IEEE Transactions on Cybernetics.

[24]  Qiuzhen Lin,et al.  An Elite Gene Guided Reproduction Operator for Many-Objective Optimization , 2019, IEEE Transactions on Cybernetics.

[25]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[26]  Xin Yao,et al.  An Empirical Investigation of the Optimality and Monotonicity Properties of Multiobjective Archiving Methods , 2019, EMO.

[27]  Hisao Ishibuchi,et al.  Reference Point Specification in Inverted Generational Distance for Triangular Linear Pareto Front , 2018, IEEE Transactions on Evolutionary Computation.

[28]  Hisao Ishibuchi,et al.  How to compare many-objective algorithms under different settings of population and archive sizes , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[29]  Silviu Maniu,et al.  Hypervolume Subset Selection with Small Subsets , 2019, Evolutionary Computation.

[30]  Hisao Ishibuchi,et al.  Benchmarking Multi- and Many-Objective Evolutionary Algorithms Under Two Optimization Scenarios , 2017, IEEE Access.

[31]  Ye Tian,et al.  PlatEMO: A MATLAB Platform for Evolutionary Multi-Objective Optimization [Educational Forum] , 2017, IEEE Computational Intelligence Magazine.