An Analysis of Quality Indicators Using Approximated Optimal Distributions in a 3-D Objective Space

Although quality indicators play a crucial role in benchmarking evolutionary multiobjective optimization algorithms, their properties are still unclear. One promising approach for understanding quality indicators is the use of the optimal distribution of objective vectors that optimizes each quality indicator. However, it is difficult to obtain the optimal distribution for each quality indicator, especially, when its theoretical property is unknown. Thus, optimal distributions for most quality indicators have not been well investigated. To address these issues, first, we propose a problem formulation of finding the optimal distribution for each quality indicator on an arbitrary Pareto front. Then, we approximate the optimal distributions for nine quality indicators using the proposed problem formulation. We analyze the nine quality indicators using their approximated optimal distributions on eight types of Pareto fronts of three-objective problems. Our analysis demonstrates that uniformly distributed objective vectors over the entire Pareto front are not optimal in many cases. Each quality indicator has its own optimal distribution for each Pareto front. We also examine the consistency among the nine quality indicators.

[1]  Tobias Friedrich,et al.  Efficient computation of two-dimensional solution sets maximizing the epsilon-indicator , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[2]  Bernard Chazelle,et al.  A minimum spanning tree algorithm with inverse-Ackermann type complexity , 2000, JACM.

[3]  Marco Laumanns,et al.  Scalable Test Problems for Evolutionary Multiobjective Optimization , 2005, Evolutionary Multiobjective Optimization.

[4]  Yaonan Wang,et al.  Multi-objective self-adaptive differential evolution with elitist archive and crowding entropy-based diversity measure , 2010, Soft Comput..

[5]  Kaisa Miettinen,et al.  Nonlinear multiobjective optimization , 1998, International series in operations research and management science.

[6]  Marjan Mernik,et al.  The impact of Quality Indicators on the rating of Multi-objective Evolutionary Algorithms , 2017, Appl. Soft Comput..

[7]  Carlos A. Coello Coello,et al.  A multi-objective evolutionary hyper-heuristic based on multiple indicator-based density estimators , 2018, GECCO.

[8]  M. Hansen,et al.  Evaluating the quality of approximations to the non-dominated set , 1998 .

[9]  Hisao Ishibuchi,et al.  Comparison of Hypervolume, IGD and IGD+ from the Viewpoint of Optimal Distributions of Solutions , 2019, EMO.

[10]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[11]  Hisao Ishibuchi,et al.  Pareto Fronts of Many-Objective Degenerate Test Problems , 2016, IEEE Transactions on Evolutionary Computation.

[12]  Hisao Ishibuchi,et al.  A new R2 indicator for better hypervolume approximation , 2018, GECCO.

[13]  Anne Auger,et al.  Theory of the hypervolume indicator: optimal μ-distributions and the choice of the reference point , 2009, FOGA '09.

[14]  Gary B. Lamont,et al.  Multiobjective evolutionary algorithms: classifications, analyses, and new innovations , 1999 .

[15]  Hisao Ishibuchi,et al.  Performance of Decomposition-Based Many-Objective Algorithms Strongly Depends on Pareto Front Shapes , 2017, IEEE Transactions on Evolutionary Computation.

[16]  Carlos A. Coello Coello,et al.  A Study of the Parallelization of a Coevolutionary Multi-objective Evolutionary Algorithm , 2004, MICAI.

[17]  Lothar Thiele,et al.  The Hypervolume Indicator Revisited: On the Design of Pareto-compliant Indicators Via Weighted Integration , 2007, EMO.

[18]  Hisao Ishibuchi,et al.  How to Specify a Reference Point in Hypervolume Calculation for Fair Performance Comparison , 2018, Evolutionary Computation.

[19]  Kiyoshi Tanaka,et al.  A Review of Features and Limitations of Existing Scalable Multiobjective Test Suites , 2019, IEEE Transactions on Evolutionary Computation.

[20]  Shengxiang Yang,et al.  Diversity Comparison of Pareto Front Approximations in Many-Objective Optimization , 2014, IEEE Transactions on Cybernetics.

[21]  Hisao Ishibuchi,et al.  A Study on Performance Evaluation Ability of a Modified Inverted Generational Distance Indicator , 2015, GECCO.

[22]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[23]  Yaochu Jin,et al.  A Critical Survey of Performance Indices for Multi-Objective Optimisation , 2003 .

[24]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[25]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[26]  Xin Yao,et al.  How to Read Many-Objective Solution Sets in Parallel Coordinates [Educational Forum] , 2017, IEEE Computational Intelligence Magazine.

[27]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[28]  Nicola Beume,et al.  SMS-EMOA: Multiobjective selection based on dominated hypervolume , 2007, Eur. J. Oper. Res..

[29]  Xin Yao,et al.  of Birmingham Quality evaluation of solution sets in multiobjective optimisation , 2019 .

[30]  Qguhm -DVNLHZLF,et al.  On the performance of multiple objective genetic local search on the 0 / 1 knapsack problem . A comparative experiment , 2000 .

[31]  E. Saff,et al.  Discretizing Manifolds via Minimum Energy Points , 2004 .

[32]  Lothar Thiele,et al.  On Set-Based Multiobjective Optimization , 2010, IEEE Transactions on Evolutionary Computation.

[33]  A. Solow,et al.  Measuring biological diversity , 2006, Environmental and Ecological Statistics.

[34]  Hisao Ishibuchi,et al.  A multi-objective genetic local search algorithm and its application to flowshop scheduling , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[35]  Alex S. Fukunaga,et al.  Improving the search performance of SHADE using linear population size reduction , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[36]  Joshua D. Knowles,et al.  On metrics for comparing nondominated sets , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[37]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[38]  Carlos A. Coello Coello,et al.  Using the Averaged Hausdorff Distance as a Performance Measure in Evolutionary Multiobjective Optimization , 2012, IEEE Transactions on Evolutionary Computation.

[39]  Bilel Derbel,et al.  A Correlation Analysis of Set Quality Indicator Values in Multiobjective Optimization , 2016, GECCO.

[40]  Xin Yao,et al.  Diversity Assessment in Many-Objective Optimization , 2017, IEEE Transactions on Cybernetics.

[41]  Ye Tian,et al.  Sampling Reference Points on the Pareto Fronts of Benchmark Multi-Objective Optimization Problems , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[42]  Lothar Thiele,et al.  Multiobjective Optimization Using Evolutionary Algorithms - A Comparative Case Study , 1998, PPSN.

[43]  Karl Bringmann,et al.  Two-dimensional subset selection for hypervolume and epsilon-indicator , 2014, GECCO.

[44]  Marco Laumanns,et al.  Scalable test problems for evolutionary multi-objective optimization , 2001 .

[45]  Lothar Thiele,et al.  A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers , 2006 .

[46]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point Based Nondominated Sorting Approach, Part II: Handling Constraints and Extending to an Adaptive Approach , 2014, IEEE Transactions on Evolutionary Computation.

[47]  Tea Tusar,et al.  Visualization of Pareto Front Approximations in Evolutionary Multiobjective Optimization: A Critical Review and the Prosection Method , 2015, IEEE Transactions on Evolutionary Computation.

[48]  John E. Dennis,et al.  Normal-Boundary Intersection: A New Method for Generating the Pareto Surface in Nonlinear Multicriteria Optimization Problems , 1998, SIAM J. Optim..

[49]  Heike Trautmann,et al.  On the properties of the R2 indicator , 2012, GECCO '12.

[50]  Thomas Stützle,et al.  An Empirical Assessment of the Properties of Inverted Generational Distance on Multi- and Many-Objective Optimization , 2017, EMO.

[51]  Hisao Ishibuchi,et al.  Modified Distance Calculation in Generational Distance and Inverted Generational Distance , 2015, EMO.

[52]  Tobias Glasmachers.  Optimized Approximation Sets for Low-Dimensional Benchmark Pareto Fronts , 2014, PPSN.

[53]  Simon Wessing,et al.  Sequential parameter optimization for multi-objective problems , 2010, IEEE Congress on Evolutionary Computation.

[54]  Hisao Ishibuchi,et al.  Indicator-based evolutionary algorithm with hypervolume approximation by achievement scalarizing functions , 2010, GECCO '10.

[55]  Lucas Bradstreet,et al.  A Fast Way of Calculating Exact Hypervolumes , 2012, IEEE Transactions on Evolutionary Computation.

[56]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[57]  Hisao Ishibuchi,et al.  Reference Point Specification in Inverted Generational Distance for Triangular Linear Pareto Front , 2018, IEEE Transactions on Evolutionary Computation.

[58]  Jie Zhang,et al.  Consistencies and Contradictions of Performance Metrics in Multiobjective Optimization , 2014, IEEE Transactions on Cybernetics.