A Performance Comparison Indicator for Pareto Front Approximations in Many-Objective Optimization

Increasing interest in simultaneously optimizing many objectives (typically more than three objectives) of problems leads to the emergence of various many-objective algorithms in the evolutionary multi-objective optimization field. However, in contrast to the development of algorithm design, how to assess many-objective algorithms has received scant concern. Many performance indicators are designed in principle for any number of objectives, but in practice are invalid or infeasible to be used in many-objective optimization. In this paper, we explain the difficulties that popular performance indicators face and propose a performance comparison indicator (PCI) to assess Pareto front approximations obtained by many-objective algorithms. PCI evaluates the quality of approximation sets with the aid of a reference set constructed by themselves. The points in the reference set are divided into many clusters, and the proposed indicator estimates the minimum moves of solutions in the approximation sets to weakly dominate these clusters. PCI has been verified both by an analytic comparison with several well-known indicators and by an empirical test on four groups of Pareto front approximations with different numbers of objectives and problem characteristics.

[1]  Dirk Thierens,et al.  The balance between proximity and diversity in multiobjective evolutionary algorithms , 2003, IEEE Trans. Evol. Comput..

[2]  Eckart Zitzler,et al.  Indicator-Based Selection in Multiobjective Search , 2004, PPSN.

[3]  Peter J. Fleming,et al.  Many-Objective Optimization: An Engineering Design Perspective , 2005, EMO.

[4]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[5]  Hisao Ishibuchi,et al.  Difficulties in specifying reference points to calculate the inverted generational distance for many-objective optimization problems , 2014, 2014 IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making (MCDM).

[6]  Hisao Ishibuchi,et al.  Behavior of Multiobjective Evolutionary Algorithms on Many-Objective Knapsack Problems , 2015, IEEE Transactions on Evolutionary Computation.

[7]  Shengxiang Yang,et al.  A Grid-Based Evolutionary Algorithm for Many-Objective Optimization , 2013, IEEE Transactions on Evolutionary Computation.

[8]  Carlos A. Coello Coello,et al.  Study of preference relations in many-objective optimization , 2009, GECCO.

[9]  David W. Corne,et al.  Techniques for highly multiobjective optimisation: some nondominated points are better than others , 2007, GECCO '07.

[10]  Gary B. Lamont,et al.  Multiobjective evolutionary algorithms: classifications, analyses, and new innovations , 1999 .

[11]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[12]  Jinhua Zheng,et al.  Enhancing Diversity for Average Ranking Method in Evolutionary Many-Objective Optimization , 2010, PPSN.

[13]  Tobias Friedrich,et al.  Speeding up many-objective optimization by Monte Carlo approximations , 2013, Artif. Intell..

[14]  Carlos A. Coello Coello,et al.  Using the Averaged Hausdorff Distance as a Performance Measure in Evolutionary Multiobjective Optimization , 2012, IEEE Transactions on Evolutionary Computation.

[15]  Arturo Hernández Aguirre,et al.  G-Metric: an M-ary quality indicator for the evaluation of non-dominated sets , 2008, GECCO '08.

[16]  Shengxiang Yang,et al.  A Comparative Study on Evolutionary Algorithms for Many-Objective Optimization , 2013, EMO.

[17]  Kalyanmoy Deb,et al.  Running performance metrics for evolutionary multi-objective optimizations , 2002 .

[18]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[19]  Peter J. Fleming,et al.  On the Evolutionary Optimization of Many Conflicting Objectives , 2007, IEEE Transactions on Evolutionary Computation.

[20]  H. Kita,et al.  Failure of Pareto-based MOEAs: does non-dominated really mean near to optimal? , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[21]  Marco Laumanns,et al.  Scalable Test Problems for Evolutionary Multiobjective Optimization , 2005, Evolutionary Multiobjective Optimization.

[22]  Nicola Beume,et al.  Pareto-, Aggregation-, and Indicator-Based Methods in Many-Objective Optimization , 2007, EMO.

[23]  David W. Corne,et al.  Properties of an adaptive archiving algorithm for storing nondominated vectors , 2003, IEEE Trans. Evol. Comput..

[24]  Shengxiang Yang,et al.  Shift-Based Density Estimation for Pareto-Based Algorithms in Many-Objective Optimization , 2014, IEEE Transactions on Evolutionary Computation.

[25]  Peter J. Fleming,et al.  Preference-Inspired Coevolutionary Algorithms for Many-Objective Optimization , 2013, IEEE Transactions on Evolutionary Computation.

[26]  Marco Laumanns,et al.  Combining Convergence and Diversity in Evolutionary Multiobjective Optimization , 2002, Evolutionary Computation.

[27]  Patrick M. Reed,et al.  Diagnostic Assessment of Search Controls and Failure Modes in Many-Objective Evolutionary Optimization , 2012, Evolutionary Computation.

[28]  Eckart Zitzler,et al.  HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization , 2011, Evolutionary Computation.

[29]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[30]  Evan J. Hughes,et al.  MSOPS-II: A general-purpose Many-Objective optimiser , 2007, 2007 IEEE Congress on Evolutionary Computation.

[31]  Shengxiang Yang,et al.  Diversity Comparison of Pareto Front Approximations in Many-Objective Optimization , 2014, IEEE Transactions on Cybernetics.

[32]  Ye Tian,et al.  A Knee Point-Driven Evolutionary Algorithm for Many-Objective Optimization , 2015, IEEE Transactions on Evolutionary Computation.

[33]  Shengxiang Yang,et al.  A test problem for visual investigation of high-dimensional multi-objective search , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[34]  Hisao Ishibuchi,et al.  Evolutionary many-objective optimization: A short review , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[35]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[36]  Jun Zhang,et al.  Fuzzy-Based Pareto Optimality for Many-Objective Evolutionary Algorithms , 2014, IEEE Transactions on Evolutionary Computation.

[37]  Hisao Ishibuchi,et al.  Modified Distance Calculation in Generational Distance and Inverted Generational Distance , 2015, EMO.