A Correlation Analysis of Set Quality Indicator Values in Multiobjective Optimization

A large spectrum of quality indicators has been proposed so far to assess the performance of discrete Pareto set approximations in multiobjective optimization. Such indicators assign, to any solution set, a real-value reflecting a given aspect of approximation quality. This is an important issue in multiobjective optimization, not only to compare the performance and assets of different approximate algorithms, but also to improve their internal selection mechanisms. In this paper, we adopt a statistical analysis to experimentally investigate by how much a selection of state-of-the-art quality indicators agree with each other for a wide range of Pareto set approximations from well-known two- and three-objective continuous benchmark functions. More particularly, we measure the correlation between the ranking of low-, medium-, and high-quality limited-size approximation sets with respect to inverted generational distance, additive epsilon, multiplicative epsilon, R2, R3, as well as hypervolume indicator values. Since no pair of indicators obtains the same ranking of approximation sets, we confirm that they emphasize different facets of approximation quality. More importantly, our statistical analysis allows the degree of compliance between these indicators to be quantified.

[1]  Lothar Thiele,et al.  On Set-Based Multiobjective Optimization , 2010, IEEE Transactions on Evolutionary Computation.

[2]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[3]  Lothar Thiele,et al.  Quality Assessment of Pareto Set Approximations , 2008, Multiobjective Optimization.

[4]  Joshua D. Knowles,et al.  On metrics for comparing nondominated sets , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[5]  Heike Trautmann,et al.  2 Indicator-Based Multiobjective Search , 2015, Evolutionary Computation.

[6]  Jie Zhang,et al.  Consistencies and Contradictions of Performance Metrics in Multiobjective Optimization , 2014, IEEE Transactions on Cybernetics.

[7]  Qingfu Zhang,et al.  Multiobjective optimization Test Instances for the CEC 2009 Special Session and Competition , 2009 .

[8]  Carlos A. Coello Coello,et al.  Using the Averaged Hausdorff Distance as a Performance Measure in Evolutionary Multiobjective Optimization , 2012, IEEE Transactions on Evolutionary Computation.

[9]  D. Corne,et al.  On Metrics for Comparing Non Dominated Sets , 2001 .

[10]  Lothar Thiele,et al.  A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers , 2006 .

[11]  Francisco Herrera,et al.  Analyzing convergence performance of evolutionary algorithms: A statistical approach , 2014, Inf. Sci..

[12]  Hadley Wickham,et al.  ggplot2 - Elegant Graphics for Data Analysis (2nd Edition) , 2017 .

[13]  Olaf Mersmann,et al.  Multiple Criteria Optimization Algorithms and Related Functions , 2014 .

[14]  Simon Wessing,et al.  Sequential parameter optimization for multi-objective problems , 2010, IEEE Congress on Evolutionary Computation.

[15]  M. Hansen,et al.  Evaluating the quality of approximations to the non-dominated set , 1998 .

[16]  Tobias Friedrich,et al.  Efficient computation of two-dimensional solution sets maximizing the epsilon-indicator , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[17]  Nicola Beume,et al.  SMS-EMOA: Multiobjective selection based on dominated hypervolume , 2007, Eur. J. Oper. Res..

[18]  Sébastien Vérel,et al.  On set-based local search for multiobjective combinatorial optimization , 2013, GECCO '13.

[19]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[20]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[21]  Hisao Ishibuchi,et al.  Modified Distance Calculation in Generational Distance and Inverted Generational Distance , 2015, EMO.

[22]  Carlos A. Coello Coello,et al.  Solving Multiobjective Optimization Problems Using an Artificial Immune System , 2005, Genetic Programming and Evolvable Machines.

[23]  Eckart Zitzler,et al.  HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization , 2011, Evolutionary Computation.

[24]  M. Kendall.  A NEW MEASURE OF RANK CORRELATION , 1938 .

[25]  David W. Corne,et al.  Properties of an adaptive archiving algorithm for storing nondominated vectors , 2003, IEEE Trans. Evol. Comput..

[26]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[27]  Anne Auger,et al.  Hypervolume-based multiobjective optimization: Theoretical foundations and practical implications , 2012, Theor. Comput. Sci..

[28]  Lothar Thiele,et al.  Multiobjective Optimization Using Evolutionary Algorithms - A Comparative Case Study , 1998, PPSN.

[29]  Heike Trautmann,et al.  Optimal averaged Hausdorff archives for bi-objective problems: theoretical and numerical results , 2016, Comput. Optim. Appl..

[30]  David A. Van Veldhuizen,et al.  Evolutionary Computation and Convergence to a Pareto Front , 1998 .