Performance assessment of multiobjective optimizers: an analysis and review

An important issue in multiobjective optimization is the quantitative comparison of the performance of different algorithms. In the case of multiobjective evolutionary algorithms, the outcome is usually an approximation of the Pareto-optimal set, which is denoted as an approximation set, and therefore the question arises of how to evaluate the quality of approximation sets. Most popular are methods that assign each approximation set a vector of real numbers that reflect different aspects of the quality. Sometimes, pairs of approximation sets are also considered. In this study, we provide a rigorous analysis of the limitations underlying this type of quality assessment. To this end, a mathematical framework is developed which allows one to classify and discuss existing techniques.

[1]  Prabuddha De,et al.  Heuristic Estimation of the Efficient Frontier for a Bi‐Criteria Scheduling Problem , 1992 .

[2]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[3]  Jason R. Schott.  Fault Tolerant Design Using Single and Multicriteria Genetic Algorithm Optimization. , 1995 .

[4]  Ernest S. Kuh,et al.  Design space exploration using the genetic algorithm , 1996, 1996 IEEE International Symposium on Circuits and Systems. Circuits and Systems Connecting the World. ISCAS 96.

[5]  Peter J. Fleming,et al.  On the Performance Assessment and Comparison of Stochastic Multiobjective Optimizers , 1996, PPSN.

[6]  Lothar Thiele,et al.  An evolutionary algorithm for multiobjective optimization: the strength Pareto approach , 1998 .

[7]  M. Hansen,et al.  Evaluating the quality of approximations to the non-dominated set , 1998 .

[8]  Piotr Czyzżak,et al.  Pareto simulated annealing—a metaheuristic technique for multiple‐objective combinatorial optimization , 1998 .

[9]  Lothar Thiele,et al.  Multiobjective Optimization Using Evolutionary Algorithms - A Comparative Case Study , 1998, PPSN.

[10]  E. L. Ulungu,et al.  MOSA method: a tool for solving multiobjective combinatorial optimization problems , 1999 .

[11]  Gary B. Lamont,et al.  Multiobjective evolutionary algorithms: classifications, analyses, and new innovations , 1999 .

[12]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[13]  Eckart Zitzler,et al.  Evolutionary algorithms for multiobjective optimization: methods and applications , 1999 .

[14]  Serpil Sayin,et al.  Measuring the quality of discrete representations of efficient sets in multiple objective mathematical programming , 2000, Math. Program..

[15]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[16]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[17]  D.A. Van Veldhuizen,et al.  On measuring multiobjective evolutionary algorithm performance , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[18]  Martin J. Oates,et al.  On the Assessment of Multiobjective Approaches to the Adaptive Distributed Database Management Problem , 2000, PPSN.

[19]  David W. Corne,et al.  Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy , 2000, Evolutionary Computation.

[20]  Shapour Azarm,et al.  Metrics for Quality Assessment of a Multiobjective Design Optimization Solution Set , 2001 .

[21]  D. Corne,et al.  On Metrics for Comparing Non Dominated Sets , 2001 .

[22]  Hans Kellerer,et al.  Approximating Multi-objective Knapsack Problems , 2001, WADS.

[23]  Carlos M. Fonseca,et al.  Inferential Performance Assessment of Stochastic Optimisers and the Attainment Function , 2001, EMO.

[24]  Tong Heng Lee,et al.  Evolutionary Algorithms for Multi-Objective Optimization: Performance Assessments and Comparisons , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[25]  Joshua D. Knowles.  Local-search and hybrid evolutionary algorithms for Pareto optimization , 2002 .

[26]  Marco Laumanns,et al.  Combining Convergence and Diversity in Evolutionary Multiobjective Optimization , 2002, Evolutionary Computation.

[27]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[28]  Marco Laumanns,et al.  Running time analysis of a multi-objective evolutionary algorithm on a simple discrete optimization problem , 2002 .

[29]  Joshua D. Knowles,et al.  On metrics for comparing nondominated sets , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[30]  Tong Heng Lee,et al.  Evolutionary Algorithms for Multi-Objective Optimization: Performance Assessments and Comparisons , 2004, Artificial Intelligence Review.