Performance comparison of NSGA-II and NSGA-III on various many-objective test problems

Recently NSGA-III has been frequently used for performance comparison of newly proposed evolutionary many-objective optimization algorithms. That is, NSGA-III has been used as a benchmark algorithm for evolutionary many-objective optimization. However, unfortunately, its source code is not available from the authors of the NSGA-III paper. This leads to an undesirable situation where a different implementation is used in a different study. Moreover, comparison is usually performed on DTLZ and WFG test problems. As a result, the performance of NSGA-III on a wide variety of many-objective test problems is still unclear whereas it has been frequently used for performance comparison in the literature. In this paper, we evaluate the performance of NSGA-III in comparison with NSGA-II on four totally different types of many-objective test problems with 3-10 objectives: DTLZ1-4 problems, their maximization variants, distance minimization problems, and knapsack problems. We use two different implementations of NSGA-II and NSGA-III. We show through computational experiments that NSGA-III does not always outperform NSGA-II even for ten-objective problems. That is, their comparison results depend not only on the number of objectives but also on the type of test problems. The choice of test problems has a larger effect on their comparison results than the number of objectives in our computational experiments. We also demonstrate that totally different results are obtained from different implementations of NSGA-III for some test problems.

[1]  Hisao Ishibuchi,et al.  A many-objective test problem for visually examining diversity maintenance behavior in a decision space , 2011, GECCO '11.

[2]  Xin Yao,et al.  Many-Objective Evolutionary Algorithms , 2015, ACM Comput. Surv..

[3]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[4]  R. Lyndon While,et al.  A review of multiobjective test problems and a scalable test problem toolkit , 2006, IEEE Transactions on Evolutionary Computation.

[5]  Xin Yao,et al.  A New Dominance Relation-Based Evolutionary Algorithm for Many-Objective Optimization , 2016, IEEE Transactions on Evolutionary Computation.

[6]  Hisao Ishibuchi,et al.  Visual examination of the behavior of EMO algorithms for many-objective optimization with many decision variables , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[7]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[8]  Hisao Ishibuchi,et al.  Behavior of Multiobjective Evolutionary Algorithms on Many-Objective Knapsack Problems , 2015, IEEE Transactions on Evolutionary Computation.

[9]  Carlos A. Coello Coello,et al.  On the Influence of the Number of Objectives on the Hardness of a Multiobjective Optimization Problem , 2011, IEEE Transactions on Evolutionary Computation.

[10]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[11]  Hisao Ishibuchi,et al.  Pareto Fronts of Many-Objective Degenerate Test Problems , 2016, IEEE Transactions on Evolutionary Computation.

[12]  Tapabrata Ray,et al.  A Decomposition-Based Evolutionary Algorithm for Many Objective Optimization , 2015, IEEE Transactions on Evolutionary Computation.

[13]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[14]  Carlos A. Brizuela,et al.  A survey on multi-objective evolutionary algorithms for many-objective problems , 2014, Computational Optimization and Applications.

[15]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[16]  Marco Laumanns,et al.  Scalable multi-objective optimization test problems , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[17]  Hisao Ishibuchi,et al.  Evolutionary many-objective optimization , 2008, 2008 3rd International Workshop on Genetic and Evolving Systems.

[18]  Qingfu Zhang,et al.  An Evolutionary Many-Objective Optimization Algorithm Based on Dominance and Decomposition , 2015, IEEE Transactions on Evolutionary Computation.

[19]  Antonio J. Nebro,et al.  jMetal: A Java framework for multi-objective optimization , 2011, Adv. Eng. Softw..