What if we Increase the Number of Objectives? Theoretical and Empirical Implications for Many-objective Optimization

The difficulty of solving a multi-objective optimization problem is impacted by the number of objectives to be optimized. The presence of many objectives typically introduces a number of challenges that affect the choice/design of optimization algorithms. This paper investigates the drivers of these challenges from two angles: (i) the influence of the number of objectives on problem characteristics and (ii) the practical behavior of commonly used procedures and algorithms for coping with many objectives. In addition to reviewing various drivers, the paper makes theoretical contributions by quantifying some drivers and/or verifying these drivers empirically by carrying out experiments on multi-objective NK landscapes and other typical benchmarks. We then make use of our theoretical and empirical findings to derive practical recommendations to support algorithm design. Finally, we discuss remaining theoretical gaps and opportunities for future research in the area of multiand many-objective optimization. Contribution: Survey, theory, empirical, implications on algorithm design.

[1]  Jana Thomann,et al.  A trust region approach for multi-objective heterogeneous optimization , 2019 .

[2]  Kaisa Miettinen,et al.  Nonlinear multiobjective optimization , 1998, International series in operations research and management science.

[3]  Mario Köppen,et al.  Fuzzy-Pareto-Dominance and its Application in Evolutionary Multi-objective Optimization , 2005, EMO.

[4]  R. W. Saaty,et al.  The analytic hierarchy process—what it is and how it is used , 1987 .

[5]  Vesa Ojalehto,et al.  Surrogate-assisted evolutionary biobjective optimization for objectives with non-uniform latencies , 2018, GECCO.

[6]  Lei Chen,et al.  A Fast Approximate Hypervolume Calculation Method by a Novel Decomposition Strategy , 2017, ICIC.

[7]  Kiyoshi Tanaka,et al.  Ieee Transactions on Evolutionary Computation Computational Cost Reduction of Non-dominated Sorting Using the M-front , 2022 .

[8]  Jonathan. E. Fieldsend,et al.  Data structures for non-dominated sets: implementations and empirical assessment of two decades of advances , 2020, GECCO.

[9]  Markus Wagner,et al.  Evolutionary many-objective optimization: A quick-start guide , 2015 .

[10]  D. Malcolm,et al.  Application of a Technique for Research and Development Program Evaluation , 1959 .

[11]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[12]  Bharat M. Deshpande,et al.  Empirical and analytical study of many-objective optimization problems: analysing distribution of nondominated solutions and population size for scalability of randomized heuristics , 2014, Memetic Computing.

[13]  Luc Crevecoeur,et al.  Optimization and inverse problems in electromagnetism , 2014 .

[14]  Carlos M. Fonseca,et al.  A Box Decomposition Algorithm to Compute the Hypervolume Indicator , 2015, Comput. Oper. Res..

[15]  L. Paquete,et al.  Easy to say they are Hard, but Hard to see they are Easy— Towards a Categorization of Tractable Multiobjective Combinatorial Optimization Problems , 2017 .

[16]  H. Riahi,et al.  Variational Methods in Partially Ordered Spaces , 2003 .

[17]  Hernán E. Aguirre.  Advances on many-objective evolutionary optimization , 2013, GECCO '13 Companion.

[18]  Evan J. Hughes,et al.  Evolutionary many-objective optimisation: many once or one many? , 2005, 2005 IEEE Congress on Evolutionary Computation.

[19]  Nicola Beume,et al.  On the Complexity of Computing the Hypervolume Indicator , 2009, IEEE Transactions on Evolutionary Computation.

[20]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[21]  Joshua D. Knowles,et al.  'Hang On a Minute': Investigations on the Effects of Delayed Objective Functions in Multiobjective Optimization , 2013, EMO.

[22]  Ignacy Kaliszewski,et al.  Quantitative Pareto Analysis by Cone Separation Technique , 1994 .

[23]  Tobias Glasmachers,et al.  A Fast Incremental BSP Tree Archive for Non-dominated Points , 2016, EMO.

[24]  Nicola Beume,et al.  Pareto-, Aggregation-, and Indicator-Based Methods in Many-Objective Optimization , 2007, EMO.

[25]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[26]  Jonathan E. Fieldsend,et al.  A feature rich distance-based many-objective visualisable test problem generator , 2019, GECCO.

[27]  Eckart Zitzler,et al.  Indicator-Based Selection in Multiobjective Search , 2004, PPSN.

[28]  Sébastien Vérel,et al.  Problem Features versus Algorithm Performance on Rugged Multiobjective Combinatorial Fitness Landscapes , 2016, Evolutionary Computation.

[29]  Carlos M. Fonseca,et al.  Computing and Updating Hypervolume Contributions in Up to Four Dimensions , 2018, IEEE Transactions on Evolutionary Computation.

[30]  Gabriele Eichfelder,et al.  Adaptive Scalarization Methods in Multiobjective Optimization , 2008, Vector Optimization.

[31]  Christiane Tammer,et al.  On Some Methods to Derive Necessary and Sufficient Optimality Conditions in Vector Optimization , 2017, J. Optim. Theory Appl..

[32]  Andrzej Jaszkiewicz,et al.  Approximate Hypervolume Calculation with Guaranteed or Confidence Bounds , 2020, PPSN.

[33]  Eckart Zitzler,et al.  HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization , 2011, Evolutionary Computation.

[34]  Yaochu Jin,et al.  Transfer learning for gaussian process assisted evolutionary bi-objective optimization for objectives with different evaluation times , 2020, GECCO.

[35]  L. Lasdon,et al.  On a bicriterion formation of the problems of integrated system identification and system optimization , 1971 .

[36]  P. Serafini,et al.  Scalarizing vector optimization problems , 1984 .

[37]  Timothy M. Chan.  Klee's Measure Problem Made Easy , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[38]  R. Geoff Dromey,et al.  An algorithm for the selection problem , 1986, Softw. Pract. Exp..

[39]  Peter J. Fleming,et al.  Evolutionary many-objective optimisation: an exploratory analysis , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[40]  Minghe Sun,et al.  Quad-Trees and Linear Lists for Identifying Nondominated Criterion Vectors , 1996, INFORMS J. Comput..

[41]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[42]  Tea Tusar,et al.  Visualization of Pareto Front Approximations in Evolutionary Multiobjective Optimization: A Critical Review and the Prosection Method , 2015, IEEE Transactions on Evolutionary Computation.

[43]  Bernard Cornet,et al.  Existence of equilibria when firms follow bounded losses pricing rules , 1988 .

[44]  Minghe Sun.  A primogenitary linked quad tree approach for solution storage and retrieval in heuristic binary optimization , 2011, Eur. J. Oper. Res..

[45]  Dipti Srinivasan,et al.  A Survey of Multiobjective Evolutionary Algorithms Based on Decomposition , 2017, IEEE Transactions on Evolutionary Computation.

[46]  Andrzej Jaszkiewicz,et al.  Improved quick hypervolume algorithm , 2016, Comput. Oper. Res..

[47]  Minghe Sun,et al.  A primogenitary linked quad tree data structure and its application to discrete multiple criteria optimization , 2006, Ann. Oper. Res..

[48]  E. Polak.  On the Approximation of Solutions to Multiple Criteria Decision Making Problems , 1976 .

[49]  Kathrin Klamroth,et al.  Navigation in multiobjective optimization methods , 2017 .

[50]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[51]  Daniel Vanderpooten,et al.  On the number of non-dominated points of a multicriteria optimization problem , 2013, Discret. Appl. Math..

[52]  D. Luenberger.  Benefit functions and duality , 1992 .

[53]  Peter J. Fleming,et al.  Many-Objective Optimization: An Engineering Design Perspective , 2005, EMO.

[54]  Patrick M. Reed,et al.  Many‐objective groundwater monitoring network design using bias‐aware ensemble Kalman filtering, evolutionary optimization, and visual analytics , 2011 .

[55]  Hisao Ishibuchi,et al.  Evolutionary many-objective optimization: A short review , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[56]  Sébastien Vérel,et al.  On the structure of multiobjective combinatorial search space: MNK-landscapes with correlated objectives , 2013, Eur. J. Oper. Res..

[57]  Bertrand Crettez,et al.  On the Characterization of Efficient Production Vectors , 2007 .

[58]  Joshua D. Knowles,et al.  Efficient discovery of anti-inflammatory small molecule combinations using evolutionary computing , 2011, Nature chemical biology.

[59]  Alessio Ishizaka,et al.  Multi-criteria decision analysis , 2013 .

[60]  Joshua D. Knowles,et al.  Multiobjective Optimization: When Objectives Exhibit Non-Uniform Latencies , 2015 .

[61]  M. Agha,et al.  Experiments with Mixtures , 1992 .

[62]  Theodor J. Stewart,et al.  A spatial optimization algorithm for geodesign , 2015 .

[63]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[64]  Peter Schönfeld,et al.  Some duality theorems for the non-linear vector maximum problem , 1970, Unternehmensforschung.

[65]  Andrzej Jaszkiewicz,et al.  ND-Tree-Based Update: A Fast Algorithm for the Dynamic Nondominance Problem , 2016, IEEE Transactions on Evolutionary Computation.

[66]  Jonathan E. Fieldsend,et al.  Using unconstrained elite archives for multiobjective optimization , 2003, IEEE Trans. Evol. Comput..

[67]  R. Newcombe.  Two-sided confidence intervals for the single proportion: comparison of seven methods. , 1998, Statistics in medicine.

[68]  Bilel Derbel,et al.  Landscape-Aware Performance Prediction for Evolutionary Multiobjective Optimization , 2020, IEEE Transactions on Evolutionary Computation.

[69]  Jürgen Teich,et al.  Quad-trees: A Data Structure for Storing Pareto Sets in Multiobjective Evolutionary Algorithms with Elitism , 2005, Evolutionary Multiobjective Optimization.

[70]  D. Luenberger.  New optimality principles for economic efficiency and equilibrium , 1992 .

[71]  Kiyoshi Tanaka,et al.  Working principles, behavior, and performance of MOEAs on MNK-landscapes , 2007, Eur. J. Oper. Res..