A tutorial on multiobjective optimization: fundamentals and evolutionary methods

In almost no other field of computer science, the idea of using bio-inspired search paradigms has been so useful as in solving multiobjective optimization problems. The idea of using a population of search agents that collectively approximate the Pareto front resonates well with processes in natural evolution, immune systems, and swarm intelligence. Methods such as NSGA-II, SPEA2, SMS-EMOA, MOPSO, and MOEA/D became standard solvers when it comes to solving multiobjective optimization problems. This tutorial will review some of the most important fundamentals in multiobjective optimization and then introduce representative algorithms, illustrate their working principles, and discuss their application scope. In addition, the tutorial will discuss statistical performance assessment. Finally, it highlights recent important trends and closely related research fields. The tutorial is intended for readers, who want to acquire basic knowledge on the mathematical foundations of multiobjective optimization and state-of-the-art methods in evolutionary multiobjective optimization. The aim is to provide a starting point for researching in this active area, and it should also help the advanced reader to identify open research topics.

[1]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[2]  Frank Kursawe,et al.  A Variant of Evolution Strategies for Vector Optimization , 1990, PPSN.

[3]  Carlos A. Coello Coello,et al.  A Micro-Genetic Algorithm for Multiobjective Optimization , 2001, EMO.

[4]  Matthias Ehrgott,et al.  Multiple criteria decision analysis: state of the art surveys , 2005 .

[5]  Heike Trautmann,et al.  Building and Using an Ontology of Preference-Based Multiobjective Evolutionary Algorithms , 2017, EMO.

[6]  Heike Trautmann,et al.  R2-EMOA: Focused Multiobjective Search Using R2-Indicator-Based Selection , 2013, LION.

[7]  Joshua D. Knowles,et al.  Bounded archiving using the lebesgue measure , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[8]  Anne Auger,et al.  Theory of the hypervolume indicator: optimal μ-distributions and the choice of the reference point , 2009, FOGA '09.

[9]  Benjamín Barán,et al.  A Multiobjective Ant Colony System for Vehicle Routing Problem with Time Windows , 2003, Applied Informatics.

[10]  Peter J. Fleming,et al.  Generalized decomposition and cross entropy methods for many-objective optimization , 2014, Inf. Sci..

[11]  Peter J. Fleming,et al.  Genetic Algorithms for Multiobjective Optimization: FormulationDiscussion and Generalization , 1993, ICGA.

[12]  Harold R. Parks,et al.  The Implicit Function Theorem , 2002 .

[13]  Feike Schieving,et al.  Functional traits determine trade-offs and niches in a tropical forest community , 2011, Proceedings of the National Academy of Sciences.

[14]  Claus Hillermeier,et al.  Nonlinear Multiobjective Optimization , 2001 .

[15]  Qguhm -DVNLHZLF,et al.  On the performance of multiple objective genetic local search on the 0 / 1 knapsack problem . A comparative experiment , 2000 .

[16]  Hao Wang,et al.  Hypervolume Indicator Gradient Ascent Multi-objective Optimization , 2017, EMO.

[17]  Yaochu Jin,et al.  Surrogate-Assisted Multicriteria Optimization: Complexities, Prospective Solutions, and Business Case , 2017 .

[18]  Oliver Schütze,et al.  On Continuation Methods for the Numerical Treatment of Multi-Objective Optimization Problems , 2005, Practical Approaches to Multi-Objective Optimization.

[19]  Kalyanmoy Deb,et al.  Multiobjective Problem Solving from Nature: From Concepts to Applications , 2008, Natural Computing Series.

[20]  Dimo Brockhoff,et al.  GECCO 2017 tutorial on evolutionary multiobjective optimization , 2017, GECCO.

[21]  Michael T. M. Emmerich,et al.  Time Complexity and Zeros of the Hypervolume Indicator Gradient Field , 2012, EVOLVE.

[22]  Theodor J. Stewart,et al.  Multiple Criteria Decision Analysis , 2001 .

[23]  Nicola Beume,et al.  Convergence rates of SMS-EMOA on continuous bi-objective problem classes , 2011, FOGA '11.

[24]  Bogdan Filipic,et al.  DEMO: Differential Evolution for Multiobjective Optimization , 2005, EMO.

[25]  Hisao Ishibuchi,et al.  Reference point specification in hypervolume calculation for fair comparison and efficient search , 2017, GECCO.

[26]  Eckart Zitzler,et al.  Indicator-Based Selection in Multiobjective Search , 2004, PPSN.

[27]  C. Hillermeier.  Nonlinear Multiobjective Optimization: A Generalized Homotopy Approach , 2001 .

[28]  Jakob Bossek,et al.  ecr 2.0: a modular framework for evolutionary computation in R , 2017, GECCO.

[29]  Charles Audet,et al.  A mesh adaptive direct search algorithm for multiobjective optimization , 2009, Eur. J. Oper. Res..

[30]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[31]  Luís N. Vicente,et al.  Direct Multisearch for Multiobjective Optimization , 2011, SIAM J. Optim..

[32]  Marco Laumanns,et al.  PISA: A Platform and Programming Language Independent Interface for Search Algorithms , 2003, EMO.

[33]  Joshua D. Knowles,et al.  ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems , 2006, IEEE Transactions on Evolutionary Computation.

[34]  Carlos M. Fonseca,et al.  Inferential Performance Assessment of Stochastic Optimisers and the Attainment Function , 2001, EMO.

[35]  Michael Emmerich,et al.  Logarithmic-Time Updates in SMS-EMOA and Hypervolume-Based Archiving , 2013 .

[36]  Heike Trautmann,et al.  OCD: Online Convergence Detection for Evolutionary Multi-Objective Algorithms Based on Statistical Testing , 2009, EMO.

[37]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[38]  Günter Rudolph,et al.  Convergence properties of some multi-objective evolutionary algorithms , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[39]  Markus Borschbach,et al.  Design Perspectives of an Evolutionary Process for Multi-objective Molecular Optimization , 2017, EMO.

[40]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[41]  Xavier Gandibleux,et al.  A survey and annotated bibliography of multiobjective combinatorial optimization , 2000, OR Spectr..

[42]  Carlos M. Fonseca,et al.  A Portfolio Optimization Approach to Selection in Multiobjective Evolutionary Algorithms , 2014, PPSN.

[43]  Kalyanmoy Deb,et al.  Introduction to Evolutionary Multiobjective Optimization , 2008, Multiobjective Optimization.

[44]  Peter J. Fleming,et al.  Liger: an open source integrated optimization environment , 2013, GECCO.

[45]  Jürgen Branke,et al.  Using Choquet integral as preference model in interactive evolutionary multiobjective optimization , 2016, Eur. J. Oper. Res..

[46]  P. M. Mateo,et al.  A mutation operator based on a Pareto ranking for multi-objective evolutionary algorithms , 2011, Journal of Heuristics.

[47]  David W. Corne,et al.  Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy , 2000, Evolutionary Computation.

[48]  Kaisa Miettinen,et al.  Nonlinear multiobjective optimization , 1998, International series in operations research and management science.

[49]  Hao Wang,et al.  Towards Analyzing Multimodality of Continuous Multiobjective Landscapes , 2016, PPSN.

[50]  A. L. Custódio,et al.  Recent Developments in Derivative-Free Multiobjective Optimisation , 2012, Computational Technology Reviews.

[51]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[52]  Carlos A. Coello Coello,et al.  Constraint-handling techniques used with evolutionary algorithms , 2017, GECCO.

[53]  Marc Sevaux,et al.  The Biobjective Inventory Routing Problem - Problem Solution and Decision Support , 2011, INOC.

[54]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[55]  Christian Grimme,et al.  Inside a predator-prey model for multi-objective optimization: a second study , 2006, GECCO '06.

[56]  Matthias Ehrgott,et al.  Multiple Criteria Decision Analysis , 2016 .

[57]  Antanas Zilinskas.  On the worst-case optimal multi-objective global optimization , 2013, Optim. Lett..

[58]  Jürgen Branke,et al.  Learning Value Functions in Interactive Evolutionary Multiobjective Optimization , 2015, IEEE Transactions on Evolutionary Computation.

[59]  M. Ehrgott.  Vilfredo Pareto and multi-objective optimization , 2012 .

[60]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[61]  Hao Wang,et al.  A Multicriteria Generalization of Bayesian Global Optimization , 2016, Advances in Stochastic and Deterministic Global Optimization.

[62]  Peter J. Fleming,et al.  Many-Objective Optimization: An Engineering Design Perspective , 2005, EMO.

[63]  Michael T. M. Emmerich,et al.  Multi-Objective Evolutionary Design of Adenosine Receptor Ligands , 2012, J. Chem. Inf. Model..

[64]  Kaisa Miettinen,et al.  Interactive multiobjective optimization system WWW-NIMBUS on the Internet , 2000, Comput. Oper. Res..

[65]  Eckart Zitzler,et al.  HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization , 2011, Evolutionary Computation.

[66]  Bernhard Sendhoff,et al.  Adapting Weighted Aggregation for Multiobjective Evolution Strategies , 2001, EMO.

[67]  Xin Yao,et al.  Many-Objective Evolutionary Algorithms , 2015, ACM Comput. Surv..

[68]  Carlos A. Coello Coello,et al.  Solving Multiobjective Optimization Problems Using an Artificial Immune System , 2005, Genetic Programming and Evolvable Machines.

[69]  Carlos M. Fonseca,et al.  Computing Hypervolume Contributions in Low Dimensions: Asymptotically Optimal Algorithm and Complexity Results , 2011, EMO.

[70]  Marco Laumanns,et al.  A Tutorial on Evolutionary Multiobjective Optimization , 2004, Metaheuristics for Multiobjective Optimisation.

[71]  Carlos M. Fonseca,et al.  Computing and Updating Hypervolume Contributions in Up to Four Dimensions , 2018, IEEE Transactions on Evolutionary Computation.

[72]  Heike Trautmann,et al.  Optimal averaged Hausdorff archives for bi-objective problems: theoretical and numerical results , 2016, Comput. Optim. Appl..

[73]  Carlos A. Coello Coello,et al.  The directed search method for multi-objective memetic algorithms , 2015, Computational Optimization and Applications.

[74]  Markus Wagner,et al.  Efficient optimization of many objectives by approximation-guided evolution , 2015, Eur. J. Oper. Res..

[75]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[76]  Christian Igel,et al.  Steady-State Selection and Efficient Covariance Matrix Update in the Multi-objective CMA-ES , 2007, EMO.

[77]  Luigi Barone,et al.  An evolution strategy with probabilistic mutation for multi-objective optimisation , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[78]  C. Fonseca,et al.  GENETIC ALGORITHMS FOR MULTI-OBJECTIVE OPTIMIZATION: FORMULATION, DISCUSSION, AND GENERALIZATION , 1993 .

[79]  M Reyes Sierra,et al.  Multi-Objective Particle Swarm Optimizers: A Survey of the State-of-the-Art , 2006 .

[80]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[81]  Andrzej Jaszkiewicz,et al.  The 'Light Beam Search' approach - an overview of methodology and applications , 1999, Eur. J. Oper. Res..

[82]  Marco Laumanns,et al.  A Spatial Predator-Prey Approach to Multi-objective Optimization: A Preliminary Study , 1998, PPSN.

[83]  Benjamín Barán,et al.  Performance metrics in multi-objective optimization , 2015, 2015 Latin American Computing Conference (CLEI).

[84]  Kalyanmoy Deb,et al.  Simulated Binary Crossover for Continuous Search Space , 1995, Complex Syst..

[85]  Matthias Ehrgott,et al.  Multicriteria Optimization , 2005 .

[86]  José Francisco Aldana Montes,et al.  Multi-objective Big Data Optimization with jMetal and Spark , 2017, EMO.

[87]  Xin Yao,et al.  Convex Hull-Based Multiobjective Genetic Programming for Maximizing Receiver Operating Characteristic Performance , 2015, IEEE Transactions on Evolutionary Computation.

[88]  John E. Dennis,et al.  Normal-Boundary Intersection: A New Method for Generating the Pareto Surface in Nonlinear Multicriteria Optimization Problems , 1998, SIAM J. Optim..

[89]  Nicola Beume,et al.  An EMO Algorithm Using the Hypervolume Measure as Selection Criterion , 2005, EMO.

[90]  Gary B. Lamont,et al.  Evolutionary algorithms for solving multi-objective problems, Second Edition , 2007, Genetic and evolutionary computation series.

[91]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[92]  M. Dellnitz,et al.  Covering Pareto Sets by Multilevel Subdivision Techniques , 2005 .

[93]  Kaisa Miettinen,et al.  Connections of reference vectors and different types of preference information in interactive multiobjective evolutionary algorithms , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[94]  Theodor J. Stewart,et al.  Multiple criteria decision analysis - an integrated approach , 2001 .

[95]  Elchanan Mossel,et al.  Sorting and Selection in Posets , 2007, SIAM J. Comput..

[96]  Michael T. M. Emmerich,et al.  Cone-Based Hypervolume Indicators: Construction, Properties, and Efficient Computation , 2013, EMO.

[97]  Hisao Ishibuchi,et al.  Multi-objective genetic local search algorithm , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[98]  Iryna Yevseyeva,et al.  Optimising anti-spam filters with evolutionary algorithms , 2013, Expert Syst. Appl..

[99]  Oliver Schütze,et al.  Hypervolume Maximization via Set Based Newton’s Method , 2014 .