Pareto Tracer: a predictor–corrector method for multi-objective optimization problems

ABSTRACT This article proposes a novel predictor–corrector (PC) method for the numerical treatment of multi-objective optimization problems (MOPs). The algorithm, Pareto Tracer (PT), is capable of performing a continuation along the set of (local) solutions of a given MOP with k objectives, and can cope with equality and box constraints. Additionally, the first steps towards a method that manages general inequality constraints are also introduced. The properties of PT are first discussed theoretically and later numerically on several examples.

[1]  H. Schwetlick.  Numerische Lösung nichtlinearer Gleichungen , 1978 .

[2]  Alexandre Goldsztejn,et al.  On continuation methods for non-linear bi-objective optimization: towards a certified interval-based approach , 2014, Journal of Global Optimization.

[3]  Bing Liang,et al.  Trust region methods for solving multiobjective optimisation , 2013, Optim. Methods Softw..

[4]  Kai-Tai Fang,et al.  Uniform design and its industrial applications , 2007 .

[5]  Fu-Rui Xiong,et al.  Parallel Cell Mapping for Unconstrained Multi-Objective Optimization Problems , 2014 .

[6]  S. Utyuzhnikov,et al.  Directed search domain: a method for even generation of the Pareto frontier in multiobjective optimization , 2011 .

[7]  M. Dellnitz,et al.  Covering Pareto Sets by Multilevel Subdivision Techniques , 2005 .

[8]  Andreas Potschka,et al.  Tracing the Pareto frontier in bi-objective optimization problems by ODE techniques , 2011, Numerical Algorithms.

[9]  E. Allgower,et al.  Numerical path following , 1997 .

[10]  Oliver Schütze,et al.  A New Predictor Corrector Variant for Unconstrained Bi-objective Optimization Problems , 2014 .

[11]  Léonard Jaillet,et al.  Randomized path planning on manifolds based on higher-dimensional continuation , 2012, Int. J. Robotics Res..

[12]  C. A. Coello Coello,et al.  Hybridizing evolutionary strategies with continuation methods for solving multi-objective problems , 2008 .

[13]  Lisa Turner,et al.  Applications of Second Order Cone Programming , 2012 .

[14]  Michael E. Henderson,et al.  Multiple Parameter Continuation: Computing Implicitly Defined k-Manifolds , 2002, Int. J. Bifurc. Chaos.

[15]  Oliver Schütze,et al.  On Continuation Methods for the Numerical Treatment of Multi-Objective Optimization Problems , 2005, Practical Approaches to Multi-Objective Optimization.

[16]  Yousef Naranjani,et al.  Simple cell mapping method for multi-objective optimal feedback control design , 2013, International Journal of Dynamics and Control.

[17]  Qingfu Zhang,et al.  Multiobjective Optimization Problems With Complicated Pareto Sets, MOEA/D and NSGA-II , 2009, IEEE Transactions on Evolutionary Computation.

[18]  Johannes Jahn,et al.  Multiobjective Search Algorithm with Subdivision Technique , 2006, Comput. Optim. Appl..

[19]  Gene H. Golub,et al.  Methods for modifying matrix factorizations , 1972, Milestones in Matrix Computation.

[20]  H. Fawcett.  Manual of Political Economy , 1995 .

[21]  Pablo Moscato,et al.  Handbook of Memetic Algorithms , 2011, Studies in Computational Intelligence.

[22]  C. Coello,et al.  MONSS: A multi-objective nonlinear simplex search approach , 2016 .

[23]  Joshua D. Knowles,et al.  Memetic Algorithms for Multiobjective Optimization: Issues, Methods and Prospects , 2004 .

[24]  Benedetta Morini,et al.  TRESNEI, a Matlab trust-region solver for systems of nonlinear equalities and inequalities , 2010, Computational Optimization and Applications.

[25]  Carlos A. Coello Coello,et al.  HCS: A New Local Search Strategy for Memetic Multiobjective Evolutionary Algorithms , 2010, IEEE Transactions on Evolutionary Computation.

[26]  A. Messac,et al.  Normal Constraint Method with Guarantee of Even Representation of Complete Pareto Frontier , 2004 .

[27]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[28]  José Castillo,et al.  Equispaced Pareto front construction for constrained bi-objective optimization , 2013, Math. Comput. Model..

[29]  Aubrey B. Poore,et al.  Numerical Continuation and Singularity Detection Methods for Parametric Nonlinear Programming , 1993, SIAM J. Optim..

[30]  Kurt M. Anstreicher,et al.  Institute for Mathematical Physics Semidefinite Programming versus the Reformulation–linearization Technique for Nonconvex Quadratically Constrained Quadratic Programming Semidefinite Programming versus the Reformulation-linearization Technique for Nonconvex Quadratically Constrained , 2022 .

[31]  Jörg Fliege,et al.  Gap-free computation of Pareto-points by quadratic scalarizations , 2004, Math. Methods Oper. Res..

[32]  Dmitry Podkopaev,et al.  Special issue on global optimization with multiple objectives , 2016, J. Glob. Optim..

[33]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[34]  Kaisa Miettinen,et al.  PAINT: Pareto front interpolation for nonlinear multiobjective optimization , 2012, Comput. Optim. Appl..

[35]  Carlos A. Coello Coello,et al.  The Gradient Free Directed Search Method as Local Search within Multi-Objective Evolutionary Algorithms , 2012, EVOLVE.

[36]  Antoine Soubeyran,et al.  A Trust-Region Method for Unconstrained Multiobjective Problems with Applications in Satisficing Processes , 2014, J. Optim. Theory Appl..

[37]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[38]  J. Greenstadt.  Revision of a derivative-free quasi-Newton method , 1978 .

[39]  Lino A. Costa,et al.  Generalized Multiobjective Evolutionary Algorithm Guided by Descent Directions , 2014, J. Math. Model. Algorithms Oper. Res..

[40]  Jong-hyun Ryu,et al.  A Derivative-Free Trust-Region Method for Biobjective Optimization , 2014, SIAM J. Optim..

[41]  Pascal Bouvry,et al.  EVOLVE - A Bridge between Probability, Set Oriented Numerics, and Evolutionary Computation VI, EVOLVE 2015, Iasi, Romania, 18-24 June 2015 , 2012, EVOLVE.

[42]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[43]  S. Schäffler,et al.  Stochastic Method for the Solution of Unconstrained Vector Optimization Problems , 2002 .

[44]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[45]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .

[46]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, STOC '84.

[47]  J. Koski.  Defectiveness of weighting method in multicriterion optimization of structures , 1985 .

[48]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[49]  Enrique Alba,et al.  A comparative study of the effect of parameter scalability in multi-objective metaheuristics , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[50]  P. Fantini,et al.  A method for generating a well-distributed Pareto set in nonlinear multiobjective optimization , 2005 .

[51]  C. Hillermeier.  Nonlinear Multiobjective Optimization: A Generalized Homotopy Approach , 2001 .

[52]  Jürgen Bräuninger.  A quasi-Newton method with Cholesky factorization , 2005, Computing.

[53]  R. T. Haftka,et al.  An active set algorithm for tracing parametrized optima , 1990 .

[54]  B. Svaiter,et al.  A steepest descent method for vector optimization , 2005 .

[55]  Qingfu Zhang,et al.  Multiobjective evolutionary algorithms: A survey of the state of the art , 2011, Swarm Evol. Comput..

[56]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[57]  Saúl Zapotecas Martínez,et al.  A Memetic Algorithm with Non Gradient-Based Local Search Assisted by a Meta-model , 2010, PPSN.

[58]  Dong-Hui Li,et al.  A new class of quasi-Newton updating formulas , 2008, Optim. Methods Softw..

[59]  P. Gill,et al.  Quasi-Newton Methods for Unconstrained Optimization , 1972 .

[60]  A. Messac,et al.  The normalized normal constraint method for generating the Pareto frontier , 2003 .

[61]  Alexandre Goldsztejn,et al.  Certified Parallelotope Continuation for One-Manifolds , 2013, SIAM J. Numer. Anal..

[62]  Jorge Nocedal,et al.  A Numerical Study of the Limited Memory BFGS Method and the Truncated-Newton Method for Large Scale Optimization , 1991, SIAM J. Optim..

[63]  Lino A. Costa,et al.  A New Hybrid Evolutionary Multiobjective Algorithm Guided by Descent Directions , 2013, J. Math. Model. Algorithms Oper. Res..

[64]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[65]  S. Ober-Blöbaum,et al.  Handling high-dimensional problems with multi-objective continuation methods via successive approximation of the tangent space , 2012 .

[66]  E. Allgower,et al.  An Algorithm for Piecewise-Linear Approximation of an Implicitly Defined Manifold , 1985 .

[67]  Isao Ono,et al.  Uniform sampling of local pareto-optimal solution curves by pareto path following and its applications in multi-objective GA , 2007, GECCO '07.

[68]  Achille Messac,et al.  Physical programming - Effective optimization for computational design , 1996 .

[69]  Gabriele Eichfelder,et al.  Adaptive Scalarization Methods in Multiobjective Optimization , 2008, Vector Optimization.

[70]  Víctor Pereyra,et al.  Fast computation of equispaced Pareto manifolds and Pareto fronts for multiobjective optimization problems , 2009, Math. Comput. Simul..

[71]  Carlos Artemio Coello-Coello,et al.  Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art , 2002 .

[72]  Lotfi A. Zadeh,et al.  Optimality and non-scalar-valued performance criteria , 1963 .

[73]  François-Xavier Le Dimet,et al.  Numerical Experience with Limited-Memory Quasi-Newton and Truncated Newton Methods , 1993, SIAM J. Optim..

[74]  Claus Hillermeier,et al.  Nonlinear Multiobjective Optimization , 2001 .

[75]  P. Papalambros,et al.  A NOTE ON WEIGHTED CRITERIA METHODS FOR COMPROMISE SOLUTIONS IN MULTI-OBJECTIVE OPTIMIZATION , 1996 .

[76]  J. Butcher.  Numerical methods for ordinary differential equations in the 20th century , 2000 .

[77]  E. Allgower,et al.  Numerical Continuation Methods , 1990 .

[78]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[79]  Layne T. Watson,et al.  Multi-Objective Control-Structure Optimization via Homotopy Methods , 1993, SIAM J. Optim..

[80]  Honggang Wang,et al.  Zigzag Search for Continuous Multiobjective Optimization , 2013, INFORMS J. Comput..

[81]  Donald Goldfarb,et al.  Second-order cone programming , 2003, Math. Program..

[82]  Maria Cristina Recchioni,et al.  A path following method for box-constrained multiobjective optimization with applications to goal programming problems , 2003, Math. Methods Oper. Res..

[83]  Oliver Schütze,et al.  Computing the Set of Approximate Solutions of a Multi-objective Optimization Problem by Means of Cell Mapping Techniques , 2013 .

[84]  Jörg Fliege,et al.  Newton's Method for Multiobjective Optimization , 2009, SIAM J. Optim..

[85]  Kaisa Miettinen,et al.  Nonlinear multiobjective optimization , 1998, International series in operations research and management science.

[86]  Honggang Wang,et al.  Direct zigzag search for discrete multi-objective optimization , 2015, Comput. Oper. Res..

[87]  Gara Miranda,et al.  Using multi-objective evolutionary algorithms for single-objective optimization , 2013, 4OR.

[88]  Enrico Miglierina,et al.  Box-constrained multi-objective optimization: A gradient-like method without "a priori" scalarization , 2008, Eur. J. Oper. Res..

[89]  Marco Laumanns,et al.  Scalable multi-objective optimization test problems , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[90]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[91]  Carlos A. Coello Coello,et al.  A Study of Multiobjective Metaheuristics When Solving Parameter Scalable Problems , 2010, IEEE Transactions on Evolutionary Computation.

[92]  Philip E. Gill,et al.  Practical optimization , 1981 .

[93]  A. Messac,et al.  Generating Well-Distributed Sets of Pareto Points for Engineering Design Using Physical Programming , 2002 .

[94]  Yong Zhang,et al.  Uniform Design: Theory and Application , 2000, Technometrics.

[95]  Alberto Lovison,et al.  Singular Continuation: Generating Piecewise Linear Approximations to Pareto Sets via Global Analysis , 2010, SIAM J. Optim..

[96]  K. Deb,et al.  A bi-objective constrained optimization algorithm using a hybrid evolutionary and penalty function approach , 2013 .

[97]  J. Greenstadt.  A Quasi-Newton Method with No Derivatives , 1972 .

[98]  W. Rheinboldt.  On the computation of multi-dimensional solution manifolds of parametrized equations , 1988 .

[99]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[100]  Jörg Fliege,et al.  Steepest descent methods for multicriteria optimization , 2000, Math. Methods Oper. Res..

[101]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[102]  Kim-Chuan Toh,et al.  Solving semidefinite-quadratic-linear programs using SDPT3 , 2003, Math. Program..

[103]  MavrotasGeorge.  Effective implementation of the ε-constraint method in Multi-Objective Mathematical Programming problems , 2009 .

[104]  Günter Rudolph,et al.  Evenly Spaced Pareto Front Approximations for Tricriteria Problems Based on Triangulation , 2013, EMO.

[105]  Jos F. Sturm,et al.  Implementation of interior point methods for mixed semidefinite and second order cone optimization problems , 2002, Optim. Methods Softw..

[106]  W. Karush.  Minima of Functions of Several Variables with Inequalities as Side Conditions , 2014 .

[107]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[108]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[109]  H. Whitney.  On the Abstract Properties of Linear Dependence , 1935 .

[110]  John E. Dennis,et al.  Normal-Boundary Intersection: A New Method for Generating the Pareto Surface in Nonlinear Multicriteria Optimization Problems , 1998, SIAM J. Optim..

[111]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[112]  S. Shreve,et al.  Stochastic differential equations , 1955, Mathematical Proceedings of the Cambridge Philosophical Society.

[113]  Ziga Povalej,et al.  Quasi-Newton's method for multiobjective optimization , 2014, J. Comput. Appl. Math..

[114]  Isao Ono,et al.  Constraint-Handling Method for Multi-objective Function Optimization: Pareto Descent Repair Operator , 2007, EMO.

[115]  Alberto Lovison,et al.  Global search perspectives for multiobjective optimization , 2013, J. Glob. Optim..

[116]  X. Mao,et al.  Stochastic Differential Equations and Applications , 1998 .

[117]  L. Lasdon,et al.  On a bicriterion formation of the problems of integrated system identification and system optimization , 1971 .

[118]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[119]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[120]  Sima Noghanian,et al.  Global Optimization: Differential Evolution, Genetic Algorithms, Particle Swarm, and Hybrid Methods , 2014 .