Complexity of gradient descent for multiobjective optimization

ABSTRACT A number of first-order methods have been proposed for smooth multiobjective optimization for which some form of convergence to first-order criticality has been proved. Such convergence is global in the sense of being independent of the starting point. In this paper, we analyse the rate of convergence of gradient descent for smooth unconstrained multiobjective optimization, and we do it for non-convex, convex, and strongly convex vector functions. These global rates are shown to be the same as for gradient descent in single-objective optimization and correspond to appropriate worst-case complexity bounds. In the convex cases, the rates are given for implicit scalarizations of the problem vector function.

[1]  José Mario Martínez,et al.  Worst-case evaluation complexity for unconstrained nonlinear optimization using high-order regularized models , 2017, Math. Program..

[2]  Pablo A. Lotito,et al.  Trust region globalization strategy for the nonconvex unconstrained multiobjective optimization problem , 2016, Math. Program..

[3]  Nicholas I. M. Gould,et al.  On the Complexity of Steepest Descent, Newton's and Regularized Newton's Methods for Nonconvex Unconstrained Optimization Problems , 2010, SIAM J. Optim..

[4]  Alfredo N. Iusem,et al.  Proximal Methods in Vector Optimization , 2005, SIAM J. Optim..

[5]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[6]  Luís Nunes Vicente,et al.  Worst case complexity of direct search , 2013, EURO J. Comput. Optim..

[7]  Ellen H. Fukuda,et al.  A SURVEY ON MULTIOBJECTIVE DESCENT METHODS , 2014 .

[8]  Jörg Fliege,et al.  Newton's Method for Multiobjective Optimization , 2009, SIAM J. Optim..

[9]  H. Attouch,et al.  A Continuous Gradient-like Dynamical Approach to Pareto-Optimization in Hilbert Spaces , 2014 .

[10]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[11]  Yu. K. Mashunin,et al.  Vector Optimization , 2017, Encyclopedia of Machine Learning and Data Mining.

[12]  Jin Yun Yuan,et al.  On the worst-case complexity of nonlinear stepsize control algorithms for convex unconstrained optimization , 2016, Optim. Methods Softw..

[13]  Jörg Fliege,et al.  Steepest descent methods for multicriteria optimization , 2000, Math. Methods Oper. Res..

[14]  L. F. Prudente,et al.  Nonlinear Conjugate Gradient Methods for Vector Optimization , 2018, SIAM J. Optim..

[15]  Amir Beck,et al.  First-Order Methods in Optimization , 2017 .

[16]  Ya-xiang Yuan,et al.  On the convergence and worst-case complexity of trust-region and regularization methods for unconstrained optimization , 2015, Math. Program..