Proximal quasi-Newton methods for nondifferentiable convex optimization

n. The method is based on Rockafellar’s proximal point algorithm and a cutting-plane technique. At each step, we use an approximate proximal point pa(xk) of xk to define a vk∈∂εkf(pa(xk)) with εk≤α∥vk∥, where α is a constant. The method monitors the reduction in the value of ∥vk∥ to identify when a line search on f should be used. The quasi-Newton step is used to reduce the value of ∥vk∥. Without the differentiability of f, the method converges globally and the rate of convergence is Q-linear. Superlinear convergence is also discussed to extend the characterization result of Dennis and Moré. Numerical results show the good performance of the method.

[1]  Krzysztof C. Kiwiel,et al.  Proximity control in bundle methods for convex nondifferentiable minimization , 1990, Math. Program..

[2]  R. Tyrrell Rockafellar,et al.  Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming , 1976, Math. Oper. Res..

[3]  Liqun Qi,et al.  A nonsmooth version of Newton's method , 1993, Math. Program..

[4]  J. Frédéric Bonnans,et al.  A family of variable metric proximal methods , 1995, Math. Program..

[5]  Defeng Sun,et al.  Quasi-Newton Bundle-Type Methods for Nondifferentiable Convex Optimization , 1998, SIAM J. Optim..

[6]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[7]  Robert Mifflin,et al.  A quasi-second-order proximal bundle algorithm , 1996, Math. Program..

[8]  Masao Fukushima,et al.  A Globally and Superlinearly Convergent Algorithm for Nonsmooth Convex Minimization , 1996, SIAM J. Optim..

[9]  R. Tyrrell Rockafellar,et al.  Generalized Hessian Properties of Regularized Nonsmooth Functions , 1996, SIAM J. Optim..

[10]  Masao Fukushima,et al.  A descent algorithm for nonsmooth convex optimization , 1984, Math. Program..

[11]  Xiaojun Chen,et al.  A preconditioning proximal newton method for nondifferentiable convex optimization , 1997, Math. Program..

[12]  James V. Burke,et al.  On the superlinear convergence of the variable metric proximal point algorithm using Broyden and BFGS matrix secant updating , 2000, Math. Program..

[13]  Claude Lemaréchal,et al.  Convergence of some algorithms for convex minimization , 1993, Math. Program..

[14]  J. Burke,et al.  A Variable Metric Proximal Point Algorithm for Monotone Operators , 1999 .

[15]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[16]  Xiaojun Chen,et al.  Convergence of the BFGS Method for LC 1 Convex Constrained Optimization , 1996 .

[17]  K. Kiwiel.  A Method for Solving Certain Quadratic Programming Problems Arising in Nonsmooth Optimization , 1986 .

[18]  Claude Lemaréchal,et al.  Variable metric bundle methods: From conceptual to implementable forms , 1997, Math. Program..

[19]  Claude Lemaréchal,et al.  Practical Aspects of the Moreau-Yosida Regularization: Theoretical Preliminaries , 1997, SIAM J. Optim..

[20]  Claude Lemaréchal,et al.  An approach to variable metric bundle methods , 1993, System Modelling and Optimization.

[21]  E. Polak,et al.  Variable-metric technique for the solution of affinely parametrized nondifferentiable optimal design problems , 1990 .

[22]  J. Burke,et al.  On the Local Super-Linear Convergence of a Matrix Secant Implementation of the Variable Metric Proximal Point Algorithm for Monotone Operators , 1998 .

[23]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[24]  Krzysztof C. Kiwiel,et al.  An aggregate subgradient method for nonsmooth convex minimization , 1983, Math. Program..

[25]  Liqun Qi,et al.  Convergence Analysis of Some Algorithms for Solving Nonsmooth Equations , 1993, Math. Oper. Res..

[26]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[27]  Shirley Dex,et al.  JR 旅客販売総合システム（マルス）における運用及び管理について , 1991 .

[28]  G. Stewart.  The Efficient Generation of Random Orthogonal Matrices with an Application to Condition Estimators , 1980 .

[29]  Ciyou Zhu,et al.  Asymptotic Convergence Analysis of Some Inexact Proximal Point Algorithms for Minimization , 1996, SIAM J. Optim..

[30]  A. Auslender.  Numerical methods for nondifferentiable convex optimization , 1987 .

[31]  Jerzy Kyparisis,et al.  Local convergence of quasi-Newton methods for B-differentiable equations , 1992, Math. Program..

[32]  J. J. Moré,et al.  A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[33]  M. Fukushima,et al.  Globally Convergent BFGS Method for Nonsmooth Convex Optimization1 , 2000 .