Manifold Sampling for Optimization of Nonconvex Functions That Are Piecewise Linear Compositions of Smooth Components

We develop a manifold sampling algorithm for the minimization of a nonsmooth composite function $f \triangleq \psi + h \circ F$ when $\psi$ is smooth with known derivatives, $h$ is a known, nonsmoo...

[1]  Ya-Xiang Yuan,et al.  Conditions for convergence of trust region algorithms for nonsmooth optimization , 1985, Math. Program..

[2]  Jason Weston,et al.  Trading convexity for scalability , 2006, ICML.

[3]  Tim Mitchell,et al.  A BFGS-SQP method for nonsmooth, nonconvex, constrained optimization and its evaluation using relative minimization profiles , 2017, Optim. Methods Softw..

[4]  Alessandro Chiuso,et al.  Convex vs non-convex estimators for regression and sparse estimation: the mean squared error properties of ARD and GLasso , 2014, J. Mach. Learn. Res..

[5]  A. Bagirov,et al.  Discrete Gradient Method: Derivative-Free Method for Nonsmooth Optimization , 2008 .

[6]  W. Hare,et al.  Compositions of convex functions and fully linear models , 2016, Optim. Lett..

[7]  Krzysztof C. Kiwiel,et al.  A Nonderivative Version of the Gradient Sampling Algorithm for Nonsmooth Nonconvex Optimization , 2010, SIAM J. Optim..

[8]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[9]  Yoshua Bengio,et al.  Scaling learning algorithms towards AI , 2007 .

[10]  Liqun Qi,et al.  Convergence Analysis of Some Algorithms for Solving Nonsmooth Equations , 1993, Math. Oper. Res..

[11]  Anastasios Kyrillidis,et al.  Dropping Convexity for Faster Semi-definite Optimization , 2015, COLT.

[12]  R. Fletcher,et al.  An algorithm for composite nonsmooth optimization problems , 1986 .

[13]  Dmitriy Drusvyatskiy,et al.  Error Bounds, Quadratic Growth, and Linear Convergence of Proximal Methods , 2016, Math. Oper. Res..

[14]  Warren Hare,et al.  A derivative-free approximate gradient sampling algorithm for finite minimax problems , 2013, Comput. Optim. Appl..

[15]  Luís Nunes Vicente,et al.  Trust-Region Methods Without Using Derivatives: Worst Case Complexity and the NonSmooth Case , 2016, SIAM J. Optim..

[16]  Krzysztof C. Kiwiel,et al.  Convergence of the Gradient Sampling Algorithm for Nonsmooth Nonconvex Optimization , 2007, SIAM J. Optim..

[17]  Michael L. Overton,et al.  A Sequential Quadratic Programming Algorithm for Nonconvex, Nonsmooth Constrained Optimization , 2012, SIAM J. Optim..

[18]  S. Scholtes.  Introduction to Piecewise Differentiable Equations , 2012 .

[19]  Stefan M. Wild,et al.  Benchmarking Derivative-Free Optimization Algorithms , 2009, SIAM J. Optim..

[20]  Stefan M. Wild.  Chapter 40: POUNDERS in TAO: Solving Derivative-Free Nonlinear Least-Squares Problems with POUNDERS , 2017 .

[21]  R. Fletcher.  A model algorithm for composite nondifferentiable optimization problems , 1982 .

[22]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[23]  John L. Nazareth,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[24]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[25]  Panagiotis Patrinos,et al.  Forward–backward quasi-Newton methods for nonsmooth optimization problems , 2016, Computational Optimization and Applications.

[26]  Dmitriy Drusvyatskiy,et al.  Nonsmooth optimization using Taylor-like models: error bounds, convergence, and termination criteria , 2016, Mathematical Programming.

[27]  R. Womersley.  Censored Discrete Linear $l_1 $ Approximation , 1986 .

[28]  Frank E. Curtis,et al.  A quasi-Newton algorithm for nonconvex, nonsmooth optimization with global convergence guarantees , 2015, Math. Program. Comput..

[29]  Po-Ling Loh,et al.  Statistical consistency and asymptotic normality for high-dimensional robust M-estimators , 2015, ArXiv.