Smoothing Method of Multipliers for Sum-Max Problems

We study a class of nonsmooth unconstrained optimization problems, which includes the problem of minimizing the sum of pairwise maxima of smooth convex functions. Minimum l1-norm approximation is a particular case of this problem. Combining the ideas of Lagrange multipliers and of smooth approximation of max-type function, we obtain an extended notion of nonquadratic augmented Lagrangian. Our approach does not require artificial variables, and preserves sparse structure of Hessian in many practical cases. We present the corresponding method of multipliers, and its convergence analysis for a dual counterpart, resulting in a proximal point maximization algorithm. The practical efficiency of the algorithm is supported by computational results for large-scale problems, arising in structural optimization.

[1]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[2]  Olvi L. Mangasarian,et al.  A class of smoothing functions for nonlinear and mixed complementarity problems , 1996, Comput. Optim. Appl..

[3]  Leonid Mosheyev,et al.  Penalty/Barrier multiplier algorthm for semidefinit programming , 2000 .

[4]  Yehoshua Y. Zeevi,et al.  Extraction of a source from multichannel data using sparse decomposition , 2002, Neurocomputing.

[5]  Paul Tseng,et al.  On the convergence of the exponential multiplier method for convex programming , 1993, Math. Program..

[6]  Marc Teboulle,et al.  Entropic Proximal Mappings with Applications to Nonlinear Programming , 1992, Math. Oper. Res..

[7]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[8]  Michael Zibulevsky,et al.  Penalty/Barrier Multiplier Methods for Convex Programming Problems , 1997, SIAM J. Optim..

[9]  R. Tyrrell Rockafellar,et al.  Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming , 1976, Math. Oper. Res..

[10]  D. Bertsekas,et al.  Multiplier methods for convex programming , 1973, CDC 1973.

[11]  M. Teboulle,et al.  A smoothing technique for nondifferentiable optimization problems , 1988 .

[12]  Barak A. Pearlmutter,et al.  Blind Source Separation by Sparse Decomposition in a Signal Dictionary , 2001, Neural Computation.

[13]  M. Bendsøe,et al.  Optimization methods for truss geometry and topology design , 1994 .

[14]  Roman A. Polyak,et al.  Modified barrier functions (theory and methods) , 1992, Math. Program..

[15]  R. Tyrrell Rockafellar,et al.  A dual approach to solving nonlinear programming problems by unconstrained optimization , 1973, Math. Program..

[16]  Martin P. Bendsøe,et al.  A New Method for Optimal Truss Topology Design , 1993, SIAM J. Optim..