Limited Memory Bundle Method and Its Variations for Large-Scale Nonsmooth Optimization

There exist a vast variety of practical problems involving nonsmooth functions with large dimensions and nonconvex characteristics. Nevertheless, most nonsmooth solution methods have been designed to solve only small- or medium scale problems and they are heavily based on the convexity of the problem. In this chapter we describe three numerical methods for solving large-scale nonconvex NSO problems. Namely, the limited memory bundle algorithm (LMBM), the diagonal bundle method (D-Bundle), and the splitting metrics diagonal bundle method (SMDB). We also recall the convergence properties of these algorithms. To demonstrate the usability of the methods in large-scale settings, numerical experiments have been made using academic NSO problems with up to million variables.

[1]  Adil M. Bagirov,et al.  A method of truncated codifferential with application to some problems of cluster analysis , 2002, J. Glob. Optim..

[2]  Adil M. Bagirov,et al.  Introduction to Nonsmooth Optimization , 2014 .

[3]  Adil M. Bagirov,et al.  New diagonal bundle method for clustering problems in large data sets , 2017, Eur. J. Oper. Res..

[4]  Ionel M. Navon,et al.  Impact of non‐smooth observation operators on variational and sequential data assimilation for a limited‐area shallow‐water equation model , 2012 .

[5]  Marjo S. Haarala.  Large-scale nonsmooth optimization : variable metric bundle method with limited memory , 2004 .

[6]  Marko M. Mäkelä,et al.  Adaptive limited memory bundle method for bound constrained large-scale nonsmooth optimization , 2010 .

[7]  G. Stavroulakis,et al.  Nonconvex Optimization in Mechanics: Algorithms, Heuristics and Engineering Applications , 1997 .

[8]  Adil M. Bagirov,et al.  Clustering in large data sets with the limited memory bundle method , 2018, Pattern Recognit..

[9]  Yu. S. Ledyaev,et al.  Nonsmooth analysis and control theory , 1998 .

[10]  Michal Kočvara,et al.  Nonsmooth approach to optimization problems with equilibrium constraints : theory, applications, and numerical results , 1998 .

[11]  Adil M. Bagirov,et al.  Introduction to Nonsmooth Optimization: Theory, Practice and Software , 2014 .

[12]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[13]  Adil M. Bagirov,et al.  Nonsmooth DC programming approach to the minimum sum-of-squares clustering problems , 2016, Pattern Recognit..

[14]  Tommi Kärkkäinen,et al.  Robust Formulations for Training Multilayer Perceptrons , 2004, Neural Computation.

[15]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Convex Nonsmooth Unconstrained Minimization1 , 1999 .

[16]  Paul S. Bradley,et al.  Mathematical Programming for Data Mining: Formulations and Challenges , 1999, INFORMS J. Comput..

[17]  Manlio Gaudioso,et al.  Diagonal bundle method with convex and concave updates for large-scale nonconvex and nonsmooth optimization , 2019, Optim. Methods Softw..

[18]  Marko M. Mäkelä,et al.  Limited memory bundle method for large bound constrained nonsmooth optimization: convergence analysis , 2010, Optim. Methods Softw..

[19]  M. M. Ali,et al.  Limited memory interior point bundle method for large inequality constrained nonsmooth minimization , 2008, Appl. Math. Comput..

[20]  Emilio Carrizosa,et al.  Supervised classification and mathematical optimization , 2013, Comput. Oper. Res..

[21]  K. Kiwiel.  Methods of Descent for Nondifferentiable Optimization , 1985 .

[22]  Napsu Karmitsa,et al.  Diagonal Bundle Method for Nonsmooth Sparse Optimization , 2015, J. Optim. Theory Appl..

[23]  Annabella Astorino,et al.  Nonsmooth Optimization Techniques for Semisupervised Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Annabella Astorino,et al.  Non-smoothness in classification problems , 2008, Optim. Methods Softw..

[25]  Adil M. Bagirov,et al.  Comparing different nonsmooth minimization methods and software , 2012, Optim. Methods Softw..

[26]  Sami Äyrämö,et al.  Knowledge mining using robust clustering , 2006 .

[27]  Kaisa Miettinen,et al.  New limited memory bundle method for large-scale nonsmooth optimization , 2004, Optim. Methods Softw..

[28]  J. Haslinger,et al.  Finite Element Approximation for Optimal Shape, Material and Topology Design , 1996 .

[29]  Kristin P. Bennett,et al.  Fast Bundle Algorithm for Multiple-Instance Learning , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Kaisa Miettinen,et al.  Globally convergent limited memory bundle method for large-scale nonsmooth optimization , 2007, Math. Program..

[31]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Nonconvex Nondifferentiable Unconstrained Minimization , 2001 .

[32]  Ionel Michael Navon,et al.  A Study of Coupling Parameter Estimation Implemented by 4D-Var and EnKF with a Simple Coupled System , 2015 .