Linearly Constrained Nonsmooth and Nonconvex Minimization

Motivated by variational models in continuum mechanics, we introduce a novel algorithm to perform nonsmooth and nonconvex minimizations with linear constraints in Euclidean spaces. We show how this algorithm is actually a natural generalization of the well-known nonstationary augmented Lagrangian method for convex optimization. The relevant features of this approach are its applicability to a large variety of nonsmooth and nonconvex objective functions, its guaranteed convergence to critical points of the objective energy independently of the choice of the initial value, and its simplicity of implementation. In fact, the algorithm results in a nested double loop iteration. In the inner loop an augmented Lagrangian algorithm performs an adaptive finite number of iterations on a fixed quadratic and strictly convex perturbation of the objective energy, depending on a parameter which is adapted by the external loop. To show the versatility of this new algorithm, we exemplify how it can be used for computing c...

[1]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[2]  M. Fornasier,et al.  Iterative thresholding algorithms , 2008 .

[3]  Stochastic Relaxation , 2014, Computer Vision, A Reference Guide.

[4]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[5]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[6]  Massimo Fornasier,et al.  Iterative Thresholding Meets Free-Discontinuity Problems , 2009, Found. Comput. Math..

[7]  H. Rauhut Compressive Sensing and Structured Random Matrices , 2009 .

[8]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[9]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[10]  Massimo Fornasier,et al.  Compressive Sensing and Structured Random Matrices , 2010 .

[11]  Holger Rauhut,et al.  Compressive Sensing with structured random matrices , 2012 .

[12]  O. Scherzer,et al.  Regularization of ill-posed linear equations by the non- stationary augmented Lagrangian method , 2010 .

[13]  Alexander Mielke,et al.  Evolution of Rate{Independent Inelasticity with Microstructure using Relaxation and Young Measures , 2003 .

[14]  Filippo Cagnetti,et al.  A VANISHING VISCOSITY APPROACH TO FRACTURE GROWTH IN A COHESIVE ZONE MODEL WITH PRESCRIBED CRACK PATH , 2008 .

[15]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[16]  Massimo Fornasier,et al.  Damping Noise-Folding and Enhanced Support Recovery in Compressed Sensing , 2013, IEEE Transactions on Signal Processing.

[17]  Gilles A. Francfort,et al.  Revisiting brittle fracture as an energy minimization problem , 1998 .

[18]  G. B. Smith,et al.  Preface to S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images” , 1987 .

[19]  Robert D. Nowak,et al.  Wavelet-based image estimation: an empirical Bayes approach using Jeffrey's noninformative prior , 2001, IEEE Trans. Image Process..

[20]  Endre Süli,et al.  An Adaptive Finite Element Approximation of a Variational Model of Brittle Fracture , 2010, SIAM J. Numer. Anal..

[21]  Mila Nikolova,et al.  Efficient Reconstruction of Piecewise Constant Images Using Nonsmooth Nonconvex Minimization , 2008, SIAM J. Imaging Sci..

[22]  Rodica Toader,et al.  A Model for the Quasi-Static Growth¶of Brittle Fractures:¶Existence and Approximation Results , 2001 .

[23]  V. T. Polyak TRET''''YAKOV: The method of penalty estimates for conditional extremum problems , 1974 .

[24]  F. Cagnetti,et al.  Quasistatic crack evolution for a cohesive zone model with different response to loading and unloading: a Young measures approach , 2011 .

[25]  A. Chambolle,et al.  Discrete approximation of the Mumford-Shah functional in dimension two , 1999, ESAIM: Mathematical Modelling and Numerical Analysis.

[26]  Rodica Toader,et al.  Quasistatic Crack Growth in Elasto-Plastic Materials: The Two-Dimensional Case , 2010 .

[27]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[28]  G. Friesecke,et al.  Minimizing atomic configurations of short range pair potentials in two dimensions: crystallization in the Wulff shape , 2009, 0909.0927.

[29]  Isabelle E. Magnin,et al.  Optimization by Stochastic Continuation , 2010, SIAM J. Imaging Sci..

[30]  G. D. Maso,et al.  An Introduction to-convergence , 1993 .

[31]  Ivan P. Gavrilyuk,et al.  Lagrange multiplier approach to variational problems and applications , 2010, Math. Comput..

[32]  Ronald F. Gariepy FUNCTIONS OF BOUNDED VARIATION AND FREE DISCONTINUITY PROBLEMS (Oxford Mathematical Monographs) , 2001 .

[33]  Mila Nikolova,et al.  Thresholding implied by truncated quadratic regularization , 2000, IEEE Trans. Signal Process..

[34]  J. Bolte,et al.  Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity , 2009 .

[35]  M. Nikolova,et al.  Stability of the Minimizers of Least Squares with a Non-Convex Regularization. Part I: Local Behavior , 2006 .

[36]  Gianni Dal Maso,et al.  A MODEL FOR THE QUASI-STATIC GROWTH OF BRITTLE FRACTURES BASED ON LOCAL MINIMIZATION , 2002 .

[37]  A. A. Griffith The Phenomena of Rupture and Flow in Solids , 1921 .

[38]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[39]  L. Ambrosio,et al.  Functions of Bounded Variation and Free Discontinuity Problems , 2000 .

[40]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[41]  C. Zanini,et al.  Quasi-static crack growth for a cohesive zone model with prescribed crack path , 2005, Proceedings of the Royal Society of Edinburgh: Section A Mathematics.

[42]  Mila Nikolova,et al.  Markovian reconstruction using a GNC approach , 1999, IEEE Trans. Image Process..

[43]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[44]  B. Bourdin Numerical implementation of the variational formulation for quasi-static brittle fracture , 2007 .

[45]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[46]  M. Fornasier,et al.  Existence of minimizers of the Mumford-Shah functional with singular operators and unbounded data , 2013 .

[47]  Antonin Chambolle,et al.  Implementation of an adaptive finite-element approximation of the Mumford-Shah functional , 2000, Numerische Mathematik.

[48]  F. Clarke Optimization And Nonsmooth Analysis , 1983 .

[49]  M. Nikolova,et al.  Stability of the Minimizers of Least Squares with a Non-Convex Regularization. Part II: Global Behavior , 2006 .

[50]  Ali Mohammad-Djafari,et al.  Inversion of large-support ill-posed linear operators using a piecewise Gaussian MRF , 1998, IEEE Trans. Image Process..

[51]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[52]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[53]  L. Ambrosio,et al.  Approximation of functional depending on jumps by elliptic functional via t-convergence , 1990 .

[54]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[55]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[56]  Antonin Chambolle,et al.  Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage , 1998, IEEE Trans. Image Process..

[57]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[58]  L. Ambrosio,et al.  Gradient Flows: In Metric Spaces and in the Space of Probability Measures , 2005 .

[59]  Wolfgang Hackbusch,et al.  Multi-grid methods and applications , 1985, Springer series in computational mathematics.