Convex constrained optimization for large-scale generalized Sylvester equations

We propose and study the use of convex constrained optimization techniques for solving large-scale Generalized Sylvester Equations (GSE). For that, we adapt recently developed globalized variants of the projected gradient method to a convex constrained least-squares approach for solving GSE. We demonstrate the effectiveness of our approach on two different applications. First, we apply it to solve the GSE that appears after applying left and right preconditioning schemes to the linear problems associated with the discretization of some partial differential equations. Second, we apply the new approach, combined with a Tikhonov regularization term, to restore some blurred and highly noisy images.

[1]  A. Goldstein Convex programming in Hilbert space , 1964 .

[2]  Alexander I. Barvinok Computing the Ehrhart quasi-polynomial of a rational simplex , 2006, Math. Comput..

[3]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[4]  Tony F. Chan,et al.  Image processing and analysis - variational, PDE, wavelet, and stochastic methods , 2005 .

[5]  Khalide Jbilou,et al.  Low rank approximate solutions to large Sylvester matrix equations , 2006, Appl. Math. Comput..

[6]  Per Christian Hansen,et al.  Regularization methods for large-scale problems , 1993 .

[7]  Boris Polyak,et al.  Constrained minimization methods , 1966 .

[8]  L. Lucy An iterative technique for the rectification of observed distributions , 1974 .

[9]  S. Hammarling Numerical Solution of the Stable, Non-negative Definite Lyapunov Equation , 1982 .

[10]  Michael A. Epton Methods for the solution ofAXD−BXC=E and its application in the numerical solution of implicit ordinary differential equations , 1980 .

[11]  Marlliny Monsalve,et al.  Block linear method for large scale Sylvester equations , 2008 .

[12]  A. Dou Method of Undetermined Coefficients in Linear Differential Systems and the Matrix Equation $YB - AY = F$ , 1966 .

[13]  G. Golub,et al.  Estimation of the L-Curve via Lanczos Bidiagonalization , 1999 .

[15]  Dianne P. O'Leary,et al.  Deblurring Images: Matrices, Spectra, and Filtering (Fundamentals of Algorithms 3) (Fundamentals of Algorithms) , 2006 .

[16]  Roger Fletcher,et al.  On the Barzilai-Borwein Method , 2005 .

[17]  E. Polak,et al.  Constrained Minimization Problems in Finite-Dimensional Spaces , 1966 .

[18]  R. Dykstra,et al.  A Method for Finding Projections onto the Intersection of Convex Sets in Hilbert Spaces , 1986 .

[19]  Kok Lay Teo,et al.  Optimal control of distributed parameter systems , 1981 .

[20]  Kok Lay Teo,et al.  Optimization and control with applications , 2005 .

[21]  Sanyang Liu,et al.  Analysis of a non-interior continuation method for second-order cone programming , 2008 .

[22]  Per Christian Hansen,et al.  Analysis of Discrete Ill-Posed Problems by Means of the L-Curve , 1992, SIAM Rev..

[23]  B. Datta Numerical methods for linear control systems : design and analysis , 2004 .

[24]  Gene H. Golub,et al.  Matrix computations , 1983 .

[25]  Lothar Reichel,et al.  GMRES, L-Curves, and Discrete Ill-Posed Problems , 2002 .

[26]  José Mario Martínez,et al.  Algorithm 813: SPG—Software for Convex-Constrained Optimization , 2001, TOMS.

[27]  M. Raydan On the Barzilai and Borwein choice of steplength for the gradient method , 1993 .

[28]  William H. Richardson,et al.  Bayesian-Based Iterative Method of Image Restoration , 1972 .

[29]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[30]  K. Jbilou,et al.  Projection methods for large Lyapunov matrix equations , 2006 .

[31]  J. Lions Optimal Control of Systems Governed by Partial Differential Equations , 1971 .

[32]  James Demmel,et al.  Accurate solutions of ill-posed problems in control theory , 1988 .

[33]  C. S. Lu Solution of the matrix equation AX+XB = C , 1971 .

[34]  L. Liao,et al.  R-linear convergence of the Barzilai and Borwein gradient method , 2002 .

[35]  D. Bertsekas On the Goldstein-Levitin-Polyak gradient projection method , 1974, CDC 1974.

[36]  J. M. Martínez,et al.  Inexact spectral projected gradient methods on convex sets , 2003 .

[37]  Richard H. Bartels,et al.  Algorithm 432 [C2]: Solution of the matrix equation AX + XB = C [F4] , 1972, Commun. ACM.

[38]  Dianne P. O'Leary,et al.  Deblurring Images: Matrices, Spectra and Filtering , 2006, J. Electronic Imaging.

[39]  I. Jaimoukha,et al.  Krylov subspace methods for solving large Lyapunov equations , 1994 .

[40]  L. Reichel,et al.  Krylov-subspace methods for the Sylvester equation , 1992 .

[41]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[42]  H. Sadok,et al.  Global FOM and GMRES algorithms for matrix equations , 1999 .

[43]  K. Jbilou,et al.  Sylvester Tikhonov-regularization methods in image restoration , 2007 .

[44]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[45]  Chun-Hua Guo,et al.  Convergence Analysis of the Doubling Algorithm for Several Nonlinear Matrix Equations in the Critical Case , 2009, SIAM J. Matrix Anal. Appl..

[46]  V. Simoncini,et al.  On the numerical solution ofAX −XB =C , 1996 .

[47]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[48]  Tony F. Chan,et al.  Image processing and analysis , 2005 .

[49]  D. Walton,et al.  The numerical solution of , 1977 .

[50]  José Mario Martínez,et al.  Spectral residual method without gradient information for solving large-scale nonlinear systems of equations , 2006, Math. Comput..

[51]  D. Calvetti,et al.  Iterative methods for X − AXB = C , 1997 .

[52]  Khalide Jbilou,et al.  Block Krylov Subspace Methods for Solving Large Sylvester Equations , 2002, Numerical Algorithms.

[53]  Arun N. Netravali,et al.  Image Restoration Based on a Subjective Criterion , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[54]  G. Golub,et al.  A Hessenberg-Schur method for the problem AX + XB= C , 1979 .

[55]  S. Gómez,et al.  The triangle method for finding the corner of the L-curve , 2002 .