Computational and sensitivity aspects of eigenvalue-based methods for the large-scale trust-region subproblem

The trust-region subproblem (TRS) of minimizing a quadratic function subject to a norm constraint arises in the context of trust-region methods in optimization and in the regularization of discrete forms of ill-posed problems, including non-negative regularization by means of interior-point methods. A class of efficient methods and software for solving large-scale trust-region subproblems (TRSs) is based on a parametric-eigenvalue formulation of the subproblem. The solution of a sequence of large symmetric eigenvalue problems is the main computation in these methods. In this work, we study the robustness and performance of eigenvalue-based methods for the large-scale TRS. We describe the eigenvalue problems and their features, and discuss the computational challenges they pose as well as some approaches to handle them. We present results from a numerical study of the sensitivity of solutions to the TRS to eigenproblem solutions.

[1]  Danny C. Sorensen,et al.  A Trust-Region Approach to the Regularization of Large-Scale Discrete Forms of Ill-Posed Problems , 2001, SIAM J. Sci. Comput..

[2]  Chao Yang,et al.  ARPACK users' guide - solution of large-scale eigenvalue problems with implicitly restarted Arnoldi methods , 1998, Software, environments, tools.

[3]  Gerard L. G. Sleijpen,et al.  A generalized Jacobi-Davidson iteration method for linear eigenvalue problems , 1998 .

[4]  D. Sorensen.  Newton's method with a model trust region modification , 1982 .

[5]  H. Voss,et al.  Solving regularized total least squares problems based on eigenproblems , 2010 .

[6]  Shiqian Ma,et al.  A fast subspace method for image deblurring , 2009, Appl. Math. Comput..

[7]  M. Chang,et al.  A new deterministic global optimization method for general twice-differentiable constrained nonlinear programming problems , 2007 .

[8]  A. N. Tikhonov,et al.  REGULARIZATION OF INCORRECTLY POSED PROBLEMS , 1963 .

[9]  Stéphane Marchand-Maillet,et al.  Evaluation of distance-based discriminant analysis and its kernelized extension in visual object recognition , 2004 .

[10]  Trond Steihaug,et al.  Solving a TRS which has linear inequality constraints , 2008 .

[11]  Nicholas I. M. Gould,et al.  Solving the Trust-Region Subproblem using the Lanczos Method , 1999, SIAM J. Optim..

[12]  G. W. Stewart,et al.  Stochastic Perturbation Theory , 1990, SIAM Rev..

[13]  P. Hansen.  Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion , 1987 .

[14]  Mi Zhang,et al.  Avoiding monotony: improving the diversity of recommendation lists , 2008, RecSys '08.

[15]  Danny C. Sorensen,et al.  A New Matrix-Free Algorithm for the Large-Scale Trust-Region Subproblem , 2000, SIAM J. Optim..

[16]  Danny C. Sorensen,et al.  Implicit Application of Polynomial Filters in a k-Step Arnoldi Method , 1992, SIAM J. Matrix Anal. Appl..

[17]  Michiel E. Hochstenbach,et al.  Discrete ill-posed least-squares problems with a solution norm constraint , 2012 .

[18]  Anders P. Eriksson,et al.  Image Segmentation with Context , 2007, SCIA.

[19]  G. H. Golub,et al.  A Stochastic Approach to Error Estimates for Iterative Linear Solvers: Part 1 , 2001 .

[20]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[21]  Thierry Pun,et al.  Distance-based discriminant analysis method and its applications , 2008, Pattern Analysis and Applications.

[22]  Amir K. Khandani,et al.  An optimized transmitter precoding scheme for synchronous DS-CDMA , 2006, IEEE Transactions on Communications.

[23]  Danny C. Sorensen,et al.  Algorithm 873: LSTRS: MATLAB software for large-scale trust-region subproblems and regularization , 2008, TOMS.

[24]  Neil J. Hurley,et al.  Analysis of Methods for Novel Case Selection , 2008, 2008 20th IEEE International Conference on Tools with Artificial Intelligence.

[25]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[26]  Anders P. Eriksson,et al.  Solving Large Scale Binary Quadratic Problems: Spectral Methods vs. Semidefinite Programming , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Danny C. Sorensen,et al.  Minimization of a Large-Scale Quadratic FunctionSubject to a Spherical Constraint , 1997, SIAM J. Optim..

[28]  Françoise Chaitin-Chatelin,et al.  Lectures on finite precision computations , 1996, Software, environments, tools.

[29]  Per Christian Hansen,et al.  Regularization Tools version 4.0 for Matlab 7.3 , 2007, Numerical Algorithms.

[30]  Pablo Nava Gabriel,et al.  Inverse sound rendering : in-situ estimation of surface acoustic impedance for acoustic simulation and design of real indoor environments , 2007 .

[31]  William W. Hager,et al.  Minimizing a Quadratic Over a Sphere , 2001, SIAM J. Optim..

[32]  Stefan M. Wild,et al.  Estimating Computational Noise , 2011, SIAM J. Sci. Comput..

[33]  Anders P. Eriksson,et al.  Improved spectral relaxation methods for binary quadratic optimization problems , 2008, Comput. Vis. Image Underst..

[34]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[35]  B. H. Fotland,et al.  A matrix-free method for regularisation with unrestricted variables , 2008 .

[36]  L. Eldén.  Algorithms for the regularization of ill-conditioned least squares problems , 1977 .

[37]  A Barbero Jimenez,et al.  Fast algorithms for total-variation based optimization , 2010 .

[38]  L G SleijpenGerard,et al.  A Jacobi--Davidson Iteration Method for Linear Eigenvalue Problems , 1996 .

[39]  J. H. Wilkinson,et al.  IMPROVING THE ACCURACY OF COMPUTED EIGENVALUES AND EIGENVECTORS , 1983 .

[40]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[41]  H. Voss.  An Arnoldi Method for Nonlinear Eigenvalue Problems , 2004 .

[42]  Gerard L. G. Sleijpen,et al.  A Jacobi-Davidson Iteration Method for Linear Eigenvalue Problems , 1996, SIAM J. Matrix Anal. Appl..

[43]  C. B. Shaw,et al.  Improvement of the resolution of an instrument by numerical solution of an integral equation , 1972 .

[44]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[45]  Danny C. Sorensen,et al.  Accelerating the LSTRS Algorithm , 2010, SIAM J. Sci. Comput..

[46]  Trond Steihaug,et al.  An interior-point trust-region-based method for large-scale non-negative regularization , 2002 .

[47]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[48]  J. M. Martínez,et al.  Practical active-set Euclidian trust-region method with spectral projected gradients for bound-constrained minimization , 2005 .

[49]  Gerard L. G. Sleijpen,et al.  A Jacobi-Davidson Iteration Method for Linear Eigenvalue Problems , 1996, SIAM Rev..

[50]  José Mario Martínez,et al.  Algorithm 813: SPG—Software for Convex-Constrained Optimization , 2001, TOMS.

[51]  G. GOLUB,et al.  Stochastic Error Estimates For Iterative Linear Solvers Part 2 , .

[52]  Serena Morigi,et al.  A hybrid multilevel-active set method for large box-constrained linear discrete ill-posed problems , 2011 .

[53]  Franz Rendl,et al.  A semidefinite framework for trust region subproblems with applications to large scale minimization , 1997, Math. Program..

[54]  A. Carasso.  Determining Surface Temperatures from Interior Observations , 1982 .

[55]  Andrzej Cichocki,et al.  Nonnegative matrix factorization with quadratic programming , 2008, Neurocomputing.

[56]  Thierry Pun,et al.  Iterative Majorization Approach to the Distance-based Discriminant Analysis , 2004, GfKl.

[57]  Henry Wolkowicz,et al.  The trust region subproblem and semidefinite programming , 2004, Optim. Methods Softw..

[58]  José Mario Martínez,et al.  On Augmented Lagrangian Methods with General Lower-Level Constraints , 2007, SIAM J. Optim..

[59]  D. Sorensen,et al.  A large-scale trust-region approach to the regularization of discrete ill-posed problems , 1999 .

[60]  Rosemary A. Renaut,et al.  Least squares problems with inequality constraints as quadratic constraints , 2010 .