Modified Iterations for Data-Sparse Solution of Linear Systems

A modification of standard linear iterative methods for the solution of linear equations is investigated aiming at improved data-sparsity with respect to a rank function. The convergence speed of the modified method is compared to the rank growth of its iterates for certain model cases. The considered general setup is common in the data-sparse treatment of high dimensional problems such as sparse approximation and low rank tensor calculus.

[1]  Lars Grasedyck,et al.  Existence and Computation of Low Kronecker-Rank Approximations for Large Linear Systems of Tensor Product Structure , 2004, Computing.

[2]  Alex Townsend,et al.  On the singular values of matrices with high displacement rank , 2017, Linear Algebra and its Applications.

[3]  Wolfgang Dahmen,et al.  Adaptive Near-Optimal Rank Tensor Approximation for High-Dimensional Operator Equations , 2013, Foundations of Computational Mathematics.

[4]  Markus Bachmayr,et al.  Iterative Methods Based on Soft Thresholding of Hierarchical Tensors , 2017, Found. Comput. Math..

[5]  Daniel Kressner,et al.  On the eigenvalue decay of solutions to operator Lyapunov equations , 2014, Syst. Control. Lett..

[6]  Boris N. Khoromskij,et al.  Approximate iterations for structured matrices , 2008, Numerische Mathematik.

[7]  K. Bredies,et al.  Linear Convergence of Iterative Soft-Thresholding , 2007, 0709.1598.

[8]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[9]  Marie Billaud-Friess,et al.  A tensor approximation method based on ideal minimal residual formulations for the solution of high-dimensional problems ∗ , 2013, 1304.6126.

[10]  Boris N. Khoromskij,et al.  Tensor Numerical Methods in Scientific Computing , 2018 .

[11]  Wolfgang Dahmen,et al.  Tensor-Sparsity of Solutions to High-Dimensional Elliptic Partial Differential Equations , 2014, Found. Comput. Math..

[12]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[13]  Reinhold Schneider,et al.  Low rank tensor recovery via iterative hard thresholding , 2016, ArXiv.

[14]  William F. Moss,et al.  Decay rates for inverses of band matrices , 1984 .

[15]  E. Zeidler.  Nonlinear functional analysis and its applications , 1988 .

[16]  A. Uschmajew,et al.  On low-rank approximability of solutions to high-dimensional operator equations and eigenvalue problems , 2014, 1406.7026.

[17]  Reinhold Schneider,et al.  Variational calculus with sums of elementary tensors of fixed rank , 2012, Numerische Mathematik.

[18]  Daniel Kressner,et al.  Krylov Subspace Methods for Linear Systems with Tensor Product Structure , 2010, SIAM J. Matrix Anal. Appl..

[19]  Ioannis Chatzigeorgiou,et al.  Bounds on the Lambert Function and Their Application to the Outage Analysis of User Cooperation , 2013, IEEE Communications Letters.

[20]  R. A. Smith.  Matrix Equation $XA + BX = C$ , 1968 .

[21]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[22]  Reinhold Schneider,et al.  Tensor completion in hierarchical tensor representations , 2014, ArXiv.

[23]  Wolfgang Hackbusch.  Solution of linear systems in high spatial dimensions , 2015, Comput. Vis. Sci..

[24]  G. Golub,et al.  Bounds for the Entries of Matrix Functions with Applications to Preconditioning , 1999 .

[25]  Martin J. Mohlenkamp,et al.  Algorithms for Numerical Analysis in High Dimensions , 2005, SIAM J. Sci. Comput..

[26]  W. Hackbusch.  Tensor Spaces and Numerical Tensor Calculus , 2012, Springer Series in Computational Mathematics.

[27]  Thilo Penzl.  Eigenvalue decay bounds for solutions of Lyapunov equations: the symmetric case , 2000 .