Stochastic algorithms for solving structured low-rank matrix approximation problems

In this paper, we investigate the complexity of the numerical construction of the Hankel structured low-rank approximation (HSLRA) problem, and develop a family of algorithms to solve this problem. Briefly, HSLRA is the problem of finding the closest (in some pre-defined norm) rank r approximation of a given Hankel matrix, which is also of Hankel structure. We demonstrate that finding optimal solutions of this problem is very hard. For example, we argue that if HSLRA is considered as a problem of estimating parameters of damped sinusoids, then the associated optimization problem is basically unsolvable. We discuss what is known as the orthogonality condition, which solutions to the HSLRA problem should satisfy, and describe how any approximation may be corrected to achieve this orthogonality. Unlike many other methods described in the literature the family of algorithms we propose has the property of guaranteed convergence.

[1]  Andrew L. Rukhin,et al.  Analysis of Time Series Structure SSA and Related Techniques , 2002, Technometrics.

[2]  James A. Cadzow,et al.  Signal enhancement-a composite property mapping algorithm , 1988, IEEE Trans. Acoust. Speech Signal Process..

[3]  S. Van Huffel,et al.  Exact and Approximate Modeling of Linear Systems: A Behavioral Approach , 2006 .

[4]  F. Andersson,et al.  Alternating Projections on Nontangential Manifolds , 2013 .

[5]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[6]  A. A. Zhigli︠a︡vskiĭ,et al.  Stochastic Global Optimization , 2007 .

[7]  Ivan Markovsky,et al.  Bibliography on total least squares and related methods , 2010 .

[8]  Fredrik Andersson,et al.  A fast alternating projection method for complex frequency estimation , 2011 .

[9]  Yaroslav D. Sergeyev,et al.  A New Applied Approach for Executing Computations with Infinite and Infinitesimal Quantities , 2008, Informatica.

[10]  Gabriele Lolli,et al.  Infinitesimals and infinites in the history of mathematics: A brief survey , 2012, Appl. Math. Comput..

[11]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[12]  Anatoly A. Zhigljavsky,et al.  Singular Spectrum Analysis for Time Series , 2013, International Encyclopedia of Statistical Science.

[13]  Anatoly A. Zhigljavsky,et al.  Computing sums of conditionally convergent and divergent series using the concept of grossone , 2012, Appl. Math. Comput..

[14]  R. Plemmons,et al.  Structured low rank approximation , 2003 .

[15]  Petar M. Djuric,et al.  Model order selection of damped sinusoids in noise by predictive densities , 1996, IEEE Trans. Signal Process..

[16]  J. B. Rosen,et al.  Low Rank Approximation of a Hankel Matrix by Structured Total Least Norm , 1999 .

[17]  S. Huffel,et al.  Efficient implementation of a structured total least squares based speech compression method , 2003 .

[18]  Sabine Van Huffel,et al.  Enhanced resolution based on minimum variance estimation and exponential data modeling , 1993, Signal Process..

[19]  Chris Chatfield,et al.  Introduction to Statistical Time Series. , 1976 .

[20]  Ivan Markovsky,et al.  Structured low-rank approximation and its applications , 2008, Autom..

[21]  Dianne P. O'Leary,et al.  Blind Deconvolution Using a Regularized Structured Total Least Norm Algorithm , 2002, SIAM J. Matrix Anal. Appl..

[22]  C. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[23]  Nina Golyandina,et al.  On the choice of parameters in Singular Spectrum Analysis and related subspace-based methods , 2010, 1005.4374.

[24]  Jonathan Gillard,et al.  Analysis of Structured Low Rank Approximation as an Optimization Problem , 2011, Informatica.

[25]  Jonathan William Gillard,et al.  Cadzow's basic algorithm, alternating projections and singular spectrum analysis , 2010 .

[26]  Yaroslav D. Sergeyev,et al.  Numerical computations and mathematical modelling with infinite and infinitesimal numbers , 2012, ArXiv.

[27]  Fredrik Andersson,et al.  Alternating projections on non-tangential manifolds , 2011, 1107.4055.

[28]  Arie Yeredor,et al.  Multiple delays estimation for chirp signals using structured total least squares , 2004 .

[29]  Sabine Van Huffel,et al.  Analysis of the Structured Total Least Squares Problem for Hankel/Toeplitz Matrices , 2004, Numerical Algorithms.

[30]  Georg Heinig,et al.  Algebraic Methods for Toeplitz-like Matrices and Operators , 1984 .

[31]  Ivan Markovsky,et al.  Low Rank Approximation - Algorithms, Implementation, Applications , 2018, Communications and Control Engineering.

[32]  J. Willems,et al.  Application of structured total least squares for system identification and model reduction , 2005, IEEE Transactions on Automatic Control.

[33]  Ivan Markovsky,et al.  Software for weighted structured low-rank approximation , 2014, J. Comput. Appl. Math..

[34]  A. Zhigljavsky.  Stochastic Global Optimization , 2008, International Encyclopedia of Statistical Science.

[35]  Yaroslav D. Sergeyev,et al.  Numerical point of view on Calculus for functions assuming finite, infinite, and infinitesimal values over finite, infinite, and infinitesimal domains , 2009, 1203.4140.

[36]  B. Moor.  Structured total least squares and L2 approximation problems , 1993 .

[37]  Antanas Zilinskas,et al.  Interval Arithmetic Based Optimization in Nonlinear Regression , 2010, Informatica.

[38]  Jonathan Gillard,et al.  Optimization challenges in the structured low rank approximation problem , 2013, J. Glob. Optim..

[39]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .