Some properties of singular value decomposition and their applications to digital signal processing

Abstract Singular value decomposition (SVD), one of the most basic and important tools of numerical linear algebra, is finding increasing applications in digital signal processing. In this paper, we first review some basic properties of SVD for matrices and then for linear operators between two Hilbert spaces. While SVD results for matrices are quite well known, applications of SVD on operators are less well known. Specifically, we propose the use of a representation for a stochastic linear system from the point of view of a stochastic SVD and show its relationship to the classical Karhunen-Loeve expansion. Furthermore, issues related to practical use of time-averaging of realizations of random processes from the SVD point of view are also considered. Then algorithms for performing SVD are briefly discussed. These include the well known QR method, the modified Jacobi method of Nash, and an extension of the modified Jacobi method applicable to SVD of linear operators. Finally, we review various applications of finite rank approximation properties of SVD to digital signal processing problems in least-squares estimations, complexity reduction of FIR digital filters, matrix rank and system order determinations, and digital image processing.

[1]  J. C. Nash,et al.  Principal Components and Regression by Singular Value Decomposition on a Small Computer , 1976 .

[2]  Gene H. Golub,et al.  Algorithm 358: singular value decomposition of a complex matrix [F1, 4, 5] , 1969, CACM.

[3]  Bhaskar D. Rao.  Perturbation analysis of an SVD-based linear prediction method for estimating the frequencies of multiple sinusoids , 1988, IEEE Trans. Acoust. Speech Signal Process..

[4]  G. Stewart.  Introduction to matrix computations , 1973 .

[5]  Naresh K. Sinha,et al.  Use of singular value decomposition in system identification , 1986 .

[6]  W. M. Carey,et al.  Digital spectral analysis: with applications , 1986 .

[7]  Tony F. Chan,et al.  An Improved Algorithm for Computing the Singular Value Decomposition , 1982, TOMS.

[8]  Brian T. Smith,et al.  Matrix Eigensystem Routines — EISPACK Guide , 1974, Lecture Notes in Computer Science.

[9]  Franklin T. Luk,et al.  Systolic Array Computation Of The Singular Value Decomposition , 1982, Other Conferences.

[10]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[11]  P. E. Castro.  Compact Numerical Methods for Computers: Linear Algebra and Function Minimization , 1978 .

[12]  Kenneth Steiglitz,et al.  Series expansion of wide-sense stationary random processes , 1968, IEEE Trans. Inf. Theory.

[13]  M. L. Eaton.  Multivariate statistics : a vector space approach , 1985 .

[14]  M Severcan.  Restoration of images of finite extent objects by a singular value decomposition technique. , 1982, Applied optics.

[15]  Sun-Yuan Kung,et al.  A new identification and model reduction algorithm via singular value decomposition , 1978 .

[16]  R. Kumaresan,et al.  Singular value decomposition and improved frequency estimation using linear prediction , 1982 .

[17]  John M. Chambers,et al.  Computational Methods for Data Analysis. , 1978 .

[18]  A. Laub,et al.  The singular value decomposition: Its computation and some applications , 1980 .

[19]  K. Fernando,et al.  Karhunen-Loève expansion with reference to singular-value decomposition and separation of variables , 1980 .

[20]  John C. Nash,et al.  A One-Sided Transformation Method for the Singular Value Decomposition and Algebraic Eigenproblem , 1975, Comput. J..

[21]  R. Kumaresan,et al.  Data adaptive signal estimation by singular value decomposition of a data matrix , 1982, Proceedings of the IEEE.

[22]  Michael A. Malcolm,et al.  Computer methods for mathematical computations , 1977 .

[23]  Franklin T. Luk,et al.  A new systolic array for the singular value decomposition , 1986 .

[24]  Konstantinos Konstantinides,et al.  Applications of singular value decomposition to system modeling in signal processing , 1984, ICASSP.

[25]  J. G. F. Francis,et al.  The QR Transformation - Part 2 , 1962, Comput. J..

[26]  R. Brent,et al.  The Solution of Singular-Value and Symmetric Eigenvalue Problems on Multiprocessor Arrays , 1985 .

[27]  Franklin T. Luk,et al.  Computing the Singular-Value Decomposition on the ILLIAC IV , 1980, TOMS.

[28]  Gene H. Golub,et al.  Matrix computations , 1983 .

[29]  Franklin T. Luk.  Architectures for Computing Eigenvalues and SVDs , 1986, Photonics West - Lasers and Applications in Science and Engineering.

[30]  C. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[31]  A. Sameh.  On Jacobi and Jacobi-like algorithms for a parallel computer , 1971 .

[32]  Harry C. Andrews,et al.  Outer Product Expansions and Their Uses in Digital Image Processing , 1976, IEEE Transactions on Computers.

[33]  Konstantinos Konstantinides,et al.  Statistical analysis of effective singular values in matrix rank determination , 1988, IEEE Trans. Acoust. Speech Signal Process..

[34]  Franklin T. Luk,et al.  Computation Of The Generalized Singular Value Decomposition Using Mesh-Connected Processors , 1983, Optics & Photonics.

[35]  J. G. F. Francis,et al.  The QR Transformation A Unitary Analogue to the LR Transformation - Part 1 , 1961, Comput. J..

[36]  H. Andrews,et al.  Singular value decompositions and digital image processing , 1976 .