Towards generic theory of data compression

In this paper, we consider an extension and rigorous justification of Karhunen-Loeve transform (KLT) which is an optimal technique for data compression. We propose and study the generic KLT which is treated as the best weighted linear estimator of a given rank under the condition that the associated covariance matrix is singular. As a result, the generic KLT is constructed in terms of the pseudo-inverse matrices that imply a development of the special technique. In particular, we give a solution of the new low-rank matrix approximation problem that provides a basis for the generic KLT. Theoretical aspects of the generic KLT are carefully studied.

[1]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[2]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[3]  Anatoli Torokhti,et al.  An optimal filter of the second order , 2001, IEEE Trans. Signal Process..

[4]  Andreas Antoniou,et al.  Design of two-dimensional digital filters using singular-value decomposition and balanced approximation method , 1991, IEEE Trans. Signal Process..

[5]  Wanquan Liu,et al.  Generalized Karhunen-Loeve transform , 1998, IEEE Signal Process. Lett..

[6]  A. Torokhti,et al.  Optimal fixed rank transform of the second degree , 2001 .

[7]  Anatoli Torokhti,et al.  Constructing fixed rank optimal estimators with method of best recurrent approximations , 2003 .

[8]  M. SIAMJ..  A COUNTEREXAMPLE TO THE POSSIBILITY OF AN EXTENSION OF THE ECKART – YOUNG LOW-RANK APPROXIMATION THEOREM FOR THE ORTHOGONAL RANK TENSOR DECOMPOSITION , 2003 .

[9]  Anatoli Torokhti,et al.  Optimal transform formed by a combination of nonlinear operators: the case of data dimensionality reduction , 2006, IEEE Transactions on Signal Processing.

[10]  S. Treitel,et al.  The Design of Multistage Separable Planar Filters , 1971 .

[11]  Anatoli Torokhti,et al.  Generalized Rank-Constrained Matrix Approximations , 2007, SIAM J. Matrix Anal. Appl..

[12]  Louis L. Scharf,et al.  The SVD and reduced rank signal processing , 1991, Signal Process..

[13]  Gene H. Golub,et al.  Matrix computations , 1983 .

[14]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[15]  Yukihiko Yamashita,et al.  Relative Karhunen-Loève transform , 1996, IEEE Trans. Signal Process..

[16]  Kari Karhunen,et al.  Über lineare Methoden in der Wahrscheinlichkeitsrechnung , 1947 .

[17]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[18]  Ana M. Aguilera,et al.  Functional Principal Components Analysis by Choice of Norm , 1999 .

[19]  Alan M. Frieze,et al.  Fast monte-carlo algorithms for finding low-rank approximations , 2004, JACM.

[20]  Adi Ben-Israel,et al.  Generalized inverses: theory and applications , 1974 .

[21]  Phil Howlett,et al.  Computational methods for modelling of nonlinear systems , 2007 .

[22]  Paul S. Wang,et al.  Weighted Low-Rank Approximation of General Complex Matrices and Its Application in the Design of 2-D Digital Filters , 1997 .