Reduced-rank estimation for ill-conditioned stochastic linear model with high signal-to-noise ratio

Abstract Reduced-rank approach has been used for decades in robust linear estimation of both deterministic and random vector of parameters in linear model y = H x + ϵ n . In practical settings, estimation is frequently performed under incomplete or inexact model knowledge, which in the stochastic case significantly increases mean-square-error (MSE) of an estimate obtained by the linear minimum mean-square-error (MMSE) estimator, which is MSE-optimal among linear estimators in the theoretical case of perfect model knowledge. However, the improved performance of reduced-rank estimators over MMSE estimator in estimation under incomplete or inexact model knowledge has been established to date only by means of numerical simulations and arguments indicating that the reduced-rank approach may provide improved performance over MMSE estimator in certain settings. In this paper we focus on the high signal-to-noise ratio (SNR) case, which has not been previously considered as a natural area of application of reduced-rank estimators. We first show explicit sufficient conditions under which familiar reduced-rank MMSE and truncated SVD estimators achieve lower MSE than MMSE estimator if singular values of array response matrix H are perturbed. We then extend these results to the case of a generic perturbation of array response matrix H, and demonstrate why MMSE estimator frequently attains higher MSE than reduced-rank MMSE and truncated SVD estimators if H is ill-conditioned. The main results of this paper are verified in numerical simulations.

[1]  T. Tao,et al.  Random Matrices: the Distribution of the Smallest Singular Values , 2009, 0903.0614.

[2]  Isao Yamada,et al.  MV-PURE estimator of dipole source signals in EEG , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[3]  Harry L. Van Trees,et al.  Optimum Array Processing , 2002 .

[4]  S. Chandrasekaran,et al.  ESTIMATION AND CONTROL WITH BOUNDED DATA UNCERTAINTIES , 1998 .

[5]  Isao Yamada,et al.  Stochastic MV-PURE Estimator— Robust Reduced-Rank Estimator for Stochastic Linear Model , 2009, IEEE Transactions on Signal Processing.

[6]  Isao Yamada,et al.  Performance of the stochastic MV-PURE estimator in highly noisy settings , 2014, J. Frankl. Inst..

[7]  G. Stewart,et al.  Matrix Perturbation Theory , 1990 .

[8]  John S. Chipman,et al.  Linear restrictions, rank reduction, and biased estimation in linear regression , 1999 .

[9]  Adi Ben-Israel,et al.  Generalized inverses: theory and applications , 1974 .

[10]  Ali H. Sayed,et al.  Linear Estimation (Information and System Sciences Series) , 2000 .

[11]  D. Luenberger Optimization by Vector Space Methods , 1968 .

[12]  B. Eynard,et al.  Random matrices. , 2015, 1510.04430.

[13]  Dave Zachariah,et al.  Estimation for the Linear Model With Uncertain Covariance Matrices , 2014, IEEE Transactions on Signal Processing.

[14]  Sabine Van Huffel,et al.  The total least squares problem , 1993 .

[15]  S. Szarek,et al.  Chapter 8 - Local Operator Theory, Random Matrices and Banach Spaces , 2001 .

[16]  Donald W. Marquaridt Generalized Inverses, Ridge Regression, Biased Linear Estimation, and Nonlinear Estimation , 1970 .

[17]  Sabine Van Huffel,et al.  Total least squares problem - computational aspects and analysis , 1991, Frontiers in applied mathematics.

[18]  Isao Yamada,et al.  An Algebraic Translation of Cayley-Dickson Linear Systems and Its Applications to Online Learning , 2014, IEEE Transactions on Signal Processing.

[19]  Isao Yamada,et al.  Why the stochastic MV-PURE estimator excels in highly noisy situations? , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[20]  Gene H. Golub,et al.  An analysis of the total least squares problem , 1980, Milestones in Matrix Computation.

[21]  Edwin K. P. Chong,et al.  The geometry of linearly and quadratically constrained optimization problems for signal processing and communications , 2010, J. Frankl. Inst..

[22]  Isao Yamada,et al.  MV-PURE Estimator: Minimum-Variance Pseudo-Unbiased Reduced-Rank Estimator for Linearly Constrained Ill-Conditioned Inverse Problems , 2008, IEEE Transactions on Signal Processing.

[23]  M. Rudelson,et al.  Non-asymptotic theory of random matrices: extreme singular values , 2010, 1003.2990.

[24]  Isao Yamada,et al.  Minimum-Variance Pseudo-Unbiased Low-Rank Estimator for Ill-Conditioned Inverse Problems , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[25]  Petre Stoica,et al.  Maximum likelihood parameter and rank estimation in reduced-rank multivariate linear regressions , 1996, IEEE Trans. Signal Process..

[26]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[27]  Jean-Marc Azaïs,et al.  Upper and Lower Bounds for the Tails of the Distribution of the Condition Number of a Gaussian Matrix , 2005, SIAM J. Matrix Anal. Appl..

[28]  Yonina C. Eldar,et al.  A competitive minimax approach to robust estimation of random parameters , 2004, IEEE Transactions on Signal Processing.

[29]  Wang,et al.  Wireless Communication Systems , 2011 .

[30]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .

[31]  G. Stewart A note on the perturbation of singular values , 1979 .

[32]  Louis L. Scharf,et al.  The SVD and reduced rank signal processing , 1991, Signal Process..

[33]  Isao Yamada,et al.  A rank selection of MV-PURE with an unbiased predicted-MSE criterion and its efficient implementation in image restoration , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[34]  Jianhong Shen On the singular values of Gaussian random matrices , 2001 .

[35]  Petre Stoica,et al.  Optimal reduced-rank estimation and filtering , 2001, IEEE Trans. Signal Process..

[36]  Isao Yamada,et al.  Performance of the stochastic MV-PURE estimator with explicit modeling of uncertainty , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[37]  Isao Yamada,et al.  Efficient Parallel Computation of the Stochastic MV-PURE Estimator by the Hybrid Steepest Descent Method , 2012, ICAISC.

[38]  Suleyman S. Kozat,et al.  Robust turbo equalization under channel uncertainties , 2011, RWS 2011.

[39]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[40]  F. M. Dopico A Note on Sin Θ Theorems for Singular Subspace Variations , 2000 .

[41]  Eric L. Miller,et al.  Robust Estimation of a Random Parameter in a Gaussian Linear Model With Joint Eigenvalue and Elementwise Covariance Uncertainties , 2010, IEEE Transactions on Signal Processing.

[42]  P. Hansen The 2-norm of random matrices , 1988 .

[43]  Yukihiko Yamashita,et al.  Relative Karhunen-Loève transform , 1996, IEEE Trans. Signal Process..

[44]  Magnus Jansson,et al.  Reduced rank linear regression and weighted low rank approximations , 2006, IEEE Transactions on Signal Processing.