Minimax MSE-ratio estimation with signal covariance uncertainties

In continuation to an earlier work, we further consider the problem of robust estimation of a random vector (or signal), with an uncertain covariance matrix, that is observed through a known linear transformation and corrupted by additive noise with a known covariance matrix. While, in the earlier work, we developed and proposed a competitive minimax approach of minimizing the worst-case mean-squared error (MSE) difference regret criterion, here, we study, in the same spirit, the minimum worst-case MSE ratio regret criterion, namely, the worst-case ratio (rather than difference) between the MSE attainable using a linear estimator, ignorant of the exact signal covariance, and the minimum MSE (MMSE) attainable by optimum linear estimation with a known signal covariance. We present the optimal linear estimator, under this criterion, in two ways: The first is as a solution to a certain semidefinite programming (SDP) problem, and the second is as an expression that is of closed form up to a single parameter whose value can be found by a simple line search procedure. We then show that the linear minimax ratio regret estimator can also be interpreted as the MMSE estimator that minimizes the MSE for a certain choice of signal covariance that depends on the uncertainty region. We demonstrate that in applications, the proposed minimax MSE ratio regret approach may outperform the well-known minimax MSE approach, the minimax MSE difference regret approach, and the "plug-in" approach, where in the latter, one uses the MMSE estimator with an estimated covariance matrix replacing the true unknown covariance.

[1]  Yonina C. Eldar,et al.  A competitive minimax approach to robust estimation of random parameters , 2004, IEEE Transactions on Signal Processing.

[2]  H. Poor On robust wiener filtering , 1980 .

[3]  Aharon Ben-Tal,et al.  Lectures on modern convex optimization , 1987 .

[4]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[5]  Yonina C. Eldar,et al.  Linear minimax regret estimation of deterministic parameters with bounded data uncertainties , 2004, IEEE Transactions on Signal Processing.

[6]  S.A. Kassam,et al.  Robust techniques for signal processing: A survey , 1985, Proceedings of the IEEE.

[7]  Yonina C. Eldar,et al.  A Competitive Minimax Approach to Robust Estimation in Linear Models , 2007 .

[8]  Robert M. Gray,et al.  Toeplitz and Circulant Matrices: A Review , 2005, Found. Trends Commun. Inf. Theory.

[9]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[10]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[11]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[12]  J. Franke Minimax-robust prediction of discrete time series , 1985 .

[13]  A.K. Katsaggelos,et al.  A general framework for frequency domain multi-channel signal processing , 1993, IEEE Trans. Image Process..

[14]  N. Merhav,et al.  A competitive Neyman-Pearson approach to universal hypothesis testing with applications , 2001, Proceedings. 2001 IEEE International Symposium on Information Theory (IEEE Cat. No.01CH37252).

[15]  K. Vastola,et al.  Robust Wiener-Kolmogorov theory , 1984, IEEE Trans. Inf. Theory.

[16]  Bruno O. Shubert,et al.  Random variables and stochastic processes , 1979 .

[17]  L. Breiman A Note on Minimax Filtering , 1973 .

[18]  Saleem A. Kassam,et al.  Robust Wiener filters , 1977 .

[19]  Zhi-Quan Luo,et al.  Robust adaptive beamforming using worst-case performance optimization: a solution to the signal mismatch problem , 2003, IEEE Trans. Signal Process..

[20]  David P. Williamson,et al.  Approximation algorithms for MAX-3-CUT and other problems via complex semidefinite programming , 2001, STOC '01.

[21]  Yonina C. Eldar,et al.  Robust mean-squared error estimation in the presence of model uncertainties , 2005, IEEE Transactions on Signal Processing.

[22]  Lee D. Davisson,et al.  Universal noiseless coding , 1973, IEEE Trans. Inf. Theory.

[23]  C. Brezinski Interpolation and Extrapolation , 2001 .

[24]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[25]  A.B. Gershman,et al.  Robust adaptive beamforming using worst-case performance optimization , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[26]  Yonina C. Eldar,et al.  Competitive Mean-Squared Error Beamforming , 2004 .

[27]  Neri Merhav,et al.  Universal Prediction , 1998, IEEE Trans. Inf. Theory.

[28]  Jian Li,et al.  On robust Capon beamforming and diagonal loading , 2003, IEEE Trans. Signal Process..

[29]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[30]  Saleem A. Kassam,et al.  Robust hypothesis testing for bounded classes of probability densities , 1981, IEEE Trans. Inf. Theory.

[31]  Zhi-Quan Luo,et al.  Robust adaptive beamforming for general-rank signal models , 2003, IEEE Trans. Signal Process..

[32]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series , 1964 .

[33]  Richard M. Schwartz,et al.  Enhancement of speech corrupted by acoustic noise , 1979, ICASSP.

[34]  Hideo Murakami,et al.  Discrete wavelet transform based on cyclic convolutions , 2004, IEEE Transactions on Signal Processing.

[35]  E. Yaz Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[36]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[37]  Yonina C. Eldar,et al.  Mean‐Squared Error Beamforming for Signal Estimation: A Competitive Approach , 2005 .

[38]  S. Boll,et al.  Suppression of acoustic noise in speech using spectral subtraction , 1979 .

[39]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[40]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[41]  Hagai Attias,et al.  Independent Factor Analysis , 1999, Neural Computation.

[42]  Neri Merhav,et al.  Universal composite hypothesis testing: A competitive minimax approach , 2002, IEEE Trans. Inf. Theory.

[43]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[44]  Stephen P. Boyd,et al.  Linear Matrix Inequalities in Systems and Control Theory , 1994 .

[45]  I. Miller Probability, Random Variables, and Stochastic Processes , 1966 .