A competitive minimax approach to robust estimation of random parameters

We consider the problem of estimating, in the presence of model uncertainties, a random vector x that is observed through a linear transformation H and corrupted by additive noise. We first assume that both the covariance matrix of x and the transformation H are not completely specified and develop the linear estimator that minimizes the worst-case mean-squared error (MSE) across all possible covariance matrices and transformations H in the region of uncertainty. Although the minimax approach has enjoyed widespread use in the design of robust methods, we show that its performance is often unsatisfactory. To improve the performance over the minimax MSE estimator, we develop a competitive minimax approach for the case where H is known but the covariance of x is subject to uncertainties and seek the linear estimator that minimizes the worst-case regret, namely, the worst-case difference between the MSE attainable using a linear estimator, ignorant of the signal covariance, and the optimal MSE attained using a linear estimator that knows the signal covariance. The linear minimax regret estimator is shown to be equal to a minimum MSE (MMSE) estimator corresponding to a certain choice of signal covariance that depends explicitly on the uncertainty region. We demonstrate, through examples, that the minimax regret approach can improve the performance over both the minimax MSE approach and a "plug in" approach, in which the estimator is chosen to be equal to the MMSE estimator with an estimated covariance matrix replacing the true unknown covariance. We then show that although the optimal minimax regret estimator in the case in which the signal and noise are jointly Gaussian is nonlinear, we often do not lose much by restricting attention to linear estimators.

[1]  J. Franke Minimax-robust prediction of discrete time series , 1985 .

[2]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[3]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[4]  I. Miller Probability, Random Variables, and Stochastic Processes , 1966 .

[5]  Robert M. Gray,et al.  Toeplitz and Circulant Matrices: A Review , 2005, Found. Trends Commun. Inf. Theory.

[6]  Saleem A. Kassam,et al.  Robust hypothesis testing for bounded classes of probability densities , 1981, IEEE Trans. Inf. Theory.

[7]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series , 1964 .

[8]  Saleem A. Kassam,et al.  Robust Wiener filters , 1977 .

[9]  C. Brezinski Interpolation and Extrapolation , 2001 .

[10]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[11]  M. Sion On general minimax theorems , 1958 .

[12]  Stephen P. Boyd,et al.  Linear Matrix Inequalities in Systems and Control Theory , 1994 .

[13]  Saleem A. Kassam,et al.  MINIMAX ROBUST EQUALIZATION FOR RANDOM SIGNALS THROUGH UNCERTAIN CHANNELS. , 1982 .

[14]  K. Vastola,et al.  Robust Wiener-Kolmogorov theory , 1984, IEEE Trans. Inf. Theory.

[15]  Yonina C. Eldar,et al.  Linear minimax regret estimation of deterministic parameters with bounded data uncertainties , 2004, IEEE Transactions on Signal Processing.

[16]  S.A. Kassam,et al.  Robust techniques for signal processing: A survey , 1985, Proceedings of the IEEE.

[17]  Neri Merhav,et al.  Universal composite hypothesis testing: A competitive minimax approach , 2002, IEEE Trans. Inf. Theory.

[18]  S. Kassam,et al.  Robust signal processing for communication systems , 1983, IEEE Communications Magazine.

[19]  L. Breiman A Note on Minimax Filtering , 1973 .

[20]  David P. Williamson,et al.  Approximation algorithms for MAX-3-CUT and other problems via complex semidefinite programming , 2001, STOC '01.

[21]  Lee D. Davisson,et al.  Universal noiseless coding , 1973, IEEE Trans. Inf. Theory.

[22]  Neri Merhav,et al.  Universal Prediction , 1998, IEEE Trans. Inf. Theory.

[23]  H. Poor On robust wiener filtering , 1980 .

[24]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[25]  E. Yaz Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[26]  H. Vincent Poor,et al.  On minimax robustness: A general approach and applications , 1984, IEEE Trans. Inf. Theory.

[27]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[28]  Richard M. Schwartz,et al.  Enhancement of speech corrupted by acoustic noise , 1979, ICASSP.

[29]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[30]  S. Boll,et al.  Suppression of acoustic noise in speech using spectral subtraction , 1979 .

[31]  N. Merhav,et al.  A competitive Neyman-Pearson approach to universal hypothesis testing with applications , 2001, Proceedings. 2001 IEEE International Symposium on Information Theory (IEEE Cat. No.01CH37252).

[32]  Bruno O. Shubert,et al.  Random variables and stochastic processes , 1979 .

[33]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.