A Competitive Minimax Approach to Robust Estimation in Linear Models

We consider the problem of estimating, in the presence of model uncertainties, a random vector x that is observed through a linear transformation H and corrupted by additive noise. We first assume that both the covariance of x and the transformation H are not completely specified, and develop the linear estimator that minimizes the worst-case mean-squared error (MSE) across all possible covariance matrices and transformations H in the region of uncertainty. Although the minimax approach has enjoyed widespread use in the design of robust methods, we show that its performance is often unsatisfactory. To improve the performance over the minimax MSE estimator, we develop a competitive minimax approach, for the case where H is known but the covariance of x is subject to uncertainties, and seek the linear estimator that minimizes the worst-case regret, namely the worst-case difference between the MSE attainable using a linear estimator, ignorant of the signal covariance, and the optimal MSE attained using a linear estimator that knows the signal covariance. The linear minimax regret estimator is shown to be equal to a minimum MSE (MMSE) estimator corresponding to a certain choice of signal covariance, that depends explicitly on the uncertainty region. We demonstrate through an example that the minimax regret approach can improve the performance over both the minimax MSE approach and a “plug in” approach, in which the estimator is chosen to be equal to the MMSE estimator with an estimated covariance matrix replacing the true unknown covariance. We then show that although the optimal minimax regret estimator in the case in which the signal and noise are jointly Gaussian is nonlinear, we often do not loose much by restricting attention to linear estimators. ∗Technion—Israel Institute of Technology, Haifa 32000, Israel. E-mail: {yonina,merhav}@ee.technion.ac.il. Fax: +972-48323041. 1