A Minimax Chebyshev Estimator for Bounded Error Estimation

We develop a nonlinear minimax estimator for the classical linear regression model assuming that the true parameter vector lies in an intersection of ellipsoids. We seek an estimate that minimizes the worst-case estimation error over the given parameter set. Since this problem is intractable, we approximate it using semidefinite relaxation, and refer to the resulting estimate as the relaxed Chebyshev center (RCC). We show that the RCC is unique and feasible, meaning it is consistent with the prior information. We then prove that the constrained least-squares (CLS) estimate for this problem can also be obtained as a relaxation of the Chebyshev center, that is looser than the RCC. Finally, we demonstrate through simulations that the RCC can significantly improve the estimation error over the CLS method.

[1]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[2]  Per Christian Hansen,et al.  Regularization methods for large-scale problems , 1993 .

[3]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[4]  Henryk Wozniakowski,et al.  Information-based complexity , 1987, Nature.

[5]  R. Tempo,et al.  Optimal algorithms theory for robust estimation and prediction , 1985 .

[6]  P. Hansen.  Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion , 1987 .

[7]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[8]  Jie Sun,et al.  Solution Methodologies for the Smallest Enclosing Circle Problem , 2003, Comput. Optim. Appl..

[9]  Yonina C. Eldar,et al.  Strong Duality in Nonconvex Quadratic Optimization with Two Quadratic Constraints , 2006, SIAM J. Optim..

[10]  Yonina C. Eldar,et al.  Covariance shaping least-squares estimation , 2003, IEEE Trans. Signal Process..

[11]  B. Efron.  Biased Versus Unbiased Estimation , 1975 .

[12]  Jianhong Shen,et al.  Deblurring images: Matrices, spectra, and filtering , 2007, Math. Comput..

[13]  Yonina C. Eldar,et al.  Blind Minimax Estimation , 2007, IEEE Transactions on Information Theory.

[14]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[15]  J. P. Norton,et al.  Identification and application of bounded-parameter models , 1985, Autom..

[16]  Donald W. Marquaridt.  Generalized Inverses, Ridge Regression, Biased Linear Estimation, and Nonlinear Estimation , 1970 .

[17]  Yonina C. Eldar,et al.  Regularization in Regression with Bounded Noise: A Chebyshev Center Approach , 2007, SIAM J. Matrix Anal. Appl..

[18]  Franz Rendl,et al.  A recipe for semidefinite relaxation for (0,1)-quadratic programming , 1995, J. Glob. Optim..

[19]  Per Christian Hansen,et al.  REGULARIZATION TOOLS: A Matlab package for analysis and solution of discrete ill-posed problems , 1994, Numerical Algorithms.

[20]  Dan Amir.  Chebyshev centers and uniform convexity , 1978 .

[21]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[22]  Zhi-Fu Wang,et al.  On Biased Estimation in Linear Models , 2006, 2006 International Conference on Machine Learning and Cybernetics.

[23]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[24]  Antonio Vicino,et al.  Optimal estimation theory for dynamic systems with set membership uncertainty: An overview , 1991, Autom..

[25]  Dianne P. O'Leary,et al.  The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems , 1993, SIAM J. Sci. Comput..

[26]  Yonina C. Eldar,et al.  Robust mean-squared error estimation in the presence of model uncertainties , 2005, IEEE Transactions on Signal Processing.

[27]  M. Milanese,et al.  Estimation theory and uncertainty intervals evaluation in presence of unknown but bounded errors: Linear families of models and estimators , 1982 .

[28]  Yonina C. Eldar,et al.  Linear minimax regret estimation of deterministic parameters with bounded data uncertainties , 2004, IEEE Transactions on Signal Processing.