Robust least-squares estimation with a relative entropy constraint

Given a nominal statistical model, we consider the minimax estimation problem consisting of finding the best least-squares estimator for the least favorable statistical model within a neighborhood of the nominal model. The neighborhood is formed by placing a bound on the Kullback-Leibler (KL) divergence between the actual and nominal models. For a Gaussian nominal model and a finite observations interval, or for a stationary Gaussian process over an infinite interval, the usual noncausal Wiener filter remains optimal. However, the worst case performance of the filter is affected by the size of the neighborhood representing the model uncertainty. On the other hand, standard causal least-squares estimators are not optimal, and a characterization is provided for the causal estimator and the corresponding least favorable model. The causal estimator takes the form of a risk-sensitive estimator with an appropriately selected risk sensitivity coefficient.

[1]  Petros G. Voulgaris,et al.  On optimal ℓ∞ to ℓ∞ filtering , 1995, Autom..

[2]  P. Khargonekar,et al.  Filtering and smoothing in an H/sup infinity / setting , 1991 .

[3]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[4]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[5]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[6]  K. Vastola,et al.  Robust Wiener-Kolmogorov theory , 1984, IEEE Trans. Inf. Theory.

[7]  P. Papantoni-Kazakos,et al.  Spectral distance measures between Gaussian processes , 1980, ICASSP.

[8]  Saleem A. Kassam,et al.  Robust Wiener filters , 1977 .

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  Anand Ganesh Dabak.  A geometry for detection theory , 1993 .

[11]  Ian R. Petersen,et al.  Robustness and risk-sensitive filtering , 2002, IEEE Trans. Autom. Control..

[12]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[13]  Wolfgang J. Runggaldier,et al.  Connections between stochastic control and dynamic games , 1996, Math. Control. Signals Syst..

[14]  S.A. Kassam,et al.  Robust techniques for signal processing: A survey , 1985, Proceedings of the IEEE.

[15]  Ali H. Sayed,et al.  A framework for state-space estimation with uncertain models , 2001, IEEE Trans. Autom. Control..

[16]  M. Basseville.  Information : entropies, divergences et moyennes , 1996 .

[17]  S. Kullback,et al.  Information Theory and Statistics , 1959 .

[18]  H. Vincent.  On Minimax Robustness: A General Approach and Applications , 1984 .

[19]  K. Glover,et al.  Minimum entropy H ∞ control , 1990 .

[20]  Ali H. Sayed,et al.  Linear Estimation (Information and System Sciences Series) , 2000 .

[21]  I. Csiszár,et al.  MEM pixel correlated solutions for generalized moment and interpolation problems , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[22]  David S. Stoffer,et al.  Time series analysis and its applications , 2000 .

[23]  H. Poor.  On robust wiener filtering , 1980 .

[24]  Guy Le Besnerais,et al.  A new look at entropy for solving linear inverse problems , 1999, IEEE Trans. Inf. Theory.

[25]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[26]  I. Csiszár.  Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems , 1991 .

[27]  Pramod P. Khargonekar,et al.  FILTERING AND SMOOTHING IN AN H" SETTING , 1991 .

[28]  P. Whittle.  Risk-Sensitive Optimal Control , 1990 .

[29]  Gjerrit Meinsma,et al.  J -spectral factorization and equalizing vectors , 1995 .

[30]  Babak Hassibi,et al.  Indefinite-Quadratic Estimation And Control , 1987 .

[31]  Rami Mangoubi.  Robust Estimation and Failure Detection: A Concise Treatment , 1998 .