Linear regression and filtering under nonstandard assumptions (arbitrary noise)

This note is devoted to parameter estimation in linear regression and filtering, where the observation noise does not possess any "nice" probabilistic properties. In particular, the noise might have an "unknown-but-bounded" deterministic nature. The basic assumption is that the model regressors (inputs) are random. Optimal rates of convergence for the modified stochastic approximation and least-squares algorithms are established under some weak assumptions. Typical behavior of the algorithms in the presence of such deterministic noise is illustrated by numerical examples.

[1]  Elias Masry,et al.  Convergence analysis of adaptive linear estimation for dependent stationary processes , 1988, IEEE Trans. Inf. Theory.

[2]  O. Granichin.  Nonminimax Filtering in Unknown Irregular Constrained Observation Noise , 2002 .

[3]  H. Robbins,et al.  A Convergence Theorem for Non Negative Almost Supermartingales and Some Applications , 1985 .

[4]  L. Ljung,et al.  The role of model validation for assessing the size of the unmodeled dynamics , 1997, IEEE Trans. Autom. Control..

[5]  L. Ljung,et al.  Necessary and sufficient conditions for stability of LMS , 1997, IEEE Trans. Autom. Control..

[6]  O. Granichin.  Estimating the Parameters of Linear Regression in an Arbitrary Noise , 2002 .

[7]  Peter C. Young,et al.  Recursive Estimation and Time Series Analysis , 1984 .

[8]  H. Robbins,et al.  A CONVERGENCE THEOREM FOR NON NEGATIVE ALMOST SUPERMARTINGALES AND SOME APPLICATIONS**Research supported by NIH Grant 5-R01-GM-16895-03 and ONR Grant N00014-67-A-0108-0018. , 1971 .

[9]  W. McEneaney.  Robust/ H ∞ filtering for nonlinear systems , 1998 .

[10]  J. I.  The Design of Experiments , 1936, Nature.

[11]  W. Fleming,et al.  Risk sensitive and robust nonlinear filtering , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[12]  A. Juditsky.  A stochastic estimation algorithm with observation averaging , 1993, IEEE Trans. Autom. Control..

[13]  Qing Zhang,et al.  Hybrid filtering for linear systems with non-Gaussian disturbances , 2000, IEEE Trans. Autom. Control..

[14]  Peter C. Young,et al.  Recursive Estimation and Time-Series Analysis: An Introduction , 1984 .

[15]  Qing Zhang.  Optimal Filtering of Discrete-Time Hybrid Systems , 1999 .

[16]  Lennart Ljung,et al.  Theory and Practice of Recursive Identification , 1983 .