Assessing the quality of identified models through the asymptotic theory - when is the result reliable?

In this paper, the problem of estimating uncertainty regions for identified models is considered. A typical approach in this context is to resort to the asymptotic theory of Prediction Error Methods for system identification, by means of which ellipsoidal uncertainty regions can be constructed for the uncertain parameters. We show that the uncertainty regions worked out through the asymptotic theory can be unreliable in certain situations, precisely characterized in the paper. Then, we critically analyze the theoretical conditions for the validity of the asymptotic theory, and prove that the asymptotic theory also applies under new assumptions which are less restrictive than the usually required ones. Thanks to this result, we single out the classes of models among standard ones (ARX, ARMAX, Box-Jenkins, etc.) where the asymptotic theory can be safely used in practical applications to assess the quality of the identified model. These results are of interest in many applications, including iterative controller design schemes.

[1]  L. Ljung,et al.  The role of model validation for assessing the size of the unmodeled dynamics , 1997, IEEE Trans. Autom. Control..

[2]  Lennart Ljung,et al.  Model Validation and Model Error Modeling , 1999 .

[3]  Lennart Ljung,et al.  Asymptotic variance expressions for estimated frequency functions , 2001, IEEE Trans. Autom. Control..

[4]  Graham C. Goodwin,et al.  Estimated Transfer Functions with Application to Model Order Selection , 1992 .

[5]  Brian D. O. Anderson,et al.  On robust performance improvement through the windsurfer approach to adaptive robust control , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[6]  L. Ljung.  Convergence analysis of parametric identification methods , 1978 .

[7]  P. V. D. Hof,et al.  Identification of probabilistic system uncertainty regions by explicit evaluation of bias and variance errors , 1997, IEEE Trans. Autom. Control..

[8]  Karl Johan Åström,et al.  Special issue on trends in system identification , 1995, Autom..

[9]  L. Ljung,et al.  Estimating the variance in case of undermodeling using bootstrap , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[10]  Håkan Hjalmarsson,et al.  The fundamental role of general orthonormal bases in system identification , 1999, IEEE Trans. Autom. Control..

[11]  Michel Gevers.  A Decade of Progress in Iterative Process Control Design: from Theory to Practice , 2000 .

[12]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[13]  Lennart Ljung,et al.  Comparing different approaches to model error modeling in robust identification , 2002, Autom..

[14]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[15]  Graham C. Goodwin,et al.  Estimation of Model Quality , 1994 .

[16]  Lennart Ljung,et al.  Model Error Modeling and Control Design , 2000 .

[17]  Giorgio Picci,et al.  Identification, Adaptation, Learning , 1996 .

[18]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.

[19]  Erik Weyer,et al.  Finite sample properties of system identification methods , 2002, IEEE Trans. Autom. Control..

[20]  Paul M. J. Van den Hof,et al.  Identi cation of Probabilistic System Uncertainty Regions byExplicit Evaluation of Bias and Variance Errors zRichard , 1997 .

[21]  Paul M.J. Van den Hof,et al.  Identification and Control - Closed Loop Issues , 1994 .

[22]  Erik Weyer,et al.  Non-asymptotic confidence ellipsoids for the least squares estimate , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[23]  L. Ljung.  Asymptotic variance expressions for identified black-box transfer function models , 1984, The 23rd IEEE Conference on Decision and Control.

[24]  Erik Weyer,et al.  Non-asymptotic confidence ellipsoids for the least-squares estimate , 2002, Autom..

[25]  Lennart Ljung,et al.  Comparing Di erent Approaches to Model Error Modeling in Robust Identi cation ∗ , 2001 .

[26]  Lennart Ljung,et al.  Estimating model variance in the case of undermodeling , 1992 .

[27]  Lennart Ljung,et al.  Estimating Linear Time-invariant Models of Nonlinear Time-varying Systems , 2001, Eur. J. Control.

[28]  L. Ljung.  Asymptotic variance expressions for identified black-box transfer function models , 1984 .

[29]  Sergio Bittanti,et al.  An iterative controller design scheme based on average robust control , 2002 .