Jackknife and bootstrap methods in the identification of dynamic models

Abstract A new criterion based on a Jackknife or a Bootstrap statistic is proposed for identifying non-parsimonious dynamic models (FIR, ARX). It is applicable for selecting the number of components in latent variable regression methods or the constraining parameter in regularized least squares regression methods. These meta parameters are used to overcome ill-conditioning caused by model over-parameterization, when fitted using prediction error or least squares methods. In all cases studied, using PLS for parameter estimation, the proposed criterion led to the selection of better models, in the mean square error sense, than when selected via cross-validation. The methodology also provides approximate confidence intervals for the model parameters and the step and impulse response of the system.

[1]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[2]  Barry M. Wise,et al.  Identification of finite impulse response models with continuum regression , 1993 .

[3]  B. Efron.  Bootstrap Methods: Another Look at the Jackknife , 1979 .

[4]  John F. MacGregor,et al.  Modeling of dynamic systems using latent variable and subspace methods , 2000 .

[5]  A. E. Hoerl,et al.  Ridge Regression: Applications to Nonorthogonal Problems , 1970 .

[6]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[7]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[8]  J. Macgregor,et al.  MULTIVARIATE IDENTIFICATION: A STUDY OF SEVERAL METHODS , 1992 .

[9]  S. Wold.  Cross-Validatory Estimation of the Number of Components in Factor and Principal Components Models , 1978 .

[10]  John F. MacGregor,et al.  Multi-output process identification , 1997 .

[11]  D. Hinkley.  Jackknifing in Unbalanced Situations , 1977 .

[12]  Rupert G. Miller.  An Unbalanced Jackknife , 1974 .

[13]  H. Akaike.  A new look at the statistical model identification , 1974 .

[14]  Vaclav James Kresta.  The application of partial least squares to problems in chemical engineering , 1992 .

[15]  N. L. Ricker.  The use of biased least-squares estimators for parameters in discrete-time pulse-response models , 1988 .

[16]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.

[17]  J. Macgregor,et al.  Identification of finite impulse response models : Methods and robustness issues , 1996 .

[18]  M. H. Quenouille.  Approximate Tests of Correlation in Time‐Series , 1949 .

[19]  H. Martens,et al.  Modified Jack-knife estimation of parameter uncertainty in bilinear modelling by partial least squares regression (PLSR) , 2000 .

[20]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[21]  S. Wold,et al.  Partial least squares analysis with cross‐validation for the two‐class problem: A Monte Carlo study , 1987 .

[22]  J. Shao,et al.  A General Theory for Jackknife Variance Estimation , 1989 .

[23]  I. Wakeling,et al.  A test of significance for partial least squares regression , 1993 .