Stochastic Embedding revisited: A modern interpretation

There is a very extensive literature on various aspects of the central Bias-Variance trade-off in linear system identification. In the 80's and 90's the focus was on bias characterization, model error models and Stochastic Embedding. Recently, there has been a new interest in Bayesian or kernel methods. This paper puts part of this literature into perspective by giving a modern interpretation of the Stochastic Embedding approach.

[1]  T. W. Anderson.  Maximum Likelihood Estimates for a Multivariate Normal Distribution when Some Observations are Missing , 1957 .

[2]  R. Elashoff,et al.  Missing Observations in Multivariate Statistics I. Review of the Literature , 1966 .

[3]  D. Rubin.  INFERENCE AND MISSING DATA , 1975 .

[4]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[5]  Arne Gabrielsen.  Consistency and identifiability , 1978 .

[6]  D.G. Dudley,et al.  Dynamic system identification experiment design and data analysis , 1979, Proceedings of the IEEE.

[7]  R. Shumway,et al.  AN APPROACH TO TIME SERIES SMOOTHING AND FORECASTING USING THE EM ALGORITHM , 1982 .

[8]  L. Ljung,et al.  Design variables for bias distribution in transfer function estimation , 1986, The 23rd IEEE Conference on Decision and Control.

[9]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[10]  G. Goodwin,et al.  A stochastic embedding approach for quantifying uncertainty in the estimation of restricted complexity models , 1989 .

[11]  G. Goodwin,et al.  Quantification of Uncertainty in Estimation , 1990 .

[12]  Tohru Katayama,et al.  Robust Identification and Smoothing for Linear System with Outliers and Missing Data , 1990 .

[13]  Antonio Vicino,et al.  Optimal estimation theory for dynamic systems with set membership uncertainty: An overview , 1991, Autom..

[14]  L. Ljung,et al.  Hard frequency-domain model error bounds from least-squares like identification techniques , 1992 .

[15]  Graham C. Goodwin,et al.  Estimated Transfer Functions with Application to Model Order Selection , 1992 .

[16]  Stefano Alliney,et al.  An algorithm for the minimization of mixed l1 and l2 norms with application to Bayesian estimation , 1994, IEEE Trans. Signal Process..

[17]  Graham C. Goodwin,et al.  Estimation of model quality , 1994, Autom..

[18]  D. Heitjan,et al.  Distinguishing “Missing at Random” and “Missing Completely at Random” , 1996 .

[19]  P. V. D. Hof,et al.  Identification of probabilistic system uncertainty regions by explicit evaluation of bias and variance errors , 1997, IEEE Trans. Autom. Control..

[20]  Lennart Ljung,et al.  Model Validation and Model Error Modeling , 1999 .

[21]  Y. Pawitan.  In all likelihood : statistical modelling and inference using likelihood , 2002 .

[22]  Lennart Ljung,et al.  Comparing different approaches to model error modeling in robust identification , 2002, Autom..

[23]  Nicole A. Lazar,et al.  Statistical Analysis With Missing Data , 2003, Technometrics.

[24]  Bruno Sinopoli,et al.  Foundations of Control and Estimation Over Lossy Networks , 2007, Proceedings of the IEEE.

[25]  R. B. Gopaluni.  A particle filter approach to identification of nonlinear processes under missing observations , 2008 .

[26]  Graham C. Goodwin,et al.  On the equivalence of time and frequency domain maximum likelihood estimation , 2010, Autom..

[27]  Giuseppe De Nicolao,et al.  A new kernel-based approach for linear system identification , 2010, Autom..

[28]  Sara van de Geer,et al.  Statistics for High-Dimensional Data , 2011 .

[29]  James G. Scott,et al.  Data augmentation for non-Gaussian regression models using variance-mean mixtures , 2011, 1103.5407.

[30]  Graham C. Goodwin,et al.  A novel approach to model error modelling using the expectation-maximization algorithm , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[31]  Graham C. Goodwin,et al.  Dual time-frequency domain system identification , 2012, Autom..

[32]  Biao Huang,et al.  System Identification , 2000, Control Theory for Physicists.

[33]  S. Kotz,et al.  The Laplace Distribution and Generalizations , 2012 .

[34]  Henrik Ohlsson,et al.  On the estimation of transfer functions, regularizations and Gaussian processes - Revisited , 2012, Autom..

[35]  也谈“nowcasting”的汉译 , 2012 .

[36]  Lennart Ljung,et al.  Kernel methods in system identification, machine learning and function estimation: A survey , 2014, Autom..

[37]  Graham C. Goodwin,et al.  Identification of sparse FIR systems using a general quantisation scheme , 2014, Int. J. Control.

[38]  Lennart Ljung,et al.  System Identification Via Sparse Multiple Kernel-Based Regularization Using Sequential Convex Optimization Techniques , 2014, IEEE Transactions on Automatic Control.