Dual time-frequency domain system identification

In this paper we obtain the maximum likelihood estimate of the parameters of discrete-time linear models by using a dual time-frequency domain approach. We propose a formulation that considers a (reduced-rank) linear transformation of the available data. Such a transformation may correspond to different options: selection of time-domain data, transformation to the frequency domain, or selection of frequency-domain data obtained from time-domain samples. We use the proposed approach to identify multivariate systems represented in state-space form by using the Expectation-Maximisation algorithm. We illustrate the benefits of the approach via numerical examples.

[1]  E. Hannan,et al.  Lagged Regression with Unknown Lags , 1973 .

[2]  Graham C. Goodwin,et al.  Estimation with Missing Data , 1999 .

[3]  Tomas McKelvey,et al.  State-space parametrizations of multivariable linear systems using tridiagonal matrix forms , 1996, Proceedings of 35th IEEE Conference on Decision and Control.

[4]  R. B. Gopaluni.  A particle filter approach to identification of nonlinear processes under missing observations , 2008 .

[5]  T. W. Anderson.  Maximum Likelihood Estimates for a Multivariate Normal Distribution when Some Observations are Missing , 1957 .

[6]  Søren Feodor Nielsen,et al.  1. Statistical Analysis with Missing Data (2nd edn). Roderick J. Little and Donald B. Rubin, John Wiley & Sons, New York, 2002. No. of pages: xv+381. ISBN: 0‐471‐18386‐5 , 2004 .

[7]  D.G. Dudley,et al.  Dynamic system identification experiment design and data analysis , 1979, Proceedings of the IEEE.

[8]  D. Heitjan,et al.  Distinguishing “Missing at Random” and “Missing Completely at Random” , 1996 .

[9]  Helmut Ltkepohl,et al.  New Introduction to Multiple Time Series Analysis , 2007 .

[10]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[11]  Yves Rolain,et al.  Minimum variance bounds for overparameterized models , 1996, IEEE Trans. Autom. Control..

[12]  G. Goodwin,et al.  Frequency domain identification of MIMO state space models using the EM algorithm , 2007, 2007 European Control Conference (ECC).

[13]  Eric Moulines,et al.  Inference in hidden Markov models , 2010, Springer series in statistics.

[14]  T. McKelvey.  Frequency domain identification methods , 2002 .

[15]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[16]  F. Vaida.  PARAMETER CONVERGENCE FOR EM AND MM ALGORITHMS , 2005 .

[17]  R. Mehra,et al.  Computational aspects of maximum likelihood estimation and reduction in sensitivity function calculations , 1974 .

[18]  Fang-kuo Sun.  On estimation with missing data , 1982, 1982 21st IEEE Conference on Decision and Control.

[19]  D. Rubin,et al.  On Jointly Estimating Parameters and Missing Data by Maximizing the Complete-Data Likelihood , 1983 .

[20]  Graham C. Goodwin,et al.  Control System Design , 2000 .

[21]  Raymond Kan.  From moments of sum to moments of product , 2008 .

[22]  Tohru Katayama,et al.  Robust Identification and Smoothing for Linear System with Outliers and Missing Data , 1990 .

[23]  Rik Pintelon,et al.  Frequency domain system identification with missing data , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[24]  Graham C. Goodwin,et al.  ROBUST IDENTIFICATION OF PROCESS MODELS FROM PLANT DATA , 2008 .

[25]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[26]  E. Hannan,et al.  The Statistical Theory of Linear Systems. , 1990 .

[27]  R. Shumway,et al.  AN APPROACH TO TIME SERIES SMOOTHING AND FORECASTING USING THE EM ALGORITHM , 1982 .

[28]  Vdi,et al.  European control conference ECC'99 , 1999 .

[29]  Rik Pintelon,et al.  System Identification: A Frequency Domain Approach , 2012 .

[30]  Graham C. Goodwin,et al.  Robust identification of continuous-time systems from sampled data , 2008 .

[31]  Bart De Moor,et al.  Subspace Identification for Linear Systems: Theory ― Implementation ― Applications , 2011 .

[32]  R. Pintelon,et al.  Frequency-domain subspace identification using FRF data from arbitrary signals , 2006 .

[33]  Brett Ninness,et al.  Maximum Likelihood estimation of state space models from frequency domain data , 2009, 2007 European Control Conference (ECC).

[34]  Bruno Sinopoli,et al.  Foundations of Control and Estimation Over Lossy Networks , 2007, Proceedings of the IEEE.

[35]  Naresh K. Sinha,et al.  Robust Identification of Continuous-Time Systems From Sampled Data , 1993 .

[36]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.

[37]  D. Rubin,et al.  Ignorability and Coarse Data , 1991 .

[38]  B. Moor,et al.  Subspace identification for linear systems , 1996 .

[39]  E. Weinstein,et al.  A new method for evaluating the log-likelihood gradient, the Hessian, and the Fisher information matrix for linear dynamic systems , 1989, IEEE Trans. Inf. Theory.

[40]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[41]  Siem Jan Koopman,et al.  Time Series Analysis by State Space Methods , 2001 .

[42]  Graham C. Goodwin,et al.  Identification of state-space systems using a dual time-frequency domain approach , 2010, 49th IEEE Conference on Decision and Control (CDC).

[43]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[44]  Manfred Deistler,et al.  Consistency and Relative Efficiency of Subspace Methods , 1994 .

[45]  Graham C. Goodwin,et al.  Some observations on robust estimation and control , 1985 .

[46]  Brett Ninness,et al.  Robust maximum-likelihood estimation of multivariable dynamic systems , 2005, Autom..

[47]  R. Elashoff,et al.  Missing Observations in Multivariate Statistics I. Review of the Literature , 1966 .

[48]  Allan Gut,et al.  An intermediate course in probability , 1995 .

[49]  V. Solo.  The Statistical Theory of Linear Systems E. J. Hannan and Manfred Deistler John Wiley & Sons, 1988 , 1992, Econometric Theory.

[50]  D. Rubin.  INFERENCE AND MISSING DATA , 1975 .

[51]  Dietmar Bauer,et al.  Comparing the CCA Subspace Method to Pseudo Maximum Likelihood Methods in the case of No Exogenous Inputs , 2005 .

[52]  George A. F. Seber,et al.  A matrix handbook for statisticians , 2007 .

[53]  G. Reinsel.  Elements of Multivariate Time Series Analysis , 1995 .

[54]  E. J. Hannan,et al.  Multiple time series , 1970 .

[55]  D. Rubin,et al.  Statistical Analysis with Missing Data. , 1989 .

[56]  R. Engle,et al.  Alternative Algorithms for the Estimation of Dynamic Factor , 1983 .

[57]  Graham C. Goodwin,et al.  On the equivalence of time and frequency domain maximum likelihood estimation , 2010, Autom..

[58]  Alf Isaksson,et al.  Identification of ARX-models subject to missing data , 1993, IEEE Trans. Autom. Control..

[59]  Graham C. Goodwin,et al.  Efficient calculation of curvature in state and parameter estimation , 1970 .

[60]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[61]  H. White.  Maximum Likelihood Estimation of Misspecified Models , 1982 .

[62]  E. Hannan.  The Identification and Parameterization of ARMAX and State Space Forms , 1976 .