On the equivalence of time and frequency domain maximum likelihood estimation

Maximum likelihood estimation has a rich history. It has been successfully applied to many problems including dynamical system identification. Different approaches have been proposed in the time and frequency domains. In this paper we discuss the relationship between these approaches and we establish conditions under which the different formulations are equivalent for finite length data. A key point in this context is how initial (and final) conditions are considered and how they are introduced in the likelihood function.

[1]  D. Brillinger.  Time series - data analysis and theory , 1981, Classics in applied mathematics.

[2]  T. McKelvey.  Frequency Domain Identification , 2000 .

[3]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[4]  B. Picinbono.  Random Signals and Systems , 1993 .

[5]  D. S. G. Pollock,et al.  A handbook of time-series analysis, signal processing and dynamics , 1999 .

[6]  Graham C. Goodwin,et al.  Some observations on robust estimation and control , 1985 .

[7]  Lennart Ljung.  Frequency Domain versus Time Domain Methods in System Identification - Revisited , 2006 .

[8]  James L. Massey,et al.  Proper complex random processes with applications to information theory , 1993, IEEE Trans. Inf. Theory.

[9]  P. Robinson,et al.  Stochastic difference equations with non-integral differences , 1974, Advances in Applied Probability.

[10]  I. Miller.  Probability, Random Variables, and Stochastic Processes , 1966 .

[11]  Bernard C. Picinbono,et al.  Second-order complex random vectors and normal distributions , 1996, IEEE Trans. Signal Process..

[12]  Denise R. Osborn,et al.  Exact and Approximate Maximum Likelihood Estimators for Vector Moving Average Processes , 1977 .

[13]  Naresh K. Sinha,et al.  Robust Identification of Continuous-Time Systems From Sampled Data , 1993 .

[14]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.

[15]  Thomas Kailath,et al.  Linear Systems , 1980 .

[16]  E. J. Hannan,et al.  Multiple time series , 1970 .

[17]  J. Schoukens,et al.  Time domain identification, frequency domain identification. Equivalencies! Differences? , 2004, Proceedings of the 2004 American Control Conference.

[18]  Lennart Ljung,et al.  Some results on identifying linear systems using frequency domain data , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[19]  G. Reinsel.  Elements of Multivariate Time Series Analysis , 1995 .

[20]  Graham C. Goodwin,et al.  Model Identification and Adaptive Control: From Windsurfing to Telecommunications , 2011 .

[21]  A. Jazwinski.  Stochastic Processes and Filtering Theory , 1970 .

[22]  Carl E. Pearson,et al.  Functions of a complex variable - theory and technique , 2005 .

[23]  E. Hannan,et al.  Lagged Regression with Unknown Lags , 1973 .

[24]  D.G. Dudley,et al.  Dynamic system identification experiment design and data analysis , 1979, Proceedings of the IEEE.

[25]  J. Schoukens,et al.  Frequency domain system identification using arbitrary signals , 1997, IEEE Trans. Autom. Control..

[26]  Manfred Deistler,et al.  System Identification - General Aspects and Structure , 2001 .

[27]  J. Schoukens,et al.  Parametric identification of transfer functions in the frequency domain-a survey , 1994, IEEE Trans. Autom. Control..

[28]  L. Ljung,et al.  Frequency Domain Maximum Likelihood Identification , 1997 .

[29]  Andrew Harvey,et al.  Linear Regression in the Frequency Domain , 1978 .

[30]  David R. Brillinger,et al.  Time Series: Data Analysis and Theory. , 1982 .

[31]  Graham C. Goodwin,et al.  ROBUST IDENTIFICATION OF PROCESS MODELS FROM PLANT DATA , 2008 .

[32]  Rik Pintelon,et al.  System Identification: A Frequency Domain Approach , 2012 .

[33]  G. Goodwin,et al.  Frequency domain identification of MIMO state space models using the EM algorithm , 2007, 2007 European Control Conference (ECC).

[34]  J. Doob.  Stochastic processes , 1953 .

[35]  Adriaan van den Bos.  The Real-Complex Normal Distribution , 1998, IEEE Trans. Inf. Theory.

[36]  Bernard C. Picinbono,et al.  On circularity , 1994, IEEE Trans. Signal Process..

[37]  Kenneth S. Miller,et al.  Complex stochastic processes: an introduction to theory and application , 1974 .

[38]  Dennis S. Bernstein,et al.  Matrix Mathematics: Theory, Facts, and Formulas with Application to Linear Systems Theory , 2005 .

[39]  Liuping Wang,et al.  Identification of Continuous-time Models from Sampled Data , 2008 .

[40]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[41]  Lennart Ljung,et al.  System identification (2nd ed.): theory for the user , 1999 .

[42]  J. Schoukens,et al.  Box-Jenkins identification revisited - Part I: Theory , 2006, Autom..

[43]  Tomas McKelvey,et al.  State-space parametrizations of multivariable linear systems using tridiagonal matrix forms , 1996, Proceedings of 35th IEEE Conference on Decision and Control.

[44]  Alex Simpkins,et al.  System Identification: Theory for the User, 2nd Edition (Ljung, L.; 1999) [On the Shelf] , 2012, IEEE Robotics & Automation Magazine.

[45]  Sofia C. Olhede,et al.  On probability density functions for complex variables , 2006, IEEE Transactions on Information Theory.

[46]  T. McKelvey.  Frequency domain identification methods , 2002 .

[47]  James L. Massey,et al.  Capacity of the discrete-time Gaussian channel with intersymbol interference , 1988, IEEE Trans. Inf. Theory.

[48]  Okan K. Ersoy,et al.  Real discrete Fourier transform , 1985, IEEE Trans. Acoust. Speech Signal Process..