Kernel-based system identification from noisy and incomplete input-output data

In this contribution, we propose a kernel-based method for the identification of linear systems from noisy and incomplete input-output datasets. We model the impulse response of the system as a Gaussian process whose covariance matrix is given by the recently introduced stable spline kernel. We adopt an empirical Bayes approach to estimate the posterior distribution of the impulse response given the data. The noiseless and missing data samples, together with the kernel hyperparameters, are estimated maximizing the joint marginal likelihood of the input and output measurements. To compute the marginal-likelihood maximizer, we build a solution scheme based on the Expectation-Maximization method. Simulations on a benchmark dataset show the effectiveness of the method.

[1]  Umberto Soverini,et al.  The frisch scheme in dynamic system identification , 1990, Autom..

[2]  R. Allen,et al.  Statistical Confluence Analysis by means of Complete Regression Systems , 1935 .

[3]  Umberto Soverini,et al.  Maximum likelihood identification of noisy input-output models , 2007, Autom..

[4]  Alessandro Chiuso,et al.  A Bayesian learning approach to linear system identification with missing data , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[5]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[6]  Rik Pintelon,et al.  Errors-in-variables identification of dynamic systems excited by arbitrary non-white input , 2013, Autom..

[7]  Alf J. Isaksson,et al.  Multiple Optima in Identification of ARX Models Subject to Missing Data , 2002, EURASIP J. Adv. Signal Process..

[8]  Torsten Söderström,et al.  Errors-in-variables methods in system identification , 2018, Autom..

[9]  Dan Fan,et al.  Frisch scheme identification for Errors-in-Variables systems , 2010, 9th IEEE International Conference on Cognitive Informatics (ICCI'10).

[10]  Torsten Söderström,et al.  Accuracy analysis of time domain maximum likelihood method and sample maximum likelihood method for errors-in-variables and output error identification , 2010, Autom..

[11]  Torsten Söderström,et al.  Perspectives on errors-in-variables estimation for dynamic systems , 2002, Signal Process..

[12]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[13]  Stephen P. Boyd,et al.  Linear models based on noisy data and the Frisch scheme , 2013, SIAM Rev..

[14]  R. Frisch.  The Foundations of Econometric Analysis: Statistical Confluence Analysis by Means of Complete Regression Systems (University Institute of Economics, Oslo, 1934, pp. 5–8) , 1995 .

[15]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[16]  Giulio Bottegal,et al.  On the identifiability of errors-in-variables models with white measurement errors , 2011, Autom..

[17]  Rik Pintelon,et al.  Identification of Linear Time-Invariant Systems From Multiple Experiments , 2015, IEEE Transactions on Signal Processing.

[18]  Anders Hansson,et al.  Maximum likelihood estimation of linear SISO models subject to missing output data and missing input data , 2014, Int. J. Control.

[19]  Zhang Liu,et al.  Nuclear norm system identification with missing inputs and outputs , 2013, Syst. Control. Lett..

[20]  Rik Pintelon,et al.  Frequency domain system identification with missing data , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[21]  K. Fernando,et al.  Identification of linear systems with input and output noise: the Koopmans-Levin method , 1985 .

[22]  Gerd Vandersteen,et al.  Frequency-domain system identification using non-parametric noise models estimated from a small number of data sets , 1997, Autom..

[23]  Giuseppe De Nicolao,et al.  A new kernel-based approach for linear system identification , 2010, Autom..

[24]  Rik Pintelon,et al.  Errors-in-variables identification of dynamic systems in general cases , 2015 .

[25]  Dennis V. Lindley,et al.  Empirical Bayes Methods , 1974 .

[26]  Chun-Bo Feng,et al.  Unbiased parameter estimation of linear systems in the presence of input and output noise , 1989 .

[27]  Ivan Markovsky,et al.  Structured Low-Rank Approximation with Missing Data , 2013, SIAM J. Matrix Anal. Appl..

[28]  Thomas A. Louis,et al.  Empirical Bayes Methods , 2006 .

[29]  Torsten Söderström,et al.  Identification of stochastic linear systems in presence of input noise , 1981, Autom..

[30]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[31]  Lennart Ljung,et al.  Kernel methods in system identification, machine learning and function estimation: A survey , 2014, Autom..

[32]  Brian D. O. Anderson,et al.  Identification of scalar errors-in-variables models with dynamics , 1985, Autom..

[33]  Lennart Ljung,et al.  Sparse multiple kernels for impulse response estimation with majorization minimization algorithms , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).