Identification of Nonlinear Processes with known Model Structure Under Missing Observations

Abstract A novel maximum likelihood solution to the problem of identifying parameters of a nonlinear model under missing observations is presented. An expectation maximization (EM) algorithm, which uses the expected value of the complete log-likelihood function including the missing observations, is developed. The expected value of the complete log-likelihood (E-step) in the EM algorithm is approximated using particle filters and smoothers. New expressions for particle filters and smoothers under missing observations are derived. The maximization step (M-step) in the EM algorithm is performed using standard optimization routines. The above nonlinear identification approach is illustrated through numerical examples.

[1]  Oliver Johnson.  A conditional entropy power inequality for dependent variables , 2004, IEEE Transactions on Information Theory.

[2]  Sirish L. Shah,et al.  Adaptive multirate state and parameter estimation strategies with application to a bioreactor , 1995 .

[3]  Dale E. Seborg,et al.  Nonlinear Process Control , 1996 .

[4]  J. Duane Morningred,et al.  An Adaptive Nonlinear Predictive Controller , 1990, 1990 American Control Conference.

[5]  Ruslan Salakhutdinov,et al.  Adaptive Overrelaxed Bound Optimization Methods , 2003, ICML.

[6]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[7]  G.C. Goodwin,et al.  Approximate EM Algorithms for Parameter and State Estimation in Nonlinear Stochastic Models , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[8]  Sirish L. Shah,et al.  Identification of chemical processes with irregular output sampling , 2006 .

[9]  Thomas Bo Schön,et al.  Maximum Likelihood Nonlinear System Estimation , 2006 .

[10]  M. Soroush,et al.  Multi‐rate nonlinear state and parameter estimation in a bioreactor , 1999 .

[11]  David S. Stoffer,et al.  Time series analysis and its applications , 2000 .

[12]  Sumeetpal S. Singh,et al.  Novel particle filter methods for recursive and batch maximum likelihood parameter estimation in general states space models , 2005 .

[13]  Peter J. Gawthrop,et al.  On the estimation of continuous time transfer functions , 2001 .

[14]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[15]  Nando de Freitas,et al.  Fast particle smoothing: if I had a million particles , 2006, ICML.

[16]  Kaare Brandt Petersen,et al.  On the Slow Convergence of EM and VBEM in Low-Noise Linear Models , 2005, Neural Computation.

[17]  Christophe Andrieu,et al.  Particle methods for change detection, system identification, and control , 2004, Proceedings of the IEEE.

[18]  Graham C. Goodwin,et al.  On sampled-data models for nonlinear systems , 2005, IEEE Transactions on Automatic Control.

[19]  Brett Ninness,et al.  Maximum-likelihood parameter estimation of bilinear systems , 2005, IEEE Transactions on Automatic Control.

[20]  Dongguang Li,et al.  Application of dual-rate modeling to CCR octane quality inferential control , 2001, IEEE Trans. Control. Syst. Technol..

[21]  Alf J. Isaksson,et al.  Analysis of identified 2-D noncausal models , 1993, IEEE Trans. Inf. Theory.