Nonlinear dynamical factor analysis for state change detection

Changes in a dynamical process are often detected by monitoring selected indicators directly obtained from the process observations, such as the mean values or variances. Standard change detection algorithms such as the Shewhart control charts or the cumulative sum (CUSUM) algorithm are often based on such first- and second-order statistics. Much better results can be obtained if the dynamical process is properly modeled, for example by a nonlinear state-space model, and then the accuracy of the model is monitored over time. The success of the latter approach depends largely on the quality of the model. In practical applications like industrial processes, the state variables, dynamics, and observation mapping are rarely known accurately. Learning from data must be used; however, methods for the simultaneous estimation of the state and the unknown nonlinear mappings are very limited. We use a novel method of learning a nonlinear state-space model, the nonlinear dynamical factor analysis (NDFA) algorithm. It takes a set of multivariate observations over time and fits blindly a generative dynamical latent variable model, resembling nonlinear independent component analysis. We compare the performance of the model in process change detection to various traditional methods. It is shown that NDFA outperforms the classical methods by a wide margin in a variety of cases where the underlying process dynamics changes.

[1]  R. A. Choudrey.  FLEXIBLE BAYESIAN INDEPENDENT COMPONENT ANALYSIS FOR BLIND SOURCE SEPARATION , 2001 .

[2]  Antti Honkela,et al.  Bayesian Non-Linear Independent Component Analysis by Multi-Layer Perceptrons , 2000 .

[3]  Jie Zhang,et al.  Process monitoring using non-linear statistical techniques , 1997 .

[4]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[5]  Juha Karhunen,et al.  An Unsupervised Ensemble Learning Method for Nonlinear Dynamic State-Space Models , 2002, Neural Computation.

[6]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[7]  Georg Dorffner,et al.  Neural Networks for Time Series Processing , 1996 .

[8]  Fredrik Gustafsson,et al.  Adaptive filtering and change detection , 2000 .

[9]  Martin T. Hagan,et al.  Gauss-Newton approximation to Bayesian learning , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[10]  E. Oja,et al.  Nonlinear Blind Source Separation by Variational Bayesian Learning , 2003, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[11]  S. Roweis,et al.  Learning Nonlinear Dynamical Systems Using the Expectation–Maximization Algorithm , 2001 .

[12]  J. Karhunen,et al.  Nonlinear Independent Factor Analysis by Hierarchical Models , 2003 .

[13]  Juha Karhunen,et al.  Accelerating Cyclic Update Algorithms for Parameter Estimation by Pattern Searches , 2003, Neural Processing Letters.

[14]  S. Haykin,et al.  Making sense of a complex world [chaotic events modeling] , 1998, IEEE Signal Process. Mag..

[15]  H. Attias.  Independent Component Analysis: ICA, graphical models and variational methods , 2001 .

[16]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[17]  Geoffrey E. Hinton,et al.  Keeping the neural networks simple by minimizing the description length of the weights , 1993, COLT '93.

[18]  B. Azimi-Sadjadi,et al.  Change detection for nonlinear systems; a particle filtering approach , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[19]  B. Moor,et al.  Subspace identification for linear systems , 1996 .

[20]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[21]  Alexander Ilin,et al.  On the Effect of the Form of the Posterior Approximation in Variational Learning of ICA Models , 2005, Neural Processing Letters.

[22]  Erkki Oja,et al.  DYNAMICAL FACTOR ANALYSIS OF RHYTHMIC MAGNETOENCEPHALOGRAPHIC ACTIVITY , 2001 .

[23]  Eric A. Wan,et al.  Dual Extended Kalman Filter Methods , 2002 .

[24]  J. W. Miskin,et al.  Ensemble Learning for Blind Source Separation , 2001 .

[25]  Jie Chen,et al.  Fault diagnosis in nonlinear dynamic systems via neural networks , 1994 .

[26]  Erkki Oja,et al.  DETECTING PROCESS STATE CHANGES BY NONLINEAR BLIND SOURCE SEPARATION , 2003 .

[27]  Zoubin Ghahramani,et al.  Learning Nonlinear Dynamical Systems Using an EM Algorithm , 1998, NIPS.

[28]  Juha Karhunen,et al.  Nonlinear static and dynamic blind source separation using ensemble learning , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[29]  Michèle Basseville,et al.  Detection of abrupt changes: theory and application , 1993 .

[30]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[31]  Jie Chen,et al.  Robust Model-Based Fault Diagnosis for Dynamic Systems , 1998, The International Series on Asian Studies in Computer and Information Science.

[32]  Juha Karhunen,et al.  Hierarchical models of variance sources , 2004, Signal Process..

[33]  Arthur C. Sanderson,et al.  Detecting change in a time-series (Corresp.) , 1980, IEEE Trans. Inf. Theory.

[34]  J. Karhunen,et al.  Building Blocks for Hierarchical Latent Variable Models , 2001 .

[35]  Nikunj C. Oza,et al.  Online Ensemble Learning , 2000, AAAI/IAAI.

[36]  Simon Haykin,et al.  Making sense of a complex world , 1998 .

[37]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[38]  Ole Winther,et al.  MEAN FIELD IMPLEMENTATION OF BAYESIAN ICA , 2001 .

[39]  Volker Tresp,et al.  Fisher Scoring and a Mixture of Modes Approach for Approximate Inference and Learning in Nonlinear State Space Models , 1998, NIPS.

[40]  P. Frank,et al.  Deterministic nonlinear observer-based approaches to fault diagnosis: A survey , 1997 .

[41]  Thomas J. McAvoy,et al.  Fault Detection and Diagnosis in Industrial Systems , 2002 .