Increasing mapping based hidden Markov model for dynamic process monitoring and diagnosis

Hidden Markov models (HMMs) perform parameter estimation based on the forward-backward (FB) procedure and the Baum-Welch (BW) algorithm. The two algorithms together may increase the computational complexity and the difficulty to understand the algorithm structure of HMMs clearly. In this study, an increasing mapping based hidden Markov model (IMHMM) is proposed. Between the observation sequence and possible state sequence an increasing mapping is established. The re-estimation formulas for the model parameters are derived straightforwardly based on these mappings instead of FB variables. The IMHMM has simpler algorithm structure and lower storage requirement than the HMM. Based on IMHMM, an expandable process monitoring and fault diagnosis framework for large-scale dynamical process is developed. To characterize the dynamic process, a novel index considering serial correlation is used to evaluate process state. The presented methodology is carried out in Tennessee Eastman process (TEP). The results show improvement over HMM in terms of memory complexity and training time of the model. Also, the power of IMHMM can be observed compared with principal component analysis (PCA) based methods.

[1]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[2]  R. Bakis.  Continuous speech recognition via centisecond acoustic states , 1976 .

[3]  Donghua Zhou,et al.  Geometric properties of partial least squares for process monitoring , 2010, Autom..

[4]  Xiang-Gen Xia,et al.  Improved hidden Markov models in the wavelet-domain , 2001, IEEE Trans. Signal Process..

[5]  Gene Ott,et al.  Compact encoding of stationary Markov sources , 1967, IEEE Trans. Inf. Theory.

[6]  Jianbo Yu,et al.  Hidden Markov models combining local and global information for nonlinear and multimodal process monitoring , 2010 .

[7]  Mohd Azlan Hussain,et al.  Fault diagnosis of Tennessee Eastman process with multi- scale PCA and ANFIS , 2013 .

[8]  Kenneth P. Bogart,et al.  Introductory Combinatorics , 1977 .

[9]  Zhi-huan Song,et al.  Process Monitoring Based on Independent Component Analysis - Principal Component Analysis ( ICA - PCA ) and Similarity Factors , 2007 .

[10]  F. Jelinek,et al.  Continuous speech recognition by statistical methods , 1976, Proceedings of the IEEE.

[11]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[12]  István Miklós,et al.  A linear memory algorithm for Baum-Welch training , 2005, BMC Bioinformatics.

[13]  L. Baum,et al.  Growth transformations for functions on manifolds. , 1968 .

[14]  Susan M. Bridges,et al.  Incremental learning of discrete hidden markov models , 2005 .

[15]  Stephen Winters-Hilt,et al.  Implementing EM and Viterbi algorithms for Hidden Markov Model in linear memory , 2007, BMC Bioinformatics.

[16]  R. Chang,et al.  On receiver structures for channels having memory , 1966, IEEE Trans. Inf. Theory.

[17]  Richard Hughey,et al.  Reduced space hidden Markov model training , 1998, Bioinform..

[18]  Dong Yang,et al.  PM2.5 concentration prediction using hidden semi-Markov model-based times series data mining , 2009, Expert Syst. Appl..

[19]  Barak A. Pearlmutter,et al.  Detecting intrusions using system calls: alternative data models , 1999, Proceedings of the 1999 IEEE Symposium on Security and Privacy (Cat. No.99CB36344).

[20]  ChangKyoo Yoo,et al.  Statistical process monitoring with independent component analysis , 2004 .

[21]  Robert Sabourin,et al.  On the memory complexity of the forward-backward algorithm , 2010, Pattern Recognit. Lett..

[22]  Irmtraud M. Meyer,et al.  Gene structure conservation aids similarity based gene prediction. , 2004, Nucleic acids research.

[23]  L. Baum,et al.  An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology , 1967 .

[24]  L. Baum,et al.  An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process , 1972 .

[25]  David He,et al.  A segmental hidden semi-Markov model (HSMM)-based diagnostics and prognostics framework and methodology , 2007 .

[26]  David M. Lane,et al.  Detecting faults in heterogeneous and dynamic systems using DSP and an agent-based architecture , 2007, Eng. Appl. Artif. Intell..

[27]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[28]  Steve Austin,et al.  The forward-backward search algorithm , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[29]  Viliam Makis,et al.  Optimal Bayesian fault prediction scheme for a partially observable system subject to random failure , 2011, Eur. J. Oper. Res..

[30]  Richard D. Braatz,et al.  Fault Detection and Diagnosis in Industrial Systems , 2001 .

[31]  Josef Raviv,et al.  Decision making in Markov chains applied to the problem of pattern recognition , 1967, IEEE Trans. Inf. Theory.

[32]  Wei Wang,et al.  Modeling program behaviors by hidden Markov models for intrusion detection , 2004, Proceedings of 2004 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.04EX826).

[33]  D. Haussler,et al.  Hidden Markov models in computational biology. Applications to protein modeling. , 1993, Journal of molecular biology.

[34]  Ben M. Herbst,et al.  Estimating the pen trajectories of static signatures using hidden Markov models , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Richard Hughey,et al.  Reduced space sequence alignment , 1997, Comput. Appl. Biosci..

[36]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[37]  Hugo Hammer,et al.  Approximate forward-backward algorithm for a switching linear Gaussian model , 2011, Comput. Stat. Data Anal..

[38]  Shunzheng Yu,et al.  Practical implementation of an efficient forward-backward algorithm for an explicit-duration hidden Markov model , 2006, IEEE Transactions on Signal Processing.

[39]  E. F. Vogel,et al.  A plant-wide industrial process control problem , 1993 .