A novel training method for HMM2 with multiple observation sequences

Second-order hidden Markov models (HMM2) have been widely used in pattern recognition, especially in speech recognition. Their main advantages are their capabilities to model noisy temporal signals of variable length. In this paper, we introduce a novel training method for HMM2 with multiple observable sequences, assuming that all the observable sequences are driven by a common hidden sequence. By generalizing Baum's auxiliary function into this framework and building up an associated objective function using Lagrange multiplier method, several new formulae solving model parametric estimation are theoretically derived.

[1]  L. Baum,et al.  Statistical Inference for Probabilistic Functions of Finite State Markov Chains , 1966 .

[2]  L. Baum,et al.  An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology , 1967 .

[3]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[4]  L. R. Rabiner,et al.  An introduction to the application of the theory of probabilistic functions of a Markov process to automatic speech recognition , 1983, The Bell System Technical Journal.

[5]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[6]  Yang He,et al.  2-D Shape Classification Using Hidden Markov Model , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[8]  Satoru Hayamizu,et al.  Prediction of protein secondary structure by the hidden Markov model , 1993, Comput. Appl. Biosci..

[9]  Jian Zhou,et al.  Off-Line Handwritten Word Recognition Using a Hidden Markov Model Type Stochastic Network , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Abdelaziz Kriouile,et al.  Automatic word recognition based on second-order hidden Markov models , 1994, IEEE Trans. Speech Audio Process..

[11]  Les E. Atlas,et al.  The challenge of spoken language systems: research directions for the nineties , 1995, IEEE Trans. Speech Audio Process..

[12]  Jean-François Mari,et al.  A second-order HMM for high performance word and phoneme-based continuous speech recognition , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[13]  Bing Feng,et al.  Off-line handwritten Chinese character recognition with hidden Markov models , 2000, WCC 2000 - ICSP 2000. 2000 5th International Conference on Signal Processing Proceedings. 16th World Computer Congress 2000.

[14]  Marc Parizeau,et al.  Training Hidden Markov Models with Multiple Observations-A Combinatorial Method , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Minh N. Do,et al.  Rotation invariant texture characterization and retrieval using steerable wavelet-domain hidden Markov models , 2002, IEEE Trans. Multim..

[16]  T. Speed,et al.  Biological Sequence Analysis , 1998 .

[17]  Xiang-Gen Xia,et al.  Wavelet-Based Texture Analysis and Synthesis Using Hidden Markov Models , 2003 .

[18]  Richard Washington,et al.  Learning to Automatically Detect Features for Mobile Robots Using Second-Order Hidden Markov Models , 2003, IJCAI 2003.

[19]  Markus Wistrand,et al.  Improving profile HMM discrimination by adapting transition probabilities. , 2004, Journal of molecular biology.

[20]  Jan Bulla,et al.  Stylized facts of financial time series and hidden semi-Markov models , 2006, Comput. Stat. Data Anal..

[21]  Zhibiao Zhao.  Parametric and nonparametric models and methods in financial econometrics , 2008, 0801.1599.

[22]  Jorng-Tzong Horng,et al.  Detecting LTR structures in human genomic sequences using profile hidden Markov models , 2009, Expert Syst. Appl..