A New Formulation of Coupled Hidden Markov Models

Among many variations of more complex hidden Markov models, coupled hidden Markov models (CHMM) have recently attracted increased interests in many practical applications. This paper describes a new CHMM formulation in which the joint transition probability is modeled as a linear combination of the marginal transition probabilities and the weights used to capture the interactions among multiple HMMs. The new formulation greatly reduces the parameter space for CHMM. New approximated forward procedure and training algorithm are proposed to reduce the computational complexity to a practical level. Experimental results show that our new CHMM formulation perform better in our recognition task on both artifical and real data compared to non-coupled HMMs. And our approximated training algorithm still converges to local maxima even though there is no theorectical proof thus provides an efficient practical algorithm for our CHMM formulation.

[1]  Pierre Baldi,et al.  Smooth On-Line Learning Algorithms for Hidden Markov Models , 1994, Neural Computation.

[2]  Steve Young,et al.  COMPETITIVE TRAINING - A CONNECTIONIST APPROACH TO THE DISCRIMINATIVE TRAINING OF HIDDEN MARKOV-MODELS , 1991 .

[3]  Fang Sun,et al.  Speech recognition based on genetic algorithm for training HMM , 1998 .

[4]  Marc Parizeau,et al.  Training Hidden Markov Models with Multiple Observations-A Combinatorial Method , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Matthew Brand,et al.  Coupled hidden Markov models for modeling interacting processes , 1997 .

[6]  Yoshua Bengio,et al.  Input-output HMMs for sequence processing , 1996, IEEE Trans. Neural Networks.

[7]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[8]  Brendan J. Frey,et al.  Event-coupled hidden Markov models , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[9]  Alex Pentland,et al.  Coupled hidden Markov models for complex action recognition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  S. Roberts,et al.  Estimation of coupled hidden Markov models with application to biosignal interaction modelling , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).

[11]  B.-H. Juang,et al.  Maximum-likelihood estimation for mixture multivariate stochastic observations of Markov chains , 1985, AT&T Technical Journal.

[12]  L. Baum,et al.  An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process , 1972 .

[13]  Lester Ingber.  Statistical mechanics of neocortical interactions: Applications of canonical momenta indicators to electroencephalography , 1997 .

[14]  L. R. Rabiner,et al.  An introduction to the application of the theory of probabilistic functions of a Markov process to automatic speech recognition , 1983, The Bell System Technical Journal.

[15]  Biing-Hwang Juang,et al.  Maximum likelihood estimation for multivariate mixture observations of markov chains , 1986, IEEE Trans. Inf. Theory.

[16]  Jaimyoung Kwon.  Modeling Freeway Traffic with Coupled HMMs , 2000 .

[17]  L. Baum,et al.  An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology , 1967 .

[18]  Amir Dembo,et al.  A minimum discrimination information approach for hidden Markov modeling , 1987, ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[19]  Sam Kwong,et al.  Optimization of HMM by a genetic algorithm , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[20]  Lalit R. Bahl,et al.  Maximum mutual information estimation of hidden Markov model parameters for speech recognition , 1986, ICASSP '86. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[21]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[22]  L. Baum,et al.  Growth transformations for functions on manifolds. , 1968 .

[23]  Sean R. Eddy,et al.  Profile hidden Markov models , 1998, Bioinform..