The Alpha-HMM Estimation Algorithm: Prior Cycle Guides Fast Paths

The estimation of generative structures for sequences is becoming increasingly important for preventing such data sources from becoming a flood of disorganized information. Obtaining hidden Markov models (HMMs) has been a central method for structuring such data. However, users have been aware of the slow speed of this algorithm. In this study, we devise generalized and fast estimation methods for HMMs by employing a geometric information measure that is associated with a function called the alpha-logarithm. Using the alpha-logarithmic likelihood ratio, we exploit prior iterations to guide rapid convergence. The parameter alpha is used to adjust the utilization of previous information. A fixed-point approach using a causal shift and a series expansion is responsible for this gain. For software implementations, we present probability scaling to avoid underflow, where we generalize flaw corrections to the de facto standard. For the update mechanism, we begin with a method called shotgun surrogates, in relation to the parameter alpha. Then, we obtain a dynamic version that employs the controlling and undoing of alpha. Experiments on biological sequences and brain signals for practical state models demonstrate that a significant speedup is achieved compared to the Baum–Welch method. The effects of restricting the state models are also reported.

[1]  S. M. Ali,et al.  A General Class of Coefficients of Divergence of One Distribution from Another , 1966 .

[2]  B.-H. Juang,et al.  Maximum-likelihood estimation for mixture multivariate stochastic observations of Markov chains , 1985, AT&T Technical Journal.

[3]  Jianming Miao Analog of the minkowski inequality for the difference of two M-matrices ∗ , 1993 .

[4]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[5]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[6]  Biing-Hwang Juang,et al.  Hidden Markov Models for Speech Recognition , 1991 .

[7]  Sean R. Eddy,et al.  Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids , 1998 .

[8]  Yasuo Matsuyama,et al.  Hidden Markov model estimation based on alpha-EM algorithm: Discrete and continuous alpha-HMMs , 2011, The 2011 International Joint Conference on Neural Networks.

[9]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[10]  R. Bakis Continuous speech recognition via centisecond acoustic states , 1976 .

[11]  M. Mézard Passing Messages Between Disciplines , 2003, Science.

[12]  S. L. Scott Bayesian Methods for Hidden Markov Models , 2002 .

[13]  Yasuo Matsuyama,et al.  The -EM Algorithm: Surrogate Likelihood Maximization Using -Logarithmic Information Measures , 2001 .

[14]  Sham M. Kakade,et al.  A spectral algorithm for learning Hidden Markov Models , 2008, J. Comput. Syst. Sci..

[15]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[16]  Yasuo Matsuyama,et al.  Brain signal's low-frequency fits the continuous authentication , 2015, Neurocomputing.

[17]  Donald B. Rubin,et al.  Max-imum Likelihood from Incomplete Data , 1972 .

[18]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[19]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[20]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition , 2012 .

[21]  Shinya Yamanaka,et al.  Promotion of direct reprogramming by transformation-deficient Myc , 2010, Proceedings of the National Academy of Sciences.

[22]  Yasuo Matsuyama,et al.  Brain Signal Recognition and Conversion towards Symbiosis with Ambulatory Humanoids , 2010, Brain Informatics.

[23]  A. Rényi On Measures of Entropy and Information , 1961 .

[24]  Yasuo Matsuyama,et al.  HMM-embedded Bayesian network for heterogeneous command integration: applications to biped humanoid operation over the network , 2008, CSTST.

[25]  Robert Sabourin,et al.  A survey of techniques for incremental learning of HMM parameters , 2012, Inf. Sci..

[26]  Louis A. Liporace,et al.  Maximum likelihood estimation for multivariate observations of Markov sources , 1982, IEEE Trans. Inf. Theory.

[27]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[28]  Yoram Singer,et al.  Training Algorithms for Hidden Markov Models using Entropy Based Distance Functions , 1996, NIPS.

[29]  William H. Majoros,et al.  Methods for computational gene prediction , 2007 .