Hidden Markov Models Training by a Particle Swarm Optimization Algorithm

In this work we consider the problem of Hidden Markov Models (HMM) training. This problem can be considered as a global optimization problem and we focus our study on the Particle Swarm Optimization (PSO) algorithm. To take advantage of the search strategy adopted by PSO, we need to modify the HMM's search space. Moreover, we introduce a local search technique from the field of HMMs and that is known as the Baum–Welch algorithm. A parameter study is then presented to evaluate the importance of several parameters of PSO on artificial data and natural data extracted from images.

[1]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[2]  Yves Lecourtier,et al.  Handwritten word recognition by image segmentation and hidden Markov models , 1993, Proceedings of IECON '93 - 19th Annual Conference of IEEE Industrial Electronics.

[3]  Mohamed Slimane,et al.  Optimizing Hidden Markov Models with a Genetic Algorithm , 1995, Artificial Evolution.

[4]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[5]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[6]  A. Kundu,et al.  Recognition of handwritten script: a hidden Markov model based approach , 1988, ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.

[7]  Lalit R. Bahl,et al.  Design of a linguistic statistical decoder for the recognition of continuous speech , 1975, IEEE Trans. Inf. Theory.

[8]  Markus Falkhausen,et al.  Calculation of distance measures between hidden Markov models , 1995, EUROSPEECH.

[9]  Mehmet Fatih Tasgetiren,et al.  Particle Swarm Optimization Algorithm for Permutation Flowshop Sequencing Problem , 2004, ANTS Workshop.

[10]  Jeng-Shyang Pan,et al.  Optimization of HMM by the Tabu Search Algorithm , 2004, J. Inf. Sci. Eng..

[11]  Mehmet Fatih Tasgetiren,et al.  Particle swarm optimization algorithm for single machine total weighted tardiness problem , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[12]  Thomas Kiel Rasmussen,et al.  Improved Hidden Markov Model training for multiple sequence alignment by a particle swarm optimization-evolutionary algorithm hybrid. , 2003, Bio Systems.

[13]  Andries Petrus Engelbrecht,et al.  Data clustering using particle swarm optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[14]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[15]  M. Do.  Fast approximation of Kullback-Leibler distance for dependence trees and hidden Markov models , 2003, IEEE Signal Processing Letters.

[16]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .

[17]  René Thomsen.  Evolving the Topology of Hidden Markov Models Using Evolutionary Algorithms , 2002, PPSN.

[18]  Nicolas Monmarché,et al.  Algorithmes de fourmis artificielles : applications à la classification et à l'optimisation. (Artificial ant based algorithms applied to clustering and optimization problems) , 2000 .

[19]  Jeff A. Bilmes,et al.  WHAT HMMS CAN'T DO , 2004 .

[20]  Gerhard Rigoll,et al.  Advanced state clustering for very large vocabulary HMM-based on-line handwriting recognition , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[21]  Yoram Singer,et al.  The Hierarchical Hidden Markov Model: Analysis and Applications , 1998, Machine Learning.

[22]  Nicolas Monmarché,et al.  On how Pachycondyla apicalis ants suggest a new search algorithm , 2000, Future Gener. Comput. Syst..

[23]  Jr. G. Forney,et al.  The viterbi algorithm , 1973 .

[24]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[25]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[26]  M. A. Abido,et al.  Optimal power flow using particle swarm optimization , 2002 .

[27]  Imtiaz Ahmad,et al.  Particle swarm optimization for task assignment problem , 2002, Microprocess. Microsystems.

[28]  Jacek M. Zurada,et al.  An approach to multimodal biomedical image registration utilizing particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[29]  Andries P. Engelbrecht,et al.  Training support vector machines with particle swarms , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[30]  Mohamed Slimane,et al.  Hybrid Genetic Learning of Hidden Markov Models for Time Series Prediction , 1998 .

[31]  Russell C. Eberhart,et al.  Swarm intelligence for permutation optimization: a case study of n-queens problem , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[32]  Jeff A. Bilmes,et al.  What HMMs Can Do , 2006, IEICE Trans. Inf. Syst..

[33]  Olivier Cappé,et al.  Ten years of HMMs , 2001 .

[34]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[35]  Andries P. Engelbrecht,et al.  Image Classification using Particle Swarm Optimization , 2002, SEAL.

[36]  Matti Vihola,et al.  Two dissimilarity measures for HMMS and their application in phoneme model clustering , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.