Acoustic model topology optimization using evolutionary methods

Currently, most of the acoustic model selection work is done empirically or heuristically or even arbitrarily. In this paper, Genetic Algorithm (GA) based and Particle Swarm Optimization (PSO) based algorithms that consider the number of states and the kernel numbers for the states simultaneously and reject the uniform allocation of Gaussian kernels are proposed to automatically optimize acoustic model topologies, and some relevant issues are also analyzed and resolved. Experiments on TIDigits corpus show that: first, our GA-based and PSO-based algorithms are effective methods to automatically optimize acoustic model topologies; second, due to the use of Bayesian Information Criterion (BIC), both of our algorithms are capable of achieving higher recognition performance with smaller number of parameters. Specifically, both of our systems with model topologies optimized using GA-based and PSO-based algorithms respectively obtain much increase in recognition performances compared with the baseline systems constructed in a conventional way and having same system complexities; moreover, if compared with baseline systems having same recognition performances, both of our optimized systems save approximate half of the parameters.

[1]  Rob A. Rutenbar,et al.  Generating small, accurate acoustic models with a modified Bayesian information criterion , 2007, INTERSPEECH.

[2]  Joseph Picone,et al.  Information theoretic approaches to model selection , 1998, ICSLP.

[3]  N. Thatphithakkul,et al.  HMM parameter optimization using tabu search [speech recognition] , 2004, IEEE International Symposium on Communications and Information Technology, 2004. ISCIT 2004..

[4]  Sam Kwong,et al.  Genetic algorithms: concepts and applications [in engineering design] , 1996, IEEE Trans. Ind. Electron..

[5]  Ananth Sankar Experiments with a Gaussian Merging-Splitting Algorithm for HMM Training for Speech Recognition , 2007 .

[6]  Ching Y. Suen,et al.  On the structure of hidden Markov models , 2004, Pattern Recognit. Lett..

[7]  Kim-Fung Man,et al.  Optimisation of HMM topology and its model parameters by genetic algorithms , 2001, Pattern Recognit..

[8]  Sam Kwong,et al.  Optimization of HMM by a genetic algorithm , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[9]  Fábio Violaro,et al.  Gaussian elimination algorithm for HMM complexity reduction in continuous speech recognition systems , 2005, INTERSPEECH.

[10]  Alain Biem,et al.  A model selection criterion for classification: application to HMM topology optimization , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[12]  Wasserman,et al.  Bayesian Model Selection and Model Averaging. , 2000, Journal of mathematical psychology.

[13]  Zhi-Jie Yan,et al.  Training Discriminative HMM by Optimal Allocation of Gaussian Kernels , 2006 .

[14]  Atiwong Suchato,et al.  A Genetic Algorithm-aided Hidden Markov Model Topology Estimation for Phoneme Recognition of Thai Continuous Speech , 2008, 2008 Ninth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing.

[15]  Yves Normandin Optimal splitting of HMM Gaussian mixture components with MMIE training , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[16]  Adam Prügel-Bennett,et al.  Evolving the structure of hidden Markov models , 2006, IEEE Transactions on Evolutionary Computation.

[17]  Frank K. Soong,et al.  Non-uniform Kernel Allocation Based Parsimonious HMM , 2006, ISCSLP.

[18]  Steve Young,et al.  The HTK book version 3.4 , 2006 .

[19]  Grünwald,et al.  Model Selection Based on Minimum Description Length. , 2000, Journal of mathematical psychology.

[20]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[21]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[22]  René Thomsen Evolving the Topology of Hidden Markov Models Using Evolutionary Algorithms , 2002, PPSN.

[23]  Sam Kwong,et al.  Genetic algorithms: concepts and applications [in engineering design] , 1996, IEEE Trans. Ind. Electron..

[24]  Gang Liu,et al.  A Research on Mixture Splitting for CHMM Based on DBC , 2009, J. Comput..

[25]  Alain Biem,et al.  A Bayesian model selection criterion for HMM topology optimization , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.