Training HMM structure with genetic algorithm for biological sequence analysis

SUMMARY Hidden Markov models (HMMs) are widely used for biological sequence analysis because of their ability to incorporate biological information in their structure. An automatic means of optimizing the structure of HMMs would be highly desirable. However, this raises two important issues; first, the new HMMs should be biologically interpretable, and second, we need to control the complexity of the HMM so that it has good generalization performance on unseen sequences. In this paper, we explore the possibility of using a genetic algorithm (GA) for optimizing the HMM structure. GAs are sufficiently flexible to allow incorporation of other techniques such as Baum-Welch training within their evolutionary cycle. Furthermore, operators that alter the structure of HMMs can be designed to favour interpretable and simple structures. In this paper, a training strategy using GAs is proposed, and it is tested on finding HMM structures for the promoter and coding region of the bacterium Campylobacter jejuni. The proposed GA for hidden Markov models (GA-HMM) allows, HMMs with different numbers of states to evolve. To prevent over-fitting, a separate dataset is used for comparing the performance of the HMMs to that used for the Baum-Welch training. The GA-HMM was capable of finding an HMM comparable to a hand-coded HMM designed for the same task, which has been published previously.

[1]  Anders Krogh,et al.  RpoD promoters in Campylobacter jejuni exhibit a strong periodic signal instead of a -35 box. , 2003, Journal of molecular biology.

[2]  Andreas Stolcke,et al.  Bayesian learning of probabilistic language models , 1994 .

[3]  A. Konagaya,et al.  Motif Extraction using an Improved Iterative Duplication Method for HMM Topology Learning , 1995 .

[4]  D. E. Goldberg,et al.  Genetic Algorithms in Search, Optimization & Machine Learning , 1989 .

[5]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[6]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[7]  矢田 哲士.  Stochastic models representing DNA sequence data : construction algorithms and their applications to prediction of gene structure and function , 1998 .

[8]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[9]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[10]  Satoru Hayamizu,et al.  Prediction of protein secondary structure by the hidden Markov model , 1993, Comput. Appl. Biosci..

[11]  R. Durbin,et al.  Biological sequence analysis: Background on probability , 1998 .

[12]  J Baker,et al.  REDUCING BIAS AND NEFFICIENCY IN THE SELECTION ALGORITHM, GENETIC ALGORITHMS AND APPLICATIONS , 2000 .

[13]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[14]  Masato Ishikawa,et al.  DNA Sequence Analysis using Hidden Markov Model and Genetic Algorithm , 1994 .

[15]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.