Hybrid Genetic Learning of Hidden Markov Models for Time Series Prediction

This paper presents how a hybrid genetic/gradient search can be used to learn hidden Markov models in the context of time series prediction. This learning algorithm called GHOSP uses a gradient search, namely the Baum Welch algorithm, as a local search operator in the main loop of a genetic algorithm, in conjunction with standard genetic operators adapted for hidden Markov models. GHOSP is able to learn efficiently the coefficients and the architecture of hidden Markov models in order to maximize the probability of generating an observation O. This observation is used to encode the recent past of a time series. Once an efficient stochastic model of the series has been learned, this model can be used to predict the next values of the series. We apply this framework to several standard series including economical ones.

[1]  Anne Lohrli Chapman and Hall , 1985 .

[2]  Samir W. Mahfoud A Comparison of Parallel and Sequential Niching Methods , 1995, ICGA.

[3]  John R. Koza,et al.  Hierarchical Genetic Algorithms Operating on Populations of Computer Programs , 1989, IJCAI.

[4]  Mohamed Slimane,et al.  Optimizing Hidden Markov Models with a Genetic Algorithm , 1995, Artificial Evolution.

[5]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[6]  Andrew J. Viterbi,et al.  Error bounds for convolutional codes and an asymptotically optimum decoding algorithm , 1967, IEEE Trans. Inf. Theory.

[7]  Oscar E. Agazzi,et al.  Keyword Spotting in Poorly Printed Documents using Pseudo 2-D Hidden Markov Models , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  L. Baum,et al.  An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process , 1972 .

[9]  Jan Torreele,et al.  Temporal Processing with Recurrent Networks: An Evolutionary Approach , 1991, ICGA.

[10]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[11]  L. Baum,et al.  An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology , 1967 .

[12]  L. R. Rabiner,et al.  An introduction to the application of the theory of probabilistic functions of a Markov process to automatic speech recognition , 1983, The Bell System Technical Journal.

[13]  E. Howard,et al.  The Application of Genetic Programming to the Investigation of Short, Noisy, Chaotic Data Series , 1994, Evolutionary Computing, AISB Workshop.

[14]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[15]  Chris Chatfield,et al.  The Analysis of Time Series: An Introduction , 1981 .

[16]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[17]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.