Evolutionary Learning of Dynamic Naive Bayesian Classifiers

Many problems such as voice recognition, speech recognition and many other tasks have been tackled with Hidden Markov Models (HMMs). These problems can also be dealt with an extension of the Naive Bayesian Classifier (NBC) known as Dynamic NBC (DNBC). From a dynamic Bayesian network (DBN) perspective, in a DNBC at each time there is a NBC. NBCs work well in data sets with independent attributes. However, they perform poorly when the attributes are dependent or when there are one or more irrelevant attributes which are dependent of some relevant ones. Therefore, to increase this classifier accuracy, we need a method to design network structures that can capture the dependencies and get rid of irrelevant attributes. Furthermore, when we deal with dynamical processes there are temporal relations that should be considered in the network design. In order to learn automatically these models from data and increase the classifier accuracy we propose an evolutionary optimization algorithm to solve this design problem. We introduce a new encoding scheme and new genetic operators which are natural extensions of previously proposed encoding and operators for grouping problems. The design methodology is applied to solve the recognition problem for nine hand gestures. Experimental results show that the evolved network has higher average classification accuracy than the basic DNBC and a HMM.

[1]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[2]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Duncan Fyfe Gillies,et al.  Probabilistic reasoning in high-level vision , 1994, Image Vis. Comput..

[4]  Brian J. Ross,et al.  Evolving dynamic Bayesian networks with Multi-objective genetic algorithms , 2007, Applied Intelligence.

[5]  H.H. Aviles-Arriaga,et al.  Visual recognition of gestures using dynamic naive Bayesian classifiers , 2003, The 12th IEEE International Workshop on Robot and Human Interactive Communication, 2003. Proceedings. ROMAN 2003..

[6]  Nir Friedman,et al.  The Bayesian Structural EM Algorithm , 1998, UAI.

[7]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[8]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[9]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[10]  Luis Enrique Sucar,et al.  Visual Recognition of Similar Gestures , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[11]  Luis Enrique Sucar,et al.  Learning Dynamic Naive Bayesian Classifiers , 2008, FLAIRS Conference.

[12]  P. Cameron.  Combinatorics: Topics, Techniques, Algorithms , 1995 .

[13]  Michael J. Pazzani,et al.  Searching for Dependencies in Bayesian Classifiers , 1995, AISTATS.

[14]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[15]  Kathryn B. Laskey,et al.  Learning Bayesian networks from incomplete data using evolutionary algorithms , 1999 .

[16]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms , 1994 .

[17]  Luis Enrique Sucar,et al.  Learning an Optimal Naive Bayes Classifier , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[18]  Kevin P. Murphy,et al.  Learning the Structure of Dynamic Probabilistic Networks , 1998, UAI.

[19]  Emanuel Falkenauer,et al.  A New Representation and Operators for Genetic Algorithms Applied to Grouping Problems , 1994, Evolutionary Computation.

[20]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[21]  Kwong-Sak Leung,et al.  A hybrid approach to learn Bayesian networks using evolutionary programming , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[22]  Eric R. Ziegel,et al.  Probability and Statistics for Engineering and the Sciences , 2004, Technometrics.

[23]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[24]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[25]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.