VOGA: Variable Ordering Genetic Algorithm for Learning Bayesian Classifiers

This work proposes a hybrid approach to help the process of learning a Bayesian Classifier (BC) from data. The proposed method named VOGA (and its variant VOGA+) uses a Genetic Algorithm to optimize the BC learning process by means of the identification of an adequate variables ordering. The main contribution of VOGA and VOGA+ is the use information about the class variable when defining the most suitable variable ordering. Trying to optimize the GA initial population, VOGA+ ranks the attributes based on the class variable. Experiments performed in a number of datasets revealed that both methods are promising and VOGA+ tends to be favored domains having higher number of variables.

[1]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[2]  Victor Zarnowitz,et al.  The Accuracy of Individual and Group Forecasts from Business Outlook Surveys , 1982 .

[3]  JingTao Yao,et al.  Forecasting the KLSE index using neural networks , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[4]  Fionn Murtagh,et al.  Combining Neural Network Forecasts on Wavelet-transformed Time Series , 1997, Connect. Sci..

[5]  Bernd Freisleben,et al.  Economic forecasting using neural networks , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[6]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[7]  Hiroaki Kitano,et al.  Empirical Studies on the Speed of Convergence of Neural Network Training Using Genetic Algorithms , 1990, AAAI.

[8]  H. M. Uhlenbein.  Evolutionary Algorithms: Theory and Applications , 1993 .

[9]  Geoffrey E. Hinton,et al.  A time-delay neural network architecture for isolated word recognition , 1990, Neural Networks.

[10]  Peter C. McCluskey,et al.  Feedforward and Recurrent Neural Networks and Genetic Programs for Stock and Time Series Forecasting , 1993 .

[11]  Ferdinand Hergert,et al.  Improving model selection by nonconvergent methods , 1993, Neural Networks.

[12]  Alireza Khotanzad,et al.  An adaptive and modular recurrent neural network based power system load forecaster , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[13]  Paulo Cortez,et al.  An evolutionary artificial neural network time series forecasting system , 1996 .

[14]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[15]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[16]  Estevam R. Hruschka,et al.  Variable Ordering for Bayesian Networks Learning from Data , 2003 .

[17]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[18]  Chris Beaumont.  Forecasting: Methods and Applications , 1984 .

[19]  Marco Sandri,et al.  Combining Singular-Spectrum Analysis and neural networks for time series forecasting , 2005, Neural Processing Letters.

[20]  E. Michael Azoff,et al.  Neural Network Time Series: Forecasting of Financial Markets , 1994 .

[21]  J. David Schaffer,et al.  Representation and Hidden Bias: Gray vs. Binary Coding for Genetic Algorithms , 1988, ML.

[22]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[23]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[24]  Woon-Seng Gan,et al.  Multivariate FOREX forecasting using artificial neural networks , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[25]  Paulo Cortez,et al.  A Neural Network Based Forecasting System , 1995 .

[26]  Alice M. Agogino,et al.  Automated Construction of Sparse Bayesian Networks from Unstructured Probabilistic Models and Domain Information , 2013, UAI.

[27]  William H. Hsu,et al.  Genetic wrappers for feature selection in decision tree induction and variable ordering in Bayesian network structure learning , 2004, Inf. Sci..

[28]  Hongjun Lu,et al.  NeuroRule: A Connectionist Approach to Data Mining , 1995, VLDB.

[29]  Marvin Minsky,et al.  Logical vs. analogical or symbolic vs. connectionist or neat vs. scruffy , 1991 .

[30]  Karsten M. DeckerCSCS,et al.  Technology Overview: a Report on Data Mining , 1995 .

[31]  Martin A. Riedmiller,et al.  Advanced supervised learning in multi-layer perceptrons — From backpropagation to adaptive learning algorithms , 1994 .

[32]  Lutz Prechelt,et al.  A Set of Neural Network Benchmark Problems and Benchmarking Rules , 1994 .

[33]  Mats Carlsson,et al.  SICStus Prolog User''s Manual , 1993 .

[34]  Stephen I. Gallant,et al.  Neural network learning and expert systems , 1993 .

[35]  Paulo Cortez,et al.  Time series forecasting in a distributed environment , 1996 .

[36]  Schloss Birlinghoven.  Evolution in Time and Space -the Parallel Genetic Algorithm , 1991 .

[37]  Scott E. Fahlman,et al.  An empirical study of learning speed in back-propagation networks , 1988 .

[38]  Robert L. Winkler,et al.  The accuracy of extrapolation (time series) methods: Results of a forecasting competition , 1982 .

[39]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[40]  Pedro Larrañaga,et al.  Learning Bayesian network structures by searching for the best ordering with genetic algorithms , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[41]  Marco Tomassini,et al.  Using Genetic Algorithms For Robust Optimization In Financial Applications , 1998 .

[42]  Julian J. Faraway,et al.  Time Series Forecasting with Neural Networks: A Case Study , 1995 .

[43]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[44]  Tony R. Martinez,et al.  A Survey of Neural Network Research and Fielded Applications , 1992 .

[45]  David Lowe,et al.  Practical methods of tracking of nonstationary time series applied to real-world data , 1996, Defense + Commercial Sensing.

[46]  Paulo Garrido,et al.  Amalgamating the Neural and Logic Computing Paradigms , 1991 .