Learning Bayesian Network Structure Using a MultiExpert Approach

The learning of a Bayesian network structure, especially in the case of wide domains, can be a complex, time-consuming and imprecise process. Therefore, the interest of the scientific community in learning Bayesian network structure from data is increasing: many techniques or disciplines such as data mining, text categorization, and ontology building, can take advantage from this process. In the literature, there are many structural learning algorithms but none of them provides good results for each dataset. This paper introduces a method for structural learning of Bayesian networks based on a MultiExpert approach. The proposed method combines five structural learning algorithms according to a majority vote combining rule for maximizing their effectiveness and, more generally, the results obtained by using of a single algorithm. This paper shows an experimental validation of the proposed algorithm on standard datasets.

[1]  Bo Thiesson,et al.  Diagnostic systems by model selection: a case study , 1994 .

[2]  Luigi Landini,et al.  Mutual Information Preconditioning Improves Structure Learning of Bayesian Networks From Medical Databases , 2009, IEEE Transactions on Information Technology in Biomedicine.

[3]  Jie Cheng,et al.  An Algorithm for Bayesian Belief Network Construction from Data , 2004 .

[4]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[5]  Kwong-Sak Leung,et al.  An efficient data mining method for learning Bayesian networks using an evolutionary algorithm-based hybrid approach , 2004, IEEE Transactions on Evolutionary Computation.

[6]  Thomas Bäck,et al.  Evolutionary computation: Toward a new philosophy of machine intelligence , 1997, Complex..

[7]  Edward H. Herskovits,et al.  Computer-based probabilistic-network construction , 1992 .

[8]  David Maxwell Chickering,et al.  Learning Bayesian Networks is , 1994 .

[9]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1999, Innovations in Bayesian Networks.

[10]  Walter Zucchini,et al.  Model Selection , 2011, International Encyclopedia of Statistical Science.

[11]  Remco R. Bouckaert,et al.  Probalistic Network Construction Using the Minimum Description Length Principle , 1993, ECSQARU.

[12]  William H. Hsu,et al.  A Survey of Algorithms for Real-Time Bayesian Network Inference , 2002 .

[13]  Wray L. Buntine.  A Guide to the Literature on Learning Probabilistic Networks from Data , 1996, IEEE Trans. Knowl. Data Eng..

[14]  Weiru Liu,et al.  Learning belief networks from data: an information theory based approach , 1997, CIKM '97.

[15]  Ching Y. Suen,et al.  Computer recognition of unconstrained handwritten numerals , 1992, Proc. IEEE.

[16]  David A. Bell,et al.  Learning Bayesian networks from data: An information-theory based approach , 2002, Artif. Intell..

[17]  Francesco Colace,et al.  An adaptive product configurator based on slow intelligence approach , 2014, Int. J. Metadata Semant. Ontologies.

[18]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[19]  Nir Friedman,et al.  The Bayesian Structural EM Algorithm , 1998, UAI.

[20]  Andrew R. Barron,et al.  Minimum complexity density estimation , 1991, IEEE Trans. Inf. Theory.

[21]  Marek J. Druzdzel,et al.  A Hybrid Anytime Algorithm for the Construction of Causal Models From Sparse Data , 1999, UAI.

[22]  Boaz Lerner,et al.  Bayesian Network Structure Learning by Recursive Autonomy Identification , 2009, J. Mach. Learn. Res..

[23]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .

[24]  Boaz Lerner,et al.  Bayesian Network Structure Learning by Recursive Autonomy Identification , 2006, SSPR/SPR.

[25]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[26]  Xin Yao,et al.  A novel evolutionary data mining algorithm with applications to churn prediction , 2003, IEEE Trans. Evol. Comput..

[27]  Richard Scheines,et al.  Discovering Causal Structure: Artificial Intelligence, Philosophy of Science, and Statistical Modeling , 1987 .

[28]  Yao Sun,et al.  Processing Continuous Queries on Sensor-Based Multimedia Data Streams by Multimedia Dependency Analysis and Ontological Filtering , 2011, Int. J. Softw. Eng. Knowl. Eng..

[29]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[30]  Gregory F. Cooper,et al.  An Entropy-driven System for Construction of Probabilistic Expert Systems from Databases , 1990, UAI.

[31]  P. Spirtes,et al.  Causation, prediction, and search , 1993 .

[32]  Judea Pearl,et al.  An Algorithm for Deciding if a Set of Observed Independencies Has a Causal Explanation , 1992, UAI.

[33]  Weimin Xiao,et al.  Evolving accurate and compact classification rules with gene expression programming , 2003, IEEE Trans. Evol. Comput..

[34]  Mario Vento,et al.  Bayesian Network Structural Learning from Data: An Algorithms Comparison , 2004, ICEIS.

[35]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[36]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  D. Heckerman,et al.  A Bayesian Approach to Causal Discovery , 2006 .

[38]  Francesco Colace,et al.  Ontology for E-Learning: A Bayesian Approach , 2010, IEEE Transactions on Education.

[39]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[40]  C. S. Wallace,et al.  Estimation and Inference by Compact Coding , 1987 .

[41]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[42]  Judea Pearl,et al.  A Theory of Inferred Causation , 1991, KR.

[43]  Dan Geiger,et al.  An Entropy-based Learning Algorithm of Bayesian Conditional Trees , 1992, UAI.

[44]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[45]  S. Kullback,et al.  Information Theory and Statistics , 1959 .

[46]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[48]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[49]  P. Spirtes,et al.  An Algorithm for Fast Recovery of Sparse Causal Graphs , 1991 .

[50]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[51]  Stuart L. Crawford,et al.  Constructor: A System for the Induction of Probabilistic Models , 1990, AAAI.

[52]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[53]  David Heckerman,et al.  Bayesian Networks for Data Mining , 2004, Data Mining and Knowledge Discovery.

[54]  Andrés Cano,et al.  A Method for Integrating Expert Knowledge When Learning Bayesian Networks From Data , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[55]  Boaz Lerner,et al.  INVESTIGATION OF THE K2 ALGORITHM IN LEARNING BAYESIAN NETWORK CLASSIFIERS , 2011, Appl. Artif. Intell..

[56]  Angelo Chianese,et al.  A Novel Approach for Semantic Interoperability in the Web Based on the Semantic Triangle Communication Model , 2011, Int. J. Softw. Eng. Knowl. Eng..

[57]  Jie Cheng,et al.  Learning Bayesian Networks from Data: An Efficient Approach Based on Information Theory , 1999 .

[58]  D. Edwards,et al.  A fast model selection procedure for large families of models , 1987 .

[59]  Russell Greiner,et al.  Learning Bayesian Belief Network Classifiers: Algorithms and System , 2001, Canadian Conference on AI.

[60]  Joe Suzuki,et al.  Learning Bayesian Belief Networks Based on the MDL Principle : An Efficient Algorithm Using the Branch and Bound Technique , 1999 .

[61]  Xue-wen Chen,et al.  Improving Bayesian Network Structure Learning with Mutual Information-Based Node Ordering in the K2 Algorithm , 2008, IEEE Transactions on Knowledge and Data Engineering.

[62]  Moninder Singh,et al.  Construction of Bayesian network structures from data: A brief survey and an efficient algorithm , 1995, Int. J. Approx. Reason..

[63]  Kwong-Sak Leung,et al.  Using Evolutionary Programming and Minimum Description Length Principle for Data Mining of Bayesian Networks , 1999, IEEE Trans. Pattern Anal. Mach. Intell..