Constraint Handling Using Tournament Selection: Abductive Inference in Partly Deterministic Bayesian Networks

Constraints occur in many application areas of interest to evolutionary computation. The area considered here is Bayesian networks (BNs), which is a probability-based method for representing and reasoning with uncertain knowledge. This work deals with constraints in BNs and investigates how tournament selection can be adapted to better process such constraints in the context of abductive inference. Abductive inference in BNs consists of finding the most probable explanation given some evidence. Since exact abductive inference is NP-hard, several approximate approaches to this inference task have been developed. One of them applies evolutionary techniques in order to find optimal or close-to-optimal explanations. A problem with the traditional evolutionary approach is this: As the number of constraints determined by the zeros in the conditional probability tables grows, performance deteriorates because the number of explanations whose probability is greater than zero decreases. To minimize this problem, this paper presents and analyzes a new evolutionary approach to abductive inference in BNs. By considering abductive inference as a constraint optimization problem, the novel approach improves performance dramatically when a BN's conditional probability tables contain a significant number of zeros. Experimental results are presented comparing the performances of the traditional evolutionary approach and the approach introduced in this work. The results show that the new approach significantly outperforms the traditional one.

[1]  William H. Hsu,et al.  An ACO Algorithm for the Most Probable Explanation Problem , 2004, Australian Conference on Artificial Intelligence.

[2]  Zbigniew Michalewicz,et al.  Evolutionary Computation 2 : Advanced Algorithms and Operators , 2000 .

[3]  José A. Gámez,et al.  Abductive Inference in Bayesian Networks: A Review , 2004 .

[4]  Solomon Eyal Shimony,et al.  Finding MAPs for Belief Networks is NP-Hard , 1994, Artif. Intell..

[5]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[6]  D E Heckerman,et al.  Toward Normative Expert Systems: Part II Probability-Based Representations for Efficient Knowledge Acquisition and Inference , 1992, Methods of Information in Medicine.

[7]  D. Heckerman,et al.  Toward Normative Expert Systems: Part I The Pathfinder Project , 1992, Methods of Information in Medicine.

[8]  David E. Goldberg,et al.  Probabilistic Crowding: Deterministic Crowding with Probabilistic Replacement , 1999 .

[9]  José A. Gámez,et al.  Partial abductive inference in Bayesian belief networks - an evolutionary computation approach by using problem-specific genetic operators , 2002, IEEE Trans. Evol. Comput..

[10]  Edwin Lughofer,et al.  Associating visual textures with human perceptions using genetic algorithms , 2010, Inf. Sci..

[11]  Luis Enrique Sucar,et al.  COMPARISON OF TWO TYPES OF EVENT BAYESIAN NETWORKS: A CASE STUDY , 2007, Appl. Artif. Intell..

[12]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[13]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[14]  Eric Horvitz,et al.  Collaborative Filtering by Personality Diagnosis: A Hybrid Memory and Model-Based Approach , 2000, UAI.

[15]  David C. Wilkins,et al.  Initialization and Restart in Stochastic Local Search: Computing a Most Probable Explanation in Bayesian Networks , 2011, IEEE Transactions on Knowledge and Data Engineering.

[16]  Jean-Louis Golmard,et al.  An algorithm directly finding the K most probable configurations in Bayesian networks , 1994, Int. J. Approx. Reason..

[17]  A. Eiben Evolutionary algorithms and constraint satisfaction: definitions, survey, methodology, and research directions , 2001 .

[18]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[19]  大西 仁,et al.  Pearl, J. (1988, second printing 1991). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan-Kaufmann. , 1994 .

[20]  Eugene Santos,et al.  Directing Genetic Algorithms for Probabilistic Reasoning Through Reinforcement Learning , 2000, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[21]  José L. Verdegay,et al.  Evolutionary Techniques for Constrained Optimization Problems , 1999 .

[22]  David B. Fogel,et al.  Evolution-ary Computation 1: Basic Algorithms and Operators , 2000 .

[23]  Carlos Artemio Coello-Coello,et al.  Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art , 2002 .

[24]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[25]  David C. Wilkins,et al.  Genetic Algorithms for Belief Network Inference: The Role of Scaling and Niching , 1998, Evolutionary Programming.

[26]  Bart Selman,et al.  Domain-Independent Extensions to GSAT : Solving Large StructuredSatis ability , 1993 .

[27]  Mark A. Kramer,et al.  An Evolutionary Computing Approach to Probabilistic Reasoning on Bayesian Networks , 1996, Evolutionary Computation.

[28]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[29]  R. K. Ursem Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[30]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[31]  Christopher M. Brown,et al.  Control of selective perception using bayes nets and decision theory , 1994, International Journal of Computer Vision.

[32]  Thomas Stützle,et al.  Stochastic Local Search: Foundations & Applications , 2004 .

[33]  Zbigniew Michalewicz,et al.  Evolutionary Algorithms for Constrained Parameter Optimization Problems , 1996, Evolutionary Computation.

[34]  José Manuel Gutiérrez,et al.  Expert Systems and Probabiistic Network Models , 1996 .

[35]  K. Deb An Efficient Constraint Handling Method for Genetic Algorithms , 2000 .

[36]  Robert L. Welch,et al.  Real Time Estimation of Bayesian Networks , 1996, UAI.

[37]  Edzard S. Gelsema,et al.  Abductive reasoning in Bayesian belief networks using a genetic algorithm , 1995, Pattern Recognit. Lett..

[38]  David C. Wilkins,et al.  Portfolios in Stochastic Local Search: Efficiently Computing Most Probable Explanations in Bayesian Networks , 2011, Journal of Automated Reasoning.

[39]  A. P. Dawid,et al.  Applications of a general propagation algorithm for probabilistic expert systems , 1992 .

[40]  Eric Horvitz,et al.  The Lumière Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users , 1998, UAI.

[41]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[42]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[43]  Carlos A. Coello Coello,et al.  Constraint-handling in genetic algorithms through the use of dominance-based tournament selection , 2002, Adv. Eng. Informatics.

[44]  Robert M. Fung,et al.  Applying Bayesian networks to information retrieval , 1995, CACM.

[45]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[46]  Alex A. Freitas,et al.  Evolutionary Computation , 2002 .

[47]  D. Nilsson,et al.  An efficient algorithm for finding the M most probable configurationsin probabilistic expert systems , 1998, Stat. Comput..

[48]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[49]  Jian Cheng,et al.  AIS-BN: An Adaptive Importance Sampling Algorithm for Evidential Reasoning in Large Bayesian Networks , 2000, J. Artif. Intell. Res..

[50]  David C. Wilkins,et al.  Controlled generation of hard and easy Bayesian networks: Impact on maximal clique size in tree clustering , 2006, Artif. Intell..

[51]  Mark A. Kramer,et al.  GALGO: A Genetic ALGOrithm Decision Support Tool for Complex Uncertain Systems Modeled with Bayesian Belief Networks , 1993, UAI.

[53]  Dirk Thierens,et al.  Convergence Models of Genetic Algorithm Selection Schemes , 1994, PPSN.

[54]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[55]  Marek J. Druzdzel,et al.  Some Properties of joint Probability Distributions , 1994, UAI.

[56]  Ashraf M. Abdelbar,et al.  Approximating MAPs for Belief Networks is NP-Hard and Other Theorems , 1998, Artif. Intell..

[57]  Michael P. Wellman,et al.  Real-world applications of Bayesian networks , 1995, CACM.