Searching for the best elimination sequence in Bayesian networks by using ant colony optimization

The knowledge base of a probabilistic expert system is usually represented as a Bayesian network. Most of the knowledge engineering tools used in the development of probabilistic expert systems do not carry out the inference process directly over the network, but in a secondary graphical structure called a junction tree. The efficiency of inference (propagation) algorithms depends on the size of the junction tree obtained, and this size depends on the elimination sequence used during the compilation/transformation of the Bayesian network into a junction tree. The problem of searching for the best elimination sequence is an NP-hard problem [W. Wen, in: P. Bonissone, M. Henrion, L. Kanal and Z. Lemmer (Eds.), Uncertainty in Artificial Intelligence, vol. 6, North-Holland, Amsterdam, 1991, pp. 209-224], and this has motivated the proliferation of approximate methods to approach it (based variously on greedy heuristics, genetic algorithms, simulated annealing, etc.). In this paper we investigate the applicability to this problem of a new combinatorial optimization technique, inspired by a natural model, which has appeared recently: ant colony optimization [M. Dorigo, Optimization, learning and natural algorithms, Ph.D. thesis, Politecnico di Milano, Italy, 1992; M. Dorigo and L. Gambardella, IEEE Trans. Evol. Comput. 1 (1997) 53; M. Dorigo and G. Di Caru, in: New Ideas in Optimization, McGraw-Hill, New York, 1999]. Our approach is validated by using a set of complex networks obtained from a repository.

[1]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[2]  Zbigniew Michalewicz,et al.  Genetic algorithms + data structures = evolution programs (2nd, extended ed.) , 1994 .

[3]  Marco Dorigo,et al.  The ant colony optimization meta-heuristic , 1999 .

[4]  Kristian G. Olesen,et al.  HUGIN - A Shell for Building Bayesian Belief Universes for Expert Systems , 1989, IJCAI.

[5]  Michael de la Maza,et al.  Book review: Genetic Algorithms + Data Structures = Evolution Programs by Zbigniew Michalewicz (Springer-Verlag, 1992) , 1993 .

[6]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[7]  José Manuel Gutiérrez,et al.  Expert Systems and Probabiistic Network Models , 1996 .

[8]  Kristian G. Olesen,et al.  HUGIN - a Shell for Building Belief Universes for Expert Systems , 1989, IJCAI 1989.

[9]  S. Moral,et al.  On the problem of performing exact partial abductive inference in Bayesian belief networks using junction trees , 2002 .

[10]  G. Di Caro,et al.  Ant colony optimization: a new meta-heuristic , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[11]  Marco Dorigo,et al.  Swarm intelligence: from natural to artificial systems , 1999 .

[12]  Riccardo Poli,et al.  New ideas in optimization , 1999 .

[13]  Finn Verner Jensen,et al.  Introduction to Bayesian Networks , 2008, Innovations in Bayesian Networks.

[14]  Wilson X. Wen,et al.  Optimal decomposition of belief networks , 1990, UAI.

[15]  Serafín Moral,et al.  Heuristic Algorithms for the Triangulation of Graphs , 1994, IPMU.

[16]  Judea Pearl,et al.  Fusion, Propagation, and Structuring in Belief Networks , 1986, Artif. Intell..

[17]  Uffe Kjærulff.  Optimal decomposition of probabilistic networks by simulated annealing , 1992 .

[18]  Judea Pearl,et al.  Distributed Revision of Composite Beliefs , 1987, Artif. Intell..

[19]  Robert E. Tarjan,et al.  Simple Linear-Time Algorithms to Test Chordality of Graphs, Test Acyclicity of Hypergraphs, and Selectively Reduce Acyclic Hypergraphs , 1984, SIAM J. Comput..

[20]  Prakash P. Shenoy,et al.  Axioms for probability and belief-function proagation , 1990, UAI.

[21]  Pedro Larrañaga,et al.  Decomposing Bayesian networks: triangulation of the moral graph with genetic algorithms , 1997, Stat. Comput..

[22]  D. Nilsson,et al.  An efficient algorithm for finding the M most probable configurationsin probabilistic expert systems , 1998, Stat. Comput..

[23]  Robert E. Tarjan,et al.  Algorithmic Aspects of Vertex Elimination on Graphs , 1976, SIAM J. Comput..

[24]  José A. Gámez,et al.  Partial abductive inference in Bayesian belief networks using a genetic algorithm , 1999, Pattern Recognit. Lett..

[25]  Kristian G. Olesen,et al.  Maximal Prime Subgraph Decomposition of Bayesian Networks , 2001, FLAIRS.

[26]  Edzard S. Gelsema,et al.  Abductive reasoning in Bayesian belief networks using a genetic algorithm , 1995, Pattern Recognit. Lett..

[27]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[28]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[29]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[30]  Marco Dorigo,et al.  Optimization, Learning and Natural Algorithms , 1992 .

[31]  Dan Geiger,et al.  A sufficiently fast algorithm for finding close to optimal junction trees , 1996, UAI.

[32]  Luis Daniel Hernández Molinero.  Diseño y validación de nuevos algoritmos para el tratamiento de grafos de dependencias , 1995 .

[33]  D. J. Smith,et al.  A Study of Permutation Crossover Operators on the Traveling Salesman Problem , 1987, ICGA.

[34]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[35]  Pedro Larrañaga,et al.  Moral Graph, Triangulation of , 2006 .

[36]  Andrew P. Sage,et al.  Uncertainty in Artificial Intelligence , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[37]  Luca Maria Gambardella,et al.  Ant Algorithms for Discrete Optimization , 1999, Artificial Life.