BAIS: A Bayesian Artificial Immune System for the effective handling of building blocks

Significant progress has been made in theory and design of Artificial Immune Systems (AISs) for solving hard problems accurately. However, an aspect not yet widely addressed by the research reported in the literature is the lack of ability of the AISs to deal effectively with building blocks (partial high-quality solutions coded in the antibody). The available AISs present mechanisms for evolving the population that do not take into account the relationship among the variables of the problem, potentially causing the disruption of high-quality partial solutions. This paper proposes a novel AIS with abilities to identify and properly manipulate building blocks in optimization problems. Instead of using cloning and mutation to generate new individuals, our algorithm builds a probabilistic model representing the joint probability distribution of the promising solutions and, subsequently, uses this model for sampling new solutions. The probabilistic model used is a Bayesian network due to its capability of properly capturing the most relevant interactions among the variables. Therefore, our algorithm, called Bayesian Artificial Immune System (BAIS), represents a significant attempt to improve the performance of immune-inspired algorithms when dealing with building blocks, and hence to solve efficiently hard optimization problems with complex interactions among the variables. The performance of BAIS compares favorably with that produced by contenders such as state-of-the-art Estimation of Distribution Algorithms.

[1]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[2]  Fernando José Von Zuben,et al.  Learning and optimization using the clonal selection principle , 2002, IEEE Trans. Evol. Comput..

[3]  Pedro Larrañaga,et al.  Analysis of the behaviour of genetic algorithms when learning Bayesian network structure from data , 1997, Pattern Recognit. Lett..

[4]  Jerne Nk Towards a network theory of the immune system. , 1974 .

[5]  Jonathan Timmis,et al.  Artificial Immune Systems: A New Computational Intelligence Approach , 2003 .

[6]  Rajkumar Roy,et al.  Advances in Soft Computing: Engineering Design and Manufacturing , 1998 .

[7]  L.N. de Castro,et al.  An artificial immune network for multimodal function optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[8]  Shumeet Baluja,et al.  Using Optimal Dependency-Trees for Combinational Optimization , 1997, ICML.

[9]  David E. Goldberg,et al.  Tightness Time for the Linkage Learning Genetic Algorithm , 2003, GECCO.

[10]  Pedro Larrañaga,et al.  Learning Bayesian Networks by Floating Search Methods , 2004 .

[11]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[12]  David Maxwell Chickering,et al.  Learning Equivalence Classes of Bayesian Network Structures , 1996, UAI.

[13]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[14]  M. Pelikán,et al.  The Bivariate Marginal Distribution Algorithm , 1999 .

[15]  Kalyanmoy Deb,et al.  RapidAccurate Optimization of Difficult Problems Using Fast Messy Genetic Algorithms , 1993, ICGA.

[16]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[17]  Pedro Larrañaga,et al.  Globally Multimodal Problem Optimization Via an Estimation of Distribution Algorithm Based on Unsupervised Learning of Bayesian Networks , 2005, Evolutionary Computation.

[18]  G L Ada,et al.  The clonal-selection theory. , 1987, Scientific American.

[19]  D. E. Goldberg,et al.  Optimization and Machine Learning , 2022 .

[20]  D. Dasgupta,et al.  Advances in artificial immune systems , 2006, IEEE Computational Intelligence Magazine.

[21]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[22]  Kalyanmoy Deb,et al.  Analyzing Deception in Trap Functions , 1992, FOGA.

[23]  Heinz Mühlenbein,et al.  FDA -A Scalable Evolutionary Algorithm for the Optimization of Additively Decomposed Functions , 1999, Evolutionary Computation.

[24]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[25]  Jonathan Timmis,et al.  Artificial immune systems - a new computational intelligence paradigm , 2002 .

[26]  David E. Goldberg,et al.  Linkage Problem, Distribution Estimation, and Bayesian Networks , 2000, Evolutionary Computation.

[27]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[28]  José A. Gámez,et al.  Advances in Bayesian networks , 2004 .

[29]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[30]  Pedro Larrañaga,et al.  Feature Subset Selection by Bayesian network-based optimization , 2000, Artif. Intell..

[31]  Doug Fisher,et al.  Learning from Data: Artificial Intelligence and Statistics V , 1996 .

[32]  David Maxwell Chickering,et al.  Learning Bayesian Networks is NP-Complete , 2016, AISTATS.

[33]  G. Bortolan,et al.  The problem of linguistic approximation in clinical decision making , 1988, Int. J. Approx. Reason..

[34]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[35]  David E. Goldberg,et al.  The compact genetic algorithm , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[36]  David Maxwell Chickering,et al.  Large-Sample Learning of Bayesian Networks is NP-Hard , 2002, J. Mach. Learn. Res..

[37]  Finn Verner Jensen,et al.  Introduction to Bayesian Networks , 2008, Innovations in Bayesian Networks.

[38]  Fernando José Von Zuben,et al.  An immune-inspired approach to Bayesian networks , 2005, Fifth International Conference on Hybrid Intelligent Systems (HIS'05).

[39]  D. Goldberg,et al.  Escaping hierarchical traps with competent genetic algorithms , 2001 .

[40]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[41]  Fabrício Olivetti de França,et al.  An artificial immune network for multimodal function optimization on dynamic environments , 2005, GECCO.

[42]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[43]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[44]  James Brian Quinn,et al.  Technology in services , 1987 .