The construction of a Boolean competitive neural network using ideas from immunology

Abstract The immune system is capable of recognizing and responding to microorganisms and molecules that cannot be perceived by our sensory mechanisms, which send stimuli straight into the brain. It performs an accessory role for nervous cognition. This paper main goals are: (1) to show how some immune principles and theories can be used as sources of inspiration to develop novel neural network learning algorithms; (2) to survey the main works from the literature that employ the immune metaphor for the development of neural network architectures; and (3) to illustrate, with a new network model, how this source of inspiration can be actually used to develop a neural network learning algorithm. The novel learning algorithm proposed has the main features of competitive learning, automatic generation of the network structure and binary representation of the connection strengths (weights). The behavior of the algorithm is primarily described using a benchmark task, and some of its potential applications are illustrated using two simple real-world problems and a binary character recognition task. The results show that the network is a promising tool for solving problems that are inherently binary, and also that the immune system provides a new paradigm to search for neural network learning algorithms.

[1]  Sung-Bae Cho,et al.  Self-Organizing Map with Dynamical Node Splitting: Application to Handwritten Digit Recognition , 1997, Neural Computation.

[2]  T. Kohonen,et al.  Self-organizing semantic maps , 1989, Biological Cybernetics.

[3]  F. Varela,et al.  Morphogenesis in shape-space. Elementary meta-dynamics in a model of the immune network. , 1991, Journal of theoretical biology.

[4]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[5]  Jerne Nk.  Towards a network theory of the immune system. , 1974 .

[6]  F T Vertosick,et al.  The immune system as a neural network: a multi-epitope approach. , 1991, Journal of theoretical biology.

[7]  Edgar Sánchez-Sinencio,et al.  A modular CMOS design of a Hamming network , 1992, IEEE Trans. Neural Networks.

[8]  A S Perelson,et al.  Pattern formation in one- and two-dimensional shape-space models of the immune system. , 1992, Journal of theoretical biology.

[9]  T. Kepler,et al.  Somatic hypermutation in B cells: an optimal control treatment. , 1993, Journal of theoretical biology.

[10]  V. K. Prabhu,et al.  Frequency-hopped multilevel FSK for mobile Radio , 1980, The Bell System Technical Journal.

[11]  N. Packard,et al.  North-Holland, Amsterdam THE IMMUNE SYSTEM, ADAPTATION, AND MACHINE LEARNING , 2004 .

[12]  Igor Aleksander,et al.  Digital general neural units with controlled transition probabilities , 1996 .

[13]  Ian H. Witten,et al.  Using Concept Learning for Knowledge Acquisition , 1988, Int. J. Man Mach. Stud..

[14]  T. Kohonen.  Self-organized formation of topographically correct feature maps , 1982 .

[15]  James Brian Quinn,et al.  Technology in services , 1987 .

[16]  A. Perelson.  Immune Network Theory , 1989, Immunological reviews.

[17]  D. Dasgupta.  Artificial Immune Systems and Their Applications , 1998, Springer Berlin Heidelberg.

[18]  Fernando José Von Zuben,et al.  An improving pruning technique with restart for the Kohonen self-organizing feature map , 1999, IJCNN.

[19]  Ron R. Hightower,et al.  Deriving shape space parameters from immunological data. , 1997, Journal of theoretical biology.

[20]  Ugur Cilingiroglu.  A charge-based neural Hamming classifier , 1993 .

[21]  Anna Maria Fanelli,et al.  An associative memory based on the immune networks , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[22]  Paul E. Kinahan,et al.  A teachable neural network based on an unorthodox neuron , 1986 .

[23]  Igor Aleksander,et al.  General neural unit: retrieval performance , 1991 .

[24]  Giorgio Vassallo,et al.  Self-Organizing Maps: A new Digital Architecture , 1991, AI*IA.

[25]  Dipankar Dasgupta,et al.  Artificial neural networks and artificial immune systems: similarities and differences , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[26]  Michel Declercq,et al.  Implementation of a learning Kohonen neuron based on a new multilevel storage technique , 1991 .

[27]  G W Hoffmann,et al.  A neural network model based on the analogy with the immune system. , 1986, Journal of theoretical biology.

[28]  Paolo Ienne,et al.  Digital systems for neural networks , 1995, Defense + Commercial Sensing.

[29]  Alan S. Perelson,et al.  The Baldwin effect in the immune system: learning by somatic hypermutation , 1996 .

[30]  J S Denker,et al.  Neural network models of learning and adaptation , 1986 .

[31]  Anthony N. Michel,et al.  A training algorithm for binary feedforward neural networks , 1992, IEEE Trans. Neural Networks.

[32]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[33]  V. Peiris,et al.  Implementation of a learning Kohonen neuron based on a pulse stream arithmetics , 1990, ESSCIRC '90: Sixteenth European Solid-State Circuits Conference.

[34]  Thomas Villmann,et al.  Growing a hypercubical output space in a self-organizing feature map , 1997, IEEE Trans. Neural Networks.

[35]  N. H. Biswas,et al.  IMS algorithm for learning representations in Boolean neural networks , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[36]  G L Ada,et al.  The clonal-selection theory. , 1987, Scientific American.

[37]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[38]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[39]  R. Gershon,et al.  "Clonal selection and after," and after. , 1979, The New England journal of medicine.

[40]  A. Perelson,et al.  Size and connectivity as emergent properties of a developing immune network. , 1991, Journal of theoretical biology.

[41]  Alan S. Perelson,et al.  The immune system, adaptation, and machine learning , 1986 .

[42]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[43]  F. Burnet.  The clonal selection theory of acquired immunity , 1959 .

[44]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[45]  Leandro Nunes de Castro,et al.  An Immunological Approach to Initialize Feedforward Neural Network Weights , 2001 .

[46]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[47]  Alan S. Perelson,et al.  The Evolution of Emergent Organization in Immune System Gene Libraries , 1995, ICGA.

[48]  Dipankar Dasgupta,et al.  An Overview of Artificial Immune Systems and Their Applications , 1993 .

[49]  F T Vertosick,et al.  Immune network theory: a role for parallel distributed processing? , 1989, Immunology.

[50]  A. Perelson,et al.  Size and connectivity of the idiotypic network are independent of the discreteness of the affinity distribution. , 1996, Journal of theoretical biology.

[51]  G. Oster,et al.  Theoretical studies of clonal selection: minimal antibody repertoire size and reliability of self-non-self discrimination. , 1979, Journal of theoretical biology.

[52]  Melanie Mitchell,et al.  An introduction to genetic algorithms , 1996 .

[53]  C. Janeway.  Immunobiology: The Immune System in Health and Disease , 1996 .

[54]  A S Perelson,et al.  Cyclic re-entry of germinal center B cells and the efficiency of affinity maturation. , 1993, Immunology today.

[55]  C. Berek,et al.  The maturation of the immune response. , 1993, Immunology today.

[56]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[57]  G. Weisbuch,et al.  Immunology for physicists , 1997 .

[58]  Alexandre Schmid,et al.  Hardware realization of a Hamming neural network with on-chip learning , 1998, ISCAS '98. Proceedings of the 1998 IEEE International Symposium on Circuits and Systems (Cat. No.98CH36187).

[59]  A. Coutinho,et al.  From an Antigen‐Centered, Clonal Perspective of Immune Responses to an Organism‐Centered, Network Perspective of Autonomous Activity in a Self‐Referential Immune System , 1984, Immunological reviews.

[60]  Patrick Thiran,et al.  Quantization effects in digitally behaving circuit implementations of Kohonen networks , 1994, IEEE Trans. Neural Networks.

[61]  G. Einarsson.  Address assignment for a time-frequency-coded, spread-spectrum system , 1980, The Bell System Technical Journal.