Internal reinforcement in a connectionist genetic programming approach

Abstract Genetic programming (GP) can learn complex concepts by searching for the target concept through evolution of a population of candidate hypothesis programs. However, unlike some learning techniques, such as Artificial Neural Networks (ANNs), GP does not have a principled procedure for changing parts of a learned structure based on that structure's performance on the training data. GP is missing a clear, locally optimal update procedure, the equivalent of gradient-descent backpropagation for ANNs. This article introduces a new algorithm, “internal reinforcement”, for defining and using performance feedback on program evolution. This internal reinforcement principled mechanism is developed within a new connectionist representation for evolving parameterized programs, namely “neural programming”. We present the algorithms for the generation of credit and blame assignment in the process of learning programs using neural programming and internal reinforcement. The article includes a comprehensive overview of genetic programming and empirical experiments that demonstrate the increased learning rate obtained by using our principled program evolution approach.

[1]  Scott Brave,et al.  The evolution of memory and mental models using genetic programming , 1996 .

[2]  Robert E. Smith,et al.  Classifier system renaissance: new analogies, new directions , 1996 .

[3]  William B. Langdon.  Evolving Data Structures with Genetic Programming , 1995, ICGA.

[4]  Peter J. Angeline.  Evolutionary algorithms and emergent intelligence , 1993 .

[5]  Peter J. Angeline,et al.  Algorithm Discovery using the Genetic Programming Paradigm: Extracting Low-Contrast Curvilinear Features from SAR Images of Arctic Ice , 1996 .

[6]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[7]  M. Veloso.  Program Evolution for Data Mining , 1995 .

[8]  William B. Langdon.  Data structures and genetic programming , 1995 .

[9]  R. Poli,et al.  Discovery of Symbolic, Neuro-Symbolic and Neural Networks with Parallel Distributed Genetic Programming , 1997, ICANNGA.

[10]  Astro Teller,et al.  PADO: a new learning architecture for object recognition , 1997 .

[11]  Astro Teller,et al.  Algorithm evolution with internal reinforcement for signal understanding , 1998 .

[12]  Astro Teller,et al.  Turing completeness in the language of genetic programming with indexed memory , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[13]  Donald A. Waterman,et al.  Pattern-Directed Inference Systems , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Nichael Lynn Cramer,et al.  A Representation for the Adaptive Generation of Simple Sequential Programs , 1985, ICGA.

[15]  Astro Teller,et al.  Algorithm Evolution for Signal Understanding , 1996, IJCAI.

[16]  Justinian P. Rosca,et al.  Discovery of subroutines in genetic programming , 1996 .

[17]  Astro Teller,et al.  A study in program response and the negative effects of introns in genetic programming , 1996 .

[18]  Manuela M. Veloso,et al.  A Contolled Experiment: Evolution for Learning Difficult Image Classification , 1995, EPIA.

[19]  David Andre,et al.  The Evolution of Agents that Build Mental Models and Create Simple Plans Using Genetic Programming , 1995, ICGA.

[20]  E. Howard,et al.  The Application of Genetic Programming to the Investigation of Short, Noisy, Chaotic Data Series , 1994, Evolutionary Computing, AISB Workshop.

[21]  Sebastian Thrun,et al.  Learning One More Thing , 1994, IJCAI.

[22]  David B. Fogel,et al.  An Evolutionary Programming Approach to Self-Adaptation on Finite State Machines , 1995, Evolutionary Programming.

[23]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[24]  Kenneth E. Kinnear,et al.  Generality and Difficulty in Genetic Programming: Evolving a Sort , 1993, ICGA.

[25]  L. Altenberg.  The evolution of evolvability in genetic programming , 1994 .

[26]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[27]  David Andre,et al.  Automatically defined features: the simultaneous evolution of 2-dimensional feature detectors and an algorithm for using them , 1994 .

[28]  Astro Teller,et al.  Algorithm evolution for face recognition: what makes a picture difficult , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[29]  Thomas S. Huang,et al.  Evolvable 3D modeling for model-based object recognition systems , 1994 .

[30]  Katsushi Ikeuchi,et al.  Symbolic visual learning , 1997 .

[31]  Peter J. Angeline,et al.  Two self-adaptive crossover operators for genetic programming , 1996 .

[32]  Astro Teller,et al.  Evolving programmers: the co-evolution of intelligent recombination operators , 1996 .

[33]  Lawrence Davis,et al.  Genetic Algorithms and Simulated Annealing , 1987 .

[34]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[35]  John H. Holland,et al.  COGNITIVE SYSTEMS BASED ON ADAPTIVE ALGORITHMS1 , 1978 .

[36]  David E. Goldberg,et al.  A Critical Review of Classifier Systems , 1989, ICGA.

[37]  Stewart W. Wilson.  Hierarchical Credit Allocation in a Classifier System , 1987, IJCAI.

[38]  Walter Alden Tackett,et al.  Genetic Programming for Feature Discovery and Image Discrimination , 1993, ICGA.

[39]  J. K. Kinnear,et al.  Advances in Genetic Programming , 1994 .

[40]  Astro Teller,et al.  PADO: Learning Tree Structured Algorithms for Orchestration into an Object Recognition System , 1995 .

[41]  Ken Sharman,et al.  Evolving signal processing algorithms by genetic programming , 1995 .

[42]  Roland Olsson,et al.  Inductive Functional Programming Using Incremental Program Transformation , 1995, Artif. Intell..

[43]  Astro Teller,et al.  The evolution of mental models , 1994 .

[44]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .