An autonomous GP-based system for regression and classification problems

The aim of this research is to develop an autonomous system for solving data analysis problems. The system, called Genetic Programming-Autonomous Solver (GP-AS) contains most of the features required by an autonomous software: it decides if it knows or not how to solve a particular problem, it can construct solutions for new problems, it can store the created solutions for later use, it can improve the existing solutions in the idle-time it can efficiently manage the computer resources for fast running speed and it can detect and handle failure cases. The generator of solutions for new problems is based on an adaptive variant of Genetic Programming. We have tested this part by solving some well-known problems in the field of symbolic regression and classification. Numerical experiments show that the GP-AS system is able to perform very well on the considered test problems being able to successfully compete with standard GP having manually set parameters.

[1]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[2]  Mihai Oltean,et al.  A-Brain: a general system for solving data analysis problems , 2007, J. Exp. Theor. Artif. Intell..

[3]  David B. Fogel,et al.  A Preliminary Investigation on Extending Evolutionary Programming to Include Self-Adaptation on Finite State , 1994, Informatica.

[4]  Una-May O'Reilly,et al.  Genetic Programming II: Automatic Discovery of Reusable Programs. , 1994, Artificial Life.

[5]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[6]  Giovanni Squillero,et al.  Exploiting Auto-adaptive 7GP for Highly Effective Test Programs Generation , 2003, ICES.

[7]  Mihai Oltean,et al.  Evolving Evolutionary Algorithms Using Multi Expression Programming , 2003, ECAL.

[8]  J. Pollack,et al.  The Evolutionary Induction of Subroutines , 1997 .

[9]  Zbigniew Michalewicz,et al.  Adaptation in evolutionary computation: a survey , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[10]  Jano I. van Hemert,et al.  Adaptive Genetic Programming Applied to New and Existing Simple Regression Problems , 2001, EuroGP.

[11]  Leo Breiman,et al.  Bias, Variance , And Arcing Classifiers , 1996 .

[12]  Dana H. Ballard,et al.  Genetic Programming with Adaptive Representations , 1994 .

[13]  Riccardo Poli,et al.  New ideas in optimization , 1999 .

[14]  J. David Schaffer,et al.  Proceedings of the third international conference on Genetic algorithms , 1989 .

[15]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[16]  Mihai Oltean,et al.  Evolving Evolutionary Algorithms with Patterns , 2007, Soft Comput..

[17]  John J. Grefenstette,et al.  Genetic algorithms and their applications , 1987 .

[18]  Christopher G. Langton,et al.  Artificial Life III , 2000 .

[19]  George S. Dulikravich,et al.  Control of Unsteady Solidification Via Optimized Magnetic Fields , 2005 .

[20]  Tughrul Arslan,et al.  2003 NASA/DoD Conference on Evolvable Hardware , 2002, NASA/DoD Conference on Evolvable Hardware, 2003. Proceedings..

[21]  Peter J. Angeline,et al.  Adaptive and Self-adaptive Evolutionary Computations , 1995 .

[22]  Peter J. Angeline,et al.  Extending Genetic Programming with Recombinative Guidance , 1996 .

[23]  John R. Koza,et al.  Evolution of Both the Architecture and the Sequence of Work-Performing Steps of a Computer Program Using Genetic Programming with Architecture-Altering Operations , 1995 .

[24]  Lee Spector,et al.  Genetic Programming and Autoconstructive Evolution with the Push Programming Language , 2002, Genetic Programming and Evolvable Machines.

[25]  J. Pollack,et al.  Coevolving High-Level Representations , 1993 .

[26]  J. Reed,et al.  Simulation of biological evolution and machine learning. I. Selection of self-reproducing numeric patterns by data processing machines, effects of hereditary control, mutation type and crossing. , 1967, Journal of theoretical biology.

[27]  Thomas Bck,et al.  Self-adaptation in genetic algorithms , 1991 .

[28]  F. Varela,et al.  Toward a Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life , 1992 .

[29]  Thomas Bäck,et al.  The Interaction of Mutation Rate, Selection, and Self-Adaptation Within a Genetic Algorithm , 1992, PPSN.

[30]  Peter J. Angeline,et al.  Evolving Programmers: The Co-evolution of Intelligent Recombination Operators , 1996 .

[31]  Mihai Oltean,et al.  Evolving Evolutionary Algorithms Using Linear Genetic Programming , 2005, Evolutionary Computation.

[32]  J. David Schaffer,et al.  An Adaptive Crossover Distribution Mechanism for Genetic Algorithms , 1987, ICGA.

[33]  Mihai Oltean,et al.  Adaptive representation for single objective optimization , 2005, Soft Comput..

[34]  Panos J. Antsaklis,et al.  Towards intelligent autonomous control systems: Architecture and fundamental issues , 1989, J. Intell. Robotic Syst..

[35]  Mihai Oltean,et al.  A Comparison of Several Linear Genetic Programming Techniques , 2003, Complex Syst..

[36]  Zbigniew Michalewicz,et al.  Parameter Control in Evolutionary Algorithms , 2007, Parameter Setting in Evolutionary Algorithms.

[37]  Hitoshi Iba,et al.  Extending genetic programming with recombinative guidance , 1996 .

[38]  Wolfgang Banzhaf,et al.  Genetic Programming: An Introduction , 1997 .

[39]  R. Rosenberg.  Simulation of genetic populations with biochemical properties : technical report , 1967 .

[40]  Jano van Hemert,et al.  SAW-ing EAs: adapting the fitness function for solving constrained problems , 1999 .

[41]  J. K. Kinnear,et al.  Advances in Genetic Programming , 1994 .

[42]  William M. Spears,et al.  Adapting Crossover in a Genetic Algorithm , 2007 .

[43]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[44]  Lawrence Davis,et al.  Adapting Operator Probabilities in Genetic Algorithms , 1989, ICGA.

[45]  John R. Koza,et al.  Genetic programming 2 - automatic discovery of reusable programs , 1994, Complex Adaptive Systems.

[46]  John Daniel. Bagley,et al.  The behavior of adaptive systems which employ genetic and correlation algorithms : technical report , 1967 .

[47]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[48]  Frank Pettersson,et al.  A genetic algorithms based multi-objective neural net applied to noisy blast furnace data , 2007, Appl. Soft Comput..

[49]  Peter J. Angeline,et al.  Two self-adaptive crossover operators for genetic programming , 1996 .

[50]  Robert E. Smith,et al.  Adaptively Resizing Populations: An Algorithm and Analysis , 1993, ICGA.

[51]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[52]  Riccardo Poli,et al.  Solving High-Order Boolean Parity Problems with Smooth Uniform Crossover, Sub-Machine Code GP and Demes , 2000, Genetic Programming and Evolvable Machines.

[53]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[54]  Wolfgang Banzhaf,et al.  A comparison of linear genetic programming and neural networks in medical data mining , 2001, IEEE Trans. Evol. Comput..

[55]  Julian Francis Miller,et al.  Principles in the Evolutionary Design of Digital Circuits—Part II , 2000, Genetic Programming and Evolvable Machines.

[56]  George S. Dulikravich,et al.  Solidification of Double-Diffusive Flows Using Thermo-Magneto-Hydrodynamics and Optimization , 2007 .

[57]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[58]  Mihai Oltean,et al.  Evolving digital circuits using multi expression programming , 2004, Proceedings. 2004 NASA/DoD Conference on Evolvable Hardware, 2004..

[59]  Marimuthu Palaniswami,et al.  Computational Intelligence: A Dynamic System Perspective , 1995 .

[60]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[61]  Astro Teller,et al.  Evolving programmers: the co-evolution of intelligent recombination operators , 1996 .

[62]  C. G. Shaefer,et al.  The ARGOT Strategy: Adaptive Representation Genetic Optimizer Technique , 1987, ICGA.