Open issues in genetic programming

It is approximately 50 years since the first computational experiments were conducted in what has become known today as the field of Genetic Programming (GP), twenty years since John Koza named and popularised the method, and ten years since the first issue appeared of the Genetic Programming & Evolvable Machines journal. In particular, during the past two decades there has been a significant range and volume of development in the theory and application of GP, and in recent years the field has become increasingly applied. There remain a number of significant open issues despite the successful application of GP to a number of challenging real-world problem domains and progress in the development of a theory explaining the behavior and dynamics of GP. These issues must be addressed for GP to realise its full potential and to become a trusted mainstream member of the computational problem solving toolkit. In this paper we outline some of the challenges and open issues that face researchers and practitioners of GP. We hope this overview will stimulate debate, focus the direction of future research to deepen our understanding of GP, and further the development of more powerful problem solving algorithms.

[1]  A. E. Eiben,et al.  A critical note on experimental research methodology in EC , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[2]  Uri Alon,et al.  Varying environments can speed up evolution , 2007, Proceedings of the National Academy of Sciences.

[3]  Conor Ryan,et al.  Survey Of Evolutionary Automatic Programming , 2003 .

[4]  Ting Hu,et al.  The Role of Population Size in Rate of Evolution in Genetic Programming , 2009, EuroGP.

[5]  Xin Yao,et al.  Evolutionary Optimization , 2002 .

[6]  Nicholas Freitag McPhee,et al.  Semantic Building Blocks in Genetic Programming , 2008, EuroGP.

[7]  Franz Rothlauf,et al.  On the Locality of Grammatical Evolution , 2006, EuroGP.

[8]  Ronald W. Morrison,et al.  Designing Evolutionary Algorithms for Dynamic Environments , 2004, Natural Computing Series.

[9]  Matthew P. Evett,et al.  Numeric Mutation Improves the Discovery of Numeric Constants in Genetic Programming , 2007 .

[10]  Peter A. Whigham,et al.  Grammatically-based Genetic Programming , 1995 .

[11]  Jason H. Moore,et al.  Development and Evaluation of an Open-Ended Computational Evolution System for the Genetic Analysis of Susceptibility to Common Human Diseases , 2008, EvoBIO.

[12]  David E. Goldberg,et al.  Where Does the Good Stuff Go, and Why? How Contextual Semantics Influences Program Structure in Simple Genetic Programming , 1998, EuroGP.

[13]  Marc Schoenauer,et al.  Evolving Genes to Balance a Pole , 2010, EuroGP.

[14]  W. B. Langdon,et al.  Genetic Programming and Data Structures , 1998, The Springer International Series in Engineering and Computer Science.

[15]  Colin G. Johnson,et al.  Semantically driven crossover in genetic programming , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[16]  Nichael Lynn Cramer,et al.  A Representation for the Adaptive Generation of Simple Sequential Programs , 1985, ICGA.

[17]  Justinian P. Rosea Towards Automatic Discovery of Building Blocks in Genetic Programming , 1995 .

[18]  Jianjun Hu,et al.  The Hierarchical Fair Competition (HFC) Framework for Sustainable Evolutionary Algorithms , 2005, Evolutionary Computation.

[19]  Anthony Brabazon,et al.  Recent Patents on Genetic Programming , 2009 .

[20]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[21]  Peter Nordin,et al.  The Effect of Extensive Use of the Mutation Operator on Generalization in Genetic Programming Using Sparse Data Sets , 1996, PPSN.

[22]  John R. Koza,et al.  Hierarchical Genetic Algorithms Operating on Populations of Computer Programs , 1989, IJCAI.

[23]  E. Jablonka,et al.  Evolution in Four Dimensions: Genetic, Epigenetic, Behavioral, and Symbolic Variation in the History of Life , 2005 .

[24]  Conor Ryan,et al.  An Analysis of Diversity of Constants of Genetic Programming , 2003, EuroGP.

[25]  Sébastien Vérel,et al.  Negative Slope Coefficient: A Measure to Characterize Genetic Programming Fitness Landscapes , 2006, EuroGP.

[26]  Subhash C. Kak,et al.  On Generalization by Neural Networks , 1998, Inf. Sci..

[27]  William B. Langdon,et al.  Genetic Programming Bloat without Semantics , 2000, PPSN.

[28]  Riccardo Poli,et al.  Exact Schema Theorems for GP with One-Point and Standard Crossover Operating on Linear Structures and Their Application to the Study of the Evolution of Size , 2001, EuroGP.

[29]  Leonardo Vanneschi,et al.  A Study of Fitness Distance Correlation as a Difficulty Measure in Genetic Programming , 2005, Evolutionary Computation.

[30]  Sara Silva,et al.  GPLAB A Genetic Programming Toolbox for MATLAB , 2004 .

[31]  Steven M. Gustafson An analysis of diversity in genetic programming , 2004 .

[32]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[33]  Nikolay I. Nikolaev,et al.  Genetic Programming and Data Structures: Genetic Programming+Data Structures=Automatic Programming , 2001, Softw. Focus.

[34]  N. Given Genetic Programming , Validation Sets , and Parsimony Pressure , 2005 .

[35]  Wolfgang Banzhaf,et al.  Linear Genetic Programming (Genetic and Evolutionary Computation) , 2006 .

[36]  Gregory Hornby,et al.  ALPS: the age-layered population structure for reducing the problem of premature convergence , 2006, GECCO.

[37]  Anthony Brabazon,et al.  Foundations in Grammatical Evolution for Dynamic Environments , 2009, Studies in Computational Intelligence.

[38]  Wolfgang Banzhaf,et al.  Linear-Graph GP - A New GP Structure , 2002, EuroGP.

[39]  Jürgen Branke,et al.  Evolutionary Optimization in Dynamic Environments , 2001, Genetic Algorithms and Evolutionary Computation.

[40]  Domagoj Jakobovic,et al.  Dynamic Scheduling with Genetic Programming , 2006, EuroGP.

[41]  Markus Brameier,et al.  On linear genetic programming , 2005 .

[42]  Stephen F. Smith,et al.  A learning system based on genetic adaptive algorithms , 1980 .

[43]  Pedro M. Domingos The Role of Occam's Razor in Knowledge Discovery , 1999, Data Mining and Knowledge Discovery.

[44]  Anthony Brabazon,et al.  Evolutionary design using grammatical evolution and shape grammars: designing a shelter , 2010 .

[45]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[46]  John R. Koza,et al.  Genetic Programming III: Darwinian Invention & Problem Solving , 1999 .

[47]  Claire Le Goues,et al.  Automatically finding patches using genetic programming , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[48]  Riccardo Poli,et al.  Extending Particle Swarm Optimisation via Genetic Programming , 2005, EuroGP.

[49]  David J. Montana,et al.  Strongly Typed Genetic Programming , 1995, Evolutionary Computation.

[50]  Malcolm I. Heywood,et al.  A Linear Genetic Programming Approach to Intrusion Detection , 2003, GECCO.

[51]  Peter A. Whigham,et al.  Grammar-based Genetic Programming: a survey , 2010, Genetic Programming and Evolvable Machines.

[52]  Leonardo Vanneschi,et al.  Using crossover based similarity measure to improve genetic programming generalization ability , 2009, GECCO.

[53]  Giancarlo Mauri,et al.  Using Subtree Crossover Distance to Investigate Genetic Programming Dynamics , 2006, EuroGP.

[54]  Leonardo Vanneschi,et al.  Genetic programming for computational pharmacokinetics in drug discovery and development , 2007, Genetic Programming and Evolvable Machines.

[55]  K. Kinnear Fitness landscapes and difficulty in genetic programming , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[56]  Vidroha Debroy,et al.  Genetic Programming , 1998, Lecture Notes in Computer Science.

[57]  E. Jablonka,et al.  Evolution in Four Dimensions , 2005 .

[58]  Michael O'Neill,et al.  Improving the Generalisation Ability of Genetic Programming with Semantic Similarity based Crossover , 2010, EuroGP.

[59]  Jacques-André Landry,et al.  Relaxed genetic programming , 2006, GECCO.

[60]  Peter A. Whigham,et al.  Grammatical bias for evolutionary learning , 1996 .

[61]  Istvan Jonyer,et al.  Improving Modularity in Genetic Programming Using Graph-Based Data Mining , 2006, FLAIRS Conference.

[62]  Juan E. Tapiador,et al.  Evolving High-Speed, Easy-to-Understand Network Intrusion Detection Rules with Genetic Programming , 2009, EvoWorkshops.

[63]  Leonardo Vanneschi,et al.  Operator-Based Distance for Genetic Programming: Subtree Crossover Distance , 2005, EuroGP.

[64]  Jason M. Daida,et al.  What Makes a Problem GP-Hard? Validating a Hypothesis of Structural Causes , 2003, GECCO.

[65]  Michael O'Neill,et al.  An Attribute Grammar Decoder for the 01 MultiConstrained Knapsack Problem , 2005, EvoCOP.

[66]  Grégory Seront gseront External Concepts Reuse in Genetic Programming , 2001 .

[67]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[68]  Wolfgang Banzhaf,et al.  Linear-Tree GP and Its Comparison with Other GP Structures , 2001, EuroGP.

[69]  S.E. Eklund,et al.  Time series forecasting using massively parallel genetic programming , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[70]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[71]  Peter J. Angeline,et al.  Two self-adaptive crossover operators for genetic programming , 1996 .

[72]  Franz Rothlauf,et al.  Representations for genetic and evolutionary algorithms , 2002, Studies in Fuzziness and Soft Computing.

[73]  William B. Langdon,et al.  Repeated Sequences in Linear Genetic Programming Genomes , 2005, Complex Syst..

[74]  Riccardo Poli,et al.  Foundations of Genetic Programming , 1999, Springer Berlin Heidelberg.

[75]  Ibrahim Kushchu,et al.  An Evaluation of EvolutionaryGeneralisation in Genetic Programming , 2002, Artificial Intelligence Review.

[76]  James V. Hansen,et al.  Genetic programming for prevention of cyberterrorism through dynamic and evolving intrusion detection , 2007, Decis. Support Syst..

[77]  Leonardo Vanneschi,et al.  Multi-optimization improves genetic programming generalization ability , 2007, GECCO '07.

[78]  Lee Spector,et al.  Evolving Control Structures with Automatically Defined Macros , 2001 .

[79]  Mengjie Zhang,et al.  Genetic Programming for Automatic Stress Detection in Spoken English , 2006, EvoWorkshops.

[80]  Peter Nordin,et al.  Benchmarking the generalization capabilities of a compiling genetic programming system using sparse data sets , 1996 .

[81]  Shengxiang Yang,et al.  Editorial to special issue on evolutionary computation in dynamic and uncertain environments , 2006, Genetic Programming and Evolvable Machines.

[82]  Kalyanmoy Deb,et al.  Multimodal Deceptive Functions , 1993, Complex Syst..

[83]  Susan Schreibman Editorial Introduction to the First Issue , 2011 .

[84]  Wo-Chiang Lee,et al.  Genetic Programming Decision Tree for Bankruptcy Prediction , 2006, JCIS.

[85]  L. Altenberg Modularity in Evolution: Some Low-Level Questions ∗ , 2005 .

[86]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[87]  Maureen A. O’Malley Evolution in Four Dimensions: Genetic, Epigenetic, Behavioral, and Symbolic Variation in the History of Life , 2006 .

[88]  Dick den Hertog,et al.  Order of Nonlinearity as a Complexity Measure for Models Generated by Symbolic Regression via Pareto Genetic Programming , 2009, IEEE Transactions on Evolutionary Computation.

[89]  J. Miller,et al.  Guidelines: From artificial evolution to computational evolution: a research agenda , 2006, Nature Reviews Genetics.

[90]  Michael O'Neill,et al.  Grammatical evolution - evolutionary automatic programming in an arbitrary language , 2003, Genetic programming.

[91]  Riccardo Poli,et al.  General Schema Theory for Genetic Programming with Subtree-Swapping Crossover: Part II , 2003, Evolutionary Computation.

[92]  Astro Teller,et al.  PADO: a new learning architecture for object recognition , 1997 .

[93]  Cyril Fonlupt,et al.  Solving the ocean color problem using a genetic programming approach , 2001, Appl. Soft Comput..

[94]  Anthony Brabazon,et al.  A Grammatical Genetic Programming Approach to Modularity in Genetic Algorithms , 2007, EuroGP.

[95]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[96]  Riccardo Poli,et al.  There Is a Free Lunch for Hyper-Heuristics, Genetic Programming and Computer Scientists , 2009, EuroGP.

[97]  John R. Koza,et al.  Genetic Programming IV: Routine Human-Competitive Machine Intelligence , 2003 .

[98]  Henry Leung,et al.  Functional reconstruction of dynamical systems from time series using genetic programming , 2000, 2000 26th Annual Conference of the IEEE Industrial Electronics Society. IECON 2000. 2000 IEEE International Conference on Industrial Electronics, Control and Instrumentation. 21st Century Technologies.

[99]  Riccardo Poli,et al.  Free lunches for function and program induction , 2009, FOGA '09.

[100]  Una-May O'Reilly,et al.  Genetic Programming II: Automatic Discovery of Reusable Programs. , 1994, Artificial Life.

[101]  Graham F. Spencer,et al.  Automatic Generation of Programs for Crawling and Walking , 1993, International Conference on Genetic Algorithms.

[102]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[103]  M. Feder Robustness and Evolvability in Living Systems. Princeton Studies in Complexity.By Andreas Wagner. Princeton (New Jersey): Princeton University Press. $49.50. xv + 367 p; ill.; index. ISBN: 0–691–12240–7. 2005. , 2006 .

[104]  Jason M. Daida,et al.  What Makes a Problem GP-Hard? Analysis of a Tunably Difficult Problem in Genetic Programming , 1999, Genetic Programming and Evolvable Machines.

[105]  Ting Hu,et al.  Neutrality and variability: two sides of evolvability in linear genetic programming , 2009, GECCO.

[106]  Richard M. Friedberg,et al.  A Learning Machine: Part I , 1958, IBM J. Res. Dev..

[107]  C. Darwin The Origin of Species by Means of Natural Selection, Or, The Preservation of Favoured Races in the Struggle for Life , 2019 .

[108]  Franz Rothlauf,et al.  Network Random KeysA Tree Representation Scheme for Genetic and Evolutionary Algorithms , 2002, Evolutionary Computation.

[109]  William B. Langdon,et al.  Repeated patterns in genetic programming , 2008, Natural Computing.

[110]  Riccardo Poli,et al.  General Schema Theory for Genetic Programming with Subtree-Swapping Crossover: Part I , 2003, Evolutionary Computation.

[111]  Zbigniew Michalewicz,et al.  Time Series Forecasting for Dynamic Environments: The DyFor Genetic Program Model , 2007, IEEE Transactions on Evolutionary Computation.

[112]  Leonardo Vanneschi,et al.  Variable size population for dynamic optimization with genetic programming , 2009, GECCO.

[113]  Leonardo Vanneschi,et al.  An Evolutionary Framework for Colorimetric Characterization of Scanners , 2008, EvoWorkshops.

[114]  P. Stadler Fitness Landscapes , 1993 .

[115]  Riccardo Poli,et al.  Toward subheuristic search , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[116]  Mengjie Zhang,et al.  Genetic Programming for Object Detection: a Two-phase Approach with an Improved Fitness Function , 2009, Progress in Computer Vision and Image Analysis.

[117]  Malcolm I. Heywood,et al.  Introducing probabilistic adaptive mapping developmental genetic programming with redundant mappings , 2007, Genetic Programming and Evolvable Machines.

[118]  Una-May O'Reilly,et al.  Integrating generative growth and evolutionary computation for form exploration , 2007, Genetic Programming and Evolvable Machines.

[119]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[120]  Bill C. White,et al.  Does Complexity Matter? Artificial Evolution, Computational Evolution and the Genetic Analysis of Epistasis in Common Human Diseases. , 2009 .

[121]  Conor Ryan,et al.  Adaptive logic programming , 2001 .

[122]  Graham Kendall,et al.  Evolving Bin Packing Heuristics with Genetic Programming , 2006, PPSN.

[123]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[124]  A. Bennett The Origin of Species by means of Natural Selection; or the Preservation of Favoured Races in the Struggle for Life , 1872, Nature.

[125]  Leonardo Vanneschi,et al.  Operator equalisation, bloat and overfitting: a study on human oral bioavailability prediction , 2009, GECCO.

[126]  A. Wagner Robustness and Evolvability in Living Systems , 2005 .

[127]  Leonardo Vanneschi,et al.  Measuring bloat, overfitting and functional complexity in genetic programming , 2010, GECCO '10.

[128]  David B. Fogel,et al.  Evolving Computer Programs , 1998 .

[129]  William B. Langdon,et al.  Genetic Programming and Data Structures: Genetic Programming + Data Structures = Automatic Programming! , 1998 .

[130]  Mihai Oltean,et al.  Evolving Evolutionary Algorithms Using Linear Genetic Programming , 2005, Evolutionary Computation.

[131]  Franz Rothlauf,et al.  Representations for genetic and evolutionary algorithms (2. ed.) , 2006 .

[132]  Anikó Ekárt,et al.  Maintaining the Diversity of Genetic Programs , 2002, EuroGP.

[133]  Melanie Mitchell,et al.  The royal road for genetic algorithms: Fitness landscapes and GA performance , 1991 .

[134]  Lee Spector,et al.  Genetic Programming and Autoconstructive Evolution with the Push Programming Language , 2002, Genetic Programming and Evolvable Machines.

[135]  U. Alon,et al.  Spontaneous evolution of modularity and network motifs. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[136]  Leonardo Vanneschi,et al.  Crossover-Based Tree Distance in Genetic Programming , 2008, IEEE Transactions on Evolutionary Computation.

[137]  Erik D. Goodman,et al.  The royal tree problem, a benchmark for single and multiple population genetic programming , 1996 .

[138]  Riccardo Poli,et al.  Theoretical results in genetic programming: the next ten years? , 2010, Genetic Programming and Evolvable Machines.

[139]  Michael O'Neill,et al.  Semantic Similarity Based Crossover in GP: The Case for Real-Valued Function Regression , 2009, Artificial Evolution.

[140]  Riccardo Poli,et al.  Genetic Programming Bloat with Dynamic Fitness , 1998, EuroGP.

[141]  William B. Langdon,et al.  A Many Threaded CUDA Interpreter for Genetic Programming , 2010, EuroGP.

[142]  David C. Wedge,et al.  Rapid prediction of optimum population size in genetic programming using a novel genotype -: fitness correlation , 2008, GECCO '08.

[143]  John R. Koza,et al.  A genetic approach to the truck backer upper problem and the inter-twined spiral problem , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[144]  Peter Nordin,et al.  Introns in Nature and in Simulated Structure Evolution , 1997, BCEC.

[145]  Riccardo Poli,et al.  Fitness-proportional negative slope coefficient as a hardness measure for genetic algorithms , 2007, GECCO '07.

[146]  Leonardo Vanneschi,et al.  Theory and practice for efficient genetic programming , 2004 .

[147]  John R. Woodward,et al.  Modularity in Genetic Programming , 2003, EuroGP.

[148]  Anthony Brabazon,et al.  Constant creation in grammatical evolution , 2007 .