Differentiable Genetic Programming

We introduce the use of high order automatic differentiation, implemented via the algebra of truncated Taylor polynomials, in genetic programming. Using the Cartesian Genetic Programming encoding we obtain a high-order Taylor representation of the program output that is then used to back-propagate errors during learning. The resulting machine learning framework is called differentiable Cartesian Genetic Programming (dCGP). In the context of symbolic regression, dCGP offers a new approach to the long unsolved problem of constant representation in GP expressions. On several problems of increasing complexity we find that dCGP is able to find the exact form of the symbolic expression as well as the constants values. We also demonstrate the use of dCGP to solve a large class of differential equations and to find prime integrals of dynamical systems, presenting, in both cases, results that confirm the efficacy of our approach.

[1]  Nicholas B. Tufillaro,et al.  Swinging Atwood's Machine : integrability and dynamics , 1990 .

[2]  Chi Zhou,et al.  Using differential evolution for symbolic regression and numerical constant creation , 2008, GECCO '08.

[3]  Alex Graves,et al.  Neural Turing Machines , 2014, ArXiv.

[4]  Gul Muhammad Khan,et al.  Fast learning neural networks using Cartesian genetic programming , 2013, Neurocomputing.

[5]  Anne Auger,et al.  Performance evaluation of an advanced local search evolutionary algorithm , 2005, 2005 IEEE Congress on Evolutionary Computation.

[6]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[7]  Julian Francis Miller,et al.  Cartesian genetic programming encoded artificial neural networks: a comparison using three benchmarks , 2013, GECCO '13.

[8]  Leonardo Vanneschi,et al.  Open issues in genetic programming , 2010, Genetic Programming and Evolvable Machines.

[9]  Michael D. Schmidt,et al.  Symbolic Regression of Implicit Equations , 2010 .

[10]  A. Topchy,et al.  Faster genetic programming based on local gradient search of numeric leaf values , 2001 .

[11]  A. Borisov,et al.  Kovalevskaya Top and Generalizations of Integrable Systems , 2005, nlin/0504002.

[12]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[13]  Isaac E. Lagaris,et al.  Solving differential equations with genetic programming , 2006, Genetic Programming and Evolvable Machines.

[14]  Francesco Biscani.  Parallel sparse polynomial multiplication on modern hardware architectures , 2012, ISSAC.

[15]  P. W. Hawkes,et al.  Modern map methods in particle beam physics , 1999 .

[16]  Julian Francis Miller,et al.  Cartesian genetic programming , 2010, GECCO.

[17]  Sergio Gomez Colmenarejo,et al.  Hybrid computing using a neural network with dynamic external memory , 2016, Nature.

[18]  Martin Berz,et al.  COSY INFINITY Version 9 , 2006 .

[19]  A. Morselli,et al.  Differential algebra space toolbox for nonlinear uncertainty propagation in space dynamics , 2016 .