AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity

We present an improved method for symbolic regression that seeks to fit data to formulas that are Pareto-optimal, in the sense of having the best accuracy for a given complexity. It improves on the previous state-of-the-art by typically being orders of magnitude more robust toward noise and bad data, and also by discovering many formulas that stumped previous methods. We develop a method for discovering generalized symmetries (arbitrary modularity in the computational graph of a formula) from gradient properties of a neural network fit. We use normalizing flows to generalize our symbolic regression method to probability distributions from which we only have samples, and employ statistical hypothesis testing to accelerate robust brute-force search.

[1]  Max Tegmark,et al.  Symbolic Pregression: Discovering Physical Laws from Raw Distorted Video , 2020, Physical review. E.

[2]  Kazumi Saito,et al.  Robust Induction of Process Models from Time-Series Data , 2003, ICML.

[3]  Kalyan Veeramachaneni,et al.  Building Predictive Models via Feature Synthesis , 2015, GECCO.

[4]  Stuart J. Russell,et al.  Research Priorities for Robust and Beneficial Artificial Intelligence , 2015, AI Mag..

[5]  Finale Doshi-Velez,et al.  Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models , 2016, ArXiv.

[6]  Friedrich Sommer,et al.  Comment on the article "Distilling free-form natural laws from experimental data" , 2012 .

[7]  John Schulman,et al.  Concrete Problems in AI Safety , 2016, ArXiv.

[8]  Pat Langley,et al.  Heuristic Induction of Rate-Based Process Models , 2015, AAAI.

[9]  Xi Zhu,et al.  Phillips-Inspired Machine Learning for Band Gap and Exciton Binding Energy Prediction. , 2019, The journal of physical chemistry letters.

[10]  Zohar Ringel,et al.  Mutual information, neural networks and the renormalization group , 2017, ArXiv.

[11]  Iain Murray,et al.  Neural Spline Flows , 2019, NeurIPS.

[12]  Mark A. Pitt,et al.  Advances in Minimum Description Length: Theory and Applications , 2005 .

[13]  Dominic P. Searson,et al.  GPTIPS: An Open Source Genetic Programming Toolbox For Multigene Symbolic Regression , 2010 .

[14]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[15]  Bryan C. Daniels,et al.  Automated adaptive inference of phenomenological dynamical models , 2015, Nature Communications.

[16]  Steven L Brunton,et al.  Sparse identification of nonlinear dynamics for rapid model recovery. , 2018, Chaos.

[17]  LEARNS OLD TRICKS,et al.  A new dog learns old tricks: RL finds classic optimization algorithms , 2018, ICLR.

[18]  Boyang Li,et al.  Game Engine Learning from Video , 2017, IJCAI.

[19]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[20]  James P. Crutchfield,et al.  Equations of Motion from a Data Series , 1987, Complex Syst..

[21]  Prafulla Dhariwal,et al.  Glow: Generative Flow with Invertible 1x1 Convolutions , 2018, NeurIPS.

[22]  Samy Bengio,et al.  Density estimation using Real NVP , 2016, ICLR.

[23]  Ivan Kobyzev,et al.  Normalizing Flows: An Introduction and Review of Current Methods , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Max Tegmark,et al.  AI Feynman: A physics-inspired method for symbolic regression , 2019, Science Advances.

[25]  Renáta Dubcáková,et al.  Eureqa: software review , 2011, Genetic Programming and Evolvable Machines.

[26]  Tom Rodden,et al.  Principles of robotics: regulating robots in the real world , 2017, Connect. Sci..

[27]  Ekaterina Vladislavleva,et al.  Separating the wheat from the chaff: on feature selection and feature importance in regression random forests and symbolic regression , 2011, GECCO.

[28]  Saso Dzeroski,et al.  Discovering dynamics: From inductive logic programming to machine discovery , 1993, Journal of Intelligent Information Systems.

[29]  Randall K. McRee,et al.  Symbolic regression using nearest neighbor indexing , 2010, GECCO '10.

[30]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[31]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[32]  Elizabeth Bradley,et al.  Reasoning about nonlinear system identification , 2001, Artif. Intell..

[33]  Michael D. Schmidt,et al.  Automated refinement and inference of analytical models for metabolic networks , 2011, Physical biology.

[34]  Marcus Hutter,et al.  A Theory of Universal Artificial Intelligence based on Algorithmic Complexity , 2000, ArXiv.

[35]  Max Tegmark,et al.  Toward an artificial intelligence physicist for unsupervised learning. , 2019, Physical review. E.