Meta-learning by the Baldwin effect

The scope of the Baldwin effect was recently called into question by two papers that closely examined the seminal work of Hinton and Nowlan. To this date there has been no demonstration of its necessity in empirically challenging tasks. Here we show that the Baldwin effect is capable of evolving few-shot supervised and reinforcement learning mechanisms, by shaping the hyperparameters and the initial parameters of deep learning algorithms. Furthermore it can genetically accommodate strong learning biases on the same set of problems as a recent machine learning algorithm called MAML "Model Agnostic Meta-Learning" which uses second-order gradients instead of evolution to learn a set of reference parameters (initial weights) that can allow rapid adaptation to tasks sampled from a distribution. Whilst in simple cases MAML is more data efficient than the Baldwin effect, the Baldwin effect is more general in that it does not require gradients to be backpropagated to the reference parameters or hyperparameters, and permits effectively any number of gradient updates in the inner loop. The Baldwin effect learns strong learning dependent biases, rather than purely genetically accommodating fixed behaviours in a learning independent manner.

[1]  J. Baldwin A New Factor in Evolution , 1896, The American Naturalist.

[2]  Geoffrey E. Hinton,et al.  How Learning Can Guide Evolution , 1996, Complex Syst..

[3]  David G. Stork,et al.  Evolution and Learning in Neural Networks: The Number and Distribution of Learning Trials Affect the Rate of Evolution , 1990, NIPS 1990.

[4]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[5]  Filippo Menczer,et al.  Maturation and the Evolution of Imitative Learning in Artificial Organisms , 1995, Adapt. Behav..

[6]  Peter D. Turney Cost-Sensitive Classification: Empirical Evaluation of a Hybrid Genetic Decision Tree Induction Algorithm , 1994, J. Artif. Intell. Res..

[7]  Peter D. Turney Myths and Legends of the Baldwin Effect , 2002, ICML 2002.

[8]  John Maynard Smith,et al.  Natural selection: when learning guides evolution , 1996 .

[9]  Jieyu Zhao,et al.  Simple Principles of Metalearning , 1996 .

[10]  Russell W. Anderson,et al.  How adaptive antibodies facilitate the evolution of natural antibodies , 1996, Immunology and cell biology.

[11]  Sebastian Thrun,et al.  Learning to Learn: Introduction and Overview , 1998, Learning to Learn.

[12]  Christian Goerick,et al.  Fast learning for problem classes using knowledge based network initialization , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[13]  John A. Bullinaria,et al.  The Evolution Of Variable Learning Rates , 2002, GECCO.

[14]  E. Jablonka,et al.  Evolution in Four Dimensions , 2005 .

[15]  Juan Julián Merelo Guervós,et al.  Lamarckian Evolution and the Baldwin Effect in Evolutionary Neural Networks , 2006, ArXiv.

[16]  Tom Schaul,et al.  Natural Evolution Strategies , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[17]  Keith L. Downing,et al.  The baldwin effect in developing neural networks , 2010, GECCO '10.

[18]  Joshua B. Tenenbaum,et al.  One shot learning of simple visual concepts , 2011, CogSci.

[19]  Tom Schaul,et al.  High dimensions and heavy tails for natural evolution strategies , 2011, GECCO '11.

[20]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Mauro Santos,et al.  Phenotypic plasticity, the Baldwin effect, and the speeding up of evolution: the computational roots of an illusion. , 2014, Journal of theoretical biology.

[22]  Gregory R. Koch,et al.  Siamese Neural Networks for One-Shot Image Recognition , 2015 .

[23]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[24]  Peter L. Bartlett,et al.  RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning , 2016, ArXiv.

[25]  Daan Wierstra,et al.  One-shot Learning with Memory-Augmented Neural Networks , 2016, ArXiv.

[26]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[27]  Zeb Kurth-Nelson,et al.  Learning to reinforcement learn , 2016, CogSci.

[28]  Hugo Larochelle,et al.  Optimization as a Model for Few-Shot Learning , 2016, ICLR.

[29]  J. Fontanari,et al.  The revival of the Baldwin effect , 2017, 1702.08411.

[30]  Max Jaderberg,et al.  Population Based Training of Neural Networks , 2017, ArXiv.

[31]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[32]  Larry Bull,et al.  The Evolution of Sex through the Baldwin Effect , 2016, Artificial Life.