Accelerating evolution via egalitarian social learning

Social learning is an extension to evolutionary algorithms that enables agents to learn from observations of others in the population. Historically, social learning algorithms have employed a student-teacher model where the behavior of one or more high-fitness agents is used to train a subset of the remaining agents in the population. This paper presents ESL, an egalitarian model of social learning in which agents are not labeled as teachers or students, instead allowing any individual receiving a sufficiently high reward to teach other agents to mimic its recent behavior. We validate our approach through a series of experiments in a robot foraging domain, including comparisons of egalitarian social learning with baseline neuroevolution and a variant of student-teacher social learning. In a complex foraging task, ESL converges to near-optimal strategies faster than either benchmark approach, outperforming both by more than an order of magnitude. The results indicate that egalitarian social learning is a promising new paradigm for social learning in intelligent agents.

[1]  Wan-Chi Siu,et al.  A study of the Lamarckian evolution of recurrent neural networks , 2000, IEEE Trans. Evol. Comput..

[2]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[3]  R. Lewontin ‘The Selfish Gene’ , 1977, Nature.

[4]  Pablo Moscato,et al.  On Evolution, Search, Optimization, Genetic Algorithms and Martial Arts : Towards Memetic Algorithms , 1989 .

[5]  C. Ember Hierarchy in the Forest: The Evolution of Egalitarian Behavior , 2001 .

[6]  Godfrey C. Onwubolu,et al.  New optimization techniques in engineering , 2004, Studies in Fuzziness and Soft Computing.

[7]  R. Reynolds AN INTRODUCTION TO CULTURAL ALGORITHMS , 2008 .

[8]  M. Tomasello,et al.  Humans Have Evolved Specialized Skills of Social Cognition: The Cultural Intelligence Hypothesis , 2007, Science.

[9]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[10]  R. Byrne,et al.  Machiavellian intelligence : social expertise and the evolution of intellect in monkeys, apes, and humans , 1990 .

[11]  Kay E. Holekamp,et al.  Questioning the social intelligence hypothesis , 2007, Trends in Cognitive Sciences.

[12]  Domenico Parisi,et al.  Cultural Evolution in a Population of Neural Networks , 1997 .

[13]  N. Humphrey The Social Function of Intellect , 1976 .

[14]  A. Goldman,et al.  Mirror neurons and the simulation theory of mind-reading , 1998, Trends in Cognitive Sciences.

[15]  L. Buşoniu Evolutionary function approximation for reinforcement learning , 2006 .

[16]  Domenico Parisi,et al.  Cultural evolution in neural networks , 1997 .

[17]  P J Webros BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[18]  G. Simpson THE BALDWIN EFFECT , 1953 .

[19]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[20]  Domenico Parisi,et al.  Cultural Transmission Between and Within Generations , 2006, J. Artif. Soc. Soc. Simul..

[21]  A. E. Eiben,et al.  Social learning in Population-based Adaptive Systems , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[22]  Stefano Nolfi,et al.  Social learning and cultural evolution in embodied and situated agents , 2007, 2007 IEEE Symposium on Artificial Life.

[23]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[24]  Thomas Stützle,et al.  Incremental Social Learning in Particle Swarms , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Lawrence J. Fogel,et al.  Intelligent decision making through a simulation of evolution. , 1966 .

[26]  A. Dickson On Evolution , 1884, Science.

[27]  J. Stevenson The cultural origins of human cognition , 2001 .

[28]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[29]  Evert Haasdijk,et al.  Modeling Social Learning of Language and Skills , 2010, Artificial Life.