How Evolution May Work Through Curiosity-Driven Developmental Process

Infants' own activities create and actively select their learning experiences. Here we review recent models of embodied information seeking and curiosity-driven learning and show that these mechanisms have deep implications for development and evolution. We discuss how these mechanisms yield self-organized epigenesis with emergent ordered behavioral and cognitive developmental stages. We describe a robotic experiment that explored the hypothesis that progress in learning, in and for itself, generates intrinsic rewards: The robot learners probabilistically selected experiences according to their potential for reducing uncertainty. In these experiments, curiosity-driven learning led the robot learner to successively discover object affordances and vocal interaction with its peers. We explain how a learning curriculum adapted to the current constraints of the learning system automatically formed, constraining learning and shaping the developmental trajectory. The observed trajectories in the robot experiment share many properties with those in infant development, including a mixture of regularities and diversities in the developmental patterns. Finally, we argue that such emergent developmental structures can guide and constrain evolution, in particular with regard to the origins of language.

[1]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[2]  Wied Ruijssenaars,et al.  Encyclopedia of the Sciences of Learning , 2012 .

[3]  Jürgen Schmidhuber,et al.  Curious model-building control systems , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[4]  S. Gould Exaptation: A Crucial Tool for an Evolutionary Psychology , 1991 .

[5]  Pierre-Yves Oudeyer,et al.  On the Impact of Robotics in Behavioral and Cognitive Sciences: From Insect Navigation to Human Cognitive Development , 2010, IEEE Transactions on Autonomous Mental Development.

[6]  Csaba Szepesvári,et al.  Exploration-exploitation tradeoff using variance estimates in multi-armed bandits , 2009, Theor. Comput. Sci..

[7]  Pierre-Yves Oudeyer,et al.  Active choice of teachers, learning strategies and goals for a socially guided intrinsic motivation learner , 2012, Paladyn J. Behav. Robotics.

[8]  Linda B. Smith It's all connected: Pathways in visual object recognition and early noun learning. , 2013, The American psychologist.

[9]  Pierre-Yves Oudeyer,et al.  What is Intrinsic Motivation? A Typology of Computational Approaches , 2007, Frontiers Neurorobotics.

[10]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[11]  D. Oller The emergence of the speech capacity , 2000 .

[12]  D. Berlyne Conflict, arousal, and curiosity , 2014 .

[13]  S. M. Arnsten Intrinsic motivation. , 1990, The American journal of occupational therapy : official publication of the American Occupational Therapy Association.

[14]  Max Lungarella,et al.  Developmental Robotics , 2009, Encyclopedia of Artificial Intelligence.

[15]  R. Mishra Pierre-Yves Oudeyer Self-Organization in the Evolution of Speech , 2011, Cognitive Systems Research.

[16]  Pierre-Yves Oudeyer,et al.  Information-seeking, curiosity, and attention: computational and neural mechanisms , 2013, Trends in Cognitive Sciences.

[17]  Robert Lickliter,et al.  Rethinking Epigenesis and Evolution in Light of Developmental Science , 2009 .

[18]  G. Loewenstein The psychology of curiosity: A review and reinterpretation. , 1994 .

[19]  M. Csíkszentmihályi Flow: The Psychology of Optimal Experience , 1990 .

[20]  Pierre-Yves Oudeyer,et al.  The progress drive hypothesis: an interpretation of early imitation , 2007 .

[21]  Pierre-Yves Oudeyer,et al.  Guest Editorial Active Learning and Intrinsically Motivated Exploration in Robots: Advances and Challenges , 2010, IEEE Trans. Auton. Ment. Dev..

[22]  Pierre-Yves Oudeyer,et al.  In Search of the Neural Circuits of Intrinsic Motivation , 2007, Front. Neurosci..

[23]  Andrew G. Barto,et al.  Intrinsic Motivation and Reinforcement Learning , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[24]  Richard N. Aslin,et al.  The Goldilocks Effect: Human Infants Allocate Attention to Visual Sequences That Are Neither Too Simple Nor Too Complex , 2012, PloS one.

[25]  Lisa Riolo Three Month Old Infants Can Learn Task Specific Patterns of Interlimb Coordination , 1995 .

[26]  Christian Abry,et al.  Vocalize to localize , 2009 .

[27]  Pierre-Yves Oudeyer,et al.  Self-organization of early vocal development in infants and machines: the role of intrinsic motivation , 2014, Front. Psychol..

[28]  Gyöorgy Benedek,et al.  A Dynamic Systems Approach to Development: Applications , 1994, Journal of Cognitive Neuroscience.

[29]  Christian Abry,et al.  Primate Communication and Human Language: Vocalisation, gestures, imitation and deixis in humans and non-humans , 2011 .

[30]  Linda B. Smith,et al.  Joint Attention without Gaze Following: Human Infants and Their Parents Coordinate Visual Attention to Objects through Eye-Hand Coordination , 2013, PloS one.

[31]  P. L. Adams THE ORIGINS OF INTELLIGENCE IN CHILDREN , 1976 .

[32]  D. Hoang FLOW: The Psychology of Optimal Experience , 2018 .

[33]  G. Gottlieb Experiential Canalization of Behavioral Development: Theory. , 1991 .

[34]  J. Triesch,et al.  Watch the hands: infants can learn to follow gaze by seeing adults manipulate objects. , 2014, Developmental science.

[35]  T. Belpraeme,et al.  Imitation and Social Learning in Robots, Humans and Animals: Behavioural, Social and Communicative Dimensions , 2006 .

[36]  C. Breazeal,et al.  Experiments in socially guided exploration: lessons learned in building robots that learn with and without human teachers , 2008, Connect. Sci..

[37]  Richard L. Lewis,et al.  Intrinsically Motivated Reinforcement Learning: An Evolutionary Perspective , 2010, IEEE Transactions on Autonomous Mental Development.

[38]  Philip H. Mirvis Flow: The Psychology of Optimal Experience , 1991 .

[39]  Marco Mirolli,et al.  Intrinsically Motivated Learning in Natural and Artificial Systems , 2013 .

[40]  Bart de Boer,et al.  The Origins of Vowel Systems , 2001 .

[41]  D. Lewkowicz,et al.  A dynamic systems approach to the development of cognition and action. , 2007, Journal of cognitive neuroscience.

[42]  Linda B. Smith,et al.  The dynamic lift of developmental process. , 2007, Developmental science.

[43]  Pierre-Yves Oudeyer,et al.  Discovering communication , 2006, Connect. Sci..

[44]  Active Learning and Intrinsically Motivated Exploration in Robots : Advances and Challenges , 2010 .

[45]  J. Kagan Motives and development. , 1972, Journal of personality and social psychology.

[46]  Linda B. Smith,et al.  A Dynamic Systems Approach to the Development of Cognition and Action , 2007, Journal of Cognitive Neuroscience.

[47]  T. Power Play and Exploration in Children and Animals , 1999 .

[48]  Karl J. Friston,et al.  Perceptions as Hypotheses: Saccades as Experiments , 2012, Front. Psychology.

[49]  A. King,et al.  Settling nature and nurture into an ontogenetic niche. , 1987, Developmental psychobiology.

[50]  Manuel Lopes,et al.  Active Learning for Autonomous Intelligent Agents: Exploration, Curiosity, and Interaction , 2014, ArXiv.