Intrinsic Motivations Drive Learning of Eye Movements: An Experiment with Human Adults

Intrinsic motivations drive the acquisition of knowledge and skills on the basis of novel or surprising stimuli or the pleasure to learn new skills. In so doing, they are different from extrinsic motivations that are mainly linked to drives that promote survival and reproduction. Intrinsic motivations have been implicitly exploited in several psychological experiments but, due to the lack of proper paradigms, they are rarely a direct subject of investigation. This article investigates how different intrinsic motivation mechanisms can support the learning of visual skills, such as “foveate a particular object in space”, using a gaze contingency paradigm. In the experiment participants could freely foveate objects shown in a computer screen. Foveating each of two “button” pictures caused different effects: one caused the appearance of a simple image (blue rectangle) in unexpected positions, while the other evoked the appearance of an always-novel picture (objects or animals). The experiment studied how two possible intrinsic motivation mechanisms might guide learning to foveate one or the other button picture. One mechanism is based on the sudden, surprising appearance of a familiar image at unpredicted locations, and a second one is based on the content novelty of the images. The results show the comparative effectiveness of the mechanism based on image novelty, whereas they do not support the operation of the mechanism based on the surprising location of the image appearance. Interestingly, these results were also obtained with participants that, according to a post experiment questionnaire, had not understood the functions of the different buttons suggesting that novelty-based intrinsic motivation mechanisms might operate even at an unconscious level.

[1]  Eric R. Ziegel,et al.  Probability and Statistics for Engineering and the Sciences , 2004, Technometrics.

[2]  J. Lisman,et al.  The Hippocampal-VTA Loop: Controlling the Entry of Information into Long-Term Memory , 2005, Neuron.

[3]  Marco Mirolli,et al.  Functions and Mechanisms of Intrinsic Motivations , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[4]  B. Velichkovsky,et al.  Distractor effect and saccade amplitudes: Further evidence on different modes of processing in free exploration of visual images , 2009 .

[5]  Scott P. Johnson,et al.  Development of object concepts in infancy: Evidence for early learning in an eye-tracking paradigm , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[6]  Gianluca Baldassarre,et al.  What are intrinsic motivations? A biological perspective , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[7]  E. Deci,et al.  Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. , 2000, Contemporary educational psychology.

[8]  Claes von Hofsten,et al.  Action in development. , 2007, Developmental science.

[9]  Richard L. Lewis,et al.  Intrinsically Motivated Reinforcement Learning: An Evolutionary Perspective , 2010, IEEE Transactions on Autonomous Mental Development.

[10]  Pierre-Yves Oudeyer,et al.  Active learning of inverse models with intrinsically motivated goal exploration in robots , 2013, Robotics Auton. Syst..

[11]  LouAnn Gerken,et al.  Infants avoid 'labouring in vain' by attending more to learnable than unlearnable linguistic patterns. , 2011, Developmental science.

[12]  T. Power.  Play and Exploration in Children and Animals , 1999 .

[13]  D. Kumaran,et al.  Which computational mechanisms operate in the hippocampus during novelty detection? , 2007, Hippocampus.

[14]  G. Baldassarre,et al.  Functions and Mechanisms of Intrinsic Motivations The Knowledge Versus Competence Distinction , 2012 .

[15]  R. W. White.  Motivation reconsidered: the concept of competence. , 1959, Psychological review.

[16]  D. Berlyne.  Curiosity and exploration. , 1966, Science.

[17]  Francesco Mannella,et al.  Intrinsically motivated action-outcome learning and goal-based action recall: a system-level bio-constrained computational model. , 2013, Neural networks : the official journal of the International Neural Network Society.

[18]  Matthew Schlesinger,et al.  Image free-viewing as intrinsically-motivated exploration: estimating the learnability of center-of-gaze image samples in infants and adults , 2013, Front. Psychol..

[19]  D. Munoz,et al.  On the importance of the transient visual response in the superior colliculus , 2008, Current Opinion in Neurobiology.

[20]  D. A. Grant.  The latin square principle in the design and analysis of psychological experiments. , 1948, Psychological bulletin.

[21]  Jürgen Schmidhuber,et al.  Curious model-building control systems , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[22]  Richard N Aslin,et al.  How Infants View Natural Scenes Gathered From a Head-Mounted Camera , 2009, Optometry and vision science : official publication of the American Academy of Optometry.

[23]  Jochen Triesch,et al.  A bio-inspired attention model of anticipation in gaze-contingency experiments with infants , 2012, 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[24]  Michael C. Frank,et al.  Measuring the Development of Social Attention Using Free-Viewing. , 2012, Infancy : the official journal of the International Society on Infant Studies.

[25]  M. Morgan,et al.  A single "stopwatch" for duration estimation, a single "ruler" for size. , 2008, Journal of vision.

[26]  J. E. Albano,et al.  Visual-motor function of the primate superior colliculus. , 1980, Annual review of neuroscience.

[27]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[28]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[29]  Marco Mirolli,et al.  Intrinsically Motivated Learning in Natural and Artificial Systems , 2013 .

[30]  O. Hikosaka,et al.  Role of the basal ganglia in the control of purposive saccadic eye movements. , 2000, Physiological reviews.

[31]  Michael Erb,et al.  Exploring the visual world: The neural substrate of spatial orienting , 2006, NeuroImage.

[32]  D. Berlyne.  Conflict, arousal, and curiosity , 2014 .

[33]  S. Reder.  On-line monitoring of eye-position signals in contingent and noncontingent paradigms , 1973 .

[34]  T. Martin McGinnity,et al.  Novelty Detection as an Intrinsic Motivation for Cumulative Learning Robots , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[35]  Jochen Triesch,et al.  Infants in Control: Rapid Anticipation of Action Outcomes in a Gaze-Contingent Paradigm , 2012, PloS one.

[36]  A. Barto,et al.  Novelty or Surprise? , 2013, Front. Psychol..

[37]  Jürgen Schmidhuber,et al.  A possibility for implementing curiosity and boredom in model-building neural controllers , 1991 .

[38]  Pierre-Yves Oudeyer,et al.  What is Intrinsic Motivation? A Typology of Computational Approaches , 2007, Frontiers Neurorobotics.

[39]  LouAnn Gerken,et al.  Surprise! Infants consider possible bases of generalization for a single input example. , 2014, Developmental science.

[40]  Marco Mirolli,et al.  Which is the best intrinsic motivation signal for learning multiple skills? , 2013, Front. Neurorobot..

[41]  G. Pezzulo,et al.  Neuroscience and Biobehavioral Reviews the Contribution of Brain Sub-cortical Loops in the Expression and Acquisition of Action Understanding Abilities , 2022 .

[42]  G. H. Freeman.  Complete Latin Squares and Related Experimental Designs , 1979 .

[43]  T. Stanford,et al.  Subcortical loops through the basal ganglia , 2005, Trends in Neurosciences.

[44]  Marco Mirolli,et al.  Intrinsically Motivated Learning Systems: An Overview , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[45]  E. Deci,et al.  On happiness and human potentials: a review of research on hedonic and eudaimonic well-being. , 2001, Annual review of psychology.

[46]  Richard N. Aslin,et al.  The Goldilocks Effect: Human Infants Allocate Attention to Visual Sequences That Are Neither Too Simple Nor Too Complex , 2012, PloS one.

[47]  Marco Mirolli,et al.  Phasic dopamine as a prediction error of intrinsic and extrinsic reinforcements driving both action acquisition and reward maximization: A simulated robotic study , 2013, Neural Networks.

[48]  Xoana G. Troncoso,et al.  Saccades and microsaccades during visual fixation, exploration, and search: foundations for a common saccadic generator. , 2008, Journal of vision.

[49]  G. Rainer,et al.  Cognitive neuroscience: Neural mechanisms for detecting and remembering novel events , 2003, Nature Reviews Neuroscience.

[50]  Nuttapong Chentanez,et al.  Intrinsically Motivated Reinforcement Learning , 2004, NIPS.

[51]  G. Baldassarre,et al.  Evolving internal reinforcers for an intrinsically motivated reinforcement-learning robot , 2007, 2007 IEEE 6th International Conference on Development and Learning.

[52]  D. Balota,et al.  The interaction of contextual constraints and parafoveal visual information in reading , 1985, Cognitive Psychology.

[53]  P. Redgrave,et al.  Functional properties of the basal ganglia's re-entrant loop architecture: selection and reinforcement , 2011, Neuroscience.