Trying anyways: How ignoring the errors may help in learning new skills

Traditional view stresses the role of errors in the learning process. The result obtained from our experiment with older infants suggested that omitting the errors during learning can also be beneficial. We propose that a temporal decrease in learning from negative feedback could be an efficient mechanism behind infant learning new skills. Herein, we claim that disregarding the errors is tightly connected to the sense of control, and results from extremely high level of self-efficacy (overconfidence). Our preliminary results with a robot simulator serve as a proof-of-concept for our approach, and suggest a possible new route for constraints balancing exploration and exploitation in intrinsically motivated reinforcement learning.

[1]  Nathaniel Daw,et al.  Tonic Dopamine Modulates Exploitation of Reward Learning , 2010, Front. Behav. Neurosci..

[2]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[3]  J. Campos,et al.  Development of Autonomy: Role of Walking Onset and its Timing , 2008, Perceptual and motor skills.

[4]  S. Heckers,et al.  Abnormal Reward System Activation in Mania , 2008, Neuropsychopharmacology.

[5]  A. Barto,et al.  Intrinsic Motivation For Reinforcement Learning Systems , 2005 .

[6]  M. Frank,et al.  From reinforcement learning models to psychiatric and neurological disorders , 2011, Nature Neuroscience.

[7]  Douglas S. Blank,et al.  An Emergent Framework For Self-Motivation In Developmental Robotics , 2004 .

[8]  Pierre-Yves Oudeyer,et al.  Intrinsically motivated goal exploration for active motor learning in robots: A case study , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Daniel S Pine,et al.  Neural circuitry engaged during unsuccessful motor inhibition in pediatric bipolar disorder. , 2007, The American journal of psychiatry.

[10]  Juyang Weng,et al.  A theory for mentally developing robots , 2002, Proceedings 2nd International Conference on Development and Learning. ICDL 2002.

[11]  Pierre-Yves Oudeyer,et al.  Maturationally-constrained competence-based intrinsically motivated learning , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[12]  Masaki Ogino,et al.  Cognitive Developmental Robotics: A Survey , 2009, IEEE Transactions on Autonomous Mental Development.

[13]  Geert-Jan M. Kruijff,et al.  Curiosity-driven acquisition of sensorimotor concepts using memory-based active learning , 2009, 2008 IEEE International Conference on Robotics and Biomimetics.

[14]  M. Appelbaum,et al.  Affective reorganization in the infant, the mother, and the dyad: the role of upright locomotion and its timing. , 1995, Child development.

[15]  W. Schultz.  The Reward Signal of Midbrain Dopamine Neurons. , 1999, News in physiological sciences : an international journal of physiology produced jointly by the International Union of Physiological Sciences and the American Physiological Society.

[16]  Nuttapong Chentanez,et al.  Intrinsically Motivated Learning of Hierarchical Collections of Skills , 2004 .

[17]  A Scher,et al.  The Onset of Upright Locomotion and Night Wakings , 1996, Perceptual and motor skills.

[18]  P. Zelazo.  The development of walking: new findings and old assumptions. , 1983, Journal of motor behavior.

[19]  Sheri L. Johnson.  Mania and dysregulation in goal pursuit: a review. , 2005, Clinical psychology review.

[20]  D. Lewkowicz,et al.  A dynamic systems approach to the development of cognition and action. , 2007, Journal of cognitive neuroscience.

[21]  M. Jackson,et al.  Stimulation of prefrontal cortex at physiologically relevant frequencies inhibits dopamine release in the nucleus accumbens , 2001, Journal of neurochemistry.

[22]  D. Shapiro,et al.  Controlling ourselves, controlling our world. Psychology's role in understanding positive and negative consequences of seeking and gaining control. , 1996, The American psychologist.

[23]  Thomas E. Hazy,et al.  PVLV: the primary value and learned value Pavlovian learning algorithm. , 2007, Behavioral neuroscience.

[24]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[25]  Paul T. P. Wong,et al.  Frustration, exploration, and learning. , 1979 .

[26]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[27]  Janet B W Williams,et al.  Diagnostic and Statistical Manual of Mental Disorders , 2013 .

[28]  M. Delgado,et al.  Perceptions of moral character modulate the neural systems of reward during the trust game , 2005, Nature Neuroscience.

[29]  Donald A. Sofge,et al.  Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches , 1992 .

[30]  Pierre-Yves Oudeyer,et al.  What is Intrinsic Motivation? A Typology of Computational Approaches , 2007, Frontiers Neurorobotics.

[31]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[32]  D. Glahn,et al.  Impulsivity and bipolar disorder , 2007, European Neuropsychopharmacology.

[33]  S. Iyengar,et al.  Born to choose: the origins and value of the need for control , 2010, Trends in Cognitive Sciences.