A Curious Emergence of Reaching

In order to perform a reaching movement towards a moving target, an autonomously learning robot must first learn several transformations, such as motion detection, coordinate transformation between the camera and the arm and the inverse model of the arm. A curious reaching robot learns them better by performing the appropriate actions so as to expedite and improve their learning speed and accuracy. We implement a model of hierarchical curiosity loops in an autonomous active learning robot, whereby each loop converges to the optimal action that maximizes the robot’s learning of the appropriate transformation. It results in the emergence of unique behaviors that ultimately lead to the capability of reaching.

[1]  Stefan Schaal,et al.  Natural Actor-Critic , 2003, Neurocomputing.

[2]  R. Shadmehr Generalization as a behavioral window to the neural mechanisms of learning internal models. , 2004, Human movement science.

[3]  Jürgen Schmidhuber,et al.  Formal Theory of Fun and Creativity , 2010, ECML/PKDD.

[4]  Per Magne Knutsen,et al.  Orthogonal coding of object location , 2009, Trends in Neurosciences.

[5]  Giulio Sandini,et al.  Exploring the world through grasping: a developmental approach , 2005, 2005 International Symposium on Computational Intelligence in Robotics and Automation.

[6]  Shalabh Bhatnagar,et al.  Incremental Natural Actor-Critic Algorithms , 2007, NIPS.

[7]  Charles C. Kemp,et al.  What can I control? A framework for robot self-discovery , 2006 .

[8]  Tao Xiong,et al.  A combined SVM and LDA approach for classification , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[9]  Giulio Sandini,et al.  Autonomous learning of 3D reaching in a humanoid robot , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Per Magne Knutsen,et al.  Object localization with whiskers , 2008, Biological Cybernetics.

[11]  Benjamin S. Lankow,et al.  Toward an Integrated Approach to Perception and Action: Conference Report and Future Directions , 2011, Front. Syst. Neurosci..

[12]  Stewart W. Wilson,et al.  A Possibility for Implementing Curiosity and Boredom in Model-Building Neural Controllers , 1991 .

[13]  Juyang Weng,et al.  Developmental Robotics: Theory and Experiments , 2004, Int. J. Humanoid Robotics.

[14]  Goren Gordon,et al.  Hierarchical Exhaustive Construction of Autonomously Learning Networks , 2011 .

[15]  Nuttapong Chentanez,et al.  Intrinsically Motivated Learning of Hierarchical Collections of Skills , 2004 .

[16]  Shalabh Bhatnagar,et al.  Incremental natural-gradient actor-critic algorithms , 2007 .

[17]  Devendra P. Garg,et al.  Camera calibration and sensor fusion in an automated flexible manufacturing multi- robot work cell , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[18]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[19]  Ehud Ahissar,et al.  Reinforcement active learning hierarchical loops , 2011, The 2011 International Joint Conference on Neural Networks.

[20]  C. Kemp,et al.  What Can I Control ? : The Development of Visual Categories for a Robot ’ s Body and the World that it Influences , 2006 .

[21]  Ehud Ahissar,et al.  Hierarchical curiosity loops and active sensing , 2012, Neural Networks.