Learning Graspability of Unknown Objects via Intrinsic Motivation

Interacting with unknown objects, and learning and producing effective grasping procedures in particular, are challenging problems for robots. This paper proposes an intrinsically motivated reinforcement learning mechanism for learning to grasp uknown objects. The mechanism uses frustration to determine when grasping of an object is not possible. The critical threshold of frustration is dynamically regulated by impulsiveness of the robot. Here, the artificial emotions regulate the learning rate according to the current task and performance of the robot. The proposed mechanism is tested in a real world scenario where the robot, using the grasp pairs generated in simulation, has to learn which objects are graspable. The results shows that the robot equipped with frustration and impulsiveness learns faster than the robot with standard action selection strategies providing some evidence that the use of artificial emotions can improve the learning time.

[1]  Aude Billard,et al.  Biologically Inspired Multimodal Integration: Interferences in a Human-Robot Interaction Game , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[3]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[4]  Jan Peters,et al.  Learning motor primitives for robotics , 2009, 2009 IEEE International Conference on Robotics and Automation.

[5]  Dan Ding,et al.  The synthesis of 3-D form-closure grasps , 2000, Robotica.

[6]  Danica Kragic,et al.  Minimum volume bounding box decomposition for shape approximation in robot grasping , 2008, 2008 IEEE International Conference on Robotics and Automation.

[7]  Jan Peters,et al.  Machine Learning for motor skills in robotics , 2008, Künstliche Intell..

[8]  N. Kruger,et al.  Learning object-specific grasp affordance densities , 2009, 2009 IEEE 8th International Conference on Development and Learning.

[9]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[10]  Pierre-Yves Oudeyer,et al.  Maturationally-constrained competence-based intrinsically motivated learning , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[11]  Oliver Kroemer,et al.  Learning robot tactile sensing for object manipulation , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Hideki Hashimoto,et al.  Dextrous hand grasping force optimization , 1996, IEEE Trans. Robotics Autom..

[13]  Henrik I. Christensen,et al.  Efficient Organized Point Cloud Segmentation with Connected Components , 2013 .

[14]  Paul T. P. Wong,et al.  Frustration, exploration, and learning. , 1979 .

[15]  Robert Platt.  Learning grasp strategies composed of contact relative motions , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[16]  Hulya Yalcin,et al.  Scene Interpretation for Lifelong Robot Learning , 2014 .

[17]  Minoru Asada,et al.  Between Frustration and Elation: Sense of Control Regulates the lntrinsic Motivation for Motor Learning , 2011, Lifelong Learning.

[18]  Jürgen Schmidhuber,et al.  A possibility for implementing curiosity and boredom in model-building neural controllers , 1991 .

[19]  Antonio Bicchi,et al.  On the Closure Properties of Robotic Grasping , 1995, Int. J. Robotics Res..

[20]  Kenji Doya,et al.  Finding intrinsic rewards by embodied evolution and constrained reinforcement learning , 2008, Neural Networks.

[21]  Douglas B. Lenat,et al.  AM, an artificial intelligence approach to discovery in mathematics as heuristic search , 1976 .

[22]  Cornelius J. König,et al.  Integrating Theories of Motivation , 2006 .

[23]  Christos Dimitrakakis,et al.  Rollout sampling approximate policy iteration , 2008, Machine Learning.