Pouring Skills with Planning and Learning Modeled from Human Demonstrations

We explore how to represent, plan and learn robot pouring. This is a case study of a complex task that has many variations and involves manipulating non-rigid materials such as liquids and granular substances. Variations of pouring we consider are the type of pouring (such as pouring into a glass or spreading a sauce on an object), material, container shapes, initial poses of containers and target amounts. The robot learns to select appropriate behaviors from a library of skills, such as tipping, shaking and tapping, to pour a range of materials from a variety of containers. The robot also learns to select behavioral parameters. Planning methods are used to adapt skills for some variations such as initial poses of containers. We show using simulation and experiments on a PR2 robot that our pouring behavior model is able to plan and learn to handle a wide variety of pouring tasks. This case study is a step towards enabling humanoid robots to perform tasks of daily living.

[1]  S. LaValle.  Rapidly-exploring random trees : a new tool for path planning , 1998 .

[2]  Leslie Pack Kaelbling,et al.  Integrated task and motion planning in belief space , 2013, Int. J. Robotics Res..

[3]  Bruno Castro da Silva,et al.  Learning Parameterized Skills , 2012, ICML.

[4]  Aude Billard,et al.  Online learning of varying stiffness through physical human-robot interaction , 2012, 2012 IEEE International Conference on Robotics and Automation.

[5]  Stefanie Tellex,et al.  Interpreting and Executing Recipes with a Cooking Robot , 2012, ISER.

[6]  TamosiunaiteMinija,et al.  Learning to pour with a robot arm combining goal and shape learning for dynamic movement primitives , 2011 .

[7]  Oliver Kroemer,et al.  Combining active learning and reactive control for robot grasping , 2010, Robotics Auton. Syst..

[8]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[9]  Oliver Kroemer,et al.  A kernel-based approach to direct action perception , 2012, 2012 IEEE International Conference on Robotics and Automation.

[10]  Darwin G. Caldwell,et al.  Robot motor skill coordination with EM-based Reinforcement Learning , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Pieter Abbeel,et al.  Cloth grasp point detection based on multiple-view geometric cues with application to robotic towel folding , 2010, 2010 IEEE International Conference on Robotics and Automation.

[12]  Jochen J. Steil,et al.  Task-level imitation learning using variance-based movement optimization , 2009, 2009 IEEE International Conference on Robotics and Automation.

[13]  Oliver Kroemer,et al.  Generalizing pouring actions between objects using warped parameters , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[14]  Jan Peters,et al.  Nonamemanuscript No. (will be inserted by the editor) Reinforcement Learning to Adjust Parametrized Motor Primitives to , 2011 .

[15]  Carme Torras,et al.  Force-based robot learning of pouring skills using parametric hidden Markov models , 2013, 9th International Workshop on Robot Motion and Control.

[16]  Nicholas Roy,et al.  E-Graphs: Bootstrapping Planning with Experience Graphs , 2013 .

[17]  Aude Billard,et al.  Robot learning by demonstration , 2013, Scholarpedia.

[18]  Christopher G. Atkeson,et al.  Learning from observation using primitives , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[19]  Stefan Schaal,et al.  Learning and generalization of motor skills by learning from demonstration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[20]  Ales Ude,et al.  Learning to pour with a robot arm combining goal and shape learning for dynamic movement primitives , 2011, Robotics Auton. Syst..

[21]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[22]  J. Andrew Bagnell,et al.  Reinforcement Planning: RL for optimal planners , 2012, 2012 IEEE International Conference on Robotics and Automation.

[23]  Tsukasa Ogasawara,et al.  Learning strategy fusion to acquire dynamic motion , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[24]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.