Reactive reaching and grasping on a humanoid: Towards closing the action-perception loop on the iCub

We propose a system incorporating a tight integration between computer vision and robot control modules on a complex, high-DOF humanoid robot. Its functionality is showcased by having our iCub humanoid robot pick-up objects from a table in front of it. An important feature is that the system can avoid obstacles - other objects detected in the visual stream - while reaching for the intended target object. Our integration also allows for non-static environments, i.e. the reaching is adapted on-the-fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. Furthermore we show that this system can be used both in autonomous and tele-operation scenarios.

[1]  G. Hirzinger,et al.  The skeleton algorithm for self-collision avoidance of a humanoid manipulator , 2007, 2007 IEEE/ASME international conference on advanced intelligent mechatronics.

[2]  Mikhail Frank,et al.  Learning spatial object localization from vision on a humanoid robot , 2012 .

[3]  Jürgen Leitner,et al.  Autonomous learning of robust visual object detection and identification on a humanoid , 2012, 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[4]  R. Johansson,et al.  Development of human precision grip I: Basic coordination of force , 2004, Experimental Brain Research.

[5]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[7]  Jürgen Leitner,et al.  Transferring spatial perception between robots operating in a shared workspace , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Giovanni Maria Farinella,et al.  Computer Vision: Detection, Recognition and Reconstruction , 2010, Computer Vision: Detection, Recognition and Reconstruction.

[9]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[10]  Jürgen Leitner,et al.  The Modular Behavioral Environment for Humanoids and other Robots (MoBeE) , 2012, ICINCO.

[11]  Giorgio Metta,et al.  YARP: Yet Another Robot Platform , 2006 .

[12]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[13]  Danica Kragic,et al.  Vision for Robotics , 2009, Found. Trends Robotics.

[14]  Alexander Dietrich,et al.  Extensions to reactive self-collision avoidance for torque and position controlled humanoids , 2011, 2011 IEEE International Conference on Robotics and Automation.

[15]  R. A. Brooks,et al.  Intelligence without Representation , 1991, Artif. Intell..

[16]  Gregor Schöner,et al.  A dynamical systems approach to task-level system integration used to plan and control autonomous vehicle motion , 1992, Robotics Auton. Syst..

[17]  Giorgio Metta,et al.  Ensuring safety of policies learned by reinforcement: Reaching objects in the presence of obstacles with the iCub , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Jürgen Leitner,et al.  Curiosity driven reinforcement learning for motion planning on humanoids , 2014, Front. Neurorobot..

[19]  Jürgen Leitner,et al.  Task-relevant roadmaps: A framework for humanoid motion planning , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Paolo Pirjanian,et al.  The vSLAM Algorithm for Robust Localization and Mapping , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[21]  Seth Hutchinson,et al.  Visual Servo Control Part I: Basic Approaches , 2006 .

[22]  François Chaumette,et al.  Visual servo control. I. Basic approaches , 2006, IEEE Robotics & Automation Magazine.

[23]  J. Spencer,et al.  The Emerging Spatial Mind , 2007 .

[24]  Jürgen Leitner,et al.  Cartesian Genetic Programming for Image Processing , 2013 .

[25]  Nikolaos G. Tsagarakis,et al.  iCub: the design and realization of an open humanoid platform for cognitive and neuroscience research , 2007, Adv. Robotics.

[26]  Pieter Abbeel,et al.  Cloth grasp point detection based on multiple-view geometric cues with application to robotic towel folding , 2010, 2010 IEEE International Conference on Robotics and Automation.

[27]  Alois Knoll,et al.  An Efficient and Robust Real-Time Contour Tracking System , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[28]  Rüdiger Dillmann,et al.  Visual servoing for humanoid grasping and manipulation tasks , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[29]  Jörg Stückler,et al.  NimbRo@Home: Winning Team of the RoboCup@Home Competition 2012 , 2012, RoboCup.

[30]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[31]  David W. Murray,et al.  Simultaneous Localization and Map-Building Using Active Vision , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Larry H. Matthies,et al.  End-to-end dexterous manipulation with deliberate interactive estimation , 2012, 2012 IEEE International Conference on Robotics and Automation.

[33]  Jürgen Leitner,et al.  icVision: A modular vision system for cognitive robotics research , 2012 .

[34]  O. Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[35]  E. Torres-Jara,et al.  Challenges for Robot Manipulation in Human Environments , 2006 .

[36]  Charles C. Kemp,et al.  Challenges for robot manipulation in human environments [Grand Challenges of Robotics] , 2007, IEEE Robotics & Automation Magazine.

[37]  Giulio Sandini,et al.  The iCub humanoid robot: An open-systems platform for research in cognitive development , 2010, Neural Networks.