Visual Grasp Affordance Localization in Point Clouds Using Curved Contact Patches

Detecting affordances on objects is one of the main open problems in robotic manipulation. This paper presents a new method to represent and localize grasp affordances as bounded curved contact patches (paraboloids) of the size of the robotic hand. In particular, given a three-dimensional (3D) point cloud from a range sensor, a set of potential grasps is localized on a detected object by a fast contact patch fitting and validation process. For the object detection, three standard methods from the literature are used and compared. The potential grasps on the object are then refined to a single affordance using their shape (size and curvature) and pose (reachability and minimum torque effort) properties, with respect to the robot and the manipulation task. We apply the proposed method to a circular valve turning task, verifying the ability to accurately and rapidly localize grasp affordances, under significant uncertainty in the environment. We experimentally validate the method with the humanoid robot COMAN on 10 circular control valves fixed on a wall, from five different viewpoints and robot poses for each valve. We compare the reliability of the introduced local grasp affordances method to the baseline that relies only on object detection, illustrating the superiority of ours for the valve turning task.

[1]  Nassir Navab,et al.  Adaptive neighborhood selection for real-time surface normal estimation from organized point cloud data using integral images , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Xiangyang Zhu,et al.  Dynamic Grasp Recognition Using Time Clustering, Gaussian Mixture Models and Hidden Markov Models , 2008, ICIRA.

[3]  Nikolaos G. Tsagarakis,et al.  Detecting object affordances with Convolutional Neural Networks , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[4]  Nikolaos G. Tsagarakis,et al.  A manipulation framework for compliant humanoid COMAN: Application to a valve turning task , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[5]  Michael Beetz,et al.  Detecting and segmenting objects for mobile manipulation , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[6]  A. Chemero.  An Outline of a Theory of Affordances , 2003, How Shall Affordances be Refined? Four Perspectives.

[7]  Marsette Vona,et al.  Curved surface contact patches with quantified uncertainty , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Jared Glover,et al.  Bingham procrustean alignment for object detection in clutter , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Danica Kragic,et al.  Learning a dictionary of prototypical grasp-predicting parts from grasping experience , 2013, 2013 IEEE International Conference on Robotics and Automation.

[10]  Abhinav Gupta,et al.  Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Kenneth Y. Goldberg,et al.  Cloud-based robot grasping with the google object recognition engine , 2013, 2013 IEEE International Conference on Robotics and Automation.

[12]  G Tsagarakis Nikos,et al.  Uncertainty analysis for curved surface contact patches , 2016 .

[13]  Manuel G. Catalano,et al.  Adaptive synergies for a humanoid robot hand , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[14]  Honglak Lee,et al.  Deep learning for detecting robotic grasps , 2013, Int. J. Robotics Res..

[15]  J. Gibson.  The Ecological Approach to Visual Perception , 1979 .

[16]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[17]  Nikolaos G. Tsagarakis,et al.  An affordance-based pilot interface for high-level control of humanoid robots in supervised autonomy , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[18]  Robert Platt,et al.  Using Geometry to Detect Grasp Poses in 3D Point Clouds , 2015, ISRR.

[19]  Ashutosh Saxena,et al.  Efficient grasping from RGBD images: Learning using a new rectangle representation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[20]  Sukhan Lee,et al.  Dual redundant arm configuration optimization with task-oriented dual arm manipulability , 1989, IEEE Trans. Robotics Autom..

[21]  Danica Kragic,et al.  Grasp Recognition for Programming by Demonstration , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[22]  Rainer Palm,et al.  Recognition of human grasps by time-clustering and fuzzy modeling , 2009, Robotics Auton. Syst..

[23]  Nikolaos G. Tsagarakis,et al.  A compact soft actuator unit for small scale human friendly robots , 2009, 2009 IEEE International Conference on Robotics and Automation.

[24]  Robert Platt,et al.  Localizing Handle-Like Grasp Affordances in 3D Point Clouds , 2014, ISER.

[25]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[26]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[27]  Marsette Vona,et al.  Bio-inspired rough terrain contact patch perception , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Dieter Fox,et al.  Interactive singulation of objects from a pile , 2012, 2012 IEEE International Conference on Robotics and Automation.

[29]  Nikolaos G. Tsagarakis,et al.  Upper-body impedance control with variable stiffness for a door opening task , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[30]  Reinhard Klein,et al.  Efficient RANSAC for Point‐Cloud Shape Detection , 2007, Comput. Graph. Forum.

[31]  J. Andrew Bagnell,et al.  Clearing a pile of unknown objects using interactive perception , 2013, 2013 IEEE International Conference on Robotics and Automation.

[32]  Henrik I. Christensen,et al.  Efficient Organized Point Cloud Segmentation with Connected Components , 2013 .

[33]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[34]  Don Ray Murray,et al.  Patchlets: Representing Stereo Vision Data with Surface Elements , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[35]  Oussama Khatib,et al.  Grasping with application to an autonomous checkout robot , 2011, 2011 IEEE International Conference on Robotics and Automation.

[36]  Dimitrios Kanoulas,et al.  Curved Surface Patches for Rough Terrain Perception , 2014, ArXiv.

[37]  Federico Tombari,et al.  Unique Signatures of Histograms for Local Surface Description , 2010, ECCV.

[38]  Stefano Chiaverini,et al.  Singularity-robust task-priority redundancy resolution for real-time kinematic control of robot manipulators , 1997, IEEE Trans. Robotics Autom..

[39]  Markus Vincze,et al.  Empty the basket - a shape based learning approach for grasping piles of unknown objects , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[40]  Marsette Vona,et al.  Sparse surface modeling with curved patches , 2013, 2013 IEEE International Conference on Robotics and Automation.

[41]  S. Gruber,et al.  Robot hands and the mechanics of manipulation , 1987, Proceedings of the IEEE.

[42]  Manuel G. Catalano,et al.  Adaptive synergies for the design and control of the Pisa/IIT SoftHand , 2014, Int. J. Robotics Res..

[43]  Nikolaos G. Tsagarakis,et al.  COMpliant huMANoid COMAN: Optimal joint stiffness tuning for modal frequency control , 2013, 2013 IEEE International Conference on Robotics and Automation.

[44]  Sergey Levine,et al.  Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection , 2016, Int. J. Robotics Res..

[45]  Edwin Olson,et al.  AprilTag: A robust and flexible visual fiducial system , 2011, 2011 IEEE International Conference on Robotics and Automation.

[46]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .