Enabling grasping of unknown objects through a synergistic use of edge and surface information

Grasping unknown objects based on visual input, where no a priori knowledge about the objects is used, is a challenging problem. In this paper, we present an Early Cognitive Vision system that builds a hierarchical representation based on edge and texture information which provides a sparse but powerful description of the scene. Based on this representation, we generate contour-based and surface-based grasps. We test our method in two real-world scenarios, as well as on a vision-based grasping benchmark providing a hybrid scenario using real-world stereo images as input and a simulator for extensive and repetitive evaluation of the grasps. The results show that the proposed method is able to generate successful grasps, and in particular that the contour and surface information are complementary for the task of grasping unknown objects. This allows for dealing with rather complex scenes.

[1]  O. Reiser,et al.  Principles Of Gestalt Psychology , 1936 .

[2]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[3]  Van-Duc Nguyen,et al.  Constructing Stable Grasps , 1989, Int. J. Robotics Res..

[4]  B. Faverjon,et al.  On computing three-finger force-closure grasps of polygonal objects , 1991 .

[5]  G. Farin.  NURB curves and surfaces: from projective geometry to practical use , 1995 .

[6]  Karun B. Shimoga,et al.  Robot Grasp Synthesis Algorithms: A Survey , 1996, Int. J. Robotics Res..

[7]  Henrik I. Christensen,et al.  Automatic grasp planning using shape primitives , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[8]  Gerd Hirzinger,et al.  Grasping the dice by dicing the grasp , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[9]  Antonio Morales Escrig.  Learning to predict grasp reliability with a multifinger robot hand by using visual features , 2004 .

[10]  Peter K. Allen,et al.  An SVM learning approach to robotic grasping , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[11]  Peter K. Allen,et al.  Graspit! A versatile simulator for robotic grasping , 2004, IEEE Robotics & Automation Magazine.

[12]  Robert B. Fisher,et al.  Visual quality measures for Characterizing Planar robot grasps , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[13]  Giulio Sandini,et al.  Exploring the world through grasping: a developmental approach , 2005, 2005 International Symposium on Computational Intelligence in Robotics and Automation.

[14]  Antonio Morales,et al.  Vision-based three-finger grasp synthesis constrained by hand geometry , 2006, Robotics Auton. Syst..

[15]  Peter K. Allen,et al.  Grasp Planning via Decomposition Trees , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[16]  Danica Kragic,et al.  Birth of the Object: Detection of Objectness and Extraction of Object Shape through Object-Action complexes , 2008, Int. J. Humanoid Robotics.

[17]  Claire Dune,et al.  Active rough shape estimation of unknown objects , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Anis Sahbani,et al.  Handling Objects by Their Handles , 2008 .

[19]  Jing Xiao,et al.  Efficient and effective grasping of novel objects through learning and adapting a knowledge base , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[21]  Mario Richtsfeld,et al.  Grasping of Unknown Objects from a Table Top , 2008 .

[22]  Danica Kragic,et al.  Minimum volume bounding box decomposition for shape approximation in robot grasping , 2008, 2008 IEEE International Conference on Robotics and Automation.

[23]  Dirk Kraft,et al.  Learning to grasp unknown objects based on 3D edge information , 2009, 2009 IEEE International Symposium on Computational Intelligence in Robotics and Automation - (CIRA).

[24]  Justus H. Piater,et al.  A Probabilistic Framework for 3D Visual Object Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Matei T. Ciocarlie,et al.  Hand Posture Subspaces for Dexterous Robotic Grasping , 2009, Int. J. Robotics Res..

[26]  Sinan Kalkan,et al.  Using multi-modal 3D contours and their relations for vision and robotics , 2010, J. Vis. Commun. Image Represent..

[27]  Danica Kragic,et al.  A strategy for grasping unknown objects based on co-planarity and colour information , 2010, Robotics Auton. Syst..

[28]  Danica Kragic,et al.  Active 3D scene segmentation and detection of unknown objects , 2010, 2010 IEEE International Conference on Robotics and Automation.

[29]  Quoc V. Le,et al.  Grasping novel objects with depth segmentation , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  Danica Kragic,et al.  Learning grasping points with shape context , 2010, Robotics Auton. Syst..

[31]  Danica Kragic,et al.  Early Cognitive Vision as a Frontend for Cognitive Systems , 2010 .

[32]  Florentin Wörgötter,et al.  Disambiguating Multi–Modal Scene Representations Using Perceptual Grouping Constraints , 2010, PloS one.

[33]  Jimmy A. Jørgensen,et al.  RobWorkSim - an Open Simulator for Sensor based Grasping , 2010, ISR/ROBOTIK.

[34]  Florentin Wörgötter,et al.  Visual Primitives: Local, Condensed, Semantically Rich Visual Descriptors and their Applications in Robotics , 2010, Int. J. Humanoid Robotics.

[35]  Advait Jain,et al.  EL-E: an assistive mobile manipulator that autonomously fetches objects from flat surfaces , 2010, Auton. Robots.

[36]  Laura Fernández Gallardo,et al.  Detection of parametrized 3-D primitives from stereo for robotic grasping , 2011, 2011 15th International Conference on Advanced Robotics (ICAR).

[37]  Mila Popovic.  Grasping Unknown Objects in an Early Cognitive Vision System , 2011 .

[38]  Oliver Kroemer,et al.  Learning grasp affordance densities , 2011, Paladyn J. Behav. Robotics.

[39]  Jimmy A. Jørgensen,et al.  Grasping unknown objects using an Early Cognitive Vision system for general scene understanding , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[40]  Jimmy A. Jørgensen,et al.  Assessing Grasp Stability Based on Learning and Haptic Data , 2011, IEEE Transactions on Robotics.

[41]  Henrik Gordon Petersen,et al.  Industrial Robot : An International Journal Ring on the hook : placing a ring on a moving and pendulating hook based on visual input , 2016 .

[42]  Jimmy A. Jørgensen,et al.  VisGraB: A benchmark for vision-based grasping , 2012, Paladyn J. Behav. Robotics.