Learning Objects and Grasp Affordances through Autonomous Exploration

We describe a system for autonomous learning of visual object representations and their grasp affordances on a robot-vision system. It segments objects by grasping and moving 3D scene features, and creates probabilistic visual representations for object detection, recognition and pose estimation, which are then augmented by continuous characterizations of grasp affordances generated through biased, random exploration. Thus, based on a careful balance of generic prior knowledge encoded in (1) the embodiment of the system, (2) a vision system extracting structurally rich information from stereo image sequences as well as (3) a number of built-in behavioral modules on the one hand, and autonomous exploration on the other hand, the system is able to generate object and grasping knowledge through interaction with its environment.

[1]  Markus Vincze,et al.  Efficient 3D Object Detection by Fitting Superquadrics to Range Image Data for Robot's Object Manipulation , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[2]  Alexander Stoytchev,et al.  Learning the Affordances of Tools Using a Behavior-Grounded Approach , 2006, Towards Affordance-Based Robot Control.

[3]  Danica Kragic,et al.  Minimum volume bounding box decomposition for shape approximation in robot grasping , 2008, 2008 IEEE International Conference on Robotics and Automation.

[4]  Florentin Wörgötter,et al.  Accumulated Visual Representation for Cognitive Vision , 2008, BMVC.

[5]  N. Kruger,et al.  Learning object-specific grasp affordance densities , 2009, 2009 IEEE 8th International Conference on Development and Learning.

[6]  Justus H. Piater,et al.  A Probabilistic Framework for 3D Visual Object Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Joachim Hertzberg,et al.  Towards Affordance-based Robot Control , 2008 .

[8]  Benjamin Kuipers,et al.  Bootstrap learning for object discovery , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[9]  Danica Kragic,et al.  Birth of the Object: Detection of Objectness and Extraction of Object Shape through Object-Action complexes , 2008, Int. J. Humanoid Robotics.

[10]  Maya Cakmak,et al.  To Afford or Not to Afford: A New Formalization of Affordances Toward Affordance-Based Robot Control , 2007, Adapt. Behav..

[11]  Markus Lappe,et al.  Biologically Motivated Multi-modal Processing of Visual Primitives , 2003 .

[12]  A. Stoytchev.  Toward Learning the Binding Affordances of Objects : A Behavior-Grounded Approach , 2022 .

[13]  A. Fagg,et al.  Learning Grasp Affordances Through Human Demonstration , 2008 .

[14]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[15]  Sukhan Lee,et al.  Recent progress in robotics : viable robotic service to human : an edition of the selected papers from the 13th International Conference on Advanced Robotics , 2008 .

[16]  Florentin Wörgötter,et al.  A Scene Representation Based on Multi-Modal 2D and 3D Features , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[17]  David G. Lowe,et al.  Fitting Parameterized Three-Dimensional Models to Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Geoffrey E. Hinton,et al.  Learning Generative Texture Models with extended Fields-of-Experts , 2009, BMVC.

[19]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[20]  E. Reed.  The Ecological Approach to Visual Perception , 1989 .

[21]  Vijay Kumar,et al.  Robotic grasping and contact: a review , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[22]  Giorgio Metta,et al.  Grounding vision through experimental manipulation , 2003, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[23]  Danica Kragic,et al.  Early reactive grasping with second order 3D feature relations , 2007 .

[24]  Nicolas Pugeault,et al.  Early cognitive vision: feedback mechanisms for the disambiguation of early visual representation , 2008 .