Perceiving, learning, and exploiting object affordances for autonomous pile manipulation

Autonomous manipulation in unstructured environments will enable a large variety of exciting and important applications. Despite its promise, autonomous manipulation remains largely unsolved. Even the most rudimentary manipulation task—such as removing objects from a pile—remains challenging for robots. We identify three major challenges that must be addressed to enable autonomous manipulation: object segmentation, action selection, and motion generation. These challenges become more pronounced when unknown man-made or natural objects are cluttered together in a pile. We present a system capable of manipulating unknown objects in such an environment. Our robot is tasked with clearing a table by removing objects from a pile and placing them into a bin. To that end, we address the three aforementioned challenges. Our robot perceives the environment with an RGB-D sensor, segmenting the pile into object hypotheses using non-parametric surface models. Our system then computes the affordances of each object, and selects the best affordance and its associated action to execute. Finally, our robot instantiates the proper compliant motion primitive to safely execute the desired action. For efficient and reliable action selection, we developed a framework for supervised learning of manipulation expertise. To verify the performance of our system, we conducted dozens of trials and report on several hours of experiments involving more than 1,500 interactions. The results show that our learning-based approach for pile manipulation outperforms a common sense heuristic as well as a random strategy, and is on par with human action selection.

[1]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[2]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[4]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[5]  Joan Serrat,et al.  Motion Segmentation from Feature Trajectories with Missing Data , 2007, IbPRIA.

[6]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[7]  René Vidal,et al.  Segmenting Motions of Different Types by Unsupervised Manifold Clustering , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Jianhua Wang,et al.  3D Motion Segmentation from Straight-Line Optical Flow , 2007, MCAM.

[9]  三嶋 博之 The theory of affordances , 2008 .

[10]  Rustam Stolkin,et al.  An EM/E-MRF algorithm for adaptive model based tracking in extremely poor visibility , 2008, Image Vis. Comput..

[11]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[12]  Oliver Brock,et al.  Manipulating articulated objects with interactive perception , 2008, 2008 IEEE International Conference on Robotics and Automation.

[13]  Oliver Brock,et al.  Learning to Manipulate Articulated Objects in Unstructured Environments Using a Grounded Relational Representation , 2008, Robotics: Science and Systems.

[14]  Siddhartha S. Srinivasa,et al.  CHOMP: Gradient optimization techniques for efficient motion planning , 2009, 2009 IEEE International Conference on Robotics and Automation.

[15]  Oliver Brock,et al.  Interactive segmentation for manipulation in unstructured environments , 2009, 2009 IEEE International Conference on Robotics and Automation.

[16]  Danica Kragic,et al.  Learning grasping affordance using probabilistic and ontological approaches , 2009, 2009 International Conference on Advanced Robotics.

[17]  Shao-Wen Yang,et al.  RANSAC matching: Simultaneous registration and segmentation , 2010, 2010 IEEE International Conference on Robotics and Automation.

[18]  C. J. Taylor,et al.  Segmentation and Analysis of RGB-D data , 2010 .

[19]  Marc Toussaint,et al.  Planning with Noisy Probabilistic Relational Rules , 2010, J. Artif. Intell. Res..

[20]  Quoc V. Le,et al.  Learning to grasp objects with multiple contact points , 2010, 2010 IEEE International Conference on Robotics and Automation.

[21]  James M. Rehg,et al.  Guided pushing for object singulation , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Martial Hebert,et al.  An integrated system for autonomous robotics manipulation , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Dieter Fox,et al.  Interactive singulation of objects from a pile , 2012, 2012 IEEE International Conference on Robotics and Automation.

[24]  Emre Ugur,et al.  Self-discovery of motor primitives and learning grasp affordances , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Gaurav S. Sukhatme,et al.  Using manipulation primitives for brick sorting in clutter , 2012, 2012 IEEE International Conference on Robotics and Automation.

[26]  J. Andrew Bagnell,et al.  Perceiving, learning, and exploiting object affordances for autonomous pile manipulation , 2013, Auton. Robots.

[27]  Oliver Kroemer,et al.  Probabilistic interactive segmentation for anthropomorphic robots in cluttered environments , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[28]  James M. Rehg,et al.  Learning contact locations for pushing and orienting unknown objects , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[29]  Zoltan-Csaba Marton,et al.  Tracking-based interactive segmentation of textureless objects , 2013, 2013 IEEE International Conference on Robotics and Automation.

[30]  J. Andrew Bagnell,et al.  Clearing a pile of unknown objects using interactive perception , 2013, 2013 IEEE International Conference on Robotics and Automation.

[31]  Nicholas Roy,et al.  Robust Object Grasping Using Force Compliant Motion Primitives , 2013 .