Autonomously Acquiring Instance-Based Object Models from Experience

A key aim of current research is to create robots that can reliably manipulate objects. However, in many applications, general-purpose object detection or manipulation is not required: the robot would be useful if it could recognize, localize, and manipulate the relatively small set of specific objects most important in that application, but do so with very high reliability. Instance-based approaches can achieve this high reliability but to work well, they require large amounts of data about the objects that are being manipulated. The first contribution of this paper is a system that automates this data collection using a robot. When the robot encounters a novel object, it collects data that enables it to detect the object, estimate its pose, and grasp it. However for some objects, information needed to infer a successful grasp is not visible to the robot’s sensors; for example, a heavy object might need to be grasped in the middle or else it will twist out of the robot’s gripper. The second contribution of this paper is an approach that allows a robot to identify the best grasp point by attempting to pick up the object and tracking its successes and failures. Because the number of grasp points is very large, we formalize grasping as an N-armed bandit problem and define a new algorithm for best arm identification in budgeted bandits that enables the robot to quickly find an arm corresponding to a good grasp without pulling all the arms. We demonstrate that a stock Baxter robot with no additional sensing can autonomously acquire models for a wide variety of objects and use the models to detect, classify, and manipulate the objects. Additionally, we show that our adaptation step significantly improves accuracy over a non-adaptive system, enabling a robot to improve its pick success rate from 55 to 75% on a collection of 30 household objects. Our instance-based approach exploits the robot’s ability to collect its own training data, enabling experience with the object to directly improve the robot’s performance during future interactions.

[1]  W. R. Thompson ON THE LIKELIHOOD THAT ONE UNKNOWN PROBABILITY EXCEEDS ANOTHER IN VIEW OF THE EVIDENCE OF TWO SAMPLES , 1933 .

[2]  R. Brooks Planning Collision- Free Motions for Pick-and-Place Operations , 1983 .

[3]  Editors , 1986, Brain Research Bulletin.

[4]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Tomás Lozano-Pérez,et al.  Task-level planning of pick-and-place robot motions , 1989, Computer.

[6]  Andrew W. Moore,et al.  Hoeffding Races: Accelerating Model Selection Search for Classification and Function Approximation , 1993, NIPS.

[7]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[8]  Antonio Morales,et al.  Experimental prediction of the performance of grasp tasks from visual features , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[9]  John N. Tsitsiklis,et al.  The Sample Complexity of Exploration in the Multi-Armed Bandit Problem , 2004, J. Mach. Learn. Res..

[10]  Hugh F. Durrant-Whyte,et al.  Simultaneous Localization, Mapping and Moving Object Tracking , 2007, Int. J. Robotics Res..

[11]  Anna Helena Reali Costa,et al.  Object-based Visual SLAM: How Object Identity Informs Geometry , 2008 .

[12]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[13]  Anis Sahbani,et al.  A hybrid approach for grasping 3D objects , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Matei T. Ciocarlie,et al.  The Columbia grasp database , 2009, 2009 IEEE International Conference on Robotics and Automation.

[15]  Rémi Munos,et al.  Pure Exploration in Multi-armed Bandits Problems , 2009, ALT.

[16]  Siddhartha S. Srinivasa,et al.  GATMO: A Generalized Approach to Tracking Movable Objects , 2009, 2009 IEEE International Conference on Robotics and Automation.

[17]  Ronald F. Boisvert,et al.  NIST Handbook of Mathematical Functions , 2010 .

[18]  Oliver Kroemer,et al.  Combining active learning and reactive control for robot grasping , 2010, Robotics Auton. Syst..

[19]  Michael Beetz,et al.  Robotic grasping of unmodeled objects using time-of-flight range data and finger torque information , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Quoc V. Le,et al.  Grasping novel objects with depth segmentation , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Justus H. Piater,et al.  Development of Object and Grasping Knowledge by Robot Exploration , 2010, IEEE Transactions on Autonomous Mental Development.

[22]  Dominik D. Freydenberger,et al.  Can We Learn to Gamble Efficiently? , 2010, COLT.

[23]  Kiriakos N. Kutulakos,et al.  Transparent and Specular Object Reconstruction , 2010, Comput. Graph. Forum.

[24]  Quoc V. Le,et al.  Learning to grasp objects with multiple contact points , 2010, 2010 IEEE International Conference on Robotics and Automation.

[25]  Dieter Fox,et al.  A large-scale hierarchical multi-view RGB-D object dataset , 2011, 2011 IEEE International Conference on Robotics and Automation.

[26]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[27]  Dieter Fox,et al.  A Scalable Tree-Based Approach for Joint Object and Pose Recognition , 2011, AAAI.

[28]  Dieter Fox,et al.  Manipulator and object tracking for in-hand 3D object modeling , 2011, Int. J. Robotics Res..

[29]  David Fofi,et al.  Review and comparison of non-conventional imaging systems for three-dimensional digitization of transparent objects , 2012, J. Electronic Imaging.

[30]  Aurélien Garivier,et al.  On Bayesian Upper Confidence Bounds for Bandit Problems , 2012, AISTATS.

[31]  Larry H. Matthies,et al.  End-to-end dexterous manipulation with deliberate interactive estimation , 2012, 2012 IEEE International Conference on Robotics and Automation.

[32]  Tamim Asfour,et al.  Discovery, segmentation and reactive grasping of unknown objects , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[33]  David Filliat,et al.  Improving object learning through manipulation and robot self-identification , 2013, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[34]  Nicholas Roy,et al.  Recognition and Pose Estimation of Rigid Transparent Objects with a Kinect Sensor , 2013 .

[35]  Paul H. J. Kelly,et al.  SLAM++: Simultaneous Localisation and Mapping at the Level of Objects , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Sonia Chernova,et al.  Construction of an object manipulation database from grasp demonstrations , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  Giorgio Metta,et al.  Three-finger precision grasp on incomplete 3D point clouds , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[38]  Sonia Chernova,et al.  Crowdsourcing the construction of a 3D object recognition database for robotic grasping , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[39]  Charles C. Kemp,et al.  Autonomously learning to visually detect where manipulation will succeed , 2012, Auton. Robots.

[40]  Wei Chen,et al.  Combinatorial Pure Exploration of Multi-Armed Bandits , 2014, NIPS.

[41]  Siddhartha S. Srinivasa,et al.  HerbDisc: Towards lifelong robotic object discovery , 2015, Int. J. Robotics Res..