Cloth grasp point detection based on multiple-view geometric cues with application to robotic towel folding

We present a novel vision-based grasp point detection algorithm that can reliably detect the corners of a piece of cloth, using only geometric cues that are robust to variation in texture. Furthermore, we demonstrate the effectiveness of our algorithm in the context of folding a towel using a general-purpose two-armed mobile robotic platform without the use of specialized end-effectors or tools. The robot begins by picking up a randomly dropped towel from a table, goes through a sequence of vision-based re-grasps and manipulations—partially in the air, partially on the table—and finally stacks the folded towel in a target location. The reliability and robustness of our algorithm enables for the first time a robot with general purpose manipulators to reliably and fully-autonomously fold previously unseen towels, demonstrating success on all 50 out of 50 single-towel trials as well as on a pile of 5 towels.

[1]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[2]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[3]  Gareth J. Monkman,et al.  Robot Grippers for Use With Fibrous Materials , 1995, Int. J. Robotics Res..

[4]  Shimon Edelman,et al.  Learning to grasp using visual information , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[5]  Karun B. Shimoga,et al.  Robot Grasp Synthesis Algorithms: A Survey , 1996, Int. J. Robotics Res..

[6]  Zoe Doulgeri,et al.  Robot handling of flat textile materials , 1997, IEEE Robotics Autom. Mag..

[7]  Kyoko Hamajima,et al.  Planning strategy for task of unfolding clothes , 1997, Robotics Auton. Syst..

[8]  Vijay Kumar,et al.  Robotic grasping and contact: a review , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[9]  Lydia E. Kavraki,et al.  Planning Paths for Elastic Objects under Manipulation Constraints , 2001, Int. J. Robotics Res..

[10]  Justus H. Piater,et al.  Developing haptic and visual perceptual categories for reaching and grasping with a humanoid robot , 2001, Robotics Auton. Syst..

[11]  Antonio Morales,et al.  Vision-based computation of three-finger grasps on unknown planar objects , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Ronald Lumia,et al.  Manipulation of unmodeled objects using intelligent grasping schemes , 2003, IEEE Trans. Fuzzy Syst..

[13]  Devin J. Balkcom,et al.  Introducing robotic origami folding , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[14]  Nobuyuki Kita,et al.  A deformable model driven visual method for handling clothes , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[15]  Kenneth Y. Goldberg,et al.  D-space and Deform Closure Grasps of Deformable Parts , 2005, Int. J. Robotics Res..

[16]  Robert Platt,et al.  Re-using schematic grasping policies , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[17]  Robert Platt,et al.  Learning Grasp Context Distinctions that Generalize , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[18]  Toshio Fukuda,et al.  Manipulation of Flexible Rope Using Topological Model Based on Sensor Information , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Lydia E. Kavraki,et al.  Path planning for deformable linear objects , 2006, IEEE Transactions on Robotics.

[20]  Hiroaki Seki,et al.  Clothes Folding Task by Tool-Using Robot , 2006, J. Robotics Mechatronics.

[21]  E. Torres-Jara,et al.  Challenges for Robot Manipulation in Human Environments , 2006 .

[22]  Y. Kamiya,et al.  Inchworm robot grippers in clothes manipulation — optimizing the tracing algorithm , 2007, 2007 International Conference on Intelligent and Advanced Systems.

[23]  Charles C. Kemp,et al.  Challenges for robot manipulation in human environments [Grand Challenges of Robotics] , 2007, IEEE Robotics & Automation Magazine.

[24]  Hiroaki Seki,et al.  Unfolding of Massive Laundry and Classification Types by Dual Manipulator , 2007, J. Adv. Comput. Intell. Intell. Informatics.

[25]  John Kenneth Salisbury,et al.  Towards a personal robotics development platform: Rationale and design of an intrinsically safe personal robot , 2008, 2008 IEEE International Conference on Robotics and Automation.

[26]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[27]  Siddhartha S. Srinivasa,et al.  Grasp synthesis in cluttered environments for dexterous hands , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[28]  Daniel Cremers,et al.  Anisotropic Huber-L1 Optical Flow , 2009, BMVC.

[29]  Siddhartha S. Srinivasa,et al.  Manipulation planning with Workspace Goal Regions , 2009, 2009 IEEE International Conference on Robotics and Automation.

[30]  Kimitoshi Yamazaki,et al.  A Cloth Detection Method Based on Image Wrinkle Feature for Daily Assistive Robots , 2009, MVA.