Manipulation of multiple objects in close proximity based on visual hierarchical relationships

This paper presents a method for daily assistive robots to manipulate objects whose contact relationships are important. In our approach, contact relationships between multiple objects such as “on” and “into” are estimated using the hierarchical relationships between the objects' regions in the images. The hierarchical states are used for failure recovery processes so that the target contact state is achieved. We tested our method in two daily tasks by HRP-2VZ humanoid robot: filing, and serving an egg with a spatula. These results suggested the effectiveness of our approach.

[1]  Philippe Salembier,et al.  Occlusion-based depth ordering on monocular images with Binary Partition Tree , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Kimitoshi Yamazaki,et al.  Recognition and manipulation integration for a daily assistive robot working on kitchen environments , 2010, 2010 IEEE International Conference on Robotics and Biomimetics.

[3]  Kimitoshi Yamazaki,et al.  System integration of a daily assistive robot and its application to tidying and cleaning rooms , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Masayuki Inaba,et al.  Task guided attention control and visual verification in tea serving by the daily assistive humanoid HRP2JSK , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Ray A. Jarvis,et al.  A Perspective on Range Finding Techniques for Computer Vision , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[7]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Shigeki Sugano,et al.  Design of human symbiotic robot TWENDY-ONE , 2009, 2009 IEEE International Conference on Robotics and Automation.

[9]  Philippe C. Cattin,et al.  Tracking the invisible: Learning where the object might be , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  T. Fukuda,et al.  Manipulation of deformable linear objects with knot invariant to classify condition , 2005, Proceedings, 2005 IEEE/ASME International Conference on Advanced Intelligent Mechatronics..

[11]  Makoto Mizukawa,et al.  Picking up operation of thin objects by robot arm with two-fingered parallel soft gripper , 2012, 2012 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO).

[12]  Kunikatsu Takase,et al.  Implementation of dynamic manipulation with visual feedback and its application to pick and place task , 2001, Proceedings of the 2001 IEEE International Symposium on Assembly and Task Planning (ISATP2001). Assembly and Disassembly in the Twenty-first Century. (Cat. No.01TH8560).

[13]  Katsushi Ikeuchi,et al.  Calculating optimal trajectories from contact transitions , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[15]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Hitoshi Akami,et al.  Robot hand with sensor for handling cloth , 1990, EEE International Workshop on Intelligent Robots and Systems, Towards a New Frontier of Applications.

[17]  Rüdiger Dillmann,et al.  An autonomous ice-cream serving robot , 2011, 2011 IEEE International Conference on Robotics and Automation.

[18]  Kimitoshi Yamazaki,et al.  End point tracking for a moving object with several attention regions by composite vision system , 2011, 2011 IEEE International Conference on Mechatronics and Automation.

[19]  Dariu Gavrila,et al.  Multi-feature hierarchical template matching using distance transforms , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[20]  Kimitoshi Yamazaki,et al.  Hierarchical estimation of multiple objects from proximity relationships arising from tool manipulation , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[21]  T. Asfour,et al.  ARMAR-III : A HUMANOID PLATFORM FOR PERCEPTION-ACTION INTEGRATION , 2006 .

[22]  Dejan Pangercic,et al.  Robotic roommates making pancakes , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.