Corner detection based real-time workpiece recognition for robot manipulation

Workpiece recognition is vital and essential for robot manipulation which is one of the most important skills for robot. In this paper, we present a corner detection based strategy to recognize the type of workpiece so as to offer important visual cues for robot manipulation. Our framework works by three steps. Firstly, the bounding-box of workpiece is detected using a multi-scale convolutional neural network according to the closed regions (enclosed by edges). Secondly, three types of corners (Y-type, A-type and L-type) are detected by employing a simple neural network and the results are further refined according to both the detection probability and geometric relationship between corners. Finally, the type of workpiece is recognized on the basis of the relative position of corners. Due to that the workpiece detection step greatly reduces the searching space for the corner detection, a real-time process is achieved. Another important characteristic of our method lies in that the detected corners can be further used as basic modelling element in reconstructing the 3D structure of the workpiece, which is beneficial for the robot to decide the grasping position and pose. With the PKU-HR6.0 platform, a manipulation controller is established based on the proposed approach. Experimental results show that our approach is comparable with some state-of-the-art work in precision of recognition, and our PKU-HR6.0 robot is able to precisely recognize and locate the Tetris-shaped building blocks so that to well accomplish the manipulation tasks.

[1]  Helge J. Ritter,et al.  Real-time hierarchical scene segmentation and classification , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[2]  Jack Bresenham,et al.  Algorithm for computer control of a digital plotter , 1965, IBM Syst. J..

[3]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[4]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[5]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[6]  H. Barlow.  Vision Science: Photons to Phenomenology by Stephen E. Palmer , 2000, Trends in Cognitive Sciences.

[7]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Gary R. Bradski,et al.  Fast 3D recognition and pose using the Viewpoint Feature Histogram , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Xihong Wu,et al.  Learning to Reconstruct 3D Structure from Object Motion , 2015, ICONIP.

[10]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[12]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Jan-Michael Frahm,et al.  Repetition-based dense single-view reconstruction , 2011, CVPR 2011.

[14]  Ales Ude,et al.  Object segmentation and learning through feature grouping and manipulation , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[15]  Michael Beetz,et al.  Multimodal autonomous tool analyses and appropriate application , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[16]  Xihong Wu,et al.  Convolutional Networks Based Edge Detector Learned via Contrast Sensitivity Function , 2015, ICONIP.

[17]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Kostas Daniilidis,et al.  Single image 3D object detection and pose estimation for grasping , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).