Real-time 3D visual sensor for robust object recognition

This paper presents a novel 3D measurement system, which yields both depth and color information in real time, by calibrating a time-of-flight and two CCD cameras. The problem of occlusions is solved by the proposed fast occluded-pixel detection algorithm. Since the system uses two CCD cameras, missing color information of occluded pixels is covered by one another. We also propose a robust object recognition using the 3D visual sensor. Multiple cues, such as color, texture and 3D (depth) information, are integrated in order to recognize various types of objects under varying lighting conditions. We have implemented the system on our autonomous robot and made the robot do recognition tasks (object learning, detection, and recognition) in various environments. The results revealed that the proposed recognition system provides far better performance than the previous system that is based only on color and texture information.

[1]  Rasmus Larsen,et al.  Cluster tracking with Time-of-Flight cameras , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[2]  Kazunori Ohno,et al.  Real-Time Robot Trajectory Estimation and 3D Map Construction using 3D Camera , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Ghassan Hamarneh,et al.  N-Sift: N-Dimensional Scale Invariant Feature Transform for Matching Medical Images , 2007, ISBI.

[4]  Thierry Oggier,et al.  Miniature 3D TOF Camera for Real-Time Imaging , 2006, PIT.

[5]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[6]  Reinhard Koch,et al.  Real-time Estimation of the Camera Path from a Sequence of Intrinsically Calibrated Pmd Depth Images , 2008 .

[7]  Carlo Tomasi,et al.  3D head tracking based on recognition and interpolation using a time-of-flight depth sensor , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[8]  Tomoaki Nakamura,et al.  Learning novel objects using out-of-vocabulary word segmentation and object extraction for home assistant robots , 2010, 2010 IEEE International Conference on Robotics and Automation.

[9]  Bernard Chazelle,et al.  Shape distributions , 2002, TOGS.

[10]  Gerd Hirzinger,et al.  Extrinsic and depth calibration of ToF-cameras , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Young Min Kim,et al.  Design and calibration of a multi-view TOF sensor fusion system , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[12]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[13]  D. Holz,et al.  3 D Pose Estimation and Mapping with Time-of-Flight Cameras , 2008 .

[14]  Reinhard Koch,et al.  Integration of a Time-of-Flight Camera into a Mixed Reality System for Handling Dynamic Scenes, Moving Viewpoints and Occlusions in Real-Time , 2008 .

[15]  Yasuo Kuniyoshi,et al.  The Development of Color CHLAC Features for Object Exploration in 3D Map , 2011 .