Texture-less object tracking with online training using an RGB-D camera

We propose a texture-less object detection and 3D tracking method which automatically extracts on the fly the information it needs from color images and the corresponding depth maps. While texture-less 3D tracking is not new, it requires a prior CAD model, and real-time methods for detection still have to be developed for robust tracking. To detect the target, we propose to rely on a fast template-based method, which provides an initial estimate of its 3D pose, and we refine this estimate using the depth and image contours information. We automatically extract a 3D model for the target from the depth information. To this end, we developed methods to enhance the depth map and to stabilize the 3D pose estimation. We demonstrate our method on challenging sequences exhibiting partial occlusions and fast motions.

[1]  Vincent Lepetit,et al.  Combining edge and texture information for real-time accurate 3D camera tracking , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[2]  Vincent Lepetit,et al.  Fast Keypoint Recognition in Ten Lines of Code , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Luc Van Gool,et al.  Object Detection by Contour Segment Networks , 2006, ECCV.

[4]  Chris Harris,et al.  Tracking with rigid models , 1993 .

[5]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  David G. Lowe,et al.  Robust model-based motion tracking through the integration of search and estimation , 1992, International Journal of Computer Vision.

[7]  Vincent Lepetit,et al.  ESM-Blur: Handling & rendering blur in 3D tracking and augmentation , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[8]  Sebastian Thrun,et al.  An Application of Markov Random Fields to Range Sensing , 2005, NIPS.

[9]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[10]  Tom Drummond,et al.  ProFORMA: Probabilistic Feature-based On-line Rapid Model Acquisition , 2009, BMVC.

[11]  Frédéric Jurie,et al.  Solution of the Simultaneous Pose and Correspondence Problem Using Gaussian Error Model , 1999, Comput. Vis. Image Underst..

[12]  Roberto Cipolla,et al.  Real-Time Visual Tracking of Complex Structures , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.

[14]  Vincent Lepetit,et al.  Dominant orientation templates for real-time detection of texture-less objects , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Tom Drummond,et al.  Fusing points and lines for high performance tracking , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[16]  Satoh Kiyohide,et al.  A Fast Initialization Method for Edge-based Registration Using an Inclination Constraint , 2008 .

[17]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .