Fusing Time-of-Flight Depth and Color for Real-Time Segmentation and Tracking

We present an improved framework for real-time segmentation and tracking by fusing depth and RGB color data. We are able to solve common problems seen in tracking and segmentation of RGB images, such as occlusions, fast motion, and objects of similar color. Our proposed real-time mean shift based algorithm outperforms the current state of the art and is significantly better in difficult scenarios.

[1]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[3]  Klaus-Dieter Kuhnert,et al.  Fusion of Stereo-Camera and PMD-Camera Data for Real-Time Suited Precise 3D Environment Reconstruction , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Marc Alexa,et al.  Combining Time-Of-Flight depth and stereo images without accurate extrinsic calibration , 2008, Int. J. Intell. Syst. Technol. Appl..

[5]  Zezhi Chen,et al.  Video Object Tracking Based on a Chamfer Distance Transform , 2007, 2007 IEEE International Conference on Image Processing.

[6]  Richard Szeliski,et al.  Noise Estimation from a Single Image , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[7]  Dorin Comaniciu,et al.  Mean shift and optimal prediction for efficient object tracking , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[8]  Ehud Rivlin,et al.  A General Framework for Combining Visual Trackers – The "Black Boxes" Approach , 2006, International Journal of Computer Vision.

[9]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[10]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[11]  Ralf Reulke,et al.  Combination of distance data with high resolution images , 2006 .

[12]  Andrew Y. Ng,et al.  Integrating Visual and Range Data for Robotic Object Detection , 2008, ECCV 2008.

[13]  Robert T. Collins,et al.  Mean-shift blob tracking through scale space , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[14]  Ryan Crabb,et al.  Real-time foreground segmentation via range and color imaging , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[15]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[16]  Q. M. Jonathan Wu,et al.  Visual Tracking Using Color Cameras and Time-of-Flight Range Imaging Sensors , 2008, J. Multim..

[17]  Hai Tao,et al.  Fusion of local appearance with stereo depth for object tracking , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.