Fusion of multimodal visual cues for model-based object tracking

While many robotic applications rely on visual tracking, conventional single-cue algorithms typically fail outside limited tracking conditions. Fusion of multimodal visual cues with complementary failure modes allows tracking to continue despite losing individual cues. While previous applications have addressed multi-cue 2D featurebased tracking, this paper develops a fusion scheme for 3D model-based tracking using a Kalman filter framework. Our algorithm fuses colour, edge and texture cues predicted from a textured CAD model of the tracked object to recover the 3D pose. The fusion framework allows additional cues to be integrated provided a suitable measurement function exists. We also show how measurements from multiple cameras can be integrated without requiring explicit correspondences between views. Experimental results demonstrate the increased robustness achieved by fusing multiple cues.

[1]  Gordon Cheng,et al.  Humanoid robot learning and game playing using PC-based vision , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Greg Welch,et al.  SCAAT: incremental tracking with incomplete information , 1997, SIGGRAPH.

[3]  Pradeep K. Khosla,et al.  An extendable framework for expectation-based visual servoing using environment models , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[4]  Danica Kragic,et al.  Model based techniques for robotic servoing and grasping , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Hans-Hellmut Nagel,et al.  Towards visually servoed manipulation of car engine parts , 1997, Proceedings of International Conference on Robotics and Automation.

[7]  Shaogang Gong,et al.  Object Tracking Using Adaptive Color Mixture Models , 1998, ACCV.

[8]  Patrick Bouthemy,et al.  Robust real-time visual tracking using a 2D-3D model-based approach , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[9]  Trevor Darrell,et al.  Integrated Person Tracking Using Stereo, Color, and Pattern Detection , 2000, International Journal of Computer Vision.

[10]  Gregory D. Hager,et al.  Incremental Focus of Attention for Robust Vision-Based Tracking , 1999, International Journal of Computer Vision.

[11]  Michael O. Kolawole,et al.  Estimation and tracking , 2002 .

[12]  Kevin Nickels,et al.  Model-based tracking of complex articulated objects , 2001, IEEE Trans. Robotics Autom..

[13]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[14]  Johnson I. Agbinya,et al.  Multi-Object Tracking in Video , 1999, Real Time Imaging.

[15]  Lindsay Kleeman,et al.  Robust Range Data Segmentation using Geometric Primitives for Robotic Applications , 2003, SIP.

[16]  Yaakov Bar-Shalom,et al.  Estimation and Tracking: Principles, Techniques, and Software , 1993 .

[17]  Andrew J. Davison,et al.  Mobile Robot Navigation Using Active Vision , 1998 .

[18]  Danica Kragic,et al.  Cue integration for visual servoing , 2001, IEEE Trans. Robotics Autom..

[19]  Bernt Schiele,et al.  Towards Robust Multi-cue Integration for Visual Tracking , 2001, ICVS.

[20]  Lindsay Kleeman,et al.  Optimal estimation of position and heading for mobile robots using ultrasonic beacons and dead-reckoning , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[21]  Trevor Darrell,et al.  Integrated Person Tracking Using Stereo, Color, and Pattern Detection , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).