Real-time tracking of complex structures with on-line camera calibration

Abstract This paper presents a novel three-dimensional model-based tracking system which has been incorporated into a visual servoing system. The tracking system combines modern graphical rendering technology with constrained active contour tracking techniques to create wire-frame-snakes. It operates in real-time at video frame rate (25 Hz) and is based on an internal CAD model of the object to be tracked. This model is rendered using a binary space partition tree to perform hidden line removal and the visible features are identified on-line at each frame and are tracked in the video feed. The tracking system has been extended to incorporate real-time on-line calibration and tracking of internal camera parameters. Results from on-line calibration and visual servoing experiments are presented.

[1]  Geoffrey D. Sullivan,et al.  Pose refinement of active models using forces in 3D , 1994, ECCV.

[2]  Chris Harris,et al.  Geometry from visual motion , 1993 .

[3]  Chris Harris,et al.  Tracking with rigid models , 1993 .

[4]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[5]  Michel Dhome,et al.  Modelled Object Pose Estimation and Tracking by Monocular Vision , 1993, BMVC.

[6]  Gerd Hirzinger,et al.  Real-time visual tracking of 3D objects with dynamic handling of occlusion , 1997, Proceedings of International Conference on Robotics and Automation.

[7]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[8]  Andrew Zisserman,et al.  Robust Object Tracking , 2001 .

[9]  Patrick Rives,et al.  A new approach to visual servoing in robotics , 1992, IEEE Trans. Robotics Autom..

[10]  Richard Szeliski,et al.  Tracking with Kalman snakes , 1993 .

[11]  David G. Lowe,et al.  Robust model-based motion tracking through the integration of search and estimation , 1992, International Journal of Computer Vision.

[12]  Roberto Cipolla,et al.  Visual tracking and control using Lie algebras , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[13]  Andrew Blake,et al.  Image Divergence and Deformation from Closed Curves , 1997, Int. J. Robotics Res..

[14]  S. P. Mudur,et al.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[15]  Patrick Bouthemy,et al.  Robust real-time visual tracking using a 2D-3D model-based approach , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[16]  F. Frances Yao,et al.  Efficient binary space partitions for hidden-surface removal and solid modeling , 1990, Discret. Comput. Geom..

[17]  Gregory D. Hager,et al.  Feature-based visual servoing and its application to telerobotics , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[18]  Hans-Hellmut Nagel,et al.  Tracking of complex driving manoeuvres in traffic image sequences , 1998, Image Vis. Comput..

[19]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[20]  T. Fukuda,et al.  Intelligent robotic system , 1996 .