Real-Time Visual Tracking of Complex Structures

Presents a framework for three-dimensional model-based tracking. Graphical rendering technology is combined with constrained active contour tracking to create a robust wire-frame tracking system. It operates in real time at video frame rate (25 Hz) on standard hardware. It is based on an internal CAD model of the object to be tracked which is rendered using a binary space partition tree to perform hidden line removal. A Lie group formalism is used to cast the motion computation problem into simple geometric terms so that tracking becomes a simple optimization problem solved by means of iterative reweighted least squares. A visual servoing system constructed using this framework is presented together with results showing the accuracy of the tracker. The paper then describes how this tracking system has been extended to provide a general framework for tracking in complex configurations. The adjoint representation of the group is used to transform measurements into common coordinate frames. The constraints are then imposed by means of Lagrange multipliers. Results from a number of experiments performed using this framework are presented and discussed.

[1]  Hans-Hellmut Nagel,et al.  Tracking of complex driving manoeuvres in traffic image sequences , 1998, Image Vis. Comput..

[2]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[3]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Larry S. Davis,et al.  3-D model-based tracking of humans in action: a multi-view approach , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[6]  Ehud Rivlin,et al.  Visual homing: Surfing on the epipoles , 1997, Block Island Workshop on Vision and Control.

[7]  Andrew Blake,et al.  Spatial Dependence in the Observation of Visual Contours , 1998, ECCV.

[8]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[9]  Patrick Bouthemy,et al.  Robust real-time visual tracking using a 2D-3D model-based approach , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[10]  Gregory D. Hager,et al.  Feature-based visual servoing and its application to telerobotics , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[11]  F. Frances Yao,et al.  Efficient binary space partitions for hidden-surface removal and solid modeling , 1990, Discret. Comput. Geom..

[12]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[13]  Michel Dhome,et al.  Modelled Object Pose Estimation and Tracking by Monocular Vision , 1993, BMVC.

[14]  Jitendra Malik,et al.  Tracking people with twists and exponential maps , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[15]  Nikolaos Papanikolopoulos,et al.  Selection of features and evaluation of visual measurements during robotic visual servoing tasks , 1995, J. Intell. Robotic Syst..

[16]  Patrick Rives,et al.  A new approach to visual servoing in robotics , 1992, IEEE Trans. Robotics Autom..

[17]  CipollaRoberto,et al.  Application of Lie Algebras to Visual Servoing , 2000 .

[18]  C. J. Taylor,et al.  Minimization on the Lie Group SO(3) and Related Manifolds , 1994 .

[19]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[20]  Olivier D. Faugeras,et al.  3D articulated models and multi-view tracking with silhouettes , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[21]  Ehud Rivlin,et al.  Visual Homing: Surfing on the Epipoles , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[22]  Gerd Hirzinger,et al.  Real-time visual tracking of 3D objects with dynamic handling of occlusion , 1997, Proceedings of International Conference on Robotics and Automation.

[23]  Patrick Rives,et al.  A new approach to visual servoing in robotics , 1992, IEEE Trans. Robotics Autom..

[24]  Chris Harris,et al.  Tracking with rigid models , 1993 .

[25]  Lee E. Weiss,et al.  Dynamic sensor-based control of robots with visual feedback , 1987, IEEE Journal on Robotics and Automation.

[26]  William J. Wilson,et al.  Relative end-effector control using Cartesian position based visual servoing , 1996, IEEE Trans. Robotics Autom..

[27]  Richard Szeliski,et al.  Tracking with Kalman snakes , 1993 .

[28]  David G. Lowe,et al.  Robust model-based motion tracking through the integration of search and estimation , 1992, International Journal of Computer Vision.

[29]  Geoffrey D. Sullivan,et al.  Pose refinement of active models using forces in 3D , 1994, ECCV.

[30]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, International Journal of Computer Vision.

[31]  D. Sattinger,et al.  Lie Groups and Algebras with Applications to Physics, Geometry and Mechanics , 1986 .

[32]  Andrew Zisserman,et al.  Robust Object Tracking , 2001 .

[33]  Roberto Cipolla,et al.  Real-Time Tracking of Multiple Articulated Structures in Multiple Views , 2000, ECCV.

[34]  Chris Harris,et al.  Geometry from visual motion , 1993 .

[35]  Jake K. Aggarwal,et al.  Nonrigid Motion Analysis: Articulated and Elastic Motion , 1998, Comput. Vis. Image Underst..

[36]  Andrew Blake,et al.  Image Divergence and Deformation from Closed Curves , 1997, Int. J. Robotics Res..

[37]  S. P. Mudur,et al.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[38]  James M. Rehg,et al.  A multiple hypothesis approach to figure tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[39]  D. Lowe.  Fitting Parameterized 3-D Models to Images , 1989 .

[40]  David G. Lowe,et al.  Fitting Parameterized Three-Dimensional Models to Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Andrew Blake,et al.  The dynamic analysis of apparent contours , 1990, [1990] Proceedings Third International Conference on Computer Vision.