Real time tracking of 3D objects: an efficient and robust approach

The main aim of this article is the description of an original approach which allows the efficient tracking of two-dimensional (2D) patterns in image sequences. This approach consists of two stages. An off-line is devoted to the computation of an interaction matrix linking the aspect variation of a pattern with its displacement in the image. In a second stage, this matrix is used to track the pattern by the following process. At first, the position, the scale and the orientation of the pattern is predicted. Then, the difference between the pattern observed at the predicted position and the reference pattern (which is to be tracked) is computed. Finally, this difference multiplied by the interaction matrix gives a correction to apply to the prediction. Computing this correction has a very low computational cost, allowing a real time implementation of the algorithm. From these principles, we have developed a real time three-dimensional object tracker; objects are modeled by sets of 2D appearances. We show that these algorithms are resistant to occlusions and changes of illumination.

[1]  Terry Caelli,et al.  Recovery of the 3-D location and motion of a rigid object through camera image (An Extended Kalman Filter Approach) , 1989, International Journal of Computer Vision.

[2]  Frédéric Jurie,et al.  Solution of the Simultaneous Pose and Correspondence Problem Using Gaussian Error Model , 1999, Comput. Vis. Image Underst..

[3]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, International Journal of Computer Vision.

[4]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[5]  Filiberto Pla,et al.  Matching Feature Points in Image Sequences through a Region-Based Method , 1997, Comput. Vis. Image Underst..

[6]  Alex Pentland,et al.  A subspace method for maximum likelihood target detection , 1995, Proceedings., International Conference on Image Processing.

[7]  Olivier D. Faugeras,et al.  What can two images tell us about a third one? , 1994, International Journal of Computer Vision.

[8]  K NayarShree,et al.  Visual learning and recognition of 3-D objects from appearance , 1995 .

[9]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[10]  Hans-Hellmut Nagel,et al.  3D Pose Estimation by Directly Matching Polyhedral Models to Gray Value Gradients , 1997, International Journal of Computer Vision.

[11]  G.-S. Young,et al.  3-D motion estimation using a sequence of noisy stereo images , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  David G. Lowe,et al.  Robust model-based motion tracking through the integration of search and estimation , 1992, International Journal of Computer Vision.

[13]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[14]  Alex Pentland,et al.  Space-time gestures , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[16]  James L. Crowley,et al.  Measuring Image Flow By Tracking Edge-lines , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[17]  Roberto Brunelli,et al.  MASSACHUSETTS INSTITUTE OF TECHNOLOGY ARTIFICIAL INTELLIGENCE LABORATORY and CENTER FOR BIOLOGICAL AND COMPUTATIONAL LEARNING DEPARTMENT OF BRAIN AND COGNITIVE SCIENCES , 2001 .

[18]  Alex Pentland,et al.  sing Interpolated Views , 1996 .

[19]  Alex Pentland,et al.  A three-dimensional model of human lip motions trained from video , 1997, Proceedings IEEE Nonrigid and Articulated Motion Workshop.

[20]  A. Pentland,et al.  Real time tracking and modeling of faces: an EKF-based analysis by synthesis approach , 1999, Proceedings IEEE International Workshop on Modelling People. MPeople'99.