A relaxation algorithm for real-time multiple view 3D-tracking

Abstract In this paper we address the problem of reliable real-time 3D-tracking of multiple objects which are observed in multiple wide-baseline camera views. Establishing the spatio-temporal correspondence is a problem with combinatorial complexity in the number of objects and views. In addition vision-based tracking suffers from the ambiguities introduced by occlusion, clutter and irregular 3D motion. In this paper we present a discrete relaxation algorithm for reducing the intrinsic combinatorial complexity by pruning the decision tree based on unreliable prior information from independent 2D-tracking for each view. The algorithm improves the reliability of spatio-temporal correspondence by simultaneous optimisation over multiple views in the case where 2D-tracking in one or more views are ambiguous. Application to the 3D reconstruction of human movement, based on tracking of skin-coloured regions in three views, demonstrates considerable improvement in reliability and performance. Results demonstrate that the optimisation over multiple views gives correct 3D reconstruction and object labelling in the presence of incorrect 2D-tracking whilst maintaining real-time performance.

[1]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[2]  Bernd Radig,et al.  Image sequence analysis using relational structures , 1984, Pattern Recognit..

[3]  Dmitry Chetverikov,et al.  Tracking feature points: a new algorithm , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[4]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Johan Philip.  Estimation Three-Dimensional Motion of Rigid Objects from Noisy Observations , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  James L. Crowley,et al.  Measurement and integration of 3-D structures by tracking edge lines , 1990, International Journal of Computer Vision.

[7]  Jeffrey B. Burl,et al.  A reduced order extended Kalman filter for sequential images containing a moving object , 1993, IEEE Trans. Image Process..

[8]  Elizabeth R. Stuck,et al.  Detecting Moving Objects Using the Rigidity Constraint , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Shaogang Gong,et al.  Dynamic Vision - From Images to Face Recognition , 2000 .

[10]  Olivier D. Faugeras,et al.  Motion from point matches: Multiplicity of solutions , 2004, International Journal of Computer Vision.

[11]  Thomas S. Huang,et al.  Motion and structure from feature correspondences: a review , 1994, Proc. IEEE.

[12]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[13]  Thomas S. Huang,et al.  Maximal matching of 3-D points for multiple-object motion estimation , 1988, Pattern Recognit..

[14]  Olivier Faugeras,et al.  3D Dynamic Scene Analysis , 1992 .

[15]  Paolo Nesi,et al.  A Robust Tracking of 3D Motion , 1994, ECCV.

[16]  Alex Waibel,et al.  Gaze Tracking Based on Face‐Color , 1995 .

[17]  David G. Lowe,et al.  Robust model-based motion tracking through the integration of search and estimation , 1992, International Journal of Computer Vision.

[18]  Ingemar J. Cox,et al.  A maximum likelihood N-camera stereo algorithm , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Ingemar J. Cox,et al.  A maximum-flow formulation of the N-camera stereo correspondence problem , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[20]  Rin-ichiro Taniguchi,et al.  Real-time human motion analysis and IK-based human figure control , 2000, Proceedings Workshop on Human Motion.

[21]  Soon Ki Jung,et al.  A model-based 3-D tracking of rigid objects from a sequence of multiple perspective views , 1998, Pattern Recognit. Lett..

[22]  Donald Reid.  An algorithm for tracking multiple targets , 1978 .

[23]  Gye-Young Kim,et al.  Model-based tracking of moving object , 1997, Pattern Recognit..

[24]  Takeo Kanade,et al.  A stereo machine for video-rate dense depth mapping and its new applications , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[26]  Ishwar K. Sethi,et al.  Finding Trajectories of Feature Points in a Monocular Image Sequence , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  David J. Fleet,et al.  Stochastic Tracking of 3D Human Figures Using 2D Image Motion , 2000, ECCV.

[28]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[29]  Shaogang Gong,et al.  Object Tracking Using Adaptive Color Mixture Models , 1998, ACCV.

[30]  Zhengyou Zhang,et al.  Token tracking in a cluttered scene , 1994, Image Vis. Comput..

[31]  Thomas S. Huang,et al.  Estimating three-dimensional motion parameters of a rigid planar patch , 1981 .

[32]  Narendra Ahuja,et al.  Motion and structure from point correspondences with error estimation: planar surfaces , 1991, IEEE Trans. Signal Process..

[33]  Derick Wood,et al.  Data structures, algorithms, and performance , 1992 .

[34]  Michael Isard,et al.  A mixed-state condensation tracker with automatic model-switching , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[35]  Mubarak Shah,et al.  Establishing motion correspondence , 1991, CVGIP Image Underst..

[36]  John K. Tsotsos,et al.  Applying temporal constraints to the dynamic stereo problem , 1986, Comput. Vis. Graph. Image Process..

[37]  Douglas F. Elliott,et al.  Moving Target Tracking Using Symbolic Registration , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Takeo Kanade,et al.  Introduction to the Special Section on Video Surveillance , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Jake K. Aggarwal,et al.  Computer analysis of dynamic scenes containing curvilinear figures , 1979, Pattern Recognit..

[40]  V. S. Hwang,et al.  Tracking feature points in time-varying images using an opportunistic selection approach , 1989, Pattern Recognit..

[41]  Olivier D. Faugeras,et al.  Improving Consistency and Reducing Ambiguity in Stochastic Labeling: An Optimization Approach , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[43]  Yang Song,et al.  Monocular Perception of Biological Motion in Johansson Displays , 2001, Comput. Vis. Image Underst..

[44]  Ishwar K. Sethi,et al.  Feature Point Correspondence in the Presence of Occlusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Yaakov Bar-Shalom,et al.  Multitarget-Multisensor Tracking: Applications and Advances , 1992 .