Using Spatio-Temporal Continuity Constraints to Enhance Visual Tracking of Moving Objects

We present a framework for annotating dynamic scenes involving occlusion and other uncertainties. Our system comprises an object tracker, an object classifier and an algorithm for reasoning about spatio-temporal continuity. The principle behind the object tracking and classifier modules is to reduce error by increasing ambiguity (by merging objects in close proximity and presenting multiple hypotheses). The reasoning engine resolves error, ambiguity and occlusion to produce a most likely hypothesis, which is consistent with global spatio-temporal continuity constraints. The system results in improved annotation over frame-by-frame methods. It has been implemented and applied to the analysis of a team sports video.

[1]  Hironobu Fujiyoshi,et al.  Moving target classification and tracking from real-time video , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[2]  Jitendra Malik,et al.  Robust Multiple Car Tracking with Occlusion Reasoning , 1994, ECCV.

[3]  Derek R. Magee,et al.  Tracking multiple vehicles using foreground, background and motion models , 2004, Image Vis. Comput..

[4]  Shaogang Gong,et al.  Resolving Visual Uncertainty and Occlusion through Probabilistic Reasoning , 2000, BMVC.

[5]  Derek R. Magee.  A Sequential Scheduling Approach to Combining Multiple Object Classifiers Using Cross-Entropy , 2003, Multiple Classifier Systems.

[6]  Anthony G. Cohn,et al.  Abducing Qualitative Spatio-Temporal Histories from Partial Observations , 2002, KR.

[7]  Philippe Muller,et al.  A Qualitative Theory of Motion Based on Spatio-Temporal Primitives , 1998, KR.

[8]  Larry S. Davis,et al.  Probabilistic framework for segmenting people under occlusion , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[9]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[10]  Leonidas J. Guibas,et al.  Counting people in crowds with a real-time network of simple image sensors , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[11]  A. M. Tekalp,et al.  Multiple camera fusion for multi-object tracking , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[12]  Azriel Rosenfeld,et al.  Tracking Groups of People , 2000, Comput. Vis. Image Underst..

[13]  Anthony G. Cohn,et al.  Describing Rigid Body Motions in a Qualitative Theory of Spatial Regions , 2000, AAAI/IAAI.

[14]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.