Detection and Tracking of Moving Objects from Overlapping EO and IR Sensors

We present an approach for tracking moving objects observed by EO and IR sensors on a moving platform. Our approach detects and tracks the moving objects after accurately recovering the geometric relationship between different sensors. We address the tracking problem by separately modeling the appearance and motion of the moving regions using stochastic models. The appearance of the detected blobs is described by multiple spatial distribution models of blobs' colors and edges from different sensors. This representation is invariant to 2D rigid and scale transformation. It provides a rich description of the object being tracked and produces an accurate blob similarity measure for tracking - especially when one of sensors fails to provide reliable information. The motion model is obtained using a Kalman Filter (KF) process, which predicts the position of the moving objects while taking into account the camera motion. Tracking is performed by the maximization of a joint probability model reflecting appearance and motion. The novelty of our approach consists in defining a Joint Probability Data Association Filter (JPDAF) for integrating multiple cues from multiple sensors, It provides an unified framework for fusing information from different types of sensors. The proposed method tracks multiple moving objects with partial and total occlusions under various illumination conditions. We demonstrate the performance of the system on several real video surveillance sequences.

[1]  Isaac Cohen,et al.  Continuous multi-views tracking using tensor voting , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[2]  R. Collins,et al.  On-line selection of discriminative tracking features , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[3]  B. S. Manjunath,et al.  A contour-based approach to multisensor image registration , 1995, IEEE Trans. Image Process..

[4]  Yaser Sheikh,et al.  ALIGNING ‘DISSIMILAR’ IMAGES DIRECTLY , 2003 .

[5]  Seong-Whan Lee,et al.  Multiple people tracking using an appearance model based on temporal color , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[6]  Larry S. Davis,et al.  Hydra: multiple people detection and tracking using silhouettes , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[7]  Gideon P. Stein,et al.  Tracking from multiple view points: Self-calibration of space and time , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[8]  Yong Rui,et al.  Better proposal distributions: object tracking using unscented particle filter , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  P. Anandan,et al.  About Direct Methods , 1999, Workshop on Vision Algorithms.

[10]  Gérard G. Medioni,et al.  Continuous tracking within and across camera streams , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[11]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[12]  Larry S. Davis,et al.  Probabilistic framework for segmenting people under occlusion , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[13]  Isaac Cohen,et al.  Inference of human postures by classification of 3D human body shape , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[14]  Jitendra Malik,et al.  Learning a discriminative classifier using shape context distances , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[15]  P. Anandan,et al.  Robust multi-sensor image alignment , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[16]  Jake K. Aggarwal,et al.  Automatic tracking of human motion in indoor scenes across multiple synchronized video streams , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[17]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.