Robust Multi-Pedestrian Tracking in Thermal-Visible Surveillance Videos

In this paper we introduce a system to track pedestrians using a combined input from RGB and thermal cameras. Two major contributions are presented here. First is the novel model of the scene background where each pixel is represented as a multi-modal distribution with the changing number of modalities for both color and thermal input. We demonstrate how to eliminate the influence of shadows with this type of fusion. Second, based on our background model we introduce a pedestrian tracker designed as a particle filter. We further develop a number of informed reversible transformations to sample the model probability space in order to maximize our model posterior probability. The novelty of our tracking approach also comes from a way we formulate observation likelihoods to account for 3D locations of the bodies with respect to the camera and occlusions by other tracked human bodies as well as static objects. The results of tracking on color and thermal sequences demonstrate that our algorithm is robust to illumination noise and performs well in the outdoor environments.

[1]  Takeo Kanade,et al.  A System for Video Surveillance and Monitoring , 2000 .

[2]  Larry S. Davis,et al.  Pedestrian Detection via Periodic Motion Analysis , 2007, International Journal of Computer Vision.

[3]  Tom Drummond,et al.  Multi-Modal Tracking using Texture Changes , 2004, BMVC.

[4]  Myron Flickner,et al.  Detection and tracking of shopping groups in stores , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[5]  Larry S. Davis,et al.  Background modeling and subtraction by codebook construction , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[6]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  James M. Rehg,et al.  Singularity analysis for articulated object tracking , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[8]  渡辺 亮平,et al.  Sequential Monte Carlo , 2005, Nonlinear Time Series Analysis.

[9]  Larry S. Davis,et al.  M2Tracker: A Multi-view Approach to Segmenting and Tracking People in a Cluttered Scene Using Region-Based Stereo , 2002, ECCV.

[10]  Larry S. Davis,et al.  Probabilistic framework for segmenting people under occlusion , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[11]  Takeo Kanade,et al.  Introduction to the Special Section on Video Surveillance , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  J. Halton Sequential Monte Carlo , 1962, Mathematical Proceedings of the Cambridge Philosophical Society.

[13]  Swarup Medasani,et al.  Classifier Swarms for Human Detection in Infrared Imagery , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[14]  Bir Bhanu,et al.  Kinematic-based human motion analysis in infrared sequences , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[15]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[16]  Xin Li,et al.  Layered Representation for Pedestrian Detection and Tracking in Infrared Imagery , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[17]  Michael Isard,et al.  BraMBLe: a Bayesian multiple-blob tracker , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[18]  Larry S. Davis,et al.  Probabilistic template based pedestrian detection in infrared videos , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[19]  Andrew Blake,et al.  Tracking through singularities and discontinuities by random sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[20]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[21]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Xia Liu,et al.  Pedestrian detection and tracking with night vision , 2005, IEEE Transactions on Intelligent Transportation Systems.

[23]  M. Tuceryan,et al.  A Vision System for Automated Customer Tracking for Marketing Analysis: Low Level Feature Extraction , 2005 .

[24]  James W. Davis,et al.  Fusion-Based Background-Subtraction using Contour Saliency , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[25]  Ramakant Nevatia,et al.  Tracking multiple humans in crowded environment , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[26]  Cristian Sminchisescu,et al.  Kinematic jump processes for monocular 3D human tracking , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[27]  Noboru Yasuda,et al.  Pedestrian Detection and Tracking in Far Infrared Images , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.