Thermal-Visible Video Fusion for Moving Target Tracking and Pedestrian Classification

The paper presents a fusion-tracker and pedestrian classifier for color and thermal cameras. The tracker builds a background model as a multi-modal distribution of colors and temperatures. It is constructed as a particle filter that makes a number of informed reversible transformations to sample the model probability space in order to maximize posterior probability of the scene model. Observation likelihoods of moving objects account their 3D locations with respect to the camera and occlusions by other tracked objects as well as static obstacles. After capturing the coordinates and dimensions of moving objects we apply a pedestrian classifier based on periodic gait analysis. To separate humans from other moving objects, such as cars, we detect, in human gait, a symmetrical double helical pattern, that can then be analyzed using the Frieze Group theory. The results of tracking on color and thermal sequences demonstrate that our algorithm is robust to illumination noise and performs well in the outdoor environments.

[1]  Riad I. Hammoud,et al.  Robust Multi-Pedestrian Tracking in Thermal-Visible Surveillance Videos , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[2]  Bir Bhanu,et al.  Kinematic-based human motion analysis in infrared sequences , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[3]  Larry S. Davis,et al.  Probabilistic framework for segmenting people under occlusion , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[4]  Ramakant Nevatia,et al.  Tracking multiple humans in crowded environment , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[5]  Larry S. Davis,et al.  Pedestrian classification from moving platforms using cyclic motion pattern , 2005, IEEE International Conference on Image Processing 2005.

[6]  Xia Liu,et al.  Pedestrian detection and tracking with night vision , 2005, IEEE Transactions on Intelligent Transportation Systems.

[7]  Xin Li,et al.  Layered Representation for Pedestrian Detection and Tracking in Infrared Imagery , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[8]  M. Tuceryan,et al.  A Vision System for Automated Customer Tracking for Marketing Analysis: Low Level Feature Extraction , 2005 .

[9]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[10]  Andrew Blake,et al.  Tracking through singularities and discontinuities by random sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  James W. Davis,et al.  Fusion-Based Background-Subtraction using Contour Saliency , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[12]  J. Little,et al.  Recognizing People by Their Gait: The Shape of Motion , 1998 .

[13]  Michael Isard,et al.  BraMBLe: a Bayesian multiple-blob tracker , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[14]  James M. Rehg,et al.  Singularity analysis for articulated object tracking , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[15]  Larry S. Davis,et al.  M2Tracker: A Multi-view Approach to Segmenting and Tracking People in a Cluttered Scene Using Region-Based Stereo , 2002, ECCV.

[16]  Tieniu Tan,et al.  Recent developments in human motion analysis , 2003, Pattern Recognit..

[17]  Takeo Kanade,et al.  Introduction to the Special Section on Video Surveillance , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Swarup Medasani,et al.  Classifier Swarms for Human Detection in Infrared Imagery , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[19]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[20]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Fang Liu,et al.  Finding periodicity in space and time , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[22]  Larry S. Davis,et al.  Pedestrian Detection via Periodic Motion Analysis , 2007, International Journal of Computer Vision.

[23]  Tom Drummond,et al.  Multi-Modal Tracking using Texture Changes , 2004, BMVC.

[24]  Myron Flickner,et al.  Detection and tracking of shopping groups in stores , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[25]  Larry S. Davis,et al.  Probabilistic template based pedestrian detection in infrared videos , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[26]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[27]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Takeo Kanade,et al.  A System for Video Surveillance and Monitoring , 2000 .

[29]  Edward H. Adelson,et al.  Analyzing and recognizing walking figures in XYT , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Cristian Sminchisescu,et al.  Kinematic jump processes for monocular 3D human tracking , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[31]  Noboru Yasuda,et al.  Pedestrian Detection and Tracking in Far Infrared Images , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.