Spatiotemporal energy modeling for foreground segmentation in multiple object tracking

In this paper, we introduce spatiotemporal energy modeling for foreground segmentation in multiple object tracking, a high accuracy and real-time foreground target extraction algorithm. From a single video sequence with multiple moving objects and stationary background, our algorithm combines spatial (color distribution) and temporal (variety between two consecutive frames) information to extract foreground objects accurately and efficiently. The key idea of our method is to employ tracking results as feedback cues for target detection in the next frame, which adaptively updates the weights and threshold. Using spatiotemporal energy modeling, the foreground extraction errors caused by ambiguous colors in foreground and background boundary and abnormal movements can be substantially reduced. Experimental results of complex scenario video demonstrate the effectiveness of our algorithm.

[1]  Wei Hua,et al.  Confidence-Based Color Modeling for Online Video Segmentation , 2009, ACCV.

[2]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[3]  Amir Averbuch,et al.  Interacting Multiple Model Methods in Target Tracking: A Survey , 1988 .

[4]  S.S. Blackman,et al.  Multiple hypothesis tracking for multiple target tracking , 2004, IEEE Aerospace and Electronic Systems Magazine.

[5]  Derek R. Magee,et al.  Tracking multiple vehicles using foreground, background and motion models , 2004, Image Vis. Comput..

[6]  Jan-Olof Eklundh,et al.  Statistical background subtraction for a mobile observer , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[7]  Yajun Fang,et al.  A layered-based fusion-based approach to detect and track the movements of pedestrians through partially occluded situations , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[8]  James Ferryman,et al.  Proceedings of the thirteenth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance , 2009 .

[9]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[10]  Sei-Wang Chen,et al.  Pedestrian detection and tracking at crossroads , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[11]  Zoran Zivkovic,et al.  Improved adaptive Gaussian mixture model for background subtraction , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[12]  Jing Zhang,et al.  Framework for Performance Evaluation of Face, Text, and Vehicle Detection and Tracking in Video: Data, Metrics, and Protocol , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Osama Masoud,et al.  A novel method for tracking and counting pedestrians in real-time using a single camera , 2001, IEEE Trans. Veh. Technol..

[14]  Harry Shum,et al.  Background Cut , 2006, ECCV.

[15]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .