Human tracking driven activity recognition in video streams

In this paper we describe a method to visually recognize activities from video streams driven by a robust tracking algorithm. The tracking algorithm uses object flow as a motion model that estimates the displacement and the direction of objects in imagery streams; as observation model, object flow is complemented by a generative prior based on the Gaussian Process Latent Variable Model for increased robustness to visual alterations of the tracked object (human). We then use tracking results as input to drive the recognition of activities by hidden Markov models-based classifiers. We evaluate the performance of the examined approach under real-life activity recognition scenarios in visually complex environments characterized by lighting changes, occlusions and outliers.

[1]  Zhihong Zeng,et al.  Audio–Visual Affective Expression Recognition Through Multistream Fused HMM , 2008, IEEE Transactions on Multimedia.

[2]  S. L. Phung,et al.  A novel skin color model in YCbCr color space and its application to human face detection , 2002, Proceedings. International Conference on Image Processing.

[3]  Athanasios Voulodimos,et al.  A System for Multicamera Task Recognition and Summarization for Structured Environments , 2013, IEEE Transactions on Industrial Informatics.

[4]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[5]  Luc Van Gool,et al.  Online classification of visual tasks for industrial workflow monitoring , 2011, Neural Networks.

[6]  Guo Li,et al.  Tracking video objects with feature points based particle filtering , 2010, Multimedia Tools and Applications.

[7]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[8]  Theodora A. Varvarigou,et al.  Efficient tracking using a robust motion estimation technique , 2012, Multimedia Tools and Applications.

[9]  James J. Little,et al.  Tracking and recognizing actions of multiple hockey players using the boosted particle filter , 2009, Image Vis. Comput..

[10]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Yanxi Liu,et al.  Online Selection of Discriminative Tracking Features , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Nassir Navab,et al.  Workflow monitoring based on 3D motion features , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[13]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[14]  Yuan Li,et al.  Tracking in Low Frame Rate Video: A Cascade Particle Filter with Discriminative Observers of Different Lifespans , 2007, CVPR.

[15]  Theodora A. Varvarigou,et al.  A Threefold Dataset for Activity and Workflow Recognition in Complex Industrial Environments , 2012, IEEE MultiMedia.

[16]  Ming Yang,et al.  Detection driven adaptive multi-cue integration for multiple human tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  Joaquin Quiñonero Candela,et al.  Local distance preservation in the GP-LVM through back constraints , 2006, ICML.

[18]  Ming-Hsuan Yang,et al.  Visual tracking with online Multiple Instance Learning , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Frank Dellaert,et al.  An MCMC-Based Particle Filter for Tracking Multiple Interacting Targets , 2004, ECCV.

[20]  Amit K. Roy-Chowdhury,et al.  Tracking and Activity Recognition Through Consensus in Distributed Camera Networks , 2010, IEEE Transactions on Image Processing.

[21]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[22]  Mubarak Shah,et al.  Human Action Recognition in Videos Using Kinematic Features and Multiple Instance Learning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Horst Bischof,et al.  On-line Boosting and Vision , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[24]  Takashi Matsuyama,et al.  Multiobject Behavior Recognition by Event Driven Selective Attention Method , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Yang Wang,et al.  Human Action Recognition by Semilatent Topic Models , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Silvio Savarese,et al.  A Unified Framework for Multi-target Tracking and Collective Activity Recognition , 2012, ECCV.

[27]  Sotirios Chatzis,et al.  Robust Sequential Data Modeling Using an Outlier Tolerant Hidden Markov Model , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Neil D. Lawrence,et al.  Probabilistic Non-linear Principal Component Analysis with Gaussian Process Latent Variable Models , 2005, J. Mach. Learn. Res..

[29]  Kyoung Mu Lee,et al.  Visual tracking via geometric particle filtering on the affine group with optimal importance functions , 2009, CVPR.

[30]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[31]  Stefan Roth,et al.  People-tracking-by-detection and people-detection-by-tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Luc Van Gool,et al.  A Hough transform-based voting framework for action recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.