Summarization and Indexing of Human Activity Sequences

In order to summarize a video consisting of a sequence of different activities, there are three fundamental problems: tracking the objects of interest, detecting the activity change times and recognizing the new activity. This paper presents an algorithm for achieving all these three tasks simultaneously and presents results on how it can used for indexing and summarizing a real-life video sequence. Human activities are represented by a model for the dynamics of the shape of the human body contour. Measures are designed for detecting both gradual transitions and sudden changes between activity models.

[1]  Lihi Zelnik-Manor,et al.  Temporal Factorization vs. Spatial Factorization , 2004, ECCV.

[2]  Henry A. Kautz,et al.  Location-Based Activity Recognition using Relational Markov Networks , 2005, IJCAI.

[3]  Michael Harville,et al.  Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[4]  W. Eric L. Grimson,et al.  Using adaptive tracking to classify and monitor activities in a site , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[5]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[6]  R. Chellappa,et al.  NonStationary “ Shape Activities ” , 2005 .

[7]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[8]  Christopher G. Atkeson,et al.  Simultaneous Tracking and Activity Recognition (STAR) Using Many Anonymous, Binary Sensors , 2005, Pervasive.

[9]  N. Vaswani,et al.  Change detection in partially observed nonlinear dynamic systems with unknown change parameters , 2004, Proceedings of the 2004 American Control Conference.

[10]  Rama Chellappa,et al.  Visual tracking and recognition using appearance-adaptive models in particle filters , 2004, IEEE Transactions on Image Processing.

[11]  Rama Chellappa,et al.  "Shape Activity": a continuous-state HMM for moving/deforming shapes with application to abnormal activity detection , 2005, IEEE Transactions on Image Processing.

[12]  Mubarak Shah,et al.  A general framework for temporal video scene segmentation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[13]  R. Chellappa,et al.  Non-Stationary "Shape Activities" , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[14]  Mubarak Shah,et al.  Integrating and employing multiple levels of zoom for activity recognition , 2004, CVPR 2004.

[15]  Mubarak Shah,et al.  Detecting group activities using rigidity of formation , 2005, MULTIMEDIA '05.

[16]  James M. Rehg,et al.  A multiple hypothesis approach to figure tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[17]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[18]  Mubarak Shah,et al.  Integrating and employing multiple levels of zoom for activity recognition , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[19]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .