"Shape Activity": a continuous-state HMM for moving/deforming shapes with application to abnormal activity detection

The aim is to model "activity" performed by a group of moving and interacting objects (which can be people, cars, or different rigid components of the human body) and use the models for abnormal activity detection. Previous approaches to modeling group activity include co-occurrence statistics (individual and joint histograms) and dynamic Bayesian networks, neither of which is applicable when the number of interacting objects is large. We treat the objects as point objects (referred to as "landmarks") and propose to model their changing configuration as a moving and deforming "shape" (using Kendall's shape theory for discrete landmarks). A continuous-state hidden Markov model is defined for landmark shape dynamics in an activity. The configuration of landmarks at a given time forms the observation vector, and the corresponding shape and the scaled Euclidean motion parameters form the hidden-state vector. An abnormal activity is then defined as a change in the shape activity model, which could be slow or drastic and whose parameters are unknown. Results are shown on a real abnormal activity-detection problem involving multiple moving objects.

[1]  Anuj Srivastava,et al.  Geometric Filtering for Subspace Tracking , 2000 .

[2]  Ramakant Nevatia,et al.  Description and tracking of moving articulated objects , 1994, Systems and Computers in Japan.

[3]  Aaron F. Bobick,et al.  Action recognition using probabilistic parsing , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[4]  Stefano Soatto,et al.  Monte Carlo filtering on Lie groups , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[5]  Stefano Soatto,et al.  DEFORMOTION: Deforming Motion, Shape Average and the Joint Registration and Segmentation of Images , 2002, ECCV.

[6]  N. Vaswani,et al.  Change detection in partially observed nonlinear dynamic systems with unknown change parameters , 2004, Proceedings of the 2004 American Control Conference.

[7]  F. Gland,et al.  STABILITY AND UNIFORM APPROXIMATION OF NONLINEAR FILTERS USING THE HILBERT METRIC AND APPLICATION TO PARTICLE FILTERS1 , 2004 .

[8]  Rikard Berthilsson,et al.  A Statistical Theory of Shape , 1998, SSPR/SPR.

[9]  Lihi Zelnik-Manor,et al.  Event-based analysis of video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[10]  Sudhir P. Mudur,et al.  Mathematical Elements for Computer Graphics , 1985, Advances in Computer Graphics.

[11]  B. Azimi-Sadjadi,et al.  Change detection for nonlinear systems; a particle filtering approach , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[12]  R. Chellappa,et al.  NonStationary “ Shape Activities ” , 2005 .

[13]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[14]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[15]  I. Miller.  Probability, Random Variables, and Stochastic Processes , 1966 .

[16]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[17]  Rama Chellappa,et al.  Probabilistic Human Recognition from Video , 2002, ECCV.

[18]  Namrata Vaswani.  Bound on errors in particle filtering with incorrect model assumptions and its implication for change detection , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[19]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[20]  Vijay Kumar,et al.  Cooperative localization and control for multi-robot manipulation , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[21]  M. Alex O. Vasilescu,et al.  Recognizing action events from multiple viewpoints , 2001, Proceedings IEEE Workshop on Detection and Recognition of Events in Video.

[22]  Arnaud Doucet,et al.  A survey of convergence results on particle filtering methods for practitioners , 2002, IEEE Trans. Signal Process..

[23]  Yi Zhou,et al.  Bayesian tangent shape model: estimating shape and pose parameters via Bayesian inference , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[24]  David G. Kendall,et al.  Shape & Shape Theory , 1999 .

[25]  Andrew Blake,et al.  A probabilistic contour discriminant for object localisation , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[26]  Wolfram Burgard,et al.  Tracking multiple moving targets with a mobile robot using particle filters and statistical data association , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[27]  Rama Chellappa,et al.  Statistical shape theory for activity modeling , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[28]  Stefano Soatto,et al.  Deformotion: Deforming Motion, Shape Average and the Joint Registration and Approximation of Structures in Images , 2003, International Journal of Computer Vision.

[29]  Bruno O. Shubert,et al.  Random variables and stochastic processes , 1979 .

[30]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[31]  W. Eric L. Grimson,et al.  Using adaptive tracking to classify and monitor activities in a site , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[32]  Namrata Vaswani,et al.  Slow and Drastic Change Detection in General HMMs Using Particle Filters with Unknown Change Parameters , 2004 .

[33]  Lorenzo Torresani,et al.  Space-Time Tracking , 2002, ECCV.

[34]  R. Chellappa,et al.  Non-Stationary "Shape Activities" , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[35]  J. Kent.  The Complex Bingham Distribution and Shape Analysis , 1994 .

[36]  Maria Huhtala,et al.  Random Variables and Stochastic Processes , 2021, Matrix and Tensor Decompositions in Signal Processing.

[37]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[38]  Timothy F. Cootes,et al.  Training Models of Shape from Sets of Examples , 1992, BMVC.

[39]  Jitendra Malik,et al.  Automatic Symbolic Traffic Scene Analysis Using Belief Networks , 1994, AAAI.

[40]  Anuj Srivastava,et al.  Bayesian and geometric subspace tracking , 2004, Advances in Applied Probability.

[41]  A. Doucet,et al.  A survey of convergence results on particle ltering for practitioners , 2002 .

[42]  Rama Chellappa,et al.  Activity recognition using the dynamics of the configuration of interacting objects , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[43]  Michèle Basseville,et al.  Detection of abrupt changes: theory and application , 1993 .

[44]  T. K. Carne,et al.  Shape and Shape Theory , 1999 .

[45]  Namrata Vaswani,et al.  Change Detection in Stochastic Shape Dynamical Models with Applications in Activity Modeling and Abnormality Detection , 2004 .

[46]  Michael J. Grimble.  IEEE conference on decision and control , 1987 .

[47]  Ralph Roskies,et al.  Fourier Descriptors for Plane Closed Curves , 1972, IEEE Transactions on Computers.

[48]  Christoph Bregler,et al.  Learning and recognizing human dynamics in video sequences , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.