Activity recognition using the dynamics of the configuration of interacting objects

Monitoring activities using video data is an important surveillance problem. A special scenario is to learn the pattern of normal activities and detect abnormal events from a very low resolution video where the moving objects are small enough to be modeled as point objects in a 2D plane. Instead of tracking each point separately, we propose to model an activity by the polygonal 'shape' of the configuration of these point masses at any time t, and its deformation over time. We learn the mean shape and the dynamics of the shape change using hand-picked location data (no observation noise) and define an abnormality detection statistic for the simple case of a test sequence with negligible observation noise. For the more practical case where observation (point locations) noise is large and cannot be ignored, we use a particle filter to estimate the probability distribution of the shape given the noisy observations up to the current time. Abnormality detection in this case is formulated as a change detection problem. We propose a detection strategy that can detect both 'drastic' and 'slow' abnormalities. Our framework can be directly applied for object location data obtained using any type of sensors - visible, radar, infrared or acoustic.

[1]  Wolfram Burgard,et al.  Tracking multiple moving targets with a mobile robot using particle filters and statistical data association , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[2]  J. Kent.  The Complex Bingham Distribution and Shape Analysis , 1994 .

[3]  Rama Chellappa,et al.  Statistical shape theory for activity modeling , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[4]  Bruno O. Shubert,et al.  Random variables and stochastic processes , 1979 .

[5]  Lorenzo Torresani,et al.  Space-Time Tracking , 2002, ECCV.

[6]  T. K. Carne,et al.  Shape and Shape Theory , 1999 .

[7]  Ramakant Nevatia,et al.  Description and tracking of moving articulated objects , 1994, Systems and Computers in Japan.

[8]  Namrata Vaswani.  A linear classifier for Gaussian class conditional distributions with unequal covariance matrices , 2002, Object recognition supported by user interaction for service robots.

[9]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[10]  Ralph Roskies,et al.  Fourier Descriptors for Plane Closed Curves , 1972, IEEE Transactions on Computers.

[11]  David Mumford,et al.  Mathematical theories of shape: do they model perception? , 1991, Optics & Photonics.

[12]  Ian L. Dryden,et al.  Statistical shape analysis in high-level vision , 2003 .

[13]  Azriel Rosenfeld,et al.  3D object tracking using shape-encoded particle propagation , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[14]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[15]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[16]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[17]  W. Eric L. Grimson,et al.  Using adaptive tracking to classify and monitor activities in a site , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[18]  Ronen Basri,et al.  Determining the similarity of deformable shapes , 1998, Vision Research.

[19]  Nicholas Ayache,et al.  Tracking Points on Deformable Objects Using Curvature Information , 1992, ECCV.

[20]  Andrew Blake,et al.  A probabilistic contour discriminant for object localisation , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[21]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[22]  Berthold K. P. Horn.  Extended Gaussian images , 1984, Proceedings of the IEEE.

[23]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[24]  B. Azimi-Sadjadi,et al.  Change detection for nonlinear systems; a particle filtering approach , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[25]  Stefano Soatto,et al.  DEFORMOTION: Deforming Motion, Shape Average and the Joint Registration and Segmentation of Images , 2002, ECCV.

[26]  David G. Kendall,et al.  Shape & Shape Theory , 1999 .

[27]  Ramakant Nevatia,et al.  Multi-agent event recognition , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.