Detecting motion patterns via direction maps with application to surveillance

Detection of motion patterns in video data can be significantly simplified by abstracting away from pixel intensity values towards representations that explicitly and compactly capture movement across space and time. A novel representation that captures the spatiotemporal distributions of motion across regions of interest, called the ''Direction Map,'' abstracts video data by assigning a two-dimensional vector, representative of local direction of motion, to quantized regions in space-time. Methods are presented for recovering direction maps from video, constructing direction map templates (defining target motion patterns of interest) and comparing templates to newly acquired video (for pattern detection and localization). These methods have been successfully implemented and tested (with real-time considerations) on over 6300 frames across seven surveillance/traffic videos, detecting potential targets of interest as they traverse the scene in specific ways. Results show an overall recognition rate of approximately 91% hits vs 8% false positives.

[1]  Hilary Buxton,et al.  Conceptual descriptions from monitoring and watching image sequences , 2000, Image Vis. Comput..

[2]  Andrew J. Bulpitt,et al.  Learning spatio-temporal patterns for predicting object behaviour , 2000, Image Vis. Comput..

[3]  Hans-Hellmut Nagel,et al.  Image sequence evaluation: 30 years and still going strong , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[4]  Vladimir Pavlovic,et al.  Discovering clusters in motion time-series data , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[5]  Tomaso A. Poggio,et al.  A pattern classification approach to dynamical object detection , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Steven S. Beauchemin,et al.  The computation of optical flow , 1995, CSUR.

[7]  Shaogang Gong,et al.  Learning pixel-wise signal energy for understanding semantics , 2003, Image Vis. Comput..

[8]  Leif Haglund,et al.  Adaptive Multidimensional Filtering , 1991 .

[9]  Patrick Bouthemy,et al.  Detection of meaningful events in videos based on a supervised classification approach , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[10]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  John K. Tsotsos,et al.  Detecting Motion Patterns via Direction Maps with Application to Surveillance , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[12]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[13]  Hans Knutsson,et al.  Signal processing for computer vision , 1994 .

[14]  Larry S. Davis,et al.  Hydra: multiple people detection and tracking using silhouettes , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[15]  Bernt Schiele,et al.  Automatic Detection and Tracking of Abandoned Objects , 2003 .

[16]  Randal C. Nelson,et al.  Recognizing activities , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[17]  B. Buxton,et al.  Monocular depth perception from optical flow by space time signal processing , 1983, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[18]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  J. Little,et al.  Recognizing People by Their Gait: The Shape of Motion , 1998 .

[20]  Larry S. Davis,et al.  A fast background scene modeling and maintenance for outdoor surveillance , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[21]  Ramakant Nevatia,et al.  Bayesian framework for video surveillance application , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[22]  Larry S. Davis,et al.  W/sup 4/: Who? When? Where? What? A real time system for detecting and tracking people , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[23]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Matthew Brand,et al.  Minimum-entropy models of scene activity , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[25]  Ramakant Nevatia,et al.  Event Detection and Analysis from Video Streams , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Ivan Laptev,et al.  Velocity adaptation of spatio-temporal receptive fields for direct recognition of activities: an experimental study , 2004, Image Vis. Comput..

[27]  Nicola J. Ferrier,et al.  Multidimensional motion segmentation and identification , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[28]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[29]  Seiji Ishikawa,et al.  Recognizing human behavior using universal eigenspace , 2002, Object recognition supported by user interaction for service robots.

[30]  Jianbo Shi,et al.  Detecting unusual activity in video , 2004, CVPR 2004.

[31]  Theodore E. Cohn,et al.  Detection and identification , 1985 .

[32]  T. Poggio,et al.  Visual hyperacuity: spatiotemporal interpolation in human vision , 1981, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[33]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[34]  James L. Crowley,et al.  A Probabilistic Sensor for the Perception and Recognition of Activities , 2000, ECCV.

[35]  Bo Yang,et al.  A real-time vision system for automatic traffic monitoring based on 2D spatio-temporal images , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[36]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[37]  Lihi Zelnik-Manor,et al.  Event-based analysis of video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[38]  Jitendra Malik,et al.  Recognizing action at a distance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[39]  Paul A. Viola,et al.  Detecting Pedestrians Using Patterns of Motion and Appearance , 2005, International Journal of Computer Vision.

[40]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[41]  Michael Isard,et al.  Learning and Classification of Complex Dynamics , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Robert Pless,et al.  Analysis of Persistent Motion Patterns Using the 3D Structure Tensor , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[43]  M. Thonnat,et al.  Video sequence interpretation for visual surveillance , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[44]  Matthew Brand,et al.  Discovery and Segmentation of Activities in Video , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[46]  Konstantinos G. Derpanis,et al.  Three-dimensional nth derivative of Gaussian separable steerable filters , 2005, IEEE International Conference on Image Processing 2005.

[47]  Lauren Christopher,et al.  A VLSI median filter for impulse noise elimination in composite or component TV signals , 1988 .

[48]  Hans-Hellmut Nagel,et al.  Combination of Edge Element and Optical Flow Estimates for 3D-Model-Based Vehicle Tracking in Traffic Image Sequences , 1999, International Journal of Computer Vision.

[49]  Adam Baumberg,et al.  Learning deformable models for tracking human motion , 1996 .

[50]  Larry S. Davis,et al.  Real Time Computer Surveillance for Crime Detection , 2002 .

[51]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[52]  François Brémond,et al.  Group behavior recognition with multiple cameras , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[53]  Larry S. Davis,et al.  Backpack: detection of people carrying objects using silhouettes , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[54]  David C. Hogg,et al.  Learning the distribution of object trajectories for event recognition , 1996, Image Vis. Comput..

[55]  Hiroshi Murase,et al.  Visual learning and recognition of 3-d objects from appearance , 2005, International Journal of Computer Vision.

[56]  Hans-Hellmut Nagel,et al.  Tracking of complex driving manoeuvres in traffic image sequences , 1998, Image Vis. Comput..

[57]  Michael J. Black,et al.  Parameterized Modeling and Recognition of Activities , 1999, Comput. Vis. Image Underst..

[58]  N. Hoose,et al.  Automatically extracting traffic data from video-tape using the CLIP4 parallel image processor , 1987, Pattern Recognit. Lett..

[59]  Patrick Bouthemy,et al.  Motion Recognition Using Nonparametric Image Motion Models Estimated from Temporal and Multiscale Cooccurrence Statistics , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[60]  Anthony G. Cohn,et al.  Constructing qualitative event models automatically from video input , 2000, Image Vis. Comput..

[61]  Jitendra Malik,et al.  Towards robust automatic traffic scene analysis in real-time , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[62]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..