Event Detection and Analysis from Video Streams

We present a system which takes as input a video stream obtained from an airborne moving platform and produces an analysis of the behavior of the moving objects in the scene. To achieve this functionality, our system relies on two modular blocks. The first one detects and tracks moving regions in the sequence. It uses a set of features at multiple scales to stabilize the image sequence, that is, to compensate for the motion of the observer, then extracts regions with residual motion and uses an attribute graph representation to infer their trajectories. The second module takes as input these trajectories, together with user-provided information in the form of geospatial context and goal context to instantiate likely scenarios. We present details of the system, together with results on a number of real video sequences and also provide a quantitative analysis of the results.

[1]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[2]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[3]  Richard Szeliski,et al.  Image mosaicing for tele-reality applications , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[4]  Hironobu Fujiyoshi,et al.  Moving target classification and tracking from real-time video , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[5]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[6]  Gérard G. Medioni,et al.  Detecting and tracking moving objects for video surveillance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[7]  P. Anandan,et al.  Robust multi-sensor image alignment , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[8]  Cordelia Schmid,et al.  Comparing and evaluating interest points , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[9]  Ramakant Nevatia,et al.  Representation and optimal recognition of human activities , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[10]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[11]  Hans-Hellmut Nagel,et al.  From image sequences towards conceptual descriptions , 1988, Image Vis. Comput..

[12]  Wolfgang Wahlster,et al.  From Visual Input to Verbal Output in the Visual Translator , 2003 .

[13]  Daphna Weinshall,et al.  Motion of disturbances: detection and tracking of multi-body non-rigid motion , 1999, Machine Vision and Applications.

[14]  W. Eric L. Grimson,et al.  Using adaptive tracking to classify and monitor activities in a site , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[15]  Thomas M. Strat,et al.  Employing Contextual Information in Computer Vision , 1993 .

[16]  Yoshiaki Shirai,et al.  Three-Dimensional Computer Vision , 1987, Symbolic Computation.

[17]  Antony Galton,et al.  Towards an Integrated Logic of Space, Time and Motion , 1993, IJCAI.

[18]  Donald Reid.  An algorithm for tracking multiple targets , 1978 .

[19]  Ingemar J. Cox,et al.  An Efficient Implementation of Reid's Multiple Hypothesis Tracking Algorithm and Its Evaluation for the Purpose of Visual Tracking , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Bernd Neumann.  Semantic Structures: Advances in Natural Language Processing , 1989 .

[21]  Daniel P. Huttenlocher,et al.  Tracking non-rigid objects in complex scenes , 1993, 1993 (4th) International Conference on Computer Vision.

[22]  P. Anandan,et al.  Adaptive-complexity registration of images , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[23]  James W. Davis,et al.  The representation and recognition of human movement using temporal templates , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Larry S. Davis,et al.  W4S : A real-time system for detecting and tracking people in 2 D , 1998, eccv 1998.

[25]  Michal Irani,et al.  Detecting and Tracking Multiple Moving Objects Using Temporal Integration , 1992, ECCV.

[26]  P. Anandan,et al.  Mosaic based representations of video sequences and their applications , 1995, Proceedings of IEEE International Conference on Computer Vision.

[27]  Isabelle Herlin,et al.  Optical Flow and Phase Portrait Methods for Environmental Satellite Image Sequences , 1996, ECCV.

[28]  Rachid Deriche,et al.  Using geometric corners to build a 2D mosaic from a set of images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Isaac Cohen,et al.  Detecting and Tracking Moving Objects in Video from an Airborne Observer , 1998 .

[30]  Hilary Buxton,et al.  Visual Surveillance Monitoring and Watching , 1996, ECCV.

[31]  L. Davis,et al.  W 4 S: a Real-time System for Detecting and Tracking People in 2 1 2 D , 1998 .

[32]  Alex Pentland,et al.  Coupled hidden Markov models for complex action recognition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  Aaron F. Bobick,et al.  Nonlinear PHMMs for the interpretation of parameterized gesture , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[34]  Shaogang Gong,et al.  Visual Surveillance in a Dynamic and Uncertain World , 1995, Artif. Intell..

[35]  Shmuel Peleg,et al.  Motion based segmentation , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[36]  Aaron F. Bobick,et al.  Visual recognition of multi-agent action using binary temporal relations , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[37]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[38]  Claudio S. Pinhanez,et al.  Human action detection using PNF propagation of temporal constraints , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[39]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[40]  Larry S. Davis,et al.  W4S: A real-time system detecting and tracking people in 2 1/2D , 1998, ECCV.

[41]  David L. Waltz.  Semantic Structures: Advances in Natural Language Processing , 1989 .

[42]  Takeo Kanade,et al.  Advances in Cooperative Multi-Sensor Video Surveillance , 1999 .

[43]  R. Chellappa,et al.  Fast 3D stabilization and mosaic construction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  Ingemar J. Cox,et al.  On Finding Ranked Assignments With Application to Multi-Target Tracking and Motion Correspondence , 1995 .

[45]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[46]  James W. Davis,et al.  Real-time closed-world tracking , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[47]  G. Medioni,et al.  Scenario Recognition in Airborne Video Imagery , 2003 .

[48]  Azriel Rosenfeld,et al.  Visual surveillance and monitoring , 1998 .

[49]  P. Anandan,et al.  A Unified Approach to Moving Object Detection in 2D and 3D Scenes , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[51]  Aaron F. Bobick,et al.  Action recognition using probabilistic parsing , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[52]  François Brémond,et al.  Issues of representing context illustrated by video-surveillance applications , 1998, Int. J. Hum. Comput. Stud..