Anatomy of a multicamera video surveillance system

Abstract.We present a framework for multicamera video surveillance. The framework consists of three phases: detection, representation, and recognition. The detection phase handles multisource spatiotemporal data fusion for efficiently and reliably extracting motion trajectories from video. The representation phase summarizes raw trajectory data to construct hierarchical, invariant, and content-rich descriptions of the motion events. Finally, the recognition phase deals with event classification and identification on the data descriptors. Through empirical study in a parking-lot-surveillance setting, we show that our spatiotemporal fusion scheme and biased sequence-data learning method are highly effective in identifying suspicious events.

[1]  Robert Grover Brown,et al.  Introduction to random signal analysis and Kalman filtering , 1983 .

[2]  Jason Weston,et al.  Mismatch String Kernels for SVM Protein Classification , 2002, NIPS.

[3]  Fadi Dornaika,et al.  Object Pose: The Link between Weak Perspective, Paraperspective, and Full Perspective , 1997, International Journal of Computer Vision.

[4]  Gang Xu,et al.  Epipolar Geometry in Stereo, Motion and Object Recognition , 1996, Computational Imaging and Vision.

[5]  Gerald Farin,et al.  Curves and surfaces for computer aided geometric design , 1990 .

[6]  Kenichi Kanatani.  Optimal Homography Computation with a Reliability Measure , 1998, MVA.

[7]  David Haussler,et al.  A Discriminative Framework for Detecting Remote Protein Homologies , 2000, J. Comput. Biol..

[8]  Vassilios Morellas,et al.  Two Examples of Indoor and Outdoor Surveillance Systems: Motivation, Design, and Testing , 2002 .

[9]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[10]  Yoshua Bengio,et al.  Markovian Models for Sequential Data , 2004 .

[11]  H.F. Durrant-Whyte,et al.  A new approach for filtering nonlinear systems , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[12]  Yoshiaki Shirai,et al.  Three-Dimensional Computer Vision , 1987, Symbolic Computation.

[13]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[15]  C. Watkins.  Dynamic Alignment Kernels , 1999 .

[16]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[17]  L.L. Lee.  A magnetic-particles display , 1975, IEEE Transactions on Electron Devices.

[18]  David Haussler,et al.  Using the Fisher Kernel Method to Detect Remote Protein Homologies , 1999, ISMB.

[19]  BozkayaTolga,et al.  Distance-based indexing for high-dimensional metric spaces , 1997 .

[20]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[21]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[22]  Christopher J. C. Burges,et al.  Geometry and invariance in kernel based methods , 1999 .

[23]  ZhangZhengyou.  A Flexible New Technique for Camera Calibration , 2000 .

[24]  J. Miller.  Numerical Analysis , 1966, Nature.

[25]  大野 義夫,et al.  Computer Graphics : Principles and Practice, 2nd edition, J.D. Foley, A.van Dam, S.K. Feiner, J.F. Hughes, Addison-Wesley, 1990 , 1991 .

[26]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[27]  Padhraic Smyth,et al.  Pattern discovery in sequences under a Markov assumption , 2002, KDD.

[28]  Geoffrey D. Sullivan,et al.  Filter for Car Tracking Based on Acceleration and Steering Angle , 1996, BMVC.

[29]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[30]  Bernhard Schölkopf,et al.  Dynamic Alignment Kernels , 2000 .

[31]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[32]  Xinhua Zhuang,et al.  Pose estimation from corresponding point data , 1989, IEEE Trans. Syst. Man Cybern..

[33]  Eleazar Eskin,et al.  The Spectrum Kernel: A String Kernel for SVM Protein Classification , 2001, Pacific Symposium on Biocomputing.

[34]  J. L. Roux.  An Introduction to the Kalman Filter , 2003 .

[36]  David Haussler,et al.  Exploiting Generative Models in Discriminative Classifiers , 1998, NIPS.

[37]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[38]  M. Pitt,et al.  Filtering via Simulation: Auxiliary Particle Filters , 1999 .

[39]  Pramod K. Varshney,et al.  Multisensor surveillance systems based on image and video data , 2002, Proceedings. International Conference on Image Processing.

[40]  Harvey Cohn,et al.  Conformal Mapping on Riemann Surfaces , 1967 .

[41]  Edward Y. Chang,et al.  Invariant feature extraction and biased statistical inference for video surveillance , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[42]  Lily Lee,et al.  Monitoring Activities from Multiple Video Streams: Establishing a Common Coordinate Frame , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Tieniu Tan,et al.  Visual vehicle tracking algorithm , 2002 .

[44]  Larry S. Davis,et al.  Model-based object pose in 25 lines of code , 1992, International Journal of Computer Vision.

[45]  Edward Y. Chang,et al.  Adaptive Feature-Space Conformal Transformation for Imbalanced-Data Learning , 2003, ICML.

[46]  Ronald Azuma,et al.  Predictive tracking for augmented reality , 1995 .

[47]  Si Wu,et al.  Improving support vector machine classifiers by modifying kernel functions , 1999, Neural Networks.

[48]  David Haussler,et al.  Probabilistic kernel regression models , 1999, AISTATS.

[49]  Ramin Zabih,et al.  Bayesian multi-camera surveillance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[50]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[51]  Jeffrey E. Boyd,et al.  MPI-Video infrastructure for dynamic environments , 1998, Proceedings. IEEE International Conference on Multimedia Computing and Systems (Cat. No.98TB100241).

[52]  Ioannis Pavlidis,et al.  Urban surveillance systems: from the laboratory to the commercial world , 2001, Proc. IEEE.

[53]  Takeo Kanade,et al.  A System for Video Surveillance and Monitoring , 2000 .

[54]  Tieniu Tan,et al.  Visual Vehicle Tracking Using An Improved EKF , 2002 .

[55]  Steven K. Feiner,et al.  Computer graphics: principles and practice (2nd ed.) , 1990 .

[56]  Michael Isard,et al.  ICONDENSATION: Unifying Low-Level and High-Level Tracking in a Stochastic Framework , 1998, ECCV.