An integrated surveillance system--human tracking and view synthesis using multiple omni-directional vision sensors

Abstract Accurate and efficient monitoring of dynamically changing environments is one of the most important requirements for visual surveillance systems. This paper describes the development of an integrated system for this monitoring purpose. The system consists of multiple omni-directional vision sensors and was developed to address two specific surveillance tasks: (1) robust tracking and profiling of human activities; (2) dynamic synthesis of virtual views for observing the environment from arbitrary vantage points.

[1]  R. Collins,et al.  Using a DEM to Determine Geospatial Object Trajectories , 1999 .

[2]  Takeo Kanade,et al.  A stereo machine for video-rate dense depth mapping and its new applications , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[4]  Mohan M. Trivedi,et al.  Omnidirectional image-based modeling: three approaches to approximated plenoptic representations , 2003, Machine Vision and Applications.

[5]  Mohan M. Trivedi,et al.  Depth extraction using a single moving camera: an integration of depth from motion and depth from stereo , 2005, Machine Vision and Applications.

[6]  H. Ishiguro,et al.  Development of low-cost and compact omnidirectional vision sensors and their applications , 1998 .

[7]  Hironobu Fujiyoshi,et al.  Moving target classification and tracking from real-time video , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[8]  Hiroshi Ishiguro.  Distributed Vision System: A Perceptual Information Infrastructure for Robot Navigation , 1997, IJCAI.

[9]  Mohan M. Trivedi,et al.  Multi-Primitive Hierarchical (MPH) Stereo Analysis , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Terrance E. Boult.  Remote Reality Demonstration , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[11]  Kiriakos N. Kutulakos,et al.  Plenoptic Image Editing , 2004, International Journal of Computer Vision.

[12]  Mohan M. Trivedi,et al.  3D ranging and virtual view generation using omniview cameras , 1999, Other Conferences.

[13]  S. B. Kang,et al.  Panoramic vision : sensors, theory, and applications , 2001 .

[14]  W. Eric L. Grimson,et al.  Using adaptive tracking to classify and monitor activities in a site , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[15]  Mohan M. Trivedi,et al.  Three-dimensional visual modeling and virtual view synthesis: a synergetic, range-space stereo approach using omni-directional images , 2000 .

[16]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[17]  Robert T. Collins,et al.  Cooperative Multi-Sensor Video Surveillance , 1999 .

[18]  Mohan M. Trivedi,et al.  3 D Ranging and Virtual View Generation using Omni-view Cameras , 1999 .

[19]  Takeo Kanade,et al.  Constructing virtual worlds using dense stereo , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[20]  Hiroshi Ishiguro,et al.  Identifying and localizing robots in a multi-robot system environment , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[21]  Jeffrey E. Boyd,et al.  MPI-Video infrastructure for dynamic environments , 1998, Proceedings. IEEE International Conference on Multimedia Computing and Systems (Cat. No.98TB100241).

[22]  Takeo Kanade,et al.  A multiple-baseline stereo , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Mohan M. Trivedi,et al.  Range-space approach for generalized multiple baseline stereo and direct virtual view synthesis , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[24]  Mohan M. Trivedi,et al.  Generalized Multiple Baseline Stereo and Direct Virtual View Synthesis Using Range-Space Search, Match, and Render , 2004, International Journal of Computer Vision.

[25]  S. Shams.  Neural network optimization for multi-target multi-sensor passive tracking , 1996 .

[26]  O. L. Davies,et al.  Statistical Methods in Research and Production, with Special Reference to the Chemical Industry. , 1948 .

[27]  Hiroshi Mizoguchi,et al.  Action recognition system based on human finder and human tracker , 1997, Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems. Innovative Robotics for Real-World Applications. IROS '97.

[28]  Mohan M. Trivedi,et al.  Intelligent environments and active camera networks , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[29]  Mohan M. Trivedi,et al.  N-Ocular stereo for real-time human tracking , 2001 .

[30]  Hiroshi Ishiguro,et al.  Omni-Directional Stereo , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Mohan M. Trivedi,et al.  Multiple Omni-Directional Vision Sensors (ODVS) based Visual Modeling Approach , 1999 .

[32]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .