Smart Camera Networks in Virtual Reality

This paper presents our research towards smart camera networks capable of carrying out advanced surveillance tasks with little or no human supervision. A unique centerpiece of our work is the combination of computer graphics, artificial life, and computer vision simulation technologies to develop such networks and experiment with them. Specifically, we demonstrate a smart camera network comprising static and active simulated video surveillance cameras that provides extensive coverage of a large virtual public space, a train station populated by autonomously self-animating virtual pedestrians. The realistically simulated network of smart cameras performs persistent visual surveillance of individual pedestrians with minimal intervention. Our innovative camera control strategy naturally addresses camera aggregation and handoff, is robust against camera and communication failures, and requires no camera calibration, detailed world model, or central controller.

[1]  C. Siva Ram Murthy,et al.  Ad Hoc Wireless Networks: Architectures and Protocols , 2004 .

[2]  A.S. Willsky,et al.  Nonparametric belief propagation for self-calibration in sensor networks , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[3]  Deborah Estrin,et al.  Directed diffusion for wireless sensor networking , 2003, TNET.

[4]  David J. Fleet,et al.  Learning Sensor Network Topology through Monte Carlo Expectation Maximization , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[5]  Mubarak Shah,et al.  KNIGHT/spl trade/: a real time surveillance system for multiple and non-overlapping cameras , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[6]  C. Diehl,et al.  Scheduling an active camera to observe people , 2004, VSSN '04.

[7]  Takeo Kanade,et al.  An active camera system for acquiring multi-view video , 2002, Proceedings. International Conference on Image Processing.

[8]  T. Kanade,et al.  A master-slave system to acquire biometric imagery of humans at distance , 2003, IWVS '03.

[9]  Takeo Kanade,et al.  Algorithms for cooperative multisensor surveillance , 2001, Proc. IEEE.

[10]  Demetri Terzopoulos,et al.  Surveillance camera scheduling: a virtual vision approach , 2005, Multimedia Systems.

[11]  Martin W. P. Savelsbergh,et al.  A combined Lagrangian, linear programming, and implication heuristic for large-scale set partitioning problems , 1996, J. Heuristics.

[12]  Makoto Yokoo,et al.  Adopt: asynchronous distributed constraint optimization with quality guarantees , 2005, Artif. Intell..

[13]  Mohan M. Trivedi,et al.  Intelligent environments and active camera networks , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[14]  Reid G. Smith,et al.  The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver , 1980, IEEE Transactions on Computers.

[15]  Makoto Yokoo,et al.  Distributed Constraint Satisfaction: Foundations of Cooperation in Multi-agent Systems , 2000 .

[16]  Makoto Yokoo,et al.  Distributed Constraint Satisfaction , 2000, Springer Series on Agent Technology.

[17]  Norman Abramson,et al.  The ALOHA SYSTEM. , 1972 .

[18]  Erwin B. Bellers,et al.  Deinterlacing-an overview , 1998, Proc. IEEE.

[19]  Roberto Brunelli,et al.  Olympus: an ambient intelligence architecture on the verge of reality , 2003, 12th International Conference on Image Analysis and Processing, 2003.Proceedings..

[20]  Demetri Terzopoulos,et al.  Animat vision: Active vision in artificial animals , 1995, Proceedings of IEEE International Conference on Computer Vision.

[21]  A. El-Rabbany,et al.  Mobile vision-based vehicle tracking and traffic control , 2002, Proceedings. The IEEE 5th International Conference on Intelligent Transportation Systems.

[22]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[23]  Isaac Cohen,et al.  Multi-views tracking within and across uncalibrated camera streams , 2003, IWVS '03.

[24]  Demetri Terzopoulos,et al.  Smart Camera Networks in Virtual Reality , 2007, Proceedings of the IEEE.

[25]  Dorin Comaniciu,et al.  Adaptive Resolution System for Distributed Surveillance , 2002, Real Time Imaging.

[26]  Sharath Pankanti,et al.  Face cataloger: multi-scale imaging for relating identity to location , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[27]  Tarak Gandhi,et al.  Calibration of a reconfigurable array of omnidirectional cameras using a moving person , 2004, VSSN '04.

[28]  Srinivasan Seshan,et al.  IrisNet: an internet-scale architecture for multimedia sensors , 2005, MULTIMEDIA '05.

[29]  Demetri Terzopoulos,et al.  Autonomous pedestrians , 2007, Graph. Model..

[30]  Augusto Sarti,et al.  Accurate and simple geometric calibration of multi-camera systems , 1999, Signal Process..

[31]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[32]  Faber Birren,et al.  Color perception in art , 1976 .

[33]  Demetri Terzopoulos.  Perceptive agents and systems in virtual reality , 2003, VRST '03.

[34]  Feng Zhao,et al.  Information-Driven Dynamic Sensor Collaboration for Tracking Applications , 2002 .

[35]  David S. Johnson,et al.  `` Strong '' NP-Completeness Results: Motivation, Examples, and Implications , 1978, JACM.

[36]  Mubarak Shah,et al.  Consistent Labeling of Tracked Objects in Multiple Cameras with Overlapping Fields of View , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  V. Michael Bove,et al.  The role of groups in smart camera networks , 2006 .

[38]  Prashant J. Shenoy,et al.  SensEye: a multi-tier camera sensor network , 2005, ACM Multimedia.

[39]  Anantha Chandrakasan,et al.  Upper bounds on the lifetime of sensor networks , 2001, ICC 2001. IEEE International Conference on Communications. Conference Record (Cat. No.01CH37240).

[40]  Feng Zhao,et al.  Information-driven dynamic sensor collaboration , 2002, IEEE Signal Process. Mag..

[41]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[42]  Haeyong Chung,et al.  Distributed metric calibration of ad hoc camera networks , 2006, TOSN.

[43]  William H. Widen.  Smart Cameras and the Right to Privacy , 2008, Proceedings of the IEEE.

[44]  F. F. Kuo,et al.  The ALOHA system , 1974, CCRV.

[45]  Leonidas J. Guibas,et al.  Collaborative signal and information processing: an information-directed approach , 2003 .