Smart Camera Networks in Virtual Reality

This paper presents our research towards smart camera networks capable of carrying out advanced surveillance tasks with little or no human supervision. A unique centerpiece of our work is the combination of computer graphics, artificial life, and computer vision simulation technologies to develop such networks and experiment with them. Specifically, we demonstrate a smart camera network comprising static and active simulated video surveillance cameras that provides extensive coverage of a large virtual public space, a train station populated by autonomously self-animating virtual pedestrians. The realistically simulated network of smart cameras performs persistent visual surveillance of individual pedestrians with minimal intervention. Our innovative camera control strategy naturally addresses camera aggregation and handoff, is robust against camera and communication failures, and requires no camera calibration, detailed world model, or central controller.

[1]  Prashant J. Shenoy,et al.  SensEye: a multi-tier camera sensor network , 2005, ACM Multimedia.

[2]  C. Diehl,et al.  Scheduling an active camera to observe people , 2004, VSSN '04.

[3]  Thomas Ertl,et al.  Computer Graphics - Principles and Practice, 3rd Edition , 2014 .

[4]  Faber Birren,et al.  Color perception in art , 1976 .

[5]  Demetri Terzopoulos,et al.  Virtual Vision and Smart Camera Networks , 2006 .

[6]  Roberto Brunelli,et al.  Olympus: an ambient intelligence architecture on the verge of reality , 2003, 12th International Conference on Image Analysis and Processing, 2003.Proceedings..

[7]  Peter Jeavons,et al.  A Survey of Tractable Constraint Satisfaction Problems , 1997 .

[8]  Demetri Terzopoulos,et al.  Active Perception in Virtual Humans , 2000 .

[9]  Anantha Chandrakasan,et al.  Upper bounds on the lifetime of sensor networks , 2001, ICC 2001. IEEE International Conference on Communications. Conference Record (Cat. No.01CH37240).

[10]  Sharath Pankanti,et al.  Face cataloger: multi-scale imaging for relating identity to location , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[11]  A.S. Willsky,et al.  Nonparametric belief propagation for self-calibration in sensor networks , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[12]  Makoto Yokoo,et al.  Adopt: asynchronous distributed constraint optimization with quality guarantees , 2005, Artif. Intell..

[13]  David J. Fleet,et al.  Learning Sensor Network Topology through Monte Carlo Expectation Maximization , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[14]  Mubarak Shah,et al.  Consistent Labeling of Tracked Objects in Multiple Cameras with Overlapping Fields of View , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Demetri Terzopoulos,et al.  Autonomous pedestrians , 2005, SCA '05.

[16]  C. Siva Ram Murthy,et al.  Ad Hoc Wireless Networks: Architectures and Protocols , 2004 .

[17]  Takeo Kanade,et al.  An active camera system for acquiring multi-view video , 2002, Proceedings. International Conference on Image Processing.

[18]  Leonidas J. Guibas,et al.  Collaborative signal and information processing: an information-directed approach , 2003 .

[19]  T. Kanade,et al.  A master-slave system to acquire biometric imagery of humans at distance , 2003, IWVS '03.

[20]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[21]  V. Michael Bove,et al.  The role of groups in smart camera networks , 2006 .

[22]  Mubarak Shah,et al.  KNIGHT/spl trade/: a real time surveillance system for multiple and non-overlapping cameras , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[23]  Isaac Cohen,et al.  Multi-views tracking within and across uncalibrated camera streams , 2003, IWVS '03.

[24]  Srinivasan Seshan,et al.  IrisNet: an internet-scale architecture for multimedia sensors , 2005, MULTIMEDIA '05.

[25]  Feng Zhao,et al.  Information-Driven Dynamic Sensor Collaboration for Tracking Applications , 2002 .

[26]  D. Terzopoulos,et al.  Surveillance camera scheduling: a virtual vision approach , 2005, VSSN@MM.

[27]  David S. Johnson,et al.  `` Strong '' NP-Completeness Results: Motivation, Examples, and Implications , 1978, JACM.

[28]  Tarak Gandhi,et al.  Calibration of a reconfigurable array of omnidirectional cameras using a moving person , 2004, VSSN '04.

[29]  Demetri Terzopoulos.  Perceptive agents and systems in virtual reality , 2003, VRST '03.

[30]  Mohan M. Trivedi,et al.  Intelligent environments and active camera networks , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[31]  Reid G. Smith,et al.  The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver , 1980, IEEE Transactions on Computers.

[32]  Feng Zhao,et al.  Information-driven dynamic sensor collaboration , 2002, IEEE Signal Process. Mag..

[33]  R. Brunelli,et al.  Synthetic Movies for Computer Vision Applications , 2003 .

[34]  Haeyong Chung,et al.  Distributed metric calibration of ad hoc camera networks , 2006, TOSN.

[35]  William H. Widen.  Smart Cameras and the Right to Privacy , 2008, Proceedings of the IEEE.

[36]  F. F. Kuo,et al.  The ALOHA system , 1974, CCRV.

[37]  David J. Fleet,et al.  Learning Sensor Network Topology through Monte Carlo , 2005 .

[38]  Neil Genzlinger.  A. and Q , 2006 .

[39]  Takeo Kanade,et al.  Algorithms for cooperative multisensor surveillance , 2001, Proc. IEEE.

[40]  Irfan Essa,et al.  Tracking Multiple People with Multiple Cameras , 1998 .

[41]  M. Shah,et al.  KNIGHT M : A REAL TIME SURVEILLANCE SYSTEM FOR MULTIPLE OVERLAPPING AND NON-OVERLAPPING CAMERAS , 2003 .

[42]  Demetri Terzopoulos,et al.  Animat vision: Active vision in artificial animals , 1995, Proceedings of IEEE International Conference on Computer Vision.

[43]  Deborah Estrin,et al.  Directed diffusion for wireless sensor networking , 2003, TNET.

[44]  Martin W. P. Savelsbergh,et al.  A combined Lagrangian, linear programming, and implication heuristic for large-scale set partitioning problems , 1996, J. Heuristics.

[45]  Dorin Comaniciu,et al.  Adaptive Resolution System for Distributed Surveillance , 2002, Real Time Imaging.

[46]  Makoto Yokoo,et al.  Distributed Constraint Satisfaction: Foundations of Cooperation in Multi-agent Systems , 2000 .

[47]  Makoto Yokoo,et al.  Distributed Constraint Satisfaction , 2000, Springer Series on Agent Technology.