Multi-Camera Coordination and Control in Surveillance Systems

The use of multiple heterogeneous cameras is becoming more common in today's surveillance systems. In order to perform surveillance tasks, effective coordination and control in multi-camera systems is very important, and is catching significant research attention these days. This survey aims to provide researchers with a state-of-the-art overview of various techniques for multi-camera coordination and control (MC3) that have been adopted in surveillance systems. The existing literature on MC3 is presented through several classifications based on the applicable architectures, frameworks and the associated surveillance tasks. Finally, a discussion on the open problems in surveillance area that can be solved effectively using MC3 and the future directions in MC3 research is presented

[1]  Hung Hai Bui,et al.  Mutliple camera coordination in a surveillance system , 2003 .

[2]  Mohamed S. Kamel,et al.  An agent-based approach to multisensor coordination , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[3]  Jake K. Aggarwal,et al.  Integrated Analysis of Thermal and Visual Images for Scene Interpretation , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Prabir Bhattacharya,et al.  A game-theoretic design for collaborative tracking in a video camera network , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[5]  Wolfgang Straßer,et al.  3D Surveillance A Distributed Network of Smart Cameras for Real-Time Tracking and its Visualization in 3D , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[6]  Mustafa Cenk Gursoy,et al.  Distributed wide-area multi-object tracking with non-overlapping camera views , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[7]  Prabhu Natarajan PhD forum: Decision-theoretic coordination and control for active multi-camera surveillance , 2012, 2012 Sixth International Conference on Distributed Smart Cameras (ICDSC).

[8]  Andrea Cavallaro,et al.  Self-Reconfigurable Smart Camera Networks , 2014, Computer.

[9]  Lily Lee,et al.  Monitoring Activities from Multiple Video Streams: Establishing a Common Coordinate Frame , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Amit K. Roy-Chowdhury,et al.  Distributed Camera Networks , 2011, IEEE Signal Processing Magazine.

[11]  Larry S. Davis,et al.  Decentralized discovery of camera network topology , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[12]  Mubarak Shah,et al.  Automated Multi-Camera Surveillance: Algorithms and Practice , 2008, The International Series in Video Computing.

[13]  C. Machy,et al.  On the performance evaluation of tracking systems using multiple pan-tilt-zoom cameras , 2007, Electronic Imaging.

[14]  Wendi B. Heinzelman,et al.  Camera selection in visual sensor networks , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[15]  Nicola Conci,et al.  Optimal configuration of PTZ camera networks based on visual quality assessment and coverage maximization , 2013, 2013 Seventh International Conference on Distributed Smart Cameras (ICDSC).

[16]  Huang Lee,et al.  Collaborative node localization in surveillance networks using opportunistic target observations , 2006, VSSN '06.

[17]  N. Nandhakumar Robust feature evaluation for multisensory computer vision , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[18]  V. M. Bove,et al.  Collaborative Knowledge Building by Smart Sensors , 2004 .

[19]  KankanhalliMohan,et al.  Multi-Camera Coordination and Control in Surveillance Systems , 2015 .

[20]  James M. Ferryman,et al.  Benchmark Datasets for Detection and Tracking , 2011, Visual Analysis of Humans.

[21]  Xin Yao,et al.  Learning to be Different: Heterogeneity and Efficiency in Distributed Smart Camera Networks , 2013, 2013 IEEE 7th International Conference on Self-Adaptive and Self-Organizing Systems.

[22]  Eric Sommerlade,et al.  Probabilistic surveillance with multiple active cameras , 2010, 2010 IEEE International Conference on Robotics and Automation.

[23]  Thomas Mauthner,et al.  Unsupervised Calibration of Camera Networks an Virtual PTZ Cameras , 2012 .

[24]  Luca Benini,et al.  An integrated multi-modal sensor network for video surveillance , 2005, VSSN '05.

[25]  Dacheng Tao,et al.  Sparse Camera Network for Visual Surveillance -- A Comprehensive Survey , 2013, ArXiv.

[26]  J.-H. Lee,et al.  Fuzzy-based camera selection for object tracking in a multi-camera system , 2008, 2008 Conference on Human System Interactions.

[27]  Mohan S. Kankanhalli,et al.  Scalable Decision-Theoretic Coordination and Control for Real-time Active Multi-Camera Surveillance , 2014, ICDSC.

[28]  Amit K. Roy-Chowdhury,et al.  Collaborative Sensing in a Distributed PTZ Camera Network , 2012, IEEE Transactions on Image Processing.

[29]  Amit K. Roy-Chowdhury,et al.  Constrained optimization for opportunistic distributed visual sensing , 2013, 2013 American Control Conference.

[30]  Demetri Terzopoulos,et al.  Surveillance camera scheduling: a virtual vision approach , 2005, Multimedia Systems.

[31]  Bernhard Rinner,et al.  Security and Privacy Protection in Visual Sensor Networks , 2014, ACM Comput. Surv..

[32]  Faisal Z. Qureshi,et al.  Multi-tasking smart cameras for intelligent video surveillance systems , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[33]  Bernhard Rinner,et al.  Video Analysis in Pan-Tilt-Zoom Camera Networks , 2010, IEEE Signal Processing Magazine.

[34]  Bernhard Rinner,et al.  Distributed resource-aware task assignment for complex monitoring scenarios in Visual Sensor Networks , 2012, 2012 Sixth International Conference on Distributed Smart Cameras (ICDSC).

[35]  Andrea Cavallaro,et al.  Multi-camera Scheduling for Video Production , 2011, 2011 Conference for Visual Media Production.

[36]  Sheng-Wen Shih,et al.  Linear production game solution to a PTZ camera network , 2010, 2010 IEEE International Conference on Image Processing.

[37]  Andreas Kriechbaum,et al.  A multisensor surveillance system for Automated Border Control (eGate) , 2013, 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[38]  No One is Left "Unwatched": Fairness in Observation of Crowds of Mobile Targets in Active Camera Surveillance , 2014, ECAI.

[39]  Prashant J. Shenoy,et al.  SensEye: a multi-tier camera sensor network , 2005, ACM Multimedia.

[40]  Zeyu Li,et al.  Wide-area external multi-camera calibration using vision graphs and virtual calibration object , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[41]  Pramod K. Varshney,et al.  Automatic camera selection and fusion for outdoor surveillance under changing weather conditions , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[42]  Bernhard Rinner,et al.  Integrating multi-camera tracking into a dynamic task allocation system for smart cameras , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[43]  Bernhard Rinner,et al.  Resource-Aware Coverage and Task Assignment in Visual Sensor Networks , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[44]  Gian Luca Foresti,et al.  An integrated surveillance system for outdoor security , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[45]  D. Warner North,et al.  A Tutorial Introduction to Decision Theory , 1968, IEEE Trans. Syst. Sci. Cybern..

[46]  Mongi A. Abidi,et al.  Survey and analysis of multimodal sensor planning and integration for wide area surveillance , 2009, CSUR.

[47]  Gian Luca Foresti,et al.  PTZ camera network reconfiguration , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[48]  Eduardo Monari,et al.  An Object- and Task-Oriented Architecture for Automated Video Surveillance in Distributed Sensor Networks , 2008, 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance.

[49]  Joonki Paik,et al.  Mutiple-view object tracking using metadata , 2007, 2007 International Conference on Wavelet Analysis and Pattern Recognition.

[50]  Hyeokho Choi,et al.  Distributed camera network localization , 2004, Conference Record of the Thirty-Eighth Asilomar Conference on Signals, Systems and Computers, 2004..

[51]  Peter H. Tu,et al.  View adaptive detection and distributed site wide tracking , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[52]  John S. Boreczky,et al.  FlySPEC: a multi-user video camera system with hybrid human and automatic control , 2002, MULTIMEDIA '02.

[53]  Mohan S. Kankanhalli,et al.  Context-Based Multimedia Sensor Selection Method , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[54]  Hamid K. Aghajan,et al.  A collaborative framework for ergonomic feedback using smart cameras , 2012, 2012 Sixth International Conference on Distributed Smart Cameras (ICDSC).

[55]  John Lygeros,et al.  Probabilistic certification of pan-tilt-zoom camera surveillance systems , 2011, IEEE Conference on Decision and Control and European Control Conference.

[56]  Bernhard Rinner,et al.  The evolution from single to pervasive smart cameras , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[57]  Ting Yu,et al.  Collaborative Real-Time Control of Active Cameras in Large Scale Surveillance Systems , 2008 .

[58]  Jie Zhang,et al.  Automatic cooperative camera system for real-time bag detection in visual surveillance , 2013, 2013 Seventh International Conference on Distributed Smart Cameras (ICDSC).

[59]  Andrea Cavallaro,et al.  Content and task-based view selection from multiple video streams , 2009, Multimedia Tools and Applications.

[60]  G. Wanielik,et al.  Shape and motion-based pedestrian detection in infrared images: a multi sensor approach , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[61]  Paolo Remagnino,et al.  Multi-Camera Color Tracking , 1999 .

[62]  Marilyn Wolf,et al.  Distributed tracking in a large-scale network of smart cameras , 2010, ICDSC '10.

[63]  Bir Bhanu,et al.  Camera pan/tilt control with multiple trackers , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[64]  Mohan S. Kankanhalli,et al.  Decision-theoretic approach to maximizing fairness in multi-target observation in multi-camera surveillance , 2014, AAMAS.

[65]  James J. Clark,et al.  Collaborative Multi-Camera Surveillance with Automated Person Detection , 2006, The 3rd Canadian Conference on Computer and Robot Vision (CRV'06).

[66]  Jake K. Aggarwal,et al.  Tracking Human Motion in Structured Environments Using a Distributed-Camera System , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[67]  F. Castanedo,et al.  Extending surveillance systems capabilities using BDI cooperative sensor agents , 2006, VSSN '06.

[68]  Demetri Terzopoulos,et al.  Planning ahead for PTZ camera assignment and handoff , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[69]  Takeo Kanade,et al.  Algorithms for cooperative multisensor surveillance , 2001, Proc. IEEE.

[70]  Edoardo Ardizzone,et al.  Enabling technologies on hybrid camera networks for behavioral analysis of unattended indoor environments and their surroundings , 2008, VNBA '08.

[71]  Anup Basu,et al.  Multi-Camera Calibration Using a Globe , 2008 .

[72]  Andrew J. Chosak,et al.  OVVV: Using Virtual Worlds to Design and Evaluate Surveillance Systems , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[73]  Michele Magno,et al.  Distributed video surveillance using hardware-friendly sparse large margin classifiers , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[74]  Bernhard Rinner,et al.  Resource coordination in Wireless Sensor Networks by combinatorial auction based method , 2012, 2012 IEEE 3rd International Conference on Networked Embedded Systems for Every Application (NESEA).

[75]  Rita Cucchiara,et al.  People reidentification in surveillance and forensics , 2013, ACM Comput. Surv..

[76]  Justus H. Piater,et al.  A modular multi-camera framework for team sports tracking , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[77]  Chao Yu,et al.  Multi-sensor Intelligent Transportation Monitoring System Based on Information Fusion Technology , 2007, 2007 International Conference on Convergence Information Technology (ICCIT 2007).

[78]  Bülent Tavli,et al.  A survey of visual sensor network platforms , 2012, Multimedia Tools and Applications.

[79]  Xin Yao,et al.  Socio-economic vision graph generation and handover in distributed smart camera networks , 2014, TOSN.

[80]  Zehang Sun,et al.  A distributed visual surveillance system , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[81]  Bernhard Rinner,et al.  Distributed Task Allocation for Visual Sensor Networks: A Market-Based Approach , 2010, 2010 Fourth IEEE International Conference on Self-Adaptive and Self-Organizing Systems Workshop.

[82]  Faisal Z. Qureshi,et al.  Learning proactive control strategies for PTZ cameras , 2011, 2011 Fifth ACM/IEEE International Conference on Distributed Smart Cameras.

[83]  Wilfried Philips,et al.  Demo: Real-time indoors people tracking in scalable camera networks , 2011, 2011 Fifth ACM/IEEE International Conference on Distributed Smart Cameras.

[84]  Nikolaos Papanikolopoulos,et al.  Multi-Camera Human Activity Monitoring , 2008, J. Intell. Robotic Syst..

[85]  Björn W. Schuller,et al.  Applying multi layer homography for multi camera person tracking , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[86]  Teresa H. Ko,et al.  Distributed calibration and tracking with low-power image sensors , 2005, TAPIA '05.

[87]  Hirotake Yamazoe Automatic camera calibration method for distributed multiple camera based human tracking system , 2002 .

[88]  Norimichi Ukita,et al.  Real-time multitarget tracking by a cooperative distributed vision system , 2002, Proc. IEEE.

[89]  I-Jeng Wang,et al.  Surveillance Camera Coordination Through Distributed Scheduling , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[90]  Andrea Cavallaro,et al.  Multi-Camera Networks: Principles and Applications , 2009 .

[91]  Ramesh Jain,et al.  Experiential Sampling for video surveillance , 2003, IWVS '03.

[92]  S. Payandeh,et al.  Cooperative hybrid multi-camera tracking for people surveillance , 2008, Canadian Journal of Electrical and Computer Engineering.

[93]  Yiming Li,et al.  Utility-Based Camera Assignment in a Video Network: A Game Theoretic Framework , 2011, IEEE Sensors Journal.

[94]  Jesús García,et al.  Agent-based Coordination of Cameras , 2005, Int. J. Comput. Sci. Appl..

[95]  R. Anido,et al.  Distributed real-time soccer tracking , 2004, VSSN '04.

[96]  Larry S. Davis,et al.  Unified multi-camera detection and tracking using region-matching , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[97]  Mohan M. Trivedi,et al.  A Survey of Vision-Based Trajectory Learning and Analysis for Surveillance , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[98]  Xiaogang Wang,et al.  Intelligent multi-camera video surveillance: A review , 2013, Pattern Recognit. Lett..

[99]  Mohan S. Kankanhalli,et al.  Multimodal fusion for multimedia analysis: a survey , 2010, Multimedia Systems.

[100]  Ig-Jae Kim,et al.  Video Surveillance using Dynamic Configuration of Mutiple Active Cameras , 2006, 2006 International Conference on Image Processing.

[101]  Faisal Z. Qureshi,et al.  A Virtual Vision Simulator for Camera Networks Research , 2012, 2012 Ninth Conference on Computer and Robot Vision.

[102]  Tarak Gandhi,et al.  Multi-camera Based Traffic Flow Characterization & Classification , 2007, 2007 IEEE Intelligent Transportation Systems Conference.

[103]  Pradeep K. Atrey,et al.  Human-coordinated camera control for monitoring public spaces , 2010, 10th International Conference on Information Science, Signal Processing and their Applications (ISSPA 2010).

[104]  Gang Wei,et al.  Multiple-camera people localization in an indoor environment , 2006, Knowledge and Information Systems.

[105]  Demetri Terzopoulos,et al.  Smart Camera Networks in Virtual Reality , 2007, Proceedings of the IEEE.

[106]  Sadiye Guler,et al.  Tracking and handoff between multiple perspective camera views , 2003, 32nd Applied Imagery Pattern Recognition Workshop, 2003. Proceedings..

[107]  A. M. Tekalp,et al.  Multiple camera fusion for multi-object tracking , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[108]  Mohan S. Kankanhalli,et al.  Coopetitive multi-camera surveillance using model predictive control , 2008, Machine Vision and Applications.

[109]  Shaogang Gong,et al.  Tracking multiple people with a multi-camera system , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[110]  Sergio A. Velastin,et al.  Intelligent distributed surveillance systems: a review , 2005 .

[111]  Carlo S. Regazzoni,et al.  Distributed Cognitive Sensor Network Approach for Surveillance Applications , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[112]  Ruggero Carli,et al.  Camera Network Coordination for Intruder Detection , 2014, IEEE Transactions on Control Systems Technology.

[113]  Mohan S. Kankanhalli,et al.  Decision-theoretic coordination and control for active multi-camera surveillance in uncertain, partially observable environments , 2012, 2012 Sixth International Conference on Distributed Smart Cameras (ICDSC).

[114]  Richard J. Radke,et al.  Distributed Metric Calibration of Large Camera Networks , 2004 .

[115]  Nicola Conci,et al.  Real-time reconfiguration of PTZ camera networks using motion field entropy and visual coverage , 2014, ICDSC.

[116]  Horst Bischof,et al.  People tracking across two distant self-calibrated cameras , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[117]  Brian C. Lovell,et al.  Tracking with Multiple Cameras for Video Surveillance , 2007, 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications (DICTA 2007).

[118]  Jin Young Choi,et al.  Intelligent visual surveillance — A survey , 2010 .

[119]  Carlos Hitoshi Morimoto,et al.  People Detection under Occlusion in Multiple Camera Views , 2008, 2008 XXI Brazilian Symposium on Computer Graphics and Image Processing.

[120]  Zeyu Li,et al.  Framework for hierarchical calibration of multi-camera systems for teleimmersion , 2009, IMMERSCOM.

[121]  Toshio Fukuda,et al.  Seamless tracking system with multiple cameras , 2000, 2000 26th Annual Conference of the IEEE Industrial Electronics Society. IECON 2000. 2000 IEEE International Conference on Industrial Electronics, Control and Instrumentation. 21st Century Technologies.

[122]  Martin Reisslein,et al.  Towards Efficient Wireless Video Sensor Networks: A Survey of Existing Node Architectures and Proposal for A Flexi-WVSNP Design , 2011, IEEE Communications Surveys & Tutorials.

[123]  Faisal Z. Qureshi Collaborative sensing via local negotiations in ad hoc networks of smart cameras , 2010, ICDSC '10.

[124]  J. Krumm,et al.  Multi-camera multi-person tracking for EasyLiving , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[125]  Miguel A. Patricio,et al.  A practical approach for active camera coordination based on a fusion-driven multi-agent system , 2014, Int. J. Syst. Sci..

[126]  Chen Wu,et al.  Multiview activity recognition in smart homes with spatio-temporal features , 2010, ICDSC '10.

[127]  Amit K. Roy-Chowdhury,et al.  Distributed multi-target tracking in a self-configuring camera network , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[128]  Tieniu Tan,et al.  Principal axis-based correspondence between multiple cameras for people tracking , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[129]  Amit K. Roy-Chowdhury,et al.  Robust Tracking in A Camera Network: A Multi-Objective Optimization Framework , 2008, IEEE Journal of Selected Topics in Signal Processing.

[130]  François Brémond,et al.  Shape recognition based on a video and multi-sensor system , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[131]  Matthijs T. J. Spaan,et al.  Decision-theoretic Planning under Uncertainty for Cooperative Active Perception , 2009 .

[132]  Mohan S. Kankanhalli,et al.  Multi-camera Skype: Enhancing the Quality of Experience of Video Conferencing , 2013 .

[133]  Greg Welch,et al.  On-line control of active camera networks for computer vision tasks , 2011, 2011 Fifth ACM/IEEE International Conference on Distributed Smart Cameras.

[134]  Sriram Sankar,et al.  Collaborative acquisition of multi-view face images in real-time using a wireless camera network , 2011, 2011 Fifth ACM/IEEE International Conference on Distributed Smart Cameras.

[135]  Amit K. Roy-Chowdhury,et al.  Camera Networks: The Acquisition and Analysis of Videos over Wide Areas , 2012, Synthesis Lectures on Computer Vision.

[136]  Amit K. Roy-Chowdhury,et al.  Decentralized camera network control using game theory , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[137]  P.B. Jones,et al.  An Iterative Algorithm for Autonomous Tasking in Sensor Networks , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[138]  Amit K. Roy-Chowdhury,et al.  Opportunistic sensing in a distributed PTZ camera network , 2012, 2012 Sixth International Conference on Distributed Smart Cameras (ICDSC).

[139]  Bir Bhanu,et al.  A comparison of techniques for camera selection and handoff in a video network , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[140]  Gian Luca Foresti,et al.  Multi-sensor Multi-cue Fusion for Object Detection in Video Surveillance , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[141]  Mohan S. Kankanhalli,et al.  Decision-theoretic approach to maximizing observation of multiple targets in multi-camera surveillance , 2012, AAMAS.