Recent Advances in Camera Planning for Large Area Surveillance

With recent advances in consumer electronics and the increasingly urgent need for public security, camera networks have evolved from their early role of providing simple and static monitoring to current complex systems capable of obtaining extensive video information for intelligent processing, such as target localization, identification, and tracking. In all cases, it is of vital importance that the optimal camera configuration (i.e., optimal location, orientation, etc.) is determined before cameras are deployed as a suboptimal placement solution will adversely affect intelligent video surveillance and video analytic algorithms. The optimal configuration may also provide substantial savings on the total number of cameras required to achieve the same level of utility. In this article, we examine most, if not all, of the recent approaches (post 2000) addressing camera placement in a structured manner. We believe that our work can serve as a first point of entry for readers wishing to start researching into this area or engineers who need to design a camera system in practice. To this end, we attempt to provide a complete study of relevant formulation strategies and brief introductions to most commonly used optimization techniques by researchers in this field. We hope our work to be inspirational to spark new ideas in the field.

[1]  Pradeep K. Khosla,et al.  Integrating Sensor Placement and Visual Tracking Strategies , 1993, ISER.

[2]  Subhashis Banerjee,et al.  Active recognition through next view planning: a survey , 2004, Pattern Recognit..

[3]  Sridha Sridharan,et al.  Determining operational measures from multi-camera surveillance systems using soft biometrics , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[4]  Sridha Sridharan,et al.  3D face verification using a free-parts approach , 2008, Pattern Recognit. Lett..

[5]  Jian Zhao,et al.  Optimal visual sensor planning , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[6]  Nael B. Abu-Ghazaleh,et al.  Coverage algorithms for visual sensor networks , 2013, TOSN.

[7]  Steve Fisk,et al.  A short proof of Chvátal's Watchman Theorem , 1978, J. Comb. Theory, Ser. B.

[8]  Nikolaos Papanikolopoulos,et al.  Multi-camera positioning to optimize task observability , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[9]  Bang Jun Lei,et al.  Camera Network Coverage Improving by Particle Swarm Optimization , 2011, EURASIP J. Image Video Process..

[10]  Thinh Nguyen,et al.  Optimal Visual Sensor Network Configuration , 2009, Multi-Camera Networks.

[11]  Mehdi Dehghan,et al.  Optimal visual sensor placement for coverage based on target location profile , 2011, Ad Hoc Networks.

[12]  Mohamed F. Younis,et al.  Strategies and techniques for node placement in wireless sensor networks: A survey , 2008, Ad Hoc Networks.

[13]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[14]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[15]  Xing Chen,et al.  An occlusion metric for selecting robust camera configurations , 2008, Machine Vision and Applications.

[16]  Santanu Chaudhury,et al.  Optimal sensor placement for surveillance of large spaces , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[17]  V. Chvátal A combinatorial theorem in plane geometry , 1975 .

[18]  Davide Scaramuzza,et al.  Omnidirectional Camera , 2014, Computer Vision, A Reference Guide.

[19]  Konstantinos A. Tarabanis,et al.  A survey of sensor planning in computer vision , 1995, IEEE Trans. Robotics Autom..

[20]  Mongi A. Abidi,et al.  Can You See Me Now? Sensor Positioning for Automated and Persistent Surveillance , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  John W. Fisher,et al.  Maximum Mutual Information Principle for Dynamic Sensor Query Problems , 2003, IPSN.

[22]  Pradeep K. Atrey,et al.  Towards optimal placement of surveillance cameras in a bus , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[23]  G. Pottie,et al.  Entropy-based sensor selection heuristic for target localization , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[24]  Raghu Machiraju,et al.  Coverage optimization to support security monitoring , 2007, Comput. Environ. Urban Syst..

[25]  Livier Reithler,et al.  Optimal deployment of cameras for video surveillance systems , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[26]  Nesa L'abbe Wu,et al.  Linear programming and extensions , 1981 .

[27]  Xiang Chen,et al.  Modeling Coverage in Camera Networks: A Survey , 2012, International Journal of Computer Vision.

[28]  Hitoshi Kitazawa,et al.  Optimum camera placement considering camera specification for security monitoring , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[29]  Hiroshi Murase,et al.  Planning of Multiple Camera Arrangement for Object Recognition in Parametric Eigenspace , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[30]  Jian Zhao,et al.  Multi-Camera Surveillance with Visual Tagging and Generic Camera Placement , 2007, 2007 First ACM/IEEE International Conference on Distributed Smart Cameras.

[31]  Fillia Makedon,et al.  Automatic sensor placement in a 3D volume , 2009, PETRA '09.

[32]  Dipankar Dasgupta,et al.  An Overview of Artificial Immune Systems and Their Applications , 1993 .

[33]  Mohan S. Kankanhalli,et al.  Decision-theoretic coordination and control for active multi-camera surveillance in uncertain, partially observable environments , 2012, 2012 Sixth International Conference on Distributed Smart Cameras (ICDSC).

[34]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[35]  Bernhard Rinner,et al.  Security and Privacy Protection in Visual Sensor Networks , 2014, ACM Comput. Surv..

[36]  Roger Y. Tsai,et al.  Analytical characterization of the feature detectability constraints of resolution, focus, and field-of-view for vision sensor planning , 1994 .

[37]  Barbara Webb,et al.  Swarm Intelligence: From Natural to Artificial Systems , 2002, Connect. Sci..

[38]  Jörg Hähner,et al.  Reinforcement Learning for Coverage Optimization Through PTZ Camera Alignment in Highly Dynamic Environments , 2014, ICDSC.

[39]  Nael B. Abu-Ghazaleh,et al.  Scalable target coverage in smart camera networks , 2010, ICDSC '10.

[40]  Pramod K. Varshney,et al.  Automatic camera selection and fusion for outdoor surveillance under changing weather conditions , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[41]  Sridha Sridharan,et al.  Activity Analysis in Complicated Scenes Using DFT Coefficients of Particle Trajectories , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[42]  Qiang Wang,et al.  On-line configuration of large scale surveillance networks using mobile smart camera , 2013, 2013 Seventh International Conference on Distributed Smart Cameras (ICDSC).

[43]  Mongi A. Abidi,et al.  Survey and analysis of multimodal sensor planning and integration for wide area surveillance , 2009, CSUR.

[44]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[45]  Dani Lischinski,et al.  Automatic camera placement for image-based modeling , 1999, Proceedings. Seventh Pacific Conference on Computer Graphics and Applications (Cat. No.PR00293).

[46]  D. T. Lee,et al.  Computational complexity of art gallery problems , 1986, IEEE Trans. Inf. Theory.

[47]  FookesClinton,et al.  Recent Advances in Camera Planning for Large Area Surveillance , 2016 .

[48]  Abas Md Said,et al.  Optimal Camera Placement for 3D Environment , 2011, ICSECS.

[49]  Bang Jun Lei,et al.  Three particle swarm algorithms to improve coverage of camera networks with mobile nodes , 2010, 2010 IEEE Fifth International Conference on Bio-Inspired Computing: Theories and Applications (BIC-TA).

[50]  Larry S. Davis,et al.  Visibility Analysis and Sensor Planning in Dynamic Environments , 2004, ECCV.

[51]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[52]  Georgios Ch. Sirakoulis,et al.  A bio-inspired multi-camera system for dynamic crowd analysis , 2014, Pattern Recognit. Lett..

[53]  James Kennedy,et al.  The particle swarm: social adaptation of knowledge , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[54]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[55]  Debasish Ghose,et al.  Glowworm swarm optimization for simultaneous capture of multiple local optima of multimodal functions , 2009, Swarm Intelligence.

[56]  Nikolaos Papanikolopoulos,et al.  Issues and solutions in surveillance camera placement , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[57]  Mongi A. Abidi,et al.  Sensor planning for automated and persistent object tracking with multiple cameras , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[58]  Mohammad Al Hasan,et al.  Optimal placement of stereo sensors , 2007, Optim. Lett..

[59]  Jun Zhang,et al.  Genetic algorithm based optimal placement of PIR sensor arrays for human localization , 2011, 2011 IEEE International Conference on Mechatronics and Automation.

[60]  Nicola Conci,et al.  Real-time reconfiguration of PTZ camera networks using motion field entropy and visual coverage , 2014, ICDSC.

[61]  Eduardo Monari,et al.  Dynamic Sensor Selection for Single Target Tracking in Large Video Surveillance Networks , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[62]  Sridha Sridharan,et al.  Textures of optical flow for real-time anomaly detection in crowds , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[63]  Nicola Conci,et al.  Camera positioning for global and local coverage optimization , 2012, 2012 Sixth International Conference on Distributed Smart Cameras (ICDSC).

[64]  J. O'Rourke Art gallery theorems and algorithms , 1987 .

[65]  G. Roth,et al.  View planning for automated three-dimensional object reconstruction and inspection , 2003, CSUR.

[66]  Dervis Karaboga,et al.  A comprehensive survey: artificial bee colony (ABC) algorithm and applications , 2012, Artificial Intelligence Review.

[67]  Y. Morsly,et al.  Particle Swarm Optimization Inspired Probability Algorithm for Optimal Camera Network Placement , 2012, IEEE Sensors Journal.

[68]  Kamyoung Kim,et al.  A Multiobjective Evolutionary Algorithm for Surveillance Sensor Placement , 2008 .

[69]  Cid C. de Souza,et al.  An exact algorithm for minimizing vertex guards on art galleries , 2011, Int. Trans. Oper. Res..

[70]  Joachim Denzler,et al.  On Optimal Camera Parameter Selection in Kalman Filter Based Object Tracking , 2002, DAGM-Symposium.

[71]  V. K. Singh,et al.  A design methodology for selection and placement of sensors in multimedia surveillance systems , 2006, VSSN '06.

[72]  Sridha Sridharan,et al.  On the Statistical Determination of Optimal Camera Configurations in Large Scale Surveillance Networks , 2012, ECCV.

[73]  Konstantinos A. Tarabanis,et al.  Computing Camera Viewpoints in an Active Robot Work Cell , 1999, Int. J. Robotics Res..

[74]  Nando de Freitas,et al.  Reversible Jump MCMC Simulated Annealing for Neural Networks , 2000, UAI.

[75]  Anton van den Hengel,et al.  Semidefinite Programming , 2014, Computer Vision, A Reference Guide.

[76]  Leonardo Lizzi,et al.  Camera placement using particle swarm optimization in visual surveillance applications , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[77]  Sridha Sridharan,et al.  Optimal Camera Planning Under Versatile User Constraints in Multi-Camera Image Processing Systems , 2014, IEEE Transactions on Image Processing.

[78]  S. Brooks,et al.  Classical model selection via simulated annealing , 2003, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[79]  Leonidas J. Guibas,et al.  Optimal Placement and Selection of Camera Network Nodes for Target Localization , 2006, DCOSS.

[80]  Nabil Aouf,et al.  On the best interceptor placement for an optimally deployed visual sensor network , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[81]  Sridha Sridharan,et al.  Evaluation of image resolution and super-resolution on face recognition performance , 2012, J. Vis. Commun. Image Represent..

[82]  José-Joel González-Barbosa,et al.  Optimal camera placement for total coverage , 2009, 2009 IEEE International Conference on Robotics and Automation.

[83]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[84]  Stan Sclaroff,et al.  Automated camera layout to satisfy task-specific and floor plan-specific coverage requirements , 2006, Comput. Vis. Image Underst..

[85]  Scott Kirkpatrick,et al.  Optimization by simulated annealing: Quantitative studies , 1984 .

[86]  Peter Slavík A Tight Analysis of the Greedy Algorithm for Set Cover , 1997, J. Algorithms.

[87]  Y.F. Li,et al.  Automatic sensor placement for model-based robot vision , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[88]  Stan Sclaroff,et al.  Optimal Placement of Cameras in Floorplans to Satisfy Task Requirements and Cost Constraints , 2004 .

[89]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[90]  KankanhalliMohan,et al.  Multi-Camera Coordination and Control in Surveillance Systems , 2015 .

[91]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[92]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[93]  Christopher S. Madden,et al.  Automatic camera placement for large scale surveillance networks , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[94]  Ailsa H. Land,et al.  An Automatic Method of Solving Discrete Programming Problems , 1960 .

[95]  Mohan S. Kankanhalli,et al.  Scalable Decision-Theoretic Coordination and Control for Real-time Active Multi-Camera Surveillance , 2014, ICDSC.

[96]  Sheldon Howard Jacobson,et al.  The Theory and Practice of Simulated Annealing , 2003, Handbook of Metaheuristics.

[97]  Roger Mohr,et al.  Optimal camera placement for accurate reconstruction , 2002, Pattern Recognit..

[98]  Wendi B. Heinzelman,et al.  Camera selection in visual sensor networks , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[99]  Bang Wang,et al.  Coverage problems in sensor networks: A survey , 2011, CSUR.

[100]  Nikolaos Papanikolopoulos,et al.  Optimal Camera Placement for Automated Surveillance Tasks , 2007, J. Intell. Robotic Syst..

[101]  Chengnian Long,et al.  Optimal coverage of camera networks using PSO algorithm , 2011, 2011 4th International Congress on Image and Signal Processing.

[102]  Nicola Conci,et al.  Optimal configuration of PTZ camera networks based on visual quality assessment and coverage maximization , 2013, 2013 Seventh International Conference on Distributed Smart Cameras (ICDSC).

[103]  共立出版株式会社 コンピュータ・サイエンス : ACM computing surveys , 1978 .

[104]  Simon Denman,et al.  Improved detection and tracking of objects in surveillance video , 2009 .

[105]  Larry S. Davis,et al.  A General Method for Sensor Planning in Multi-Sensor Systems: Extension to Random Occlusion , 2007, International Journal of Computer Vision.

[106]  Benoit Debaque,et al.  Optimal video camera network deployment to support security monitoring , 2009, 2009 12th International Conference on Information Fusion.

[107]  Sridha Sridharan,et al.  Gait energy volumes and frontal gait recognition using depth images , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[108]  Monique Laurent,et al.  A Comparison of the Sherali-Adams, Lovász-Schrijver, and Lasserre Relaxations for 0-1 Programming , 2003, Math. Oper. Res..

[109]  Antonis A. Argyros,et al.  Fast positioning of limited-visibility guards for the inspection of 2D workspaces , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[110]  Bang Jun Lei,et al.  Constrained particle swarm algorithms for optimizing coverage of large-scale camera networks with mobile nodes , 2013, Soft Comput..

[111]  Mirko Krivánek,et al.  Simulated Annealing: A Proof of Convergence , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[112]  Pierre David,et al.  A Sensor Placement Approach for the Monitoring of Indoor Scenes , 2007, EuroSSC.

[113]  R. Lienhart,et al.  On the optimal placement of multiple visual sensors , 2006, VSSN '06.

[114]  Y.-C. Lee,et al.  The optimal locations of surveillance cameras on straight lanes , 2011, Expert Syst. Appl..

[115]  Eduardo Monari,et al.  A knowledge-based camera selection approach for object tracking in large sensor networks , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[116]  Bo Yang,et al.  Omnidirectional camera , 2013, 2013 IEEE 9th International Conference on Intelligent Computer Communication and Processing (ICCP).

[117]  Larry S. Davis,et al.  COST: An Approach for Camera Selection and Multi-Object Inference Ordering in Dynamic Scenes , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[118]  A. Gasteratos,et al.  Optimum multi-camera arrangement using a bee colony algorithm , 2012, 2012 IEEE International Conference on Imaging Systems and Techniques Proceedings.

[119]  Nael B. Abu-Ghazaleh,et al.  Target-oriented coverage maximization in visual sensor networks , 2011, MobiWac '11.

[120]  Gavin S. P. Miller,et al.  Hierarchical Z-buffer visibility , 1993, SIGGRAPH.

[121]  Liang Liu,et al.  Optimal Node Selection for Target Localization in Wireless Camera Sensor Networks , 2010, IEEE Transactions on Vehicular Technology.

[122]  Jian Zhao,et al.  Camera Planning and Fusion in a Heterogeneous Camera Network , 2012 .

[123]  Cid C. de Souza,et al.  Experimental Evaluation of an Exact Algorithm for the Orthogonal Art Gallery Problem , 2008, WEA.

[124]  Luca Di Gaspero,et al.  Virtual Camera Composition with Particle Swarm Optimization , 2008, Smart Graphics.

[125]  Rainer Lienhart,et al.  Optimal Placement of Multiple Visual Sensors , 2009, Multi-Camera Networks.

[126]  Simone Gasparini,et al.  Camera Models and Fundamental Concepts Used in Geometric Computer Vision , 2011, Found. Trends Comput. Graph. Vis..