Wireless Video Surveillance: A Survey

A wireless video surveillance system consists of three major components: 1) the video capture and preprocessing; 2) the video compression and transmission in wireless sensor networks; and 3) the video analysis at the receiving end. A myriad of research works have been dedicated to this field due to its increasing popularity in surveillance applications. This survey provides a comprehensive overview of existing state-of-the-art technologies developed for wireless video surveillance, based on the in-depth analysis of the requirements and challenges in current systems. Specifically, the physical network infrastructure for video transmission over wireless channel is analyzed. The representative technologies for video capture and preliminary vision tasks are summarized. For video compression and transmission over the wireless networks, the ultimate goal is to maximize the received video quality under the resource limitation. This is also the main focus of this survey. We classify different schemes into categories including unequal error protection, error resilience, scalable video coding, distributed video coding, and cross-layer control. Cross-layer control proves to be a desirable measure for system-level optimal resource allocation. At the receiver's end, the received video is further processed for higher-level vision tasks, and the security and privacy issues in surveillance applications are also discussed.

[1]  Tim Ellis,et al.  A multi-view surveillance system , 2003 .

[2]  Christophe Tillier,et al.  3D, 3-band, 3-tap temporal lifting for scalable video coding , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[3]  W. Sweldens The Lifting Scheme: A Custom - Design Construction of Biorthogonal Wavelets "Industrial Mathematics , 1996 .

[4]  P. Petrov,et al.  Face detection and tracking with an active camera , 2008, 2008 4th International IEEE Conference Intelligent Systems.

[5]  Aggelos K. Katsaggelos,et al.  Super Resolution of Images and Video , 2006, Super Resolution of Images and Video.

[6]  Christophe Tillier,et al.  Distributed Temporal Multiple Description Coding for Robust Video Transmission , 2008, EURASIP J. Wirel. Commun. Netw..

[7]  Lin Cai,et al.  Scalable Video Coding with Compressive Sensing for Wireless Videocast , 2011, 2011 IEEE International Conference on Communications (ICC).

[8]  Mihaela van der Schaar,et al.  Cross-layer resource allocation for delay constrained wireless video transmission , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[9]  Chuohao Yeo,et al.  Robust Distributed Multiview Video Compression for Wireless Camera Networks , 2010, IEEE Transactions on Image Processing.

[10]  Nalini Venkatasubramanian,et al.  Privacy-protecting video surveillance , 2005, IS&T/SPIE Electronic Imaging.

[11]  Anthony Vetro,et al.  View Synthesis for Multiview Video Compression , 2006 .

[12]  Feng Wu,et al.  In-Scale Motion Compensation for Spatially Scalable Video Coding , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  K. Plataniotis,et al.  Privacy Protected Surveillance Using Secure Visual Object Coding , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Kannan Ramchandran,et al.  PRISM: A new robust video coding architecture based on distributed compression principles , 2002 .

[15]  Peter Meer,et al.  Point matching under large image deformations and illumination changes , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  David S. Taubman,et al.  A flexible structure for fully scalable motion-compensated 3-D DWT with emphasis on the impact of spatial scalability , 2006, IEEE Transactions on Image Processing.

[17]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[18]  Song Ci,et al.  Dynamic video object detection with single PTU camera , 2011, 2011 Visual Communications and Image Processing (VCIP).

[19]  Minghua Chen,et al.  A fragile watermark error detection scheme for wireless video communications , 2005, IEEE Transactions on Multimedia.

[20]  Rastislav Lukac,et al.  SPIHT-Based Coding of the Shape and Texture of Arbitrarily Shaped Visual Objects , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Mourad Ouaret,et al.  Recent Advances in Multi-view Distributed Video Coding , 2007 .

[22]  Marta Karczewicz,et al.  The SP- and SI-frames design for H.264/AVC , 2003, IEEE Trans. Circuits Syst. Video Technol..

[23]  Wesley De Neve,et al.  Privacy Protection in Video Surveillance Systems: Analysis of Subband-Adaptive Scrambling in JPEG XR , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Aggelos K. Katsaggelos,et al.  Cost-distortion optimized unequal error protection for object-based video communications , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Wenjun Zeng,et al.  Efficient frequency domain selective scrambling of digital video , 2003, IEEE Trans. Multim..

[26]  Haohong Wang,et al.  Video Surveillance Over Wireless Sensor and Actuator Networks Using Active Cameras , 2011, IEEE Transactions on Automatic Control.

[27]  Aggelos K. Katsaggelos,et al.  Content-aware resource allocation and packet scheduling for video transmission over wireless networks , 2007, IEEE Journal on Selected Areas in Communications.

[28]  Shunsuke Kamijo,et al.  Smart camera network system for use in railway stations , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[29]  Heiko Schwarz,et al.  Overview of the Scalable Video Coding Extension of the H.264/AVC Standard , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Wireless Video Surveillance , 2011 .

[31]  Ning Luo A Wireless Traffic Surveillance System Using Video Analytics , 2011 .

[32]  Aggelos K. Katsaggelos,et al.  Bayesian resolution enhancement of compressed video , 2004, IEEE Transactions on Image Processing.

[33]  Mihaela van der Schaar,et al.  Cross-Layer Optimized Video Streaming Over Wireless Multihop Mesh Networks , 2006, IEEE Journal on Selected Areas in Communications.

[34]  Jack K. Wolf,et al.  Noiseless coding of correlated information sources , 1973, IEEE Trans. Inf. Theory.

[35]  Yanjiang Wang,et al.  An improved adaptive background modeling algorithm based on Gaussian Mixture Model , 2008, 2008 9th International Conference on Signal Processing.

[36]  Aggelos K. Katsaggelos,et al.  Binocular video object tracking with fast disparity estimation , 2013, 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[37]  Rachid Deriche,et al.  Geodesic active regions and level set methods for motion estimation and tracking , 2005, Comput. Vis. Image Underst..

[38]  Yi Wang,et al.  Barrier coverage in camera sensor networks , 2011, MobiHoc '11.

[39]  Tetsuya Takiguchi,et al.  Object recognition and segmentation using SIFT and Graph Cuts , 2008, 2008 19th International Conference on Pattern Recognition.

[40]  Peng Tang,et al.  Video object segmentation based on graph cut with dynamic shape prior constraint , 2008, 2008 19th International Conference on Pattern Recognition.

[41]  Thomas Wiegand,et al.  SVC-based multisource streaming for robust video transmission in mobile ad hoc networks , 2006, IEEE Wireless Communications.

[42]  Chen-Nee Chuah,et al.  Rate-Distortion Optimized Joint Source/Channel Coding of WWAN Multicast Video for a Cooperative Peer-to-Peer Collective , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[43]  Feng Wu,et al.  Channel Distortion Modeling for Multi-View Video Transmission Over Packet-Switched Networks , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[44]  Rui Zhang,et al.  Video coding with optimal inter/intra-mode switching for packet loss resilience , 2000, IEEE Journal on Selected Areas in Communications.

[45]  Jean-Yves Guillemaut,et al.  Robust graph-cut scene segmentation and reconstruction for free-viewpoint video of complex dynamic scenes , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[46]  Sergio A. Velastin,et al.  Fusing Visual and Audio Information in a Distributed Intelligent Surveillance System for Public Transport Systems , 2003 .

[47]  Vivek K. Goyal,et al.  Multiple description coding: compression meets the network , 2001, IEEE Signal Process. Mag..

[48]  Muppuri Siva Goutham,et al.  Resource Reservation Protocol , 2012 .

[49]  Takashi Watanabe,et al.  A User Dependent System for Multi-view Video Transmission , 2011, 2011 IEEE International Conference on Advanced Information Networking and Applications.

[50]  Wen Gao,et al.  Low-delay View Random Access for Multi-view Video Coding , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[51]  Peter Lambert,et al.  Flexible macroblock ordering as a content adaptation tool in H.264/AVC , 2005, SPIE Optics East.

[52]  Xue Wang,et al.  Distributed Visual-Target-Surveillance System in Wireless Sensor Networks , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[53]  Stefano Tubaro,et al.  Multiple description video coding for scalable and robust transmission over IP , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[54]  Haohong Wang,et al.  Cross-layer optimization for video summary transmission over wireless networks , 2007, IEEE Journal on Selected Areas in Communications.

[55]  Shlomo Shamai,et al.  Systematic Lossy Source/Channel Coding , 1998, IEEE Trans. Inf. Theory.

[56]  Wen Gao,et al.  Distributed multi-view video coding , 2006, Electronic Imaging.

[57]  Rajiv Chakravorty,et al.  MobiStream: Error-Resilient Video Streaming in Wireless WANs Using Virtual Channels , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[58]  Aggelos K. Katsaggelos,et al.  Quantization optimized H.264 encoding for traffic video tracking applications , 2010, 2010 IEEE International Conference on Image Processing.

[59]  X. Artigas,et al.  Side Information Generation for Multiview Distributed Video Coding Using a Fusion Approach , 2006, Proceedings of the 7th Nordic Signal Processing Symposium - NORSIG 2006.

[60]  Mihaela van der Schaar,et al.  Cross-layer wireless multimedia transmission: challenges, principles, and new paradigms , 2005, IEEE Wirel. Commun..

[61]  Shipeng Li,et al.  Motion compensated lifting wavelet and its application in video coding , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[62]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[63]  Jacco R. Taal,et al.  Asymmetric Multiple Description Coding using Layered Coding and Lateral Error Correction , 2006 .

[64]  John G. Apostolopoulos,et al.  Unbalanced multiple description video communication using path diversity , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[65]  Qingming Huang,et al.  Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model , 2009, Signal Process. Image Commun..

[66]  Gian Luca Foresti,et al.  Special issue on video communications, processing, and understanding for third generation surveillance systems , 2001 .

[67]  Richard Han,et al.  FireWxNet: a multi-tiered portable wireless system for monitoring weather conditions in wildland fire environments , 2006, MobiSys '06.

[68]  Heiko Schwarz,et al.  MCTF and scalability extension of H.264/AVC and its application to video transmission, storage, and surveillance , 2005, Visual Communications and Image Processing.

[69]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[70]  C. Guillemot,et al.  Distributed Monoview and Multiview Video Coding , 2007, IEEE Signal Processing Magazine.

[71]  J. D. Han,et al.  Moving Target Tracking and Measurement with a Binocular Vision System , 2008, 2008 15th International Conference on Mechatronics and Machine Vision in Practice.

[72]  Bo Yan,et al.  Design and implementation of a sensor-based wireless camera system for continuous monitoring in assistive environments , 2010, Personal and Ubiquitous Computing.

[73]  Martin Reisslein,et al.  A survey of multimedia streaming in wireless sensor networks , 2008, IEEE Communications Surveys & Tutorials.

[74]  Mihaela van der Schaar,et al.  Complexity scalable motion compensated wavelet video encoding , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[75]  Roberto Rinaldo,et al.  Comparison Between Multiple Description and Single Description Video Coding With Forward Error Correction , 2005, 2005 IEEE 7th Workshop on Multimedia Signal Processing.

[76]  Aggelos K. Katsaggelos,et al.  Unequal Error Protection for Robust Streaming of Scalable Video Over Packet Lossy Networks , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[77]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[78]  Andrea J. Goldsmith,et al.  Cross-Layer Design for Lifetime Maximization in Interference-Limited Wireless Sensor Networks , 2006, IEEE Transactions on Wireless Communications.

[79]  Sufen Fong,et al.  MeshEye: A Hybrid-Resolution Smart Camera Mote for Applications in Distributed Intelligent Surveillance , 2007, 2007 6th International Symposium on Information Processing in Sensor Networks.

[80]  Marco Dalai,et al.  The DISCOVER codec: Architecture, Techniques and Evaluation , 2007, PCS 2007.

[81]  Touradj Ebrahimi,et al.  Multi-view video segmentation and tracking for video surveillance , 2009, Defense + Commercial Sensing.

[82]  Gary Bradski,et al.  Computer Vision Face Tracking For Use in a Perceptual User Interface , 1998 .

[83]  Sheldon Leader TELECOMMUNICATIONS HANDBOOK FOR TRANSPORTATION PROFESSIONALS: THE BASICS OF TELECOMMUNICATIONS , 2004 .

[84]  A.J. Goldsmith,et al.  Cross-layer optimization of sensor networks based on cooperative MIMO techniques with rate adaptation , 2005, IEEE 6th Workshop on Signal Processing Advances in Wireless Communications, 2005..

[85]  William A. Pearlman,et al.  A new, fast, and efficient image codec based on set partitioning in hierarchical trees , 1996, IEEE Trans. Circuits Syst. Video Technol..

[86]  A. Murat Tekalp,et al.  Scalable multiple description video coding with flexible number of descriptions , 2005, IEEE International Conference on Image Processing 2005.

[87]  Timo Kohlberger,et al.  Real-Time Optic Flow Computation with Variational Methods , 2003, CAIP.

[88]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[89]  Huijun Di,et al.  Background modeling from a free-moving camera by Multi-Layer Homography Algorithm , 2008, 2008 15th IEEE International Conference on Image Processing.

[90]  Andrea J. Goldsmith,et al.  Cross-layer design of ad hoc networks for real-time video streaming , 2005, IEEE Wireless Communications.

[91]  Tim J. Ellis,et al.  Illumination-Invariant Motion Detection Using Colour Mixture Models , 2001, BMVC.

[92]  Jianfei Cai,et al.  Joint source channel rate-distortion analysis for adaptive mode selection and rate control in wireless video coding , 2002, IEEE Trans. Circuits Syst. Video Technol..

[93]  M. van der Schaar,et al.  Cross-layer wireless multimedia transmission: challenges, principles, and new paradigms , 2005, IEEE Wireless Communications.

[94]  Jo Yew Tham,et al.  Pattern selection for error-resilient slice interleaving based on receiver error concealment technique , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[95]  Amotz Bar-Noy,et al.  Pan and scan: Configuring cameras for coverage , 2011, 2011 Proceedings IEEE INFOCOM.

[96]  Rui Zhang,et al.  Wyner-Ziv coding of motion video , 2002, Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers, 2002..

[97]  Hong Jiang,et al.  Surveillance Video Processing Using Compressive Sensing , 2012, ArXiv.

[98]  Montse Pardàs,et al.  Bayesian foreground segmentation and tracking using pixel-wise background model and region based foreground model , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[99]  Catarina Brites,et al.  Studying the GOP Size Impact on the Performance of a Feedback Channel-Based Wyner-Ziv Video Codec , 2007, PSIVT.

[100]  Dapeng Wu,et al.  Rate-Distortion Optimized Cross-Layer Rate Control in Wireless Video Communication , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[101]  Christoph Stiller,et al.  Fusing optical flow and stereo disparity for object tracking , 2002, Proceedings. The IEEE 5th International Conference on Intelligent Transportation Systems.

[102]  Chee-Yee Chong,et al.  Sensor networks: evolution, opportunities, and challenges , 2003, Proc. IEEE.

[103]  Prashant J. Shenoy,et al.  SensEye: a multi-tier camera sensor network , 2005, ACM Multimedia.

[104]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[105]  Mauro Barni,et al.  Robust video watermarking for wireless multimedia communications , 2000, 2000 IEEE Wireless Communications and Networking Conference. Conference Record (Cat. No.00TH8540).

[106]  Ted Morris,et al.  Advanced Portable Wireless Measurement and Observation Station , 2005 .

[107]  Wu-chi Feng,et al.  Panoptes: A Scalable Architecture for Video Sensor Networking Applications , 2004 .

[108]  Yi Wang,et al.  On full-view coverage in camera sensor networks , 2011, 2011 Proceedings IEEE INFOCOM.

[109]  Aggelos K. Katsaggelos,et al.  MPEG-4 and rate-distortion-based shape-coding techniques , 1998, Proc. IEEE.

[110]  Mourad Ouaret,et al.  Iterative Multiview Side Information for Enhanced Reconstruction in Distributed Video Coding , 2009, EURASIP J. Image Video Process..

[111]  Thrasyvoulos N. Pappas,et al.  Spatiotemporal Algorithm for Background Subtraction , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[112]  Janusz Konrad,et al.  Space-time image sequence analysis: object tunnels and occlusion volumes , 2006, IEEE Transactions on Image Processing.

[113]  Jiang Li,et al.  A real-time interactive multi-view video system , 2005, MULTIMEDIA '05.

[114]  Gozde Bozdagi Akar,et al.  Rate-Distortion Optimization for Stereoscopic Video Streaming with Unequal Error Protection , 2009, EURASIP J. Adv. Signal Process..

[115]  Yaser Sheikh,et al.  Bayesian modeling of dynamic scenes for object detection , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[116]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[117]  Anthony Vetro,et al.  An overview of scalable video streaming , 2007, Wirel. Commun. Mob. Comput..

[118]  Aggelos K. Katsaggelos,et al.  Error resilient video coding techniques , 2000, IEEE Signal Process. Mag..