Distributed architectures and logical-task decomposition in multimedia surveillance systems

In the past few years, the development of complex surveillance systems has captured the interest of both the research and industrial worlds. Strong and challenging requirements of modern society are involved in this problem, which aims to increase safety and security in several application domains such as transport, tourism, home and bank security, military applications, etc. At the same time, fast improvements in microelectronics, telecommunications, and computer science make it necessary to consider new perspectives in this field. The main objective of this paper is to investigate, discuss, and evaluate the impact of distributed processing and new communication techniques on multimedia surveillance systems, which represent the so-called third-generation surveillance systems (3 GSSs). In particular, aspects related to the distribution of intelligence among multiple-processing and wide-bandwidth resources are discussed in detail. It is shown how distribution of intelligence can be obtained by a hierarchical architecture that partitions, in a dynamic way, the main logical processing tasks (i.e., representation, recognition, and communication) performed in a 3 GSS physical architecture made up of intelligent cameras, hubs, and central control rooms. The advantages of this solution are pointed out in terms of 1) increased flexibility and reconfigurability and 2) optimal allocation of available processing and bandwidth resources. Finally, a case study is analyzed that allows one to gain a deeper insight into a distributed surveillance system.

[1]  Hans-Hellmut Nagel,et al.  Incremental recognition of traffic situations from video image sequences , 2000, Image Vis. Comput..

[2]  Gian Luca Foresti,et al.  Guest Editorial: Video Processing and Communications in Real-Time Surveillance Systems , 2001, Real Time Imaging.

[3]  Carlo S. Regazzoni,et al.  Distributed data fusion for real-time crowding estimation , 1996, Signal Process..

[4]  Ian H. Witten,et al.  Arithmetic coding for data compression , 1987, CACM.

[5]  R. Mattone,et al.  A New Solution Philosophy for Complex Pattern Recognition Problems: Application to Advanced Video-Surveillance , 2000 .

[6]  Gary Friedman,et al.  The trustworthy digital camera: restoring credibility to the photographic image , 1993 .

[7]  Gian Luca Foresti,et al.  Special issue on video communications, processing, and understanding for third generation surveillance systems , 2001 .

[8]  Shyang Chang,et al.  Statistical change detection with moments under time-varying illumination , 1998, IEEE Trans. Image Process..

[9]  Tieniu Tan,et al.  Recognizing objects on the ground-plane , 1994, Image Vis. Comput..

[10]  Ioannis Pavlidis,et al.  Urban surveillance systems: from the laboratory to the commercial world , 2001, Proc. IEEE.

[11]  Franco Oberti,et al.  Performance Evaluation Criterion for Characterizing Video-Surveillance Systems , 2001, Real Time Imaging.

[12]  François Brémond,et al.  Tracking multiple nonrigid objects in video sequences , 1998, IEEE Trans. Circuits Syst. Video Technol..

[13]  Shaogang Gong,et al.  Visual Surveillance in a Dynamic and Uncertain World , 1995, Artif. Intell..

[14]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[15]  C. P. Liu A New Two Successive Process Image Compression Technique Using Subband Coding and JPEG Discrete Cosine Transform Coding , 1997, CVGIP Graph. Model. Image Process..

[16]  Carlo S. Regazzoni,et al.  Advanced Video-Based Surveillance Systems , 1998 .

[17]  Hironobu Fujiyoshi,et al.  A System for Video Surveillance and Monitoring CMU VSAM Final Report , 1999 .

[18]  A. Teuner,et al.  Surveillance sensor systems using CMOS imagers , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[19]  L. Hanzo,et al.  Adaptive multicarrier modulation: a convenient framework for time-frequency processing in wireless communications , 2000, Proceedings of the IEEE.

[20]  John G. Proakis,et al.  Digital Communications , 1983 .

[21]  JEFFREY WOOD,et al.  Invariant pattern recognition: A review , 1996, Pattern Recognit..

[22]  Carlo S. Regazzoni,et al.  “Long-Memory” Matching of Interacting Complex Objects from Real Image Sequences , 1997 .

[23]  Donald G. Chandler Applications for a Bidirectional Broadband Coaxial Cable Communications System , 1970 .

[24]  Eiichi Tanaka A Metric Between Unrooted and Unordered Trees and its Bottom-Up Computing Method , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  U. Oppelt,et al.  New possibilities for video applications in the security field , 1995, Proceedings The Institute of Electrical and Electronics Engineers. 29th Annual 1995 International Carnahan Conference on Security Technology.

[26]  Anil K. Jain,et al.  Three-Dimensional Object Recognition Systems , 1993 .

[27]  Terrance E. Boult,et al.  Into the woods: visual surveillance of noncooperative and camouflaged targets in complex outdoor settings , 2001, Proc. IEEE.

[28]  S. Maitra Moment invariants , 1979, Proceedings of the IEEE.

[29]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Gian Luca Foresti,et al.  Object detection and tracking in time-varying and badly illuminated outdoor environments , 1998 .

[31]  Shih-Fu Chang,et al.  Next-generation content representation, creation, and searching for new-media applications in education , 1998 .

[32]  Haim Schweitzer,et al.  Organizing image databases as visual-content search trees , 1999, Image Vis. Comput..

[33]  Michael W. Marcellin,et al.  Image coding using adaptive recursive interpolative DPCM , 1995, IEEE Trans. Image Process..

[34]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Tatsuya Suda,et al.  Credit-based source-adaptive multilayered video multicast , 2000, Perform. Evaluation.

[36]  Bruce S. Davie,et al.  Computer Networks: A System Approach , 1998, IEEE Communications Magazine.

[37]  Jeffrey E. Boyd,et al.  MPI-Video infrastructure for dynamic environments , 1998, Proceedings. IEEE International Conference on Multimedia Computing and Systems (Cat. No.98TB100241).

[38]  Gordon L. Stuber,et al.  Principles of Mobile Communication , 1996 .

[39]  Aggelos K. Katsaggelos,et al.  MPEG-4 and rate-distortion-based shape-coding techniques , 1998, Proc. IEEE.

[40]  Lilly Spirkovska Three-dimensional object recognition using similar triangles and decision trees , 1993, Pattern Recognit..

[41]  Takeo Kanade,et al.  Algorithms for cooperative multisensor surveillance , 2001, Proc. IEEE.

[42]  Jörn Ostermann,et al.  Detection of Moving Cast Shadows for Object Segmentation , 1999, IEEE Trans. Multim..

[43]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[44]  Zeno Geradts,et al.  Forensic photo/videogrammetry: Monte Carlo simulation of pixel and measurement errors , 1999, Other Conferences.

[45]  G. Van Sickle,et al.  Aircraft self reports for military air surveillance , 1999 .

[46]  Ioannis Pitas,et al.  Nonlinear Digital Filters - Principles and Applications , 1990, The Springer International Series in Engineering and Computer Science.

[47]  Franck Luthon,et al.  Real-time DSP implementation for MRF-based video motion detection , 1999, IEEE Trans. Image Process..

[48]  G. Agrawal Fiber‐Optic Communication Systems , 2021 .

[49]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[50]  Jonathan Schaeffer,et al.  Best-First Fixed-Depth Minimax Algorithms , 1996, J. Int. Comput. Games Assoc..

[51]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[52]  D. Miller,et al.  Sensor link protocol: linking sensor systems to the digital battlefield , 1998, IEEE Military Communications Conference. Proceedings. MILCOM 98 (Cat. No.98CH36201).

[53]  Paolo Remagnino,et al.  An Agent Society for Scene Interpretation , 2000 .

[54]  Carlo S. Regazzoni,et al.  Advanced image-processing tools for counting people in tourist site-monitoring applications , 2001, Signal Process..

[55]  Gregory A. Baxes,et al.  Digital image processing - principles and applications , 1994 .

[56]  大野 義夫,et al.  Computer Graphics : Principles and Practice, 2nd edition, J.D. Foley, A.van Dam, S.K. Feiner, J.F. Hughes, Addison-Wesley, 1990 , 1991 .

[57]  Minoru Etoh,et al.  MPEG-4, Part 2: Submitted papers , 1997, Signal Process. Image Commun..

[58]  Yoshiaki Shirai,et al.  Real-Time Surveillance System Detecting Persons in Complex Scenes , 2001, Real Time Imaging.

[59]  Michael Meyer,et al.  Video surveillance applications using multiple views of a scene , 1999 .

[60]  Yuval Shavitt,et al.  Active networks for efficient distributed network management , 2000 .

[61]  Borko Furht,et al.  Video and Image Processing in Multimedia Systems , 1995 .

[62]  A. Bruce Carlson,et al.  Communication systems: an introduction to signals and noise in electrical communication , 1975 .

[63]  R. P. Wishner,et al.  Battlefield awareness via synergistic SAR and MTI exploitation , 1998 .

[64]  Andrea Cavallaro,et al.  Image Analysis for Video Surveillance Based on Spatial Regularization of a Statistical Model-Based Change Detection , 2001, Real Time Imaging.

[65]  G. Thiel Automatic CCTV surveillance-towards the VIRTUAL GUARD , 1999, Proceedings IEEE 33rd Annual 1999 International Carnahan Conference on Security Technology (Cat. No.99CH36303).

[66]  Aggelos K. Katsaggelos,et al.  Signal Recovery Techniques for Image and Video Compression and Transmission , 1998, Springer US.

[67]  A. D. Williams,et al.  Noise and intermodulation problems in multichannel closed-circuit television systems , 1961, Transactions of the American Institute of Electrical Engineers, Part I: Communication and Electronics.

[68]  Vittorio Murino,et al.  A distributed approach to 3D road scene recognition , 1994 .

[69]  Ramesh C. Jain,et al.  Three-dimensional object recognition , 1985, CSUR.

[70]  Kaveh Pahlavan,et al.  Wireless Information Networks , 1995 .

[71]  Gian Luca Foresti,et al.  Outdoor Scene Classification by a Neural Tree-Based Approach , 1999, Pattern Analysis & Applications.

[72]  Gian Luca Foresti,et al.  Multimedia Video-Based Surveillance Systems: Requirements, Issues and Solutions , 2000 .

[73]  Miroslaw Pawlak,et al.  On the reconstruction aspects of moment descriptors , 1992, IEEE Trans. Inf. Theory.

[74]  Bir Bhanu,et al.  The specification of distributed sensing and control , 1985, J. Field Robotics.

[75]  Jan M. Rabaey,et al.  Analysis of multidimensional DSP specifications , 1996, IEEE Trans. Signal Process..

[76]  Roldano Cattoni,et al.  The Advanced Visual Monitoring Project at IRST , 1999 .

[77]  Jon Siegel,et al.  OMG overview: CORBA and the OMA in enterprise computing , 1998, CACM.

[78]  C. S. Regazzoni,et al.  Object Detection and Tracking in Distributed Surveillance Systems Using Multiple Cameras , 2002 .

[79]  Klaus Illgner DSPs for image and video processing , 2000, Signal Process..

[80]  Carlo S. Regazzoni,et al.  Use of Advanced Video Surveillance and Communication Technologies for Remote Monitoring of Protected Sites , 1999 .

[81]  Carlo S. Regazzoni,et al.  A distributed surveillance system for detection of abandoned objects in unmanned railway environments , 2000, IEEE Trans. Veh. Technol..

[82]  Rick S. Blum,et al.  A categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application , 1999, Proc. IEEE.

[83]  Gian Luca Foresti,et al.  Distributed spatial reasoning for multisensory image interpretation , 1993, Signal Process..

[84]  Touradj Ebrahimi,et al.  Evaluation of video segmentation methods for surveillance applications , 2000, 2000 10th European Signal Processing Conference.

[85]  James Llinas,et al.  An introduction to multisensor data fusion , 1997, Proc. IEEE.

[86]  David C. Hogg,et al.  Generating Spatiotemporal Models from Examples , 1995, BMVC.

[87]  Gian Luca Foresti,et al.  Multimedia Video-Based Surveillance Systems , 2000 .

[88]  N. Garcia,et al.  Motorway surveillance through stereo computer vision , 1999, Proceedings IEEE 33rd Annual 1999 International Carnahan Conference on Security Technology (Cat. No.99CH36303).

[89]  Fernando Pereira MPEG-4: Why, what, how and when? , 2000, Signal Process. Image Commun..

[90]  Touradj Ebrahimi,et al.  High-performance compression of visual information-a tutorial review. I. Still pictures , 1999, Proc. IEEE.

[91]  C. L. TAN,et al.  An analysis of a distributed multiresolution vision system , 1989, Pattern Recognit..

[92]  Ramin Zabih,et al.  Bayesian multi-camera surveillance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[93]  I. Kuroda,et al.  Multimedia processors , 1998, Proc. IEEE.

[94]  Alexander Reinefeld,et al.  Enhanced Iterative-Deepening Search , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[95]  M. Kunt,et al.  Block coding of graphics: A tutorial review , 1980, Proceedings of the IEEE.

[96]  Minoru Etoh,et al.  MPEG-4, part 1: Invited papers , 1997, Signal Process. Image Commun..

[97]  Demetri Psaltis,et al.  Recognitive Aspects of Moment Invariants , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[98]  Aggelos K. Katsaggelos,et al.  SNR scalable video coder using progressive transmission of DCT coefficients , 1998, Electronic Imaging.

[99]  T. Ebrahimi,et al.  Change detection and background extraction by linear algebra , 2001, Proc. IEEE.

[100]  Carlo S. Regazzoni,et al.  Localisation and tracking of multiple unknown objects in real environments , 1995 .

[101]  Farzin Mokhtarian,et al.  Robust Image Corner Detection Through Curvature Scale Space , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[102]  Carlo S. Regazzoni,et al.  Dynamic Shape Detection for Multiple Camera Systems , 2000 .

[103]  Patrick Bouthemy,et al.  Moving object detection in color image sequences using region-level graph labeling , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[104]  Michael J. Flynn,et al.  Producer-consumer communication in distributed shared memory multiprocessors , 1999, Proc. IEEE.

[105]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[106]  A. Murat Tekalp,et al.  Face and 2-D mesh animation in MPEG-4 , 2000, Signal Process. Image Commun..

[107]  Andrew J. Viterbi,et al.  Principles of Digital Communication and Coding , 1979 .

[108]  Stan Sclaroff,et al.  Improved Tracking of Multiple Humans with Trajectory Predcition and Occlusion Modeling , 1998 .

[109]  Carlo S. Regazzoni,et al.  Real-time video-shot detection for scene surveillance applications , 2000, IEEE Trans. Image Process..

[110]  Fabio Roli,et al.  Learning and Classification of Suspicious Events for Advanced Visual-Based Surveillance , 2000 .

[111]  Carlo S. Regazzoni,et al.  Network Management Within an Architecture for Distributed Hierarchial Digital Surveillance Systems , 2000 .

[112]  Monique Thonnat,et al.  The PASSWORDS Project [intelligent video image analysis system] , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[113]  Alireza Khotanzad,et al.  Invariant Image Recognition by Zernike Moments , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[114]  David G. Stork,et al.  Pattern Classification , 1973 .

[115]  Ann E. Nicholson,et al.  Dynamic Belief Networks for Discrete Monitoring , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[116]  L. Hanzo,et al.  Interactive cellular and cordless video telephony: State-of-the-art system design principles and expected performance , 2000, Proceedings of the IEEE.

[117]  A. M. Tekalp Special Issue On Multimedia Signal Processing, Part I [Scanning the Issue] , 1998 .