Event-driven video adaptation: A powerful tool for industrial video supervision

Efficient video content adaptation requires techniques for content analysis and understanding as well as the development of appropriate mechanisms for content scaling in terms of the network properties, terminal devices characteristics and users’ preferences. This is particularly evident in industrial surveillance applications, due to the huge amount of data needed to be stored, delivered and handled. In this paper, we address both issues by incorporating (a) computer vision tools that allows efficient tracking of salient visual objects for long time regardless of the dynamics of the visual environment –via a self initialized tracking algorithm—and (b) an adaptive optimal rate distortion scheme able to allocate different priorities for each detected video object with respect to users’ needs, network platforms capabilities and terminal characteristics. The self initialized tracker firstly appropriately describes visual content, secondly incorporates adaptive mechanisms for automatically update the tracker to adjust to the current conditions and thirdly includes an efficient decision mechanism that estimates the time instances in which adaptation should be activated. For the rate distortion algorithm, an optimal adaptive framework is adopted which is capable of allocating the desired quality to objects of users’ interest without violating the target bit rate of the sequence. The Wavelet Packet Transform (WPT) is adopted towards this purpose. The advantage of the WPT is that it localizes the frequency components of each video object and therefore it offers additionally content adaptability according to video object texture coding. The WPT tree is transmitted only at the first frame of each shot and thus dew bits are required for its encoding. Experimental results and comparisons with other approaches are presented to illustrate the good performance of the proposed architecture. The results cover real-world and complex industrial environments.

[1]  Theodora A. Varvarigou,et al.  A dataset for workflow recognition in industrial scenes , 2011, 2011 18th IEEE International Conference on Image Processing.

[2]  Stefanos D. Kollias,et al.  Efficient Unsupervised Content-Based Segmentation in Stereoscopic Video Sequences , 2000, Int. J. Artif. Intell. Tools.

[3]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[4]  Anastasios Doulamis,et al.  Scene detection methods for MPEG-encoded video signals , 2000, 2000 10th Mediterranean Electrotechnical Conference. Information Technology and Electrotechnology for the Mediterranean Countries. Proceedings. MeleCon 2000 (Cat. No.00CH37099).

[5]  Nilesh V. Patel,et al.  Video shot detection and characterization for video databases , 1997, Pattern Recognit..

[6]  Stefanos D. Kollias,et al.  Low bit-rate coding of image sequences using adaptive regions of interest , 1998, IEEE Trans. Circuits Syst. Video Technol..

[7]  Touradj Ebrahimi,et al.  Video segmentation based on multiple features for interactive multimedia applications , 1998, IEEE Trans. Circuits Syst. Video Technol..

[8]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[9]  Truong Q. Nguyen,et al.  A Fully Scalable Motion Model for Scalable Video Coding , 2007, IEEE Transactions on Image Processing.

[10]  Athanasios Voulodimos,et al.  Bayesian filter based behavior recognition in workflows allowing for user feedback , 2012, Comput. Vis. Image Underst..

[11]  Shih-Chia Huang,et al.  An Advanced Motion Detection Algorithm With Video Quality Analysis for Video Surveillance Systems , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Pankaj Batra,et al.  Modeling and efficient optimization for object-based scalability and some related problems , 2000, IEEE Trans. Image Process..

[13]  Yonghong Zeng,et al.  Integer DCTs and fast algorithms , 2001, IEEE Trans. Signal Process..

[14]  Anthony Vetro,et al.  Introduction to the Special Section on MPEG-21 , 2005 .

[15]  Theodora A. Varvarigou,et al.  An Industrial Visual Surveillance Framework Based on a Pre-Configured Behavior Repertoire: A Practical Approach , 2011, 2011 UkSim 13th International Conference on Computer Modelling and Simulation.

[16]  Mohamed Abdel-Mottaleb,et al.  Multimedia descriptions based on MPEG-7: extraction and applications , 2004, IEEE Transactions on Multimedia.

[17]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[18]  Xiaoqin Zhang,et al.  Multiple Object Tracking Via Species-Based Particle Swarm Optimization , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Nikolaos D. Doulamis Coupled multi-object tracking and labeling for vehicle trajectory estimation and matching , 2009, Multimedia Tools and Applications.

[20]  Daniel Cremers,et al.  The Elastic Ratio: Introducing Curvature Into Ratio-Based Image Segmentation , 2011, IEEE Transactions on Image Processing.

[21]  E. Kreyszig Introductory Functional Analysis With Applications , 1978 .

[22]  Anil K. Jain,et al.  Object tracking using deformable templates , 2000 .

[23]  Alejandro Linares-Barranco,et al.  Video surveillance at an industrial environment using an address event vision sensor: Comparative between two different video sensor based on a bioinspired retina , 2011, Proceedings of the International Conference on Signal Processing and Multimedia Applications.

[24]  Theodora A. Varvarigou,et al.  IMPROVING MULTI-CAMERA ACTIVITY RECOGNITION BY EMPLOYING NEURAL NETWORK BASED READJUSTMENT , 2012, Appl. Artif. Intell..

[25]  Jean-Marc Odobez,et al.  Embedding Motion in Model-Based Stochastic Tracking , 2004, IEEE Transactions on Image Processing.

[26]  Anastasios D. Doulamis,et al.  Dynamic tracking re-adjustment: a method for automatic tracking recovery in complex visual environments , 2010, Multimedia Tools and Applications.

[27]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[28]  Mihaela van der Schaar,et al.  A hybrid temporal-SNR fine-granular scalability for Internet video , 2001, IEEE Trans. Circuits Syst. Video Technol..

[29]  Nikolaos F. Matsatsinis,et al.  Sensor Networks and Multi-Agents inIndustrial Workflows , 2011 .

[30]  Jens-Rainer Ohm,et al.  Advances in Scalable Video Coding , 2005, Proceedings of the IEEE.

[31]  Pierre Machart Morphological Segmentation , 2009 .

[32]  Nikolaos F. Matsatsinis,et al.  Visual understanding industrial workflows under uncertainty on distributed service oriented architectures , 2012, Future Gener. Comput. Syst..

[33]  Luc Van Gool,et al.  Unsupervised workflow discovery in industrial environments , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[34]  Ruth Bergman,et al.  Perceptual Segmentation: Combining Image Segmentation With Object Tagging , 2011, IEEE Transactions on Image Processing.

[35]  Ehud Rivlin,et al.  Tracking by Affine Kernel Transformations Using Color and Boundary Cues , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Seong-Won Lee,et al.  Combined shape and feature-based video analysis and its application to non-rigid object tracking , 2011 .

[37]  Adrian N. Evans,et al.  A median centred difference gradient operator and its application in watershed segmentation , 2011 .

[38]  Pierre Duhamel,et al.  Iterative backward segmentation for hierarchical wavelet image coding , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[39]  Hendry,et al.  Archive and Preservation of Media Content Using MPEG-A , 2010, IEEE MultiMedia.

[40]  B. S. Manjunath,et al.  Variable Length Open Contour Tracking Using a Deformable Trellis , 2011, IEEE Transactions on Image Processing.

[41]  Rik Van de Walle,et al.  MPEG-21 digital item Processing , 2005, IEEE Trans. Multim..

[42]  Stefanos D. Kollias,et al.  A fuzzy video content representation for video summarization and content-based retrieval , 2000, Signal Process..

[43]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[44]  Thomas Sikora,et al.  The MPEG-4 video standard verification model , 1997, IEEE Trans. Circuits Syst. Video Technol..

[45]  Y. Yang,et al.  Rate-distortion optimizations for region and object based wavelet video coding , 2000, Conference Record of the Thirty-Fourth Asilomar Conference on Signals, Systems and Computers (Cat. No.00CH37154).

[46]  John S. Baras,et al.  Scalable coding of video objects , 1998, ISCAS '98. Proceedings of the 1998 IEEE International Symposium on Circuits and Systems (Cat. No.98CH36187).

[47]  Shih-Fu Chang,et al.  Introduction to the special issue on MPEG-7 , 2001, IEEE Trans. Circuits Syst. Video Technol..

[48]  Boon-Lock Yeo,et al.  Rapid scene analysis on compressed video , 1995, IEEE Trans. Circuits Syst. Video Technol..

[49]  Narendra Ahuja,et al.  A scheme for spatial scalability using nonscalable encoders , 2003, IEEE Trans. Circuits Syst. Video Technol..

[50]  Anil K. Jain,et al.  Object Tracking Using Deformable Templates , 1998, ICCV.

[51]  Hujun Bao,et al.  Robust Bilayer Segmentation and Motion/Depth Estimation with a Handheld Camera , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.