An integrated approach to summarization and adaptation using H.264/MPEG-4 SVC

The huge amount of multimedia content and the variety of terminals and networks make video summarization and video adaptation two key technologies to provide effective access and browsing. With scalable video coding, the adaptation of video to heterogeneous terminals and networks can be efficiently achieved using together a layered coding hierarchy and bitstream extraction. On the other hand, many video summarization techniques can be seen as a special case of structural adaptation. This paper describes how some of them can be modified and included in the adaptation framework of the scalable extension of H.264/AVC. The advantage of this approach is that summarization and adaptation are integrated into the same efficient framework. The utility of this approach is demonstrated with experimental results for the generation of storyboards and video skims, showing that the proposed framework can generate the adapted bitstream of the summary faster than a conventional transcoding approach.

[1]  Stéphane Pateux,et al.  Optimized Rate-Distortion Extraction with Quality Layers , 2006, 2006 International Conference on Image Processing.

[2]  Liang-Tien Chia,et al.  MPEG-21 digital item adaptation by applying perceived motion energy to H.264 video , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[3]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[4]  Aggelos K. Katsaggelos,et al.  Rate-distortion optimal video summary generation , 2005, IEEE Transactions on Image Processing.

[5]  Chia-Hung Yeh,et al.  Techniques for movie content analysis and skimming: tutorial and overview on video abstraction techniques , 2006, IEEE Signal Processing Magazine.

[6]  Ahmet M. Kondoz,et al.  Fast analysis of scalable video for adaptive browsing interfaces , 2009, Comput. Vis. Image Underst..

[7]  Wesley De Neve,et al.  Temporal Video Segmentation on H.264/AVC Compressed Bitstreams , 2007, MMM.

[8]  Sang Uk Lee,et al.  Efficient video indexing scheme for content-based retrieval , 1999, IEEE Trans. Circuits Syst. Video Technol..

[9]  Ahmed K. Elmagarmid,et al.  InsightVideo: toward hierarchical video content organization for efficient browsing, summarization and retrieval , 2005, IEEE Transactions on Multimedia.

[10]  Lie Lu,et al.  A generic framework of user attention model and its application in video summarization , 2005, IEEE Trans. Multim..

[11]  Gary Marchionini,et al.  How fast is too fast? evaluating fast forward surrogates for digital video , 2003, 2003 Joint Conference on Digital Libraries, 2003. Proceedings..

[12]  Yong Wang,et al.  Optimal adaptation decision-taking for terminal and network quality-of-service , 2005, IEEE Transactions on Multimedia.

[13]  R. Van de Walle,et al.  Enhanced Shot-Based Video Adaptation using MPEG-21 generic Bitstream Syntax Schema , 2007, 2007 IEEE Symposium on Computational Intelligence in Image and Signal Processing.

[14]  Boon-Lock Yeo,et al.  Video visualization for compact presentation and fast browsing of pictorial content , 1997, IEEE Trans. Circuits Syst. Video Technol..

[15]  Jun Xin,et al.  Video Adaptation : Concepts , Technologies , and Open Issues , .

[16]  Jesús Bescós,et al.  Content-Driven Adaptation of On-Line Video , 2007, 2007 International Workshop on Content-Based Multimedia Indexing.

[17]  Ajay Divakaran,et al.  Constant pace skimming and temporal sub-sampling of video using motion activity , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[18]  Takeo Kanade,et al.  Video skimming and characterization through the combination of image and language understanding , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[19]  Akio Yamada,et al.  The MPEG-7 color layout descriptor: a compact image feature description for high-speed image/video segment retrieval , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[20]  Anthony Vetro,et al.  MPEG-21 digital item adaptation: enabling universal multimedia access , 2004, IEEE MultiMedia.

[21]  Wolfgang Effelsberg,et al.  Abstracting Digital Movies Automatically , 1996, J. Vis. Commun. Image Represent..

[22]  Avideh Zakhor,et al.  Applications of Video-Content Analysis and Retrieval , 2002, IEEE Multim..

[23]  Michael A. Smith,et al.  Video skimming and characterization through the combination of image and language understanding techniques , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Shih-Fu Chang,et al.  Survey of compressed-domain features used in audio-visual indexing and analysis , 2003, J. Vis. Commun. Image Represent..

[25]  Luis Herranz,et al.  Integrating semantic analysis and scalable video coding for efficient content-based adaptation , 2007, Multimedia Systems.

[26]  Heiko Schwarz,et al.  Overview of the Scalable Video Coding Extension of the H.264/AVC Standard , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Yu Sun,et al.  Video transcoding: an overview of various techniques and research issues , 2005, IEEE Transactions on Multimedia.

[28]  Sethuraman Panchanathan,et al.  Adaptive Video Transmission Schemes Using MPEG-7 Motion Intensity Descriptor , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Christian Timmerer,et al.  Performance Analysis of Scalable Video Adaptation: Generic versus Specific Approach , 2008, 2008 Ninth International Workshop on Image Analysis for Multimedia Interactive Services.

[30]  Stéphane Pateux,et al.  Optimized Rate-Distortion Extraction With Quality Layers in the Scalable Extension of H.264/AVC , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Gary J. Sullivan,et al.  Video Compression - From Concepts to the H.264/AVC Standard , 2005, Proceedings of the IEEE.

[32]  Chong-Wah Ngo,et al.  Automatic video summarization by graph modeling , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[33]  Janko Calic,et al.  Efficient Layout of Comic-Like Video Summaries , 2007, IEEE Transactions on Circuits and Systems for Video Technology.