VISON: VIdeo Summarization for ONline applications

Recent advances in technology have increased the availability of video data, creating a strong requirement for efficient systems to manage those materials. Making efficient use of video information requires that data to be accessed in a user-friendly way. This has been the goal of a quickly evolving research area known as video summarization. Most of existing techniques to address the problem of summarizing a video sequence have focused on the uncompressed domain. However, decoding and analyzing of a video sequence are two extremely time-consuming tasks. Thus, video summaries are usually produced off-line, penalizing any user interaction. The lack of customization is very critical, as users often have different demands and resources. Since video data are usually available in compressed form, it is desirable to directly process video material without decoding. In this paper, we present VISON, a novel approach for video summarization that works in the compressed domain and allows user interaction. The proposed method is based on both exploiting visual features extracted from the video stream and on using a simple and fast algorithm to summarize the video content. Results from a rigorous empirical comparison with a subjective evaluation show that our technique produces video summaries with high quality relative to the state-of-the-art solutions and in a computational time that makes it suitable for online usage.

[1]  Stephen R. Gulliver,et al.  Introduction to special issue on eye-tracking applications in multimedia systems , 2007, TOMCCAP.

[2]  J. Crowley,et al.  Experimental Comparison of Correlation Techniques , 2007 .

[3]  Jurandy Almeida,et al.  Making colors worth more than a thousand words , 2008, SAC '08.

[4]  Raj Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[5]  Jurandy Almeida,et al.  Rapid Cut Detection on Compressed Video , 2011, CIARP.

[6]  Boon-Lock Yeo,et al.  Rapid scene analysis on compressed video , 1995, IEEE Trans. Circuits Syst. Video Technol..

[7]  Federico Tombari,et al.  Reliable rejection of mismatching candidates for efficient ZNCC template matching , 2008, 2008 15th IEEE International Conference on Image Processing.

[8]  Changming Sun,et al.  Fast optical flow using 3D shortest path techniques , 2002, Image Vis. Comput..

[9]  Du-Ming Tsai,et al.  The evaluation of normalized cross correlations for defect detection , 2003, Pattern Recognit. Lett..

[10]  Ba Tu Truong,et al.  Video abstraction: A systematic review and classification , 2007, TOMCCAP.

[11]  Arnaldo de Albuquerque Araújo,et al.  VSUMM: A mechanism designed to produce static video summaries and a novel evaluation method , 2011, Pattern Recognit. Lett..

[12]  Allan Kuchinsky,et al.  Quality is in the eye of the beholder: meeting users' requirements for Internet quality of service , 2000, CHI.

[13]  Jurandy Almeida,et al.  Rapid Video Summarization on Compressed Video , 2010, 2010 IEEE International Symposium on Multimedia.

[14]  Paul Over,et al.  The trecvid 2007 BBC rushes summarization evaluation pilot , 2007, TVS '07.

[15]  Changming Sun,et al.  Fast Stereo Matching Using Rectangular Subregioning and 3D Maximum-Surface Techniques , 2002, International Journal of Computer Vision.

[16]  David S. Doermann,et al.  Video summarization by curve simplification , 1998, MULTIMEDIA '98.

[17]  Marco Pellegrini,et al.  STIMO: STIll and MOving video storyboard for the web scenario , 2009, Multimedia Tools and Applications.

[18]  Jurandy Almeida,et al.  Comparison of video sequences with histograms of motion patterns , 2011, 2011 18th IEEE International Conference on Image Processing.

[19]  Alan Hanjalic,et al.  An integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis , 1999, IEEE Trans. Circuits Syst. Video Technol..

[20]  Jurandy Almeida,et al.  Robust Estimation of Camera Motion Using Optical Flow Models , 2009, ISVC.

[21]  José María Martínez Sanchez,et al.  An efficient summarization algorithm based on clustering and bitstream extraction , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[22]  Paul Over,et al.  The trecvid 2008 BBC rushes summarization evaluation , 2008, TVS '08.

[23]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[24]  Mark D. Dunlop,et al.  Subjectivity and Notions of Time and Value in Interactive Information Retrieval , 1998, Interact. Comput..

[25]  Georgios S. Paschos,et al.  Perceptually uniform color spaces for color texture analysis: an empirical evaluation , 2001, IEEE Trans. Image Process..

[26]  Tao Mei,et al.  Video collage , 2007, ACM Multimedia.

[27]  Harry W. Agius,et al.  Video summarisation: A conceptual framework and survey of the state of the art , 2008, J. Vis. Commun. Image Represent..

[28]  Janko Calic,et al.  Efficient Layout of Comic-Like Video Summaries , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Masaru Sugano,et al.  Automated MPEG audio-video summarization and description , 2002, Proceedings. International Conference on Image Processing.

[30]  Yelena Yesha,et al.  Keyframe-based video summarization using Delaunay clustering , 2006, International Journal on Digital Libraries.

[31]  Xin Liu,et al.  Video summarization using singular value decomposition , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[32]  Mario A. Nascimento,et al.  Techniques for Color-Based Image Retrieval , 2003 .

[33]  Yueting Zhuang,et al.  Adaptive key frame extraction using unsupervised clustering , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[34]  N. J. Leite,et al.  Estimation of Camera Parameters in Video Sequences with a Large Amount of Scene Motion , 2010 .

[35]  Shih-Fu Chang,et al.  A fully automated content-based video search engine supporting spatiotemporal queries , 1998, IEEE Trans. Circuits Syst. Video Technol..

[36]  Amarnath Gupta,et al.  Virage video engine , 1997, Electronic Imaging.

[37]  Lawrence A. Rowe,et al.  Multimedia systems and applications , 2010, 2010 International Conference on Signal Processing and Multimedia Applications (SIGMAP).

[38]  Mohan S. Kankanhalli,et al.  Semantic video summarization in compressed domain MPEG video , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[39]  Federico Tombari,et al.  ZNCC-based template matching using bounded partial correlation , 2004 .