Two-stage hierarchical video summary extraction to match low-level user browsing preferences

A compact summary of video that conveys visual content at various levels of detail enhances user interaction significantly. In this paper, we propose a two-stage framework to generate MPEG-7-compliant hierarchical key frame summaries of video sequences. At the first stage, which is carried out off-line at the time of content production, fuzzy clustering and data pruning methods are applied to given video segments to obtain a nonredundant set of key frames that comprise the finest level of the hierarchical summary. The number of key frames allocated to each shot or segment is determined dynamically and without user supervision through the use of cluster validation techniques. A coarser summary is generated on-demand in the second stage by reducing the number of key frames to match the low-level browsing preferences of a user. The proposed method has been validated by experimental results on a collection of video programs.

[1]  Dragutin Petkovic,et al.  Content-Based Representation and Retrieval of Visual Media: A State-of-the-Art Review , 1996 .

[2]  B. S. Manjunath,et al.  Color and texture descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..

[3]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Alan Hanjalic,et al.  A New Method for Key Frame Based Video Content Representation , 1998, Image Databases and Multi-Media Search.

[5]  M. Ibrahim Sezan,et al.  Hierarchical video summarization , 1998, Electronic Imaging.

[6]  G H Ball,et al.  A clustering technique for summarizing multivariate data. , 1967, Behavioral science.

[7]  John R. Smith,et al.  MPEG-7 multimedia description schemes , 2001, IEEE Trans. Circuits Syst. Video Technol..

[8]  Shih-Fu Chang,et al.  Overview of the MPEG-7 standard , 2001, IEEE Trans. Circuits Syst. Video Technol..

[9]  Charles A. Bouman,et al.  ViBE: a video indexing and browsing environment , 1999, Optics East.

[10]  James C. Bezdek,et al.  On cluster validity for the fuzzy c-means model , 1995, IEEE Trans. Fuzzy Syst..

[11]  Thomas S. Huang,et al.  Constructing table-of-content for videos , 1999, Multimedia Systems.

[12]  J. Bednar,et al.  Alpha-trimmed means and their relationship to median filters , 1984 .

[13]  Alan Hanjalic,et al.  An integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis , 1999, IEEE Trans. Circuits Syst. Video Technol..

[14]  Mohan S. Kankanhalli,et al.  Content-based representative frame extraction for digital video , 1998, Proceedings. IEEE International Conference on Multimedia Computing and Systems (Cat. No.98TB100241).

[15]  Stephen W. Smoliar,et al.  An integrated system for content-based video retrieval and browsing , 1997, Pattern Recognit..

[16]  Shih-Fu Chang,et al.  Clustering methods for video browsing and annotation , 1996, Electronic Imaging.

[17]  A. Murat Tekalp,et al.  Robust color histogram descriptors for video segment retrieval and identification , 2002, IEEE Trans. Image Process..

[18]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[19]  Borko Furht,et al.  Video and Image Processing in Multimedia Systems , 1995 .

[20]  A. Murat Tekalp,et al.  Efficient Filtering and Clustering Methods for Temporal Video Segmentation and Visual Summarization , 1998, J. Vis. Commun. Image Represent..

[21]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[22]  Gary Marchionini,et al.  Key frame preview techniques for video browsing , 1998, DL '98.

[23]  J. Dunn.  Well-Separated Clusters and Optimal Fuzzy Partitions , 1974 .

[24]  Marco Ceccarelli,et al.  Automation of systems enabling search on stored video data , 1997, Electronic Imaging.

[25]  Gerardo Beni,et al.  A Validity Measure for Fuzzy Clustering , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  M. P. Windham.  Cluster validity for fuzzy clustering algorithms , 1981 .

[27]  R. Gunderson.  Application of Fuzzy Isodata Algorithms to Star Tracker Pointing Systems , 1978 .

[28]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  A. Murat Tekalp,et al.  Group-of-frames/pictures color histogram descriptors for multimedia applications , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[30]  Touradj Ebrahimi,et al.  A new algorithm for shot boundary detection , 2000, 2000 10th European Signal Processing Conference.