Clip-based similarity measure for query-dependent clip retrieval and video summarization

This paper proposes a new approach and algorithm for the similarity measure of video clips. The similarity is mainly based on two bipartite graph matching algorithms: maximum matching (MM) and optimal matching (OM). MM is able to rapidly filter irrelevant video clips, while OM is capable of ranking the similarity of clips according to visual and granularity factors. We apply the similarity measure for two tasks: retrieval and summarization. In video retrieval, a hierarchical retrieval framework is constructed based on MM and OM. The validity of the framework is theoretically proved and empirically verified on a video database of 21 h. A query-dependent clip segmentation algorithm is also proposed to automatically locate the potential boundaries of clips in videos. In video summarization, a graph-based clustering algorithm, incorporated with the proposed similarity measure, is adopted to detect the highlighted events reported by different newscasts.

[1]  Shih-Fu Chang,et al.  Story boundary detection in large broadcast news video archives: techniques, experience and trends , 2004, MULTIMEDIA '04.

[2]  Avideh Zakhor,et al.  Fast similarity search and clustering of video sequences on the world-wide-web , 2005, IEEE Transactions on Multimedia.

[3]  J. R. Kender,et al.  Mosaic-based clustering of scene locations in videos , 2001, Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries (CBAIVL 2001).

[4]  Avideh Zakhor,et al.  Efficient video similarity measurement with video signature , 2002, Proceedings. International Conference on Image Processing.

[5]  Sanjeev R. Kulkarni,et al.  A framework for measuring video similarity and its application to video query by example , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[6]  Wolfgang Effelsberg,et al.  VisualGREP: A Systematic Method to Compare and Retrieve Video Sequences , 2004, Multimedia Tools and Applications.

[7]  Alexander Schrijver,et al.  Combinatorial optimization. Polyhedra and efficiency. , 2003 .

[8]  John R. Kender,et al.  Video scene segmentation via continuous video coherence , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[9]  Alan Hanjalic,et al.  Automated high-level movie segmentation for advanced video-retrieval systems , 1999, IEEE Trans. Circuits Syst. Video Technol..

[10]  Yueting Zhuang,et al.  Content-based video similarity model , 2000, MM 2000.

[11]  Wei Xiong,et al.  Query by video clip , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[12]  Yueting Zhuang,et al.  A new approach to retrieve video by example video clip , 1999, MULTIMEDIA '99.

[13]  Chong-Wah Ngo,et al.  Motion-Based Video Representation for Scene Change Detection , 2004, International Journal of Computer Vision.

[14]  George Karypis,et al.  A Comparison of Document Clustering Techniques , 2000 .

[15]  Kunio Kashino,et al.  A quick search method for audio and video signals based on histogram pruning , 2003, IEEE Trans. Multim..

[16]  Chong-Wah Ngo,et al.  Video partitioning by temporal slice coherency , 2001, IEEE Trans. Circuits Syst. Video Technol..

[17]  Donald A. Adjeroh,et al.  A Distance Measure for Video Sequences , 1999, Comput. Vis. Image Underst..

[18]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  John R. Kender,et al.  Video Summaries through Mosaic-Based Shot and Scene Clustering , 2002, ECCV.

[20]  Tat-Seng Chua,et al.  A match and tiling approach to content-based video retrieval , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[21]  Boon-Lock Yeo,et al.  Video visualization for compact presentation and fast browsing of pictorial content , 1997, IEEE Trans. Circuits Syst. Video Technol..

[22]  Mohamed Abdel-Mottaleb,et al.  Content-based video retrieval by example video clip , 1997, Electronic Imaging.

[23]  Milind R. Naphade,et al.  Novel scheme for fast and efficent video sequence matching using compact signatures , 1999, Electronic Imaging.

[24]  Justin Zobel,et al.  Fast video matching with signature alignment , 2003, MIR '03.

[25]  Wolfgang Effelsberg,et al.  VisualGREP: a systematic method to compare and retrieve video sequences , 1997, Electronic Imaging.

[26]  Takao Nishizeki,et al.  Graph Theory and Algorithms , 1981, Lecture Notes in Computer Science.

[27]  Dimitri P. Bertsekas,et al.  Auction algorithms for network flow problems: A tutorial introduction , 1992, Comput. Optim. Appl..