Key frame vector and its application to shot retrieval

This paper proposes a video representation method named key frame vector (KFV) to support video shot retrieval. By considering temporal correlation between frames reflects the relations between visual contents of frames, and video content weighting has influence on the video comparison performance, the proposed KFV incorporates the temporal correlation and spatial relations between frames, as well as content weighting. It employs a block-based frame comparison scheme, and uses EMD to compare shots represented by KFVs. The proposed algorithm has been demonstrated to be effective in comparison with state-of-the-arts methods.

[1]  Patrick Gros,et al.  A Geometrical Key-Frame Selection Method Exploiting Dominant Motion Estimation in Video , 2004, CIVR.

[2]  Jintao Li,et al.  Interactive key frame selection model , 2006, J. Vis. Commun. Image Represent..

[3]  Chong-Wah Ngo,et al.  OM-based video shot retrieval by one-to-one matching , 2007, Multimedia Tools and Applications.

[4]  Mubarak Shah,et al.  Detection and representation of scenes in videos , 2005, IEEE Transactions on Multimedia.

[5]  Zi Huang,et al.  Batch Nearest Neighbor Search for Video Retrieval , 2008, IEEE Transactions on Multimedia.

[6]  John R. Kender,et al.  Optimization Algorithms for the Selection of Key Frame Sequences of Variable Length , 2002, ECCV.

[7]  Chong-Wah Ngo,et al.  Motion-Based Video Representation for Scene Change Detection , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[8]  Tie-Yan Liu,et al.  Shot reconstruction degree: a novel criterion for key frame selection , 2004, Pattern Recognit. Lett..

[9]  Dipti Prasad Mukherjee,et al.  Key Frame Estimation in Video Using Randomness Measure of Feature Point Pattern , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Jenq-Neng Hwang,et al.  Object-based video abstraction for video surveillance systems , 2002, IEEE Trans. Circuits Syst. Video Technol..

[12]  Rong Yan,et al.  Learning query-class dependent weights in automatic video retrieval , 2004, MULTIMEDIA '04.

[13]  Guoliang Fan,et al.  Combined key-frame extraction and object-based video segmentation , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Tao Mei,et al.  Video annotation based on temporally consistent Gaussian random field , 2007 .

[15]  Qi Tian,et al.  Multilevel video representation with application to keyframe extraction , 2004, 10th International Multimedia Modelling Conference, 2004. Proceedings..

[16]  Yueting Zhuang,et al.  Content-based video similarity model , 2000, ACM Multimedia.

[17]  Rainer Lienhart,et al.  The Holy Grail of Multimedia Information Retrieval: So Close or Yet So Far Away? , 2008 .

[18]  Mohan S. Kankanhalli,et al.  Application Potential of Multimedia Information Retrieval , 2008, Proceedings of the IEEE.

[19]  Akio Nagasaka,et al.  Automatic Video Indexing and Full-Video Search for Object Appearances , 1991, VDB.

[20]  Chia-Wen Lin,et al.  Fast coarse-to-fine video retrieval using shot-level spatio-temporal statistics , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Li Zhao,et al.  Key-frame extraction and shot retrieval using nearest feature line (NFL) , 2000, MULTIMEDIA '00.

[22]  Kin-Man Lam,et al.  A new key frame representation for video segment retrieval , 2005, IEEE Transactions on Circuits and Systems for Video Technology.