Travel Video Scene Detection by Search

We propose an approach to conduct video scene detection especially for travel videos captured by amateur photographers in journeys. The correlation between a travel video and its corresponding text-based travel schedule is discovered. Because scene boundaries are clearly defined in schedules, we segment videos into scenes by checking the discovered cross-media correlation. To make these two modalities comparable, photos related to the visited scenic spots are retrieved from image search engines, by the keywords extracted from text-based schedules. Sequences of key frames and retrieved photos are represented as visual word histograms, and the problem of correlation determination is then transformed as an approximate sequence matching problem. The experimental results verify the effectiveness of the proposed idea, and show the promising research direction of utilizing cross-media correlation in media analysis.

[1]  Alessandro Vinciarelli,et al.  Broadcast news story segmentation using social network analysis and hidden markov models , 2007, ACM Multimedia.

[2]  Kuan-Yu Chen,et al.  On the range maximum-sum segment query problem , 2007, Discret. Appl. Math..

[3]  Nikolas P. Galatsanos,et al.  Scene Detection in Videos Using Shot Clustering and Sequence Alignment , 2009, IEEE Transactions on Multimedia.

[4]  Nikos A. Vlassis,et al.  The global k-means clustering algorithm , 2003, Pattern Recognit..

[5]  Wei-Ta Chu,et al.  Using cross-media correlation for scene detection in travel videos , 2009, CIVR '09.

[6]  Wei-Ying Ma,et al.  AnnoSearch: Image Auto-Annotation by Search , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[7]  Hanghang Tong,et al.  Blur detection for digital images using wavelet transform , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[8]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[9]  Andrew Zisserman,et al.  Efficient Visual Search for Objects in Videos , 2008, Proceedings of the IEEE.