Video copy detection using multiple visual cues and MPEG-7 descriptors

We propose a video copy detection framework that detects copy segments by fusing the results of three different techniques: facial shot matching, activity subsequence matching, and non-facial shot matching using low-level features. In facial shot matching part, a high-level face detector identifies facial frames/shots in a video clip. Matching faces with extended body regions gives the flexibility to discriminate the same person (e.g., an anchor man or a political leader) in different events or scenes. In activity subsequence matching part, a spatio-temporal sequence matching technique is employed to match video clips/segments that are similar in terms of activity. Lastly, the non-facial shots are matched using low-level MPEG-7 descriptors and dynamic-weighted feature similarity calculation. The proposed framework is tested on the query and reference dataset of CBCD task of TRECVID 2008. Our results are compared with the results of top-8 most successful techniques submitted to this task. Promising results are obtained in terms of both effectiveness and efficiency.

[1]  Pinar Duygulu Sahin,et al.  Searching for repeated video sequences , 2007, MIR '07.

[2]  B. S. Manjunath,et al.  Efficient and Robust Detection of Duplicate Videos in a Large Database , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Luc Van Gool,et al.  Spatio-temporal features for robust content-based video copy detection , 2008, MIR '08.

[4]  Olivier Buisson,et al.  Content-Based Copy Retrieval Using Distortion-Based Probabilistic Similarity Search , 2007, IEEE Transactions on Multimedia.

[5]  Dong Wang,et al.  THU and ICRC at TRECVID 2007 , 2007, TRECVID.

[6]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[7]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[8]  Changsheng Xu,et al.  Segmentation, categorization, and identification of commercial clips from TV streams using multimodal analysis , 2006, MM '06.

[10]  Binoy Pinto,et al.  Speeded Up Robust Features , 2011 .

[11]  Chong-Wah Ngo,et al.  Measuring novelty and redundancy with multiple modalities in cross-lingual broadcast news , 2008, Comput. Vis. Image Underst..

[12]  Iwan Setyawan,et al.  Watermarking digital image and video data. A state-of-the-art overview , 2000 .

[13]  Shin Satoh.  News video analysis based on identical shot detection , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[14]  Olivier Buisson,et al.  Robust voting algorithm based on labels of behavior for video copy detection , 2006, MM '06.

[15]  Edoardo Ardizzone,et al.  Video indexing using MPEG motion compensation vectors , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[16]  Olivier Buisson,et al.  Robust Content-Based Video Copy Identification in a Large Reference Database , 2003, CIVR.

[17]  Shih-Fu Chang,et al.  Topic Tracking Across Broadcast News Videos with Visual Duplicates and Semantic Concepts , 2006, 2006 International Conference on Image Processing.

[18]  Qingming Huang,et al.  Near-duplicate video matching with transformation recognition , 2009, MM '09.

[19]  Video Copy Detection , 2008, Encyclopedia of Multimedia.

[20]  Olivier Buisson,et al.  Content-based video copy detection in large databases: a local fingerprints statistical similarity search approach , 2005, IEEE International Conference on Image Processing 2005.

[21]  B. S. Manjunath,et al.  Introduction to mpeg-7 , 2002 .

[22]  B. S. Manjunath,et al.  Introduction to MPEG-7: Multimedia Content Description Interface , 2002 .

[23]  Ruud M. Bolle,et al.  Comparison of distance measures for video copy detection , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[24]  Özgür Ulusoy,et al.  Fuzzy color histogram-based video segmentation , 2010, Comput. Vis. Image Underst..

[25]  Alberto Del Bimbo,et al.  Video Clip Matching Using MPEG-7 Descriptors and Edit Distance , 2006, CIVR.

[26]  G. C. Langelaar,et al.  Watermarking digital image and video data , 2000 .

[27]  Hung-Khoon Tan,et al.  Near-Duplicate Keyframe Identification With Interest Point Matching and Pattern Learning , 2007, IEEE Transactions on Multimedia.

[28]  Rakesh Mohan,et al.  Video sequence matching , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[29]  Sheng Tang,et al.  Visual words based spatiotemporal sequence matching in video copy detection , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[30]  Julien Law-To,et al.  INRIA-IMEDIA TRECVID 2008: Video Copy Detection , 2008, TRECVID.

[31]  Mubarak Shah,et al.  Content based video matching using spatiotemporal volumes , 2008, Comput. Vis. Image Underst..

[32]  Ruud M. Bolle,et al.  Comparison of sequence matching techniques for video copy detection , 2001, IS&T/SPIE Electronic Imaging.

[33]  Mei-Chen Yeh,et al.  Video copy detection by fast sequence matching , 2009, CIVR '09.

[34]  Mubarak Shah,et al.  Tracking news stories across different sources , 2005, MULTIMEDIA '05.

[35]  Olivier Buisson,et al.  Statistical similarity search applied to content-based video copy detection , 2005, 21st International Conference on Data Engineering Workshops (ICDEW'05).

[36]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[37]  Li Chen,et al.  Video copy detection: a comparative study , 2007, CIVR '07.

[38]  Paul Over,et al.  TRECVID-2008 content-based copy detection task overview (slides) , 2008 .

[39]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[40]  Alexis Joly,et al.  New local descriptors based on dissociated dipoles , 2007, CIVR '07.

[41]  Vasudev Bhaskaran,et al.  Spatiotemporal sequence matching for efficient video copy detection , 2005, IEEE Trans. Circuits Syst. Video Technol..

[42]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[43]  Cordelia Schmid,et al.  INRIA-LEAR'S Video Copy Detection System , 2008, TRECVID.

[44]  Olivier Buisson,et al.  Local Behaviours Labelling for Content Based Video Copy Detection , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[45]  Sid-Ahmed Berrani,et al.  The Orange Labs Real Time Video Copy Detection System - TrecVid 2008 Results , 2008, TRECVID.