Content-based Video Copy Detection using Binary Object Fingerprints

Content-based video copy detection in large-scale databases is still an open issue. One of the main reasons is that geometrical attack can easily surpass the global features of the frame while local features are inefficient in terms of compact representation and computational complexity. In this paper, we propose to use binary object fingerprints to represent video frame for improving the robustness of the video copy detection system. It is because salient object can be robustly detected using advanced convolutional neural network (CNN) based object detector. We proposed to use the well-known RetinaNet for generating object regions from the input frame and then these regions are used to generate binary fingerprints for fast copy detection in the database. This approach can maintain compact representation of video frame and high searching speed by binary fingerprint searching scheme. Experimental results show that the proposed approach can achieve about 10% higher recall rate with only sacrificing 1% prediction rate on VCDB dataset.

[1]  Zi Huang,et al.  Near-duplicate video retrieval: Current research and future trends , 2013, CSUR.

[2]  Yiannis Kompatsiaris,et al.  Near-Duplicate Video Retrieval by Aggregating Intermediate CNN Layers , 2017, MMM.

[3]  Olivier Buisson,et al.  Content-Based Copy Retrieval Using Distortion-Based Probabilistic Similarity Search , 2007, IEEE Transactions on Multimedia.

[4]  Mengyang Liu,et al.  A novel inverted index file based searching strategy for video copy detection , 2017, 2017 Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA).

[5]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[6]  Jian Lu,et al.  Video fingerprinting for copy identification: from research to industry applications , 2009, Electronic Imaging.

[7]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Bian Yang,et al.  Block Mean Value Based Image Perceptual Hashing , 2006, 2006 International Conference on Intelligent Information Hiding and Multimedia.

[9]  Mengyang Liu,et al.  Shearlet Based Video Fingerprint for Content-Based Copy Detection , 2016 .

[10]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Nasir D. Memon,et al.  Spatio–Temporal Transform Based Video Hashing , 2006, IEEE Transactions on Multimedia.

[12]  Hung-Khoon Tan,et al.  Scalable detection of partial near-duplicate videos by visual-temporal consistency , 2009, ACM Multimedia.

[13]  Jiajun Wang,et al.  VCDB: A Large-Scale Database for Partial Copy Detection in Videos , 2014, ECCV.

[14]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[15]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Haojie Li,et al.  Compact CNN Based Video Representation for Efficient Video Copy Detection , 2017, MMM.

[17]  Neslihan Serap Sengör,et al.  Content-based copy detection by a subspace learning based video fingerprinting scheme , 2012, Multimedia Tools and Applications.

[18]  Cordelia Schmid,et al.  An Image-Based Approach to Video Copy Detection With Spatio-Temporal Post-Filtering , 2010, IEEE Transactions on Multimedia.

[19]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Xingming Sun,et al.  Effective and Efficient Global Context Verification for Image Copy Detection , 2017, IEEE Transactions on Information Forensics and Security.

[21]  Tao Liu,et al.  AT&T Research at TRECVID 2009 Content-based Copy Detection , 2009, TRECVID.

[22]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[23]  Mohamed Hefeeda,et al.  Spatio-temporal video copy detection , 2012, MMSys '12.

[24]  Rabab Kreidieh Ward,et al.  A Robust and Fast Video Copy Detection System Using Content-Based Fingerprinting , 2011, IEEE Transactions on Information Forensics and Security.

[25]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[26]  Wei Liu,et al.  DSSD : Deconvolutional Single Shot Detector , 2017, ArXiv.

[27]  Qi Tian,et al.  Large-scale video copy retrieval with temporal-concentration SIFT , 2016, Neurocomputing.

[28]  Jiajun Wang,et al.  Partial Copy Detection in Videos: A Benchmark and an Evaluation of Popular Methods , 2016, IEEE Transactions on Big Data.

[29]  Qingming Huang,et al.  A Rotation Invariant Descriptor for Robust Video Copy Detection , 2013 .