Adaptive Near Duplicate Image Retrieval Using SURF and CNN Features

In this paper, we present an adaptive approach in order to match and retrieve near duplicate images at different scales. Matching only local Features does not necessarily identify visually similar images. Global features are fast at matching but less accurate. Many existing methods either use local features or CNN features for image or video retrieval task. In this paper, we combined the use of SURF local points and CNN features extracted around SURF points in order to match near duplicate image pairs. Image pairs are segmented into blocks and CNN features of the image block containing matched SURF features are extracted and matched. Regions around matched image blocks are grown adaptively and matching is carried out until CNN mismatch is observed. To verify our proposed approach, experiments are carried out on benchmarking California-ND and Holiday dataset. Compared to traditional approaches for image retrieval, our approach not only retrieves relevant images but also provides detail of localized matched patch. For California-ND dataset and Holiday dataset, we achieve remarkable mAP (mean average precision) score up to 0.86 and 0.74 respectively.

[1]  Yan Ke,et al.  An efficient parts-based near-duplicate and sub-image retrieval system , 2004, MULTIMEDIA '04.

[2]  Xing Xie,et al.  Coherent Phrase Model for Efficient Image Near-Duplicate Retrieval , 2009, IEEE Transactions on Multimedia.

[3]  Yonghong Tian,et al.  CNN vs. SIFT for Image Retrieval: Alternative or Complementary? , 2016, ACM Multimedia.

[4]  Cordelia Schmid,et al.  Aggregating local descriptors into a compact image representation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Shichao Kan,et al.  Fusion of multiple VLAD vectors based on different features for image retrieval , 2016, 2016 IEEE 13th International Conference on Signal Processing (ICSP).

[6]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[7]  R. Venkatesh Babu,et al.  Object level deep feature pooling for compact image representation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Mohammed Ahmed,et al.  A Novel Technique Using Multiple K-Shingling Based Weighted Dissimilarity Score for Web Content Outlier Mining , 2019 .

[9]  Cordelia Schmid,et al.  Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search , 2008, ECCV.

[10]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[11]  V. Pimenov,et al.  Fast Image Matching with Visual Attention and SURF Descriptors , 2009 .

[12]  Qi Tian,et al.  Exploiting Hierarchical Activations of Neural Network for Image Retrieval , 2016, ACM Multimedia.

[13]  Shih-Fu Chang,et al.  Detecting image near-duplicate by stochastic attributed relational graph matching with learning , 2004, MULTIMEDIA '04.

[14]  Grigorios Tsoumakas,et al.  A Comprehensive Study Over VLAD and Product Quantization in Large-Scale Image Retrieval , 2014, IEEE Transactions on Multimedia.

[15]  Jie Lin,et al.  A practical guide to CNNs and Fisher Vectors for image instance retrieval , 2015, Signal Process..

[16]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[17]  Cordelia Schmid,et al.  DeepMatching: Hierarchical Deformable Dense Matching , 2015, International Journal of Computer Vision.

[18]  Florent Perronnin,et al.  Large-scale image retrieval with compressed Fisher vectors , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Qi Tian,et al.  Image Classification and Retrieval are ONE , 2015, ICMR.

[20]  Yanning Zhang,et al.  Learning Near Duplicate Image Pairs using Convolutional Neural Networks , 2018, International Journal of Performability Engineering.

[21]  Cordelia Schmid,et al.  Aggregating Local Image Descriptors into Compact Codes , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  C. Lawrence Zitnick,et al.  Edge Boxes: Locating Object Proposals from Edges , 2014, ECCV.

[23]  Ce Liu,et al.  Duplicate Discovery on 2 Billion Internet Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[24]  Stefan Winkler,et al.  California-ND: An annotated dataset for near-duplicate detection in personal photo collections , 2013, 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX).

[25]  Qi Tian,et al.  Accurate Image Search with Multi-Scale Contextual Evidences , 2016, International Journal of Computer Vision.

[26]  David Stutz,et al.  Neural Codes for Image Retrieval , 2015 .

[27]  Andrew Zisserman,et al.  Triangulation Embedding and Democratic Aggregation for Image Search , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Chong-Wah Ngo,et al.  Practical elimination of near-duplicates from web video search , 2007, ACM Multimedia.

[29]  Chao Xu,et al.  DCT Inspired Feature Transform for Image Retrieval and Reconstruction , 2016, IEEE Transactions on Image Processing.

[30]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[31]  Jun Jie Foo,et al.  Pruning SIFT for Scalable Near-duplicate Image Matching , 2007, ADC.

[32]  Chong-Wah Ngo,et al.  Visual word proximity and linguistics for semantic video indexing and near-duplicate retrieval , 2009, Comput. Vis. Image Underst..

[33]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[34]  Svetlana Lazebnik,et al.  Multi-scale Orderless Pooling of Deep Convolutional Activation Features , 2014, ECCV.

[35]  Chng Eng Siong,et al.  Improved Keypoint Matching Method for Near-Duplicate Keyframe Retrieval , 2009, 2009 11th IEEE International Symposium on Multimedia.

[36]  Atsuto Maki,et al.  Visual Instance Retrieval with Deep Convolutional Networks , 2014, ICLR.

[37]  Chong-Wah Ngo,et al.  Scale-Rotation Invariant Pattern Entropy for Keypoint-Based Near-Duplicate Detection , 2009, IEEE Transactions on Image Processing.

[38]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..