Image Retrieval with Attribute-Associated Auxiliary References

This paper develops a general framework of image retrieval, named A3, by introducing an auxiliary set of samples (object references), each of which is annotated with semantic attributes (tags). Given a query image (without tags), we first map it into the references by a non-convex sparse coding formulation, which jointly optimizes appearance reconstruction of the query and semantics consistency among references. A novel algorithm, namely semantic coherent coding (SCC), is presented to solve this optimization. As a result, a subset of references is selected to augment the query for retrieval. Then we can rank images to be retrieved by intuitively matching them together with the query and the constructed subset. The advantages of our approach are validated on the public datasets (Caltech-256 and ImageNet), and summarized as follows: (1) The auxiliary set of references gives rise to an alternating way for query augmentation while utilizing semantics. (2) The semantics of references can also be propagated into the query and database images.

[1]  Hanjiang Lai,et al.  Supervised Hashing for Image Retrieval via Image Representation Learning , 2014, AAAI.

[2]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Shiguang Shan,et al.  Deep Supervised Hashing for Fast Image Retrieval , 2016, International Journal of Computer Vision.

[4]  Qi Tian,et al.  Recent Advance in Content-based Image Retrieval: A Literature Survey , 2017, ArXiv.

[5]  Inderjit S. Dhillon,et al.  Online Metric Learning and Fast Similarity Search , 2008, NIPS.

[6]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[7]  Shiliang Zhang,et al.  An Attribute-Assisted Reranking Model for Web Image Search , 2015, IEEE Transactions on Image Processing.

[8]  Samy Bengio,et al.  Large Scale Online Learning of Image Similarity through Ranking , 2009, IbPRIA.

[9]  Grigorios Tsoumakas,et al.  A Comprehensive Study Over VLAD and Product Quantization in Large-Scale Image Retrieval , 2014, IEEE Transactions on Multimedia.

[10]  Liqing Zhang,et al.  Edgel index for large-scale sketch-based image search , 2011, CVPR 2011.

[11]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[12]  Abbes Amira,et al.  Semantic content-based image retrieval: A comprehensive study , 2015, J. Vis. Commun. Image Represent..

[13]  Alberto Del Bimbo,et al.  Socializing the Semantic Gap , 2015, ACM Comput. Surv..

[14]  Qi Tian,et al.  Correlated attribute transfer with multi-task graph-guided fusion , 2012, ACM Multimedia.

[15]  Lei Zhang,et al.  Bit-Scalable Deep Hashing With Regularized Similarity Learning for Image Retrieval and Person Re-Identification , 2015, IEEE Transactions on Image Processing.

[16]  Atsuto Maki,et al.  Visual Instance Retrieval with Deep Convolutional Networks , 2014, ICLR.

[17]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[18]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[19]  Fei-Fei Li,et al.  Hierarchical semantic indexing for large scale image retrieval , 2011, CVPR 2011.

[20]  G. Griffin,et al.  Caltech-256 Object Category Dataset , 2007 .

[21]  Ji Wan,et al.  Deep Learning for Content-Based Image Retrieval: A Comprehensive Study , 2014, ACM Multimedia.

[22]  Xiangyang Wang,et al.  Content-based image retrieval by integrating color and texture features , 2012, Multimedia Tools and Applications.