Learning to Combine Ad-hoc Ranking Functions for Image Retrieval

Along with the success of "bag of visual words" scheme in content based image retrieval (CBIR), various technologies in text information retrieval realm have been transferred into image retrieval system and obtain promising performance. However, how to select the suitable ranking technology, such as ranking model, for a specific image database is still an open question. Because most ranking models are data-dependent, it is hard to find an optimal model for all the applications. In this paper, we propose to resolve this problem for CBIR with the learning to rank approach which has been widely utilized in text retrieval. Specifically, we consider several well performed ad-hoc ranking models and use their ranking scores to construct the ranking features for the Ranking SVM framework. To best preserve the spatial structures existed in the visual words of image, we split the image into different size blocks, and design the ranking features with a pyramid approach from large blocks to small blocks. Experimental results on both Oxford and Image Net databases demonstrate the effectiveness of proposed method compared with the performance that individual ranking model is adopted. Moreover, the proposed method brings little computational burden to the system and the efficiency analysis proves its scalability.

[1]  W. Bruce Croft,et al.  Cross-lingual relevance models , 2002, SIGIR '02.

[2]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[3]  W. Bruce Croft,et al.  Search Engines - Information Retrieval in Practice , 2009 .

[4]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[5]  Michael Isard,et al.  Total Recall: Automatic Query Expansion with a Generative Feature Model for Object Retrieval , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[6]  Frederick Jelinek,et al.  Interpolated estimation of Markov source parameters from sparse data , 1980 .

[7]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[8]  Cordelia Schmid,et al.  Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search , 2008, ECCV.

[9]  Zhigang Luo,et al.  Manifold Regularized Discriminative Nonnegative Matrix Factorization With Fast Gradient Descent , 2011, IEEE Transactions on Image Processing.

[10]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[12]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Michael Isard,et al.  Lost in quantization: Improving particular object retrieval in large scale image databases , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[15]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[16]  Xian-Sheng Hua,et al.  Ranking Model Adaptation for Domain-Specific Search , 2009, IEEE Transactions on Knowledge and Data Engineering.

[17]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[18]  Dacheng Tao,et al.  Max-Min Distance Analysis by Using Sequential SDP Relaxation for Dimension Reduction , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Xian-Sheng Hua,et al.  Content-aware Ranking for visual search , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Michael Isard,et al.  Bundling features for large scale partial-duplicate web image search , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  John D. Lafferty,et al.  Information retrieval as statistical translation , 1999, SIGIR '99.

[22]  Xian-Sheng Hua,et al.  Ensemble Manifold Regularization , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Bo Geng,et al.  DAML: Domain Adaptation Metric Learning , 2011, IEEE Transactions on Image Processing.

[24]  Ricardo da Silva Torres,et al.  Learning to rank for content-based image retrieval , 2010, MIR '10.

[25]  Hans Ulrich Simon,et al.  Robust Trainability of Single Neurons , 1995, J. Comput. Syst. Sci..

[26]  Zhigang Luo,et al.  Non-Negative Patch Alignment Framework , 2011, IEEE Transactions on Neural Networks.

[27]  Bo Geng,et al.  A Study of Language Model for Image Retrieval , 2009, 2009 IEEE International Conference on Data Mining Workshops.

[28]  Xuelong Li,et al.  Geometric Mean for Subspace Selection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.