Query difficulty estimation via relevance prediction for image retrieval

Query difficulty estimation (QDE) attempts to automatically predict the performance of the search results returned for a given query. QDE has long been of interest in text retrieval. However, few research works have been conducted in image retrieval. Existing QDE methods in image retrieval mainly explore the statistical characteristics (coherence, specificity, etc.) of the returned images to derive a value for indicating the query difficulty degree. To the best of our knowledge, little research has been done to directly estimate the real search performance, such as average precision. In this paper, we propose a novel query difficulty estimation approach which automatically estimates the average precision of the image search results. Specifically, we first adaptively select a set of query relevant and query irrelevant images for each query via modified pseudo relevance feedback. Then a simple but effective voting scheme and two estimation methods (hard estimation and soft estimation) are proposed to estimate the relevance probability of each image in the search results. Based on the images' relevance probabilities, the average precision for each query is derived. The experimental results on two benchmark image search datasets demonstrate the effectiveness of the proposed method. We automatically predict the image search performance instead of only an indicator.We propose an adaptive pseudo positive image selection method.We propose an efficient voting scheme to estimate images' relevance probability.

[1]  Xian-Sheng Hua,et al.  MSRA-MM: Bridging Research and Industrial Societies for Multimedia Information Retrieval , 2009 .

[2]  Jean Dickinson Gibbons,et al.  Nonparametric Statistical Inference , 1972, International Encyclopedia of Statistical Science.

[3]  Jun Yu,et al.  Exploiting Click Constraints and Multi-view Features for Image Re-ranking , 2014, IEEE Transactions on Multimedia.

[4]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[5]  Meng Wang,et al.  Multimodal Graph-Based Reranking for Web Image Search , 2012, IEEE Transactions on Image Processing.

[6]  Ricardo Baeza-Yates,et al.  Improved query difficulty prediction for the web , 2008, CIKM '08.

[7]  Tao Mei,et al.  Learning to video search rerank via pseudo preference feedback , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[8]  Elad Yom-Tov,et al.  Learning to estimate query difficulty: including applications to missing content detection and distributed information retrieval , 2005, SIGIR '05.

[9]  R. Manmatha,et al.  Automatic Image Annotation and Retrieval using CrossMedia Relevance Models , 2003 .

[10]  R. Krishnamoorthy,et al.  Generalized adaptive Bayesian Relevance Feedback for image retrieval in the Orthogonal Polynomials Transform domain , 2012, Signal Process..

[11]  Aditi Sharan,et al.  Co-occurrence based predictors for estimating query difficulty , 2010, 2010 IEEE International Conference on Data Mining Workshops.

[12]  Xinmei Tian,et al.  Query Difficulty Prediction for Web Image Search , 2012, IEEE Transactions on Multimedia.

[13]  Yiming Yang,et al.  Translingual Information Retrieval: A Comparative Evaluation , 1997, IJCAI.

[14]  W. Bruce Croft,et al.  Predicting query performance , 2002, SIGIR '02.

[15]  Yong Luo,et al.  Query Difficulty Guided Image Retrieval System , 2011, MMM.

[16]  Yuan Yan Tang,et al.  Multiview Hessian discriminative sparse coding for image annotation , 2013, Comput. Vis. Image Underst..

[17]  Thomas S. Huang,et al.  Small sample learning during multimedia retrieval using BiasMap , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[18]  Hong Liu,et al.  A comprehensive study on learning to rank for content-based image retrieval , 2013, Signal Process..

[19]  Yi Zhang,et al.  Query Difficulty Prediction for Contextual Image Retrieval , 2010, ECIR.

[20]  Jun Yu,et al.  Click Prediction for Web Image Reranking Using Multimodal Sparse Coding , 2014, IEEE Transactions on Image Processing.

[21]  Weifeng Liu,et al.  Multiview Hessian Regularization for Image Annotation , 2013, IEEE Transactions on Image Processing.

[22]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[23]  F. W. Kellaway,et al.  Advanced Engineering Mathematics , 1969, The Mathematical Gazette.

[24]  Fernando Diaz,et al.  Performance prediction using spatial autocorrelation , 2007, SIGIR.

[25]  Meng Wang,et al.  Oracle in Image Search: A Content-Based Approach to Performance Prediction , 2012, TOIS.

[26]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[27]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[28]  Shumeet Baluja,et al.  VisualRank: Applying PageRank to Large-Scale Image Search , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  M. Kendall Rank Correlation Methods , 1949 .

[30]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[31]  Xian-Sheng Hua,et al.  Bayesian Visual Reranking , 2011, IEEE Transactions on Multimedia.

[32]  Iadh Ounis,et al.  Query performance prediction , 2006, Inf. Syst..

[33]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[34]  Rong Yan,et al.  Multimedia Search with Pseudo-relevance Feedback , 2003, CIVR.

[35]  Rong Yan,et al.  Co-retrieval: A Boosted Reranking Approach for Video Retrieval , 2004, CIVR.

[36]  Stevan Rudinac,et al.  Exploiting Result Consistency to Select Query Expansions for Spoken Content Retrieval , 2010, ECIR.

[37]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[38]  W. Bruce Croft,et al.  Query performance prediction in web search environments , 2007, SIGIR.