Leveraging User Interaction Signals for Web Image Search

User interfaces for web image search engine results differ significantly from interfaces for traditional (text) web search results, supporting a richer interaction. In particular, users can see an enlarged image preview by hovering over a result image, and an `image preview' page allows users to browse further enlarged versions of the results, and to click-through to the referral page where the image is embedded. No existing work investigates the utility of these interactions as implicit relevance feedback for improving search ranking, beyond using clicks on images displayed in the search results page. In this paper we propose a number of implicit relevance feedback features based on these additional interactions: hover-through rate, 'converted-hover' rate, referral page click through, and a number of dwell time features. Also, since images are never self-contained, but always embedded in a referral page, we posit that clicks on other images that are embedded on the same referral webpage as a given image can carry useful relevance information about that image. We also posit that query-independent versions of implicit feedback features, while not expected to capture topical relevance, will carry feedback about the quality or attractiveness of images, an important dimension of relevance for web image search. In an extensive set of ranking experiments in a learning to rank framework, using a large annotated corpus, the proposed features give statistically significant gains of over 2% compared to a state of the art baseline that uses standard click features.

[1]  W. Bruce Croft,et al.  A Markov random field model for term dependencies , 2005, SIGIR '05.

[2]  Nicholas J. Belkin,et al.  Display time as implicit feedback: understanding task effects , 2004, SIGIR '04.

[3]  Ryen W. White,et al.  No search result left behind: branching behavior with browser tabs , 2012, WSDM '12.

[4]  Rossano Schifanella,et al.  A Large-Scale Study of User Image Search Behavior on the Web , 2015, CHI.

[5]  Xian-Sheng Hua,et al.  The role of attractiveness in web image search , 2011, ACM Multimedia.

[6]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[7]  Suju Rajan,et al.  Beyond clicks: dwell time for personalization , 2014, RecSys '14.

[8]  Christos Diou,et al.  Image annotation using clickthrough data , 2009, CIVR '09.

[9]  Helen Ashman,et al.  Evaluating implicit judgements from Image search interactions , 2009 .

[10]  Hugo Zaragoza,et al.  The Probabilistic Relevance Framework: BM25 and Beyond , 2009, Found. Trends Inf. Retr..

[11]  Nick Craswell,et al.  Random walks on the click graph , 2007, SIGIR.

[12]  Yiqun Liu,et al.  Incorporating vertical results into search click models , 2013, SIGIR.

[13]  Vidit Jain,et al.  Learning to re-rank: query-dependent image re-ranking using click data , 2011, WWW.

[14]  Thorsten Joachims,et al.  Accurately interpreting clickthrough data as implicit feedback , 2005, SIGIR '05.

[15]  Y. Rui,et al.  Learning to Rank Using User Clicks and Visual Features for Image Retrieval , 2015, IEEE Transactions on Cybernetics.

[16]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[17]  Jacek Gwizdka,et al.  Using dwell time as an implicit measure of usefulness in different task types , 2011, ASIST.

[18]  Desney S. Tan,et al.  Designing Novel Image Search Interfaces by Understanding Unique Characteristics and Usage , 2009, INTERACT.

[19]  Ryen W. White,et al.  Modeling dwell time to predict click-level satisfaction , 2014, WSDM.

[20]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[21]  Yang Song,et al.  Exploring and exploiting user search behavior on mobile and tablet devices to improve search relevance , 2013, WWW '13.

[22]  Luca Chiarandini,et al.  Image ranking based on user browsing behavior , 2012, SIGIR '12.

[23]  Ryen W. White,et al.  Assessing the scenic route: measuring the value of search trails in web logs , 2010, SIGIR.

[24]  Hao Jiang,et al.  Mining User Dwell Time for Personalized Web Search Re-Ranking , 2011, IJCAI.

[25]  Luca Chiarandini,et al.  Search behaviour on photo sharing platforms , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[26]  Hongyuan Zha,et al.  A regression framework for learning ranking functions using relative relevance judgments , 2007, SIGIR.

[27]  Amanda Spink,et al.  The Effect of Specialized Multimedia Collections on Web Searching , 2004, J. Web Eng..

[28]  Nick Craswell,et al.  An experimental comparison of click position-bias models , 2008, WSDM '08.

[29]  Hao Jiang,et al.  A User-Oriented Webpage Ranking Algorithm Based on User Attention Time , 2008, AAAI.

[30]  Ryen W. White,et al.  Mining the search trails of surfing crowds: identifying relevant websites from user activity , 2008, WWW.