Improving retrieval relevance using users' explicit feedback

Purpose – The purpose of this paper is to improve users’ search results relevancy by manipulating their explicit feedback. Design/methodology/approach – CoRRe – an explicit feedback model integrating three popular feedback, namely, Comment-Rating-Referral is proposed in this study. The model is further enhanced using case-based reasoning in retrieving the top-5 results. A search engine prototype was developed using Text REtrieval Conference as the document collection, and results were evaluated at three levels (i.e. top-5, 10 and 15). A user evaluation involving 28 students was administered, focussing on 20 queries. Findings – Both Mean Average Precision and Normalized Discounted Cumulative Gain results indicate CoRRe to have the highest retrieval precisions at all the three levels compared to the other feedback models. Furthermore, independent t-tests showed the precision differences to be significant. Rating was found to be the most popular technique among the participants, producing the best precision ...

[1]  Nicholas J. Belkin,et al.  Cases, scripts, and information-seeking strategies: On the design of interactive information retrieval systems , 1995 .

[2]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[3]  Barry Smyth,et al.  Mining the real-time web: A novel approach to product recommendation , 2012, Knowl. Based Syst..

[4]  Edwina L. Rissland,et al.  A case-based approach to intelligent information retrieval , 1995, SIGIR '95.

[5]  Jonathan L. Herlocker,et al.  Click data as implicit relevance feedback in web search , 2007, Inf. Process. Manag..

[6]  Kunio Kashino,et al.  A Quick Search Method for Audio Signals Based on a Piecewise Linear Representation of Feature Trajectories , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[7]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[8]  Thorsten Joachims,et al.  Accurately Interpreting Clickthrough Data as Implicit Feedback , 2017 .

[9]  Pradeep Rusiya,et al.  GRF : Greedy Based Relevance Feedback Algorithm for Retrieval of Multimedia Object , 2013 .

[10]  Ryen W. White,et al.  Large-scale analysis of individual and task differences in search result page examination strategies , 2012, WSDM '12.

[11]  Hang Li,et al.  Improving quality of training data for learning to rank using click-through data , 2010, WSDM '10.

[12]  Katashi Nagao,et al.  Video Scene Retrieval Using Online Video Annotation , 2007, JSAI.

[13]  Emanuele Della Valle,et al.  An Introduction to Information Retrieval , 2013 .

[14]  Bing Liu,et al.  Sentiment Analysis and Opinion Mining , 2012, Synthesis Lectures on Human Language Technologies.

[15]  M. Takimoto.  The effects of explicit feedback on the development of pragmatic proficiency , 2006 .

[16]  Jian Wang,et al.  Utilizing re-finding for personalized information retrieval , 2010, CIKM '10.

[17]  Mao Chen.  A Greedy Algorithm with Forward-Looking Strategy , 2008 .

[18]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[19]  Sophie Ahrens,et al.  Recommender Systems , 2012 .

[20]  Alberto Costa,et al.  Optimization Algorithm for Improving the Efficacy of an Information Retrieval Model , 2010 .

[21]  Eugene Agichtein,et al.  ViewSer: enabling large-scale remote user studies of web search examination and interaction , 2011, SIGIR.

[22]  Rashmi R. Sinha,et al.  Comparing Recommendations Made by Online Systems and Friends , 2001, DELOS.

[23]  Hsi-Peng Lu,et al.  An examination of the celebrity endorsements and online customer reviews influence female consumers' shopping behavior , 2013, Comput. Hum. Behav..

[24]  M Ashok Kumar.  Clustering Analysis based on Greedy Heuristic Algorithm , 2013 .

[25]  Donna K. Harman,et al.  Overview of the Ninth Text REtrieval Conference (TREC-9) , 2000, TREC.

[26]  Anita Sharma,et al.  Personality and Patterns of Facebook Usage , 2016 .

[27]  Tetsuya Sakai,et al.  On the reliability of information retrieval metrics based on graded relevance , 2007, Inf. Process. Manag..

[28]  Janet L. Kolodner,et al.  An introduction to case-based reasoning , 1992, Artificial Intelligence Review.

[29]  Enrique Herrera-Viedma,et al.  Sentiment analysis: A review and comparative analysis of web services , 2015, Inf. Sci..

[30]  Nava Tintarev,et al.  Rate it again: increasing recommendation accuracy by user re-rating , 2009, RecSys '09.

[31]  Ryen W. White,et al.  Text selections as implicit relevance feedback , 2012, SIGIR '12.

[32]  Douglas W. Oard,et al.  Implicit Feedback for Recommender Systems , 1998 .

[33]  David Schuff,et al.  What Makes a Helpful Review? A Study of Customer Reviews on Amazon.com , 2010 .

[34]  Jaime Teevan,et al.  Implicit feedback for inferring user preference: a bibliography , 2003, SIGF.

[35]  Patty Kostkova,et al.  Modeling User Preferences in Recommender Systems , 2014, ACM Trans. Interact. Intell. Syst..

[36]  Wu He,et al.  The effects of conceptual description and search practice on users' mental models and information seeking in a case-based reasoning retrieval system , 2008, Inf. Process. Manag..

[37]  Frank Hopfgartner,et al.  Evaluating the implicit feedback models for adaptive video retrieval , 2007, MIR '07.

[38]  Chunxiao Zhang,et al.  Memory and forgetting: An improved dynamic maintenance method for case-based reasoning , 2014, Inf. Sci..

[39]  Lei Zhang,et al.  Sentiment Analysis and Opinion Mining , 2017, Encyclopedia of Machine Learning and Data Mining.

[40]  Kristina Lerman,et al.  Social Networks and Social Information Filtering on Digg , 2006, ICWSM.

[41]  Vimala Balakrishnan,et al.  IMPROVING THE RELEVANCY OF DOCUMENT SEARCH USING THE MULTI-TERM ADJACENCY KEYWORD-ORDER MODEL , 2012 .

[42]  Qi Li,et al.  Personalized web exploration with task models , 2008, WWW.

[43]  Heesop Kim,et al.  Retrieval effectiveness of controlled and uncontrolled index terms in INSPEC database , 2014 .

[44]  Pierre-Emmanuel Leni,et al.  Case-Based Reasoning adaptation of numerical representations of human organs by interpolation , 2014, Expert Syst. Appl..

[45]  T. Graepel,et al.  Private traits and attributes are predictable from digital records of human behavior , 2013, Proceedings of the National Academy of Sciences.

[46]  Yue Lu,et al.  Rated aspect summarization of short comments , 2009, WWW '09.

[47]  Ryen W. White,et al.  Explicit feedback in local search tasks , 2013, SIGIR.

[48]  D. Bixby,et al.  Managing inadequate responses to frontline treatment of chronic myeloid leukemia: a case-based review. , 2013, Cancer treatment reviews.

[49]  Thomas L. Ngo-Ye,et al.  Analyzing Online Review Helpfulness Using a Regressional ReliefF-Enhanced Text Mining Method , 2012, TMIS.

[50]  Eugene Agichtein,et al.  Beyond dwell time: estimating document relevance from cursor movements and other post-click searcher behavior , 2012, WWW.

[51]  Mark Claypool,et al.  Inferring User Interest , 2001, IEEE Internet Comput..

[52]  Agnar Aamodt,et al.  CASE-BASED REASONING: FOUNDATIONAL ISSUES, METHODOLOGICAL VARIATIONS, AND SYSTEM APPROACHES AICOM - ARTIFICIAL INTELLIGENCE COMMUNICATIONS , 1994 .

[53]  Wolfgang Nejdl,et al.  How useful are your comments?: analyzing and predicting youtube comments and comment ratings , 2010, WWW '10.

[54]  Martin Szomszor,et al.  Comparison of implicit and explicit feedback from an online music recommendation service , 2010, HetRec '10.

[55]  Eric Brill,et al.  Improving web search ranking by incorporating user behavior information , 2006, SIGIR.

[56]  Mark T. Maybury,et al.  Information Storage and Retrieval Systems: Theory and Implementation , 2000 .

[57]  Vimala Balakrishnan,et al.  Implicit user behaviours to improve post-retrieval document relevancy , 2014, Comput. Hum. Behav..

[58]  José T. Palma,et al.  Evaluating Case-Base Maintenance algorithms , 2014, Knowl. Based Syst..

[59]  Vimala Balakrishnan,et al.  Critical Product Features' Identification Using an Opinion Analyzer , 2014, TheScientificWorldJournal.

[60]  Yi Zhu,et al.  User Interest Modeling and Self-Adaptive Update Using Relevance Feedback Technology , 2012 .

[61]  Yiqun Liu,et al.  How do users describe their information need: Query recommendation based on snippet click model , 2011, Expert Syst. Appl..

[62]  Thorsten Joachims,et al.  Online learning to diversify from implicit feedback , 2012, KDD.

[63]  Juan Manuel Cueva Lovelle,et al.  Implicit feedback techniques on recommender systems applied to electronic books , 2012, Comput. Hum. Behav..

[64]  Ting Yu,et al.  Revealing and incorporating implicit communities to improve recommender systems , 2014, EC.

[65]  Nicole Beebe,et al.  Post-retrieval search hit clustering to improve information retrieval effectiveness: Two digital forensics case studies , 2011, Decis. Support Syst..

[66]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[67]  Nasser Yazdani,et al.  A3CRank: An adaptive ranking method based on connectivity, content and click-through data , 2010, Inf. Process. Manag..

[68]  Andreas Hotho,et al.  Tagging data as implicit feedback for learning-to-rank , 2011 .

[69]  Kazutoshi Sumiya,et al.  Scene extraction system for video clips using attached comment interval and pointing region , 2010, Multimedia Tools and Applications.

[70]  Ron Kohavi.  KDD-2004 : proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 22-25, 2004, Seattle, Washington, USA , 2004 .

[71]  Steve Fox,et al.  Evaluating implicit measures to improve web search , 2005, TOIS.

[72]  Wei Yuan,et al.  Smoothing clickthrough data for web search ranking , 2009, SIGIR.

[73]  Jiajun Bu,et al.  Who Also Likes It? Generating the Most Persuasive Social Explanations in Recommender Systems , 2014, AAAI.