The Use of Relevance Feedback on the Web: Implications for Web IR System Design

Abstract: We conducted a transaction log analysis of 51,473 queries from 18,113 users of Excite, a major Web search engine. Approximately 2,500 (approximately 5%) of these queries were from the use of relevance feedback. Given the high level of research activity and historical success of relevance feedback in assisting users in locating relevant information, this is a surprising small percentage of usage. In order to investigate this phenomenon on the Web, we analyzed users sessions that contained relevance feedback queries. We identified states and patterns in these sessions. We also attempt to classify the sessions as successfully or not. This analysis provides insight on the current use of relevance feedback on the Web, its success or failure, and why it is so seldom utilized. We relate our conclusions to system design of information retrieval systems on the Web.