Using Negative Information in Search

Consider a user searching for information on the World Wide Web. If the information need of the user is somewhat specific, and if the user is permitted to provide a detailed description of his precise need, then it is quite likely that this description will include negative constraints, i.e., specifications of what the user is 'not' looking for. A search engine that makes use of such constraints is likely to return more accurate results. In this paper, we consider the problem of identifying such negative constraints from verbose queries. A maximum-entropy classifier is trained to identify negative sentences in verbose queries with about 90\% accuracy. We next study how retrieval effectiveness is affected when these negative sentences are eliminated from the queries. We find that this step results in modest improvements in retrieval accuracy, but our analysis suggests that significant improvements can be obtained if negative sentences are properly handled during query processing.

[1]  Gerard Salton,et al.  The SMART Retrieval System—Experiments in Automatic Document Processing , 1971 .

[2]  W. Bruce Croft,et al.  Research Paper: Ad Hoc Classification of Radiology Reports , 1999, J. Am. Medical Informatics Assoc..

[3]  Wendy W. Chapman,et al.  A Simple Algorithm for Identifying Negated Findings and Diseases in Discharge Summaries , 2001, J. Biomed. Informatics.

[4]  Prakash M. Nadkarni,et al.  Research Paper: Use of General-purpose Negation Detection to Augment Concept Indexing of Medical Documents: A Quantitative Study Using the UMLS , 2001, J. Am. Medical Informatics Assoc..

[5]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[6]  Long H. Ngo,et al.  Implementation and Evaluation of Four Different Methods of Negation Detection , 2007 .

[7]  Andrew Trotman,et al.  Advances in Focused Retrieval, 7th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2008, Dagstuhl Castle, Germany, December 15-18, 2008. Revised and Selected Papers , 2009, INEX.

[8]  Sukomal Pal,et al.  Parameter Tuning in Pivoted Normalization for XML Retrieval: ISI@INEX09 Adhoc Focused Task , 2009, INEX.

[9]  W. Bruce Croft,et al.  Analysis of long queries in a large scale search log , 2009, WSCD '09.

[10]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[11]  W. Bruce Croft,et al.  Evaluating verbose query processing techniques , 2010, SIGIR.

[12]  Andrew Trotman,et al.  Focused Retrieval and Evaluation, 8th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2009, Brisbane, Australia, December 7-9, 2009, Revised and Selected Papers , 2010, INEX.