Naive (Bayes) at Forty: The Independence Assumption in Information Retrieval

The naive Bayes classifier, currently experiencing a renaissance in machine learning, has long been a core technique in information retrieval. We review some of the variations of naive Bayes models used for text retrieval and classification, focusing on the distributional assumptions made about word occurrences in documents.

[1]  Samuel B. Williams,et al.  ASSOCIATION FOR COMPUTING MACHINERY , 2000 .

[2]  M. E. Maron,et al.  On Relevance, Probabilistic Indexing and Information Retrieval , 1960, JACM.

[3]  M. E. Maron,et al.  Automatic Indexing: An Experimental Inquiry , 1961, JACM.

[4]  Robert R. Korfhage,et al.  Information Storage and Retrieval , 1963 .

[5]  J. J. Rocchio,et al.  Relevance feedback in information retrieval , 1971 .

[6]  Gerard Salton,et al.  The SMART Retrieval System—Experiments in Automatic Document Processing , 1971 .

[7]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[8]  Stephen P. Harter,et al.  A probabilistic approach to automatic keyword indexing , 1974 .

[9]  Peter E. Hart,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[10]  Stephen P. Harter,et al.  A probabilistic approach to automatic keyword indexing. Part II. An algorithm for probabilistic indexing , 1975, J. Am. Soc. Inf. Sci..

[11]  Don R. Swanson,et al.  A decision theoretic foundation for indexing , 1975, J. Am. Soc. Inf. Sci..

[12]  Stephen P. Harter,et al.  A probabilistic approach to automatic keyword indexing. Part I. On the Distribution of Specialty Words in a Technical Literature , 1975, J. Am. Soc. Inf. Sci..

[13]  Stephen E. Robertson,et al.  Relevance weighting of search terms , 1976, J. Am. Soc. Inf. Sci..

[14]  Donald H. Kraft,et al.  Operations Research Applied to Document Indexing and Retrieval Decisions , 1977, JACM.

[15]  Van Rijsbergen,et al.  A theoretical basis for the use of co-occurence data in information retrieval , 1977 .

[16]  C. J. van Rijsbergen,et al.  An Evaluation of feedback in Document Retrieval using Co‐Occurrence Data , 1978, J. Documentation.

[17]  Karen Spärck Jones.  Search Term Relevance Weighting given Little Relevance Information , 1997, J. Documentation.

[18]  Stephen E. Robertson,et al.  Probabilistic models of indexing and searching , 1980, SIGIR '80.

[19]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[20]  Frederick Mosteller,et al.  Applied Bayesian and classical inference : the case of the Federalist papers , 1984 .

[21]  W. Bruce Croft.  Boolean Queries and Term Dependencies in Probabilistic Retrieval Models. , 1986 .

[22]  Robert M. Losee,et al.  Parameter Estimation for Probabilistic Document-Retrieval Models. , 1988 .

[23]  Clement T. Yu,et al.  Two learning schemes in information retrieval , 1988, SIGIR '88.

[24]  Marvin Minsky,et al.  Perceptrons: expanded edition , 1988 .

[25]  Norbert Fuhr,et al.  Models for retrieval with probabilistic indexing , 1989, Inf. Process. Manag..

[26]  Gerard Salton,et al.  Improving retrieval performance by relevance feedback , 1997, J. Am. Soc. Inf. Sci..

[27]  W. Bruce Croft,et al.  Evaluation of an inference network-based retrieval model , 1991, TOIS.

[28]  L. R. Rasmussen,et al.  In information retrieval: data structures and algorithms , 1992 .

[29]  Donna K. Harman,et al.  Relevance Feedback and Other Query Modification Techniques , 1992, Information retrieval (Boston).

[30]  David D. Lewis.  Text representation for intelligent text retrieval: a classification-oriented view , 1992 .

[31]  David Yarowsky,et al.  A method for disambiguating word senses in a large corpus , 1992, Comput. Humanit..

[32]  Donna Harman,et al.  The First Text REtrieval Conference (TREC-1) , 1993 .

[33]  Robert M. Fung,et al.  Bayesian Inference with Node Aggregation for Information Retrieval , 1993, TREC.

[34]  Eugene L. Margulis,et al.  Modelling Documents with Multiple Poisson Distributions , 1993, Inf. Process. Manag..

[35]  Louise Guthrie,et al.  Document Classification By Machine: Theory and Practice , 1994, COLING.

[36]  Donna K. Harman,et al.  Overview of the Third Text REtrieval Conference (TREC-3) , 1995, TREC.

[37]  William A. Gale,et al.  A sequential algorithm for training text classifiers , 1994, SIGIR '94.

[38]  Peter Norvig,et al.  Text-Based Intelligent Systems , 1994, Artif. Intell..

[39]  Stephen E. Robertson,et al.  Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval , 1994, SIGIR '94.

[40]  William S. Cooper,et al.  Some inconsistencies and misidentified modeling assumptions in probabilistic information retrieval , 1995, TOIS.

[41]  Kenneth Ward Church.  One term or two? , 1995, SIGIR '95.

[42]  Donna Harman,et al.  The Second Text Retrieval Conference (TREC-2) , 1995, Inf. Process. Manag..

[43]  David D. Lewis,et al.  Text categorization of low quality images , 1995 .

[44]  David D. Lewis,et al.  Evaluating and optimizing autonomous text classification systems , 1995, SIGIR '95.

[45]  Amit Singhal,et al.  Pivoted document length normalization , 1996, SIGIR 1996.

[46]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[47]  Slava M. Katz.  Distribution of content words and phrases in text and language modelling , 1996, Natural Language Engineering.

[48]  Yoram Singer,et al.  Context-sensitive learning methods for text categorization , 1996, SIGIR '96.

[49]  Karen Spärck Jones,et al.  Natural language processing for information retrieval , 1996, CACM.

[50]  Donna Harman,et al.  The fourth text REtrieval conference , 1996 .

[51]  Hang Li,et al.  Document Classification Using a Finite Mixture Model , 1997, ACL.

[52]  Prabhakar Raghavan,et al.  Using Taxonomy, Discriminants, and Signatures for Navigating in Text Databases , 1997, VLDB.

[53]  Hang Li,et al.  Document Classification Using a Finite Mixture Model , 1997, ACL.

[54]  Ellen M. Voorhees,et al.  Information Technology: The Fifth Text REtrieval Conference [TREC-5] | NIST , 1997 .

[55]  Gerald Kowalski,et al.  Information Retrieval Systems: Theory and Implementation , 1997 .

[56]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.