Discrimination Decisions for 100,000-Dimensional Spaces

Discrimination decisions arise in many natural language processing tasks. Three classical tasks are discriminating texts by their authors (author identification), discriminating documents by their relevance to some query (information retrieval), and discriminating multi-meaning words by their meanings (sense discrimination). Many other discrimination tasks arise regularly, such as determining whether a particular proper noun represents a person or a place, or whether a given word from some teletype text would be capitalized if both cases had been used.

[1]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[2]  David Yarowsky,et al.  A method for disambiguating word senses in a large corpus , 1992, Comput. Humanit..

[3]  David Yarowsky,et al.  Word-Sense Disambiguation Using Statistical Models of Roget’s Categories Trained on Large Corpora , 2010, COLING.

[4]  Gerald Salton,et al.  Automatic text processing , 1988 .

[5]  S. Fienberg,et al.  Inference and Disputed Authorship: The Federalist , 1966 .

[6]  Kenneth Ward Church,et al.  Identifying word correspondence in parallel texts , 1991 .

[7]  Geoffrey K. Pullum,et al.  Category Structures , 1988, Comput. Linguistics.

[8]  Steven J. DeRose,et al.  Grammatical Category Disambiguation by Statistical Optimization , 1988, CL.

[9]  B. Merialdo,et al.  Tagging text with a probabilistic model , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[10]  Richard A. Harshman,et al.  Indexing by Latent Structure Analysis , 1990 .

[11]  Alon Itai,et al.  Two Languages Are More Informative Than One , 1991, ACL.

[12]  Kenneth Ward Church,et al.  Estimation Procedures for Language Context: Poor Estimates are Worse than None , 1990 .

[13]  Kenneth Ward Church,et al.  A Program for Aligning Sentences in Bilingual Corpora , 1993, CL.

[14]  R. Burchfield.  Frequency Analysis of English Usage: Lexicon and Grammar. By W. Nelson Francis and Henry Kučera with the assistance of Andrew W. Mackie. Boston: Houghton Mifflin. 1982. x + 561 , 1985 .

[15]  Donna Harman,et al.  How effective is suffixing , 1991 .

[16]  Zellig S. Harris,et al.  Mathematical structures of language , 1968, Interscience tracts in pure and applied mathematics.

[17]  Kenneth Ward Church.  A Stochastic Parts Program and Noun Phrase Parser for Unrestricted Text , 1988, ANLP.

[18]  Robert L. Mercer,et al.  Word-Sense Disambiguation Using Statistical Methods , 1991, ACL.

[19]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[20]  Gerard Salton,et al.  On the Specification of Term Values in Automatic Indexing , 1973 .