Natural language processing for information retrieval: the time is ripe (again)

Paraphrasing van Rijsbergen [37], the time is ripe for another attempt at using natural language processing (NLP) for information retrieval (IR). This paper introduces my dissertation study, which will explore methods for integrating modern NLP with state-of-the-art IR techniques. In addition to text, I will also apply retrieval to conversational speech data, which poses a unique set of considerations in comparison to text. Greater use of NLP has potential to improve both text and speech retrieval.

[1]  Bowen Hui.  Applying NLP to IR: Why and How , 1998 .

[2]  Alan F. Smeaton,et al.  Using NLP or NLP Resources for Information Retrieval Tasks , 1999 .

[3]  Eugene Charniak,et al.  Effective Self-Training for Parsing , 2006, NAACL.

[4]  Oren Etzioni,et al.  Machine Reading , 2006, AAAI.

[5]  CHENGXIANG ZHAI,et al.  A study of smoothing methods for language models applied to information retrieval , 2004, TOIS.

[6]  W. Bruce Croft,et al.  Relevance-Based Language Models , 2001, SIGIR '01.

[7]  W. Bruce Croft,et al.  Linear feature-based models for information retrieval , 2007, Information Retrieval.

[8]  James Allan.  Perspectives on Information Retrieval and Speech , 2001, SIGIR Workshop: Information Retrieval Techniques for Speech Applications.

[9]  Jianfeng Gao,et al.  Dependence language model for information retrieval , 2004, SIGIR '04.

[10]  Stephen E. Robertson,et al.  A probabilistic model of information retrieval: development and comparative experiments - Part 1 , 2000, Inf. Process. Manag..

[11]  S. Robertson.  The probability ranking principle in IR , 1997 .

[12]  Richard Sproat,et al.  Lattice-Based Search for Spoken Utterance Retrieval , 2004, NAACL.

[13]  Jianfeng Gao,et al.  Linear discriminant model for information retrieval , 2005, SIGIR '05.

[14]  Douglas A. Reynolds,et al.  Measuring the readability of automatic speech-to-text transcripts , 2003, INTERSPEECH.

[15]  Karen Sparck Jones.  What is the Role of NLP in Text Retrieval , 1999 .

[16]  Malik Magdon-Ismail,et al.  Detecting conversing groups of chatters: a model, algorithms, and tests , 2005, IADIS AC.

[17]  Gary Geunbae Lee,et al.  Dependency Structure Applied to Language Modeling for Information Retrieval , 2006 .

[18]  Jian-Yun Nie,et al.  Query expansion using term relationships in language models for information retrieval , 2005, CIKM '05.

[19]  Joshua Goodman,et al.  A bit of progress in language modeling , 2001, Comput. Speech Lang..

[20]  Ryen W. White,et al.  Overview of the CLEF-2006 Cross-Language Speech Retrieval Track , 2006, CLEF.

[21]  James Allan,et al.  Capturing term dependencies using a language model based on sentence trees , 2002, CIKM '02.

[22]  Jian-Yun Nie,et al.  Integrating word relationships into language models , 2005, SIGIR '05.

[23]  Matthew Lease,et al.  Brown at CL-SR'07: Retrieving Conversational Speech in English and Czech , 2007, CLEF.

[24]  Ellen M. Voorhees,et al.  The TREC Spoken Document Retrieval Track: A Success Story , 2000, TREC.

[25]  Thorsten Brants,et al.  Natural Language Processing in Information Retrieval , 2003, CLIN.

[26]  David Carmel,et al.  Spoken document retrieval from call-center conversations , 2006, SIGIR.

[27]  Tao Tao,et al.  A formal study of information retrieval heuristics , 2004, SIGIR '04.

[28]  W. Bruce Croft,et al.  A language modeling approach to information retrieval , 1998, SIGIR '98.

[29]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[30]  W. Bruce Croft,et al.  A general language model for information retrieval , 1999, CIKM '99.

[31]  Stephen E. Robertson,et al.  A probabilistic model of information retrieval: development and comparative experiments - Part 2 , 2000, Inf. Process. Manag..

[32]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[33]  Ryen W. White,et al.  Overview of the CLEF-2005 Cross-Language Speech Retrieval Track , 2005, CLEF.

[34]  ChengXiang Zhai,et al.  A Brief Review of Information Retrieval Modesl , 2007 .

[35]  Alexander Dekhtyar,et al.  Information Retrieval , 2018, Lecture Notes in Computer Science.

[36]  Mary P. Harper,et al.  2005 Johns Hopkins Summer Workshop Final Report on Parsing and Spoken Structural Event Detection , 2005 .

[37]  Matthew Lease,et al.  A Look at Parsing and Its Applications , 2006, AAAI.

[38]  Matthew Lease,et al.  Recognizing disfluencies in conversational speech , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[39]  Providen e RIe.  Immediate-Head Parsing for Language Models , 2001 .

[40]  Dongsong Zhang,et al.  NLPIR: a Theoretical Framework for Applying Natural Language Processing to Information Retrieval , 2003, J. Assoc. Inf. Sci. Technol..