Information Retrieval and Artificial Intelligence

Abstract This paper addresses the relations between information retrieval (IR) and AI. It examines document retrieval, summarising its essential features and illustrating the state of its art by presenting one probabilistic model in detail, with some test results showing its value. The paper then analyses this model and related successful approaches, concentrating on and justifying their use of weak, redundant representation and reasoning. It goes on to other information management tasks and considers how the concepts and methods developed for retrieval may be applied to these, concluding by arguing that such ways of dealing with information may also have wider relevance to AI.

[1]  Douglas E. Appelt,et al.  FASTUS: A Finite-state Processor for Information Extraction from Real-world Text , 1993, IJCAI.

[2]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[3]  Philip J. Hayes,et al.  Intelligent high-volume text processing using shallow, domain-specific techniques , 1992 .

[4]  Hans Peter Luhn,et al.  The Automatic Creation of Literature Abstracts , 1958, IBM J. Res. Dev..

[5]  David R. Karger,et al.  Scatter/Gather: a cluster-based approach to browsing large document collections , 1992, SIGIR '92.

[6]  Norbert Fuhr,et al.  The automatic indexing system AIR/PHYS - from research to applications , 1988, SIGIR '88.

[7]  Therese Firmin Hand,et al.  A Proposal for Task-based Evaluation of Text Summarization Systems , 1997, Workshop On Intelligent Scalable Text Summarization.

[8]  Stephen P. Harter,et al.  A probabilistic approach to automatic keyword indexing , 1974 .

[9]  Donna Harman,et al.  The First Text REtrieval Conference (TREC-1) , 1993 .

[10]  Stephen E. Robertson,et al.  On Term Selection for Query Expansion , 1991, J. Documentation.

[11]  David A. Hull.  The TREC-6 Filtering Track: Description and Analysis , 1997, TREC.

[12]  M. E. Maron,et al.  On Relevance, Probabilistic Indexing and Information Retrieval , 1960, JACM.

[13]  Karen Sparck Jones.  What is the Role of NLP in Text Retrieval , 1999 .

[14]  Cyril Cleverdon,et al.  The Cranfield tests on index language devices , 1997 .

[15]  Peter Norvig,et al.  Text-Based Intelligent Systems , 1994, Artif. Intell..

[16]  Gerard Salton,et al.  Associative Document Retrieval Techniques Using Bibliographic Information , 1963, JACM.

[17]  Stephen E. Robertson,et al.  Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval , 1994, SIGIR '94.

[18]  Calvin N. Mooers,et al.  Zatocoding applied to mechanical organization of knowledge , 1951 .

[19]  S. Robertson.  The probability ranking principle in IR , 1997 .

[20]  Joel L. Fagan.  The effectiveness of a nonsyntatic approach to automatic phrase indexing for document retrieval , 1989 .

[21]  W. Bruce Croft,et al.  Evaluation of an inference network-based retrieval model , 1991, TOIS.

[22]  K. Sparck Jones,et al.  A Probabilistic Model of Information Retrieval : Development and Status , 1998 .

[23]  Stephen P. Harter,et al.  A probabilistic approach to automatic keyword indexing. Part I. On the Distribution of Specialty Words in a Technical Literature , 1975, J. Am. Soc. Inf. Sci..

[24]  Chris D. Paice,et al.  Constructing literature abstracts by computer: Techniques and prospects , 1990, Inf. Process. Manag..

[25]  Claire Cardie,et al.  An Analysis of Statistical and Syntactic Phrases , 1997, RIAO.

[26]  Hiyan Alshawi,et al.  Memory and context for language interpretation , 1987 .

[27]  W. Bruce Croft,et al.  Computationally tractable probabilistic modeling of Boolean operators , 1997, SIGIR '97.

[28]  Tomek Strzalkowski.  Natural Language Information Retrieval , 1995, Inf. Process. Manag..

[29]  Nicholas J. Belkin,et al.  Ask for Information Retrieval: Part I. Background and Theory , 1997, J. Documentation.

[30]  James Allan,et al.  Selective text utilization and text traversal , 1993, Int. J. Hum. Comput. Stud..

[31]  Charles L. A. Clarke,et al.  Relevance ranking for one to three term queries , 1997, Inf. Process. Manag..

[32]  J. J. Rocchio,et al.  Relevance feedback in information retrieval , 1971 .

[33]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.

[34]  Alexander Dekhtyar,et al.  Information Retrieval , 2018, Lecture Notes in Computer Science.

[35]  Joel L. Fagan,et al.  The effectiveness of a nonsyntactic approach to automatic phrase indexing for document retrieval , 1989, JASIS.

[36]  Karen Spärck Jones.  Further reflections on TREC , 2000, Inf. Process. Manag..

[37]  David Hawking,et al.  Overview of TREC-7 Very Large Collection Track , 1997, TREC.

[38]  Pattie Maes,et al.  Agents that reduce work and information overload , 1994, CACM.

[39]  Gerard Salton,et al.  Improving retrieval performance by relevance feedback , 1997, J. Am. Soc. Inf. Sci..

[40]  Justin Zobel,et al.  Passage retrieval revisited , 1997, SIGIR '97.

[41]  Alina Vickery,et al.  A Reference and referral System using Expert System Techniques , 1987, J. Documentation.

[42]  Mortimer Taube Associates.  Storage and retrieval of information by means of the association of ideas , 1955 .

[43]  Sheryl R. Young,et al.  Automatic Classification and Summarization of Banking Telexes , 1985, CAIA.

[44]  Re..  Techniques for Information Retrieval from Speech Messages , 1991 .

[45]  S CooperWilliam.  Some inconsistencies and misidentified modeling assumptions in probabilistic information retrieval , 1995 .

[46]  Karen Spärck Jones.  Automatic summarising: factors and directions , 1998, ArXiv.

[47]  Lisa F. Rau,et al.  Automatic Condensation of Electronic Publications by Sentence Selection , 1995, Inf. Process. Manag..

[48]  Gerard Salton,et al.  A theory of indexing , 1975, Regional conference series in applied mathematics.

[49]  Daniel E. Rose,et al.  V-Twin: A Lightweight Engine for Interactive Use , 1996, TREC.