The TREC-8 Question Answering Track Report

The TREC-8 Question Answering track was the first large-scale evaluation of domain-independent question answering systems. This paper summarizes the results of the track by giving a brief overview of the different approaches taken to solve the problem. The most accurate systems found a correct response for more than 2/3 of the questions. Relatively simple bag-of-words approaches were adequate for finding answers when responses could be as long as a paragraph (250 bytes), but more sophisticated processing was necessary for more direct responses (50 bytes). The TREC-8 Question Answering track was an initial e ort to bring the bene ts of large-scale evaluation to bear on a question answering (QA) task. The goal in the QA task is to retrieve small snippets of text that contain the actual answer to a question rather than the document lists traditionally returned by text retrieval systems. The assumption is that users would usually prefer to be given the answer rather than and the answer themselves in a document. This paper summarizes the retrieval results of the track; a companion paper (\The TREC-8 Question Answering Track Evaluation") gives details about how the evaluation was implemented. By necessity, a track report can give only an overview of the different approaches used in the track. Readers are urged to consult the participants' papers elsewhere in the Proceedings for details regarding a particular approach.

[1]  Editors , 1986, Brain Research Bulletin.

[2]  Mark Sanderson,et al.  University of Sheffield TREC-8 Q&A System , 1999, TREC.

[3]  Padmini Srinivasan,et al.  Filters, Webs and Answers: The University of Iowa TREC-8 Results , 1999, TREC.

[4]  Jianqiang Wang,et al.  TREC-8 Experiments at Maryland: CLIR, QA and Routing , 1999, TREC.

[5]  Hsin-Hsi Chen,et al.  Description of Preliminary Results to TREC-8 QA Task , 1999, TREC.

[6]  Wei Li,et al.  Information Extraction Supported Question Answering , 1999, TREC.

[7]  Michael Collins,et al.  AT&T at TREC-8 , 1999, TREC.

[8]  Charles L. A. Clarke,et al.  Fast Automatic Passage Ranking (MultiText Experiments for TREC-8) , 1999, TREC.

[9]  Joel D. Martin,et al.  Ask Me Tomorrow: The NRC and University of Ottawa Question Answering System , 1999, TREC.

[10]  Thomas S. Morton,et al.  Using Coreference for Question Answering , 1999, TREC.

[11]  Ross Wilkinson,et al.  The RMIT/CSIRO Ad Hoc, Q&A, Web, Interactive, and Speech Experiments at TREC 8 , 1999, TREC.

[12]  Toru Takaki.  NTT DATA: Overview of system approach at TREC-8 ad-hoc and question answering , 1999, TREC.

[13]  Inderjeet Mani,et al.  A Sys Called Qanda , 1999, TREC.

[14]  Kenneth C. Litkowski.  Question-Answering Using Semantic Relation Triples , 1999, TREC.

[15]  Christian Jacquemin,et al.  QALC - the Question-Answering program of the Language and Cognition group at LIMSI-CNRS , 1999, TREC.

[16]  Dragomir R. Radev,et al.  The Use of Predictive Annotation for Question Answering in TREC8 , 1999, TREC.

[17]  David A. Hull.  Xerox TREC-8 Question Answering Track Report , 1999, TREC.

[18]  Sergei Nirenburg,et al.  CRL's TREC-8 Systems Cross-Lingual IR, and Q&A , 1999, TREC.

[19]  Ellen M. Voorhees,et al.  The Eighth Text REtrieval Conference (TREC-8) , 2000 .