The University of Sheffield's TREC 2003 Q&A Experiments

As a natural language processing group (NLP) our original approach to question answering was linguistically motivated culminating in the development of the QA-LaSIE system (Humphreys et al., 1999). In its original form QA-LaSIE would only propose answers which were linked via syntactic/semantic relations to the information missing from the question (for example “Who released the Internet worm?” is missing a person). While the answers proposed by the system were often correct, the system was frequently unable to suggest any answer. The next version of the system loosened the requirement for a link between question and answer which improved performance (Scott and Gaizauskas, 2000). There are still a number of open questions from the development of the QA-LaSIE system: does the use of parsing and discourse interpretation to determine links between questions and proposed answers result in better performance than simpler systems which adopt a shallower approach? Is it simply that the performance of our parser is below the level at which it could contribute to question answering? Are there questions which can only be answered using deep linguistic techniques? With the continued development of a second QA system at Sheffield which uses shallower techniques (Gaizauskas et al., 2005) we believe that we are now in a position to investigate these and related questions. Our entries to the 2006 TREC QA evaluation are designed to help us answer some of these questions and to investigate further the possible benefits of linguistic processing over shallower techniques.

[1]  Robert J. Gaizauskas,et al.  Evaluating Passage Retrieval Approaches for Question Answering , 2004, ECIR.

[2]  Horacio Saggion,et al.  A pattern based approach to answering factoid, list and definition questions , 2004 .

[3]  M. F. Porter,et al.  An algorithm for suffix stripping , 1997 .

[4]  John M. Conroy,et al.  Performance of a Three-Stage System for Multi-Document Summarization , 2003 .

[5]  Mark Sanderson,et al.  University of Sheffield TREC-8 Q&A System , 1999, TREC.

[6]  Stephen E. Robertson,et al.  Okapi/Keenbow at TREC-8 , 1999, TREC.

[7]  H. Markov,et al.  An algorithm to , 1997 .

[8]  Jimmy J. Lin,et al.  Data-Intensive Question Answering , 2001, TREC.

[9]  Horacio Saggion,et al.  Exploring the Performance of Boolean Retrieval Strategies for Open Domain Question Answering , 2004 .

[10]  Robert J. Gaizauskas,et al.  Mining On-line Sources for Definition Knowledge , 2004, FLAIRS.

[11]  Eduard H. Hovy,et al.  Question Answering in Webclopedia , 2000, TREC.

[12]  Sarah A. Goodman.  Performance of a Three-Stage System for Multi-Document Summarization , 2003 .

[13]  Gideon S. Mann,et al.  Analyses for elucidating current question answering technology , 2001, Natural Language Engineering.

[14]  Mark Andrew Greenwood,et al.  Open-domain question answering , 2005 .

[15]  Robert J. Gaizauskas,et al.  SUPPLE: A Practical Parser for Natural Language Engineering Applications , 2005, IWPT.

[16]  Mark A. Greenwood,et al.  Using a Named Entity Tagger to Generalise Surface Matching Text Patterns for Question Answering , 2003 .

[17]  John M. Conroy.  Left-Brain/Right-Brain Multi-Document Summarization , 2004 .

[18]  Jinxi Xu,et al.  TREC 2003 QA at BBN: Answering Definitional Questions , 2003, TREC.

[19]  Robert J. Gaizauskas,et al.  The University of Sheffield TREC 2002 Q&A System , 2002, TREC.

[20]  M. Greenwood.  AnswerFinder : Question Answering from your Desktop , 2003 .

[21]  Julian Kupiec,et al.  MURAX: a robust linguistic approach for question answering using an on-line encyclopedia , 1993, SIGIR.

[22]  Sanda M. Harabagiu,et al.  FALCON: Boosting Knowledge for Answer Engines , 2000, TREC.

[23]  Mitchell P. Marcus,et al.  Text Chunking using Transformation-Based Learning , 1995, VLC@ACL.