论文信息 - Answer Attenuation in Question Answering

Answer Attenuation in Question Answering

Research in Question Answering (QA) has been dominated by the TREC methodology of black-box system evaluation. This makes it difficult to evaluate the effectiveness of individual components and requires human involvement. We have collected a set of answer locations within the AQUAINT corpus for a sample of TREC questions, in doing so we also analyse the ability of humans to retrieve answers. Our answer corpus allows us to track answer attenuation through a QA system. We use this method to evaluate the Pronto QA system (Bos et al., 2007).

James R. Curran | Katie Bell

[1] Christof Monz,et al. Document Retrieval in the Context of Question Answering , 2003, ECIR.

[2] Lynette Hirschman,et al. Natural language question answering: the view from here , 2001, Natural Language Engineering.

[3] James R. Curran,et al. Parsing the WSJ Using CCG and Log-Linear Models , 2004, ACL.

[4] Sanda M. Harabagiu,et al. Performance issues and error analysis in an open-domain question answering system , 2003, TOIS.

[5] Johan Bos. Towards Wide-Coverage Semantic Interpretation , 2005 .

[6] Susan T. Dumais,et al. An Analysis of the AskMSR Question-Answering System , 2002, EMNLP.

[7] Johan Bos,et al. The Pronto QA System at TREC 2007: Harvesting Hyponyms, Using Nominalisation Patterns, and Computing Answer Cardinality , 2007, TREC.

[8] Gideon S. Mann,et al. Analyses for elucidating current question answering technology , 2001, Natural Language Engineering.

[9] Jimmy J. Lin. Evaluation of resources for question answering evaluation , 2005, SIGIR '05.

[10] Ellen M. Voorhees,et al. Overview of the TREC 2004 Novelty Track. , 2005 .

[11] Peter Thanisch,et al. Natural language interfaces to databases – an introduction , 1995, Natural Language Engineering.

[12] Jimmy J. Lin,et al. Quantitative evaluation of passage retrieval algorithms for question answering , 2003, SIGIR.

[13] Inderjeet Mani,et al. How to Evaluate Your Question Answering System Every Day ... and Still Get Real Work Done , 2000, LREC.