论文信息 - Evaluating Web-based Question Answering Systems

Evaluating Web-based Question Answering Systems

The official evaluation of TREC-style Q&A systems is done manually, which is quite expensive and not scalable to web-based Q&A systems. An automatic evaluation technique is needed for dynamic Q&A systems. This paper presents a set of metrics that have been implemented in our web-based Q&A system, namely NSIR. It also shows the correlations between the different metrics.

[1] Ellen M. Voorhees,et al. The TREC-8 Question Answering Track Evaluation , 2000, TREC.

[2] Dragomir R. Radev,et al. The Use of Predictive Annotation for Question Answering in TREC8 , 1999, TREC.

[3] Oren Etzioni,et al. Scaling question answering to the Web , 2001, WWW '01.

[4] Luis Gravano,et al. Learning search engine specific query transformations for question answering , 2001, WWW '01.

[5] Harris Wu,et al. Probabilistic question answering on the web , 2002, WWW '02.