论文信息 - Two Levels of valuation in a Complex NL System

Two Levels of valuation in a Complex NL System

The QALC question-answering system, developed at LIMSI, has been a participant for two years in the QA track of the TREC conference. In this paper, we present a quantitative evaluation of various modules in our system, based on two criteria: first, the numbers of documents containing the correct answer and selected by the system; secondly, the number of answers found. The first criterion is used for evaluating locally the modules in the system, which contribute in selecting documents that are likely to contain the answer. The second one provides a global evaluation of the system. As such, it also serves for an indirect evaluation of various modules.

[1] Christiane Fellbaum,et al. Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[2] Christian Jacquemin,et al. Boosting Variant Recognition with Light Semantics , 2000, COLING.

[3] Mark Sanderson,et al. University of Sheffield TREC-8 Q&A System , 1999, TREC.

[4] Christian Jacquemin,et al. QALC--The Question-Answering System of LIMSI-CNRS , 2000, TREC.

[5] P. Bellot,et al. Evaluation of Document Retrieval Systems and Query Difficulty , 2022 .

[6] Sanda M. Harabagiu,et al. Experiments with Open-Domain Textual Question Answering , 2000, COLING.

[7] Christian Jacquemin,et al. Syntagmatic and Paradigmatic Representations of Term Variation , 1999, ACL.

[8] Στέλλα Μαρκαντωνάτου,et al. Proceedings of the Second International Conference on Language Resources and Evaluation , 2000 .

[9] Margaret King,et al. Evaluation of natural language processing systems , 1991 .

[10] Marc El-Bèze,et al. Query Expansion and Classification of Retrieved Documents , 1998, TREC.