论文信息 - A Study of the Assessment of Relevance for the INEX'02 Test Collection

A Study of the Assessment of Relevance for the INEX'02 Test Collection

We investigate possible assessment trends and inconsistencies within the collected relevance assessments of the INEX’02 test collection in order to provide a critical analysis of the employed relevance criterion and assessment procedure for the evaluation of content-oriented XML retrieval approaches.

[1] Ellen M. Voorhees,et al. The Tenth Text REtrieval Conference, TREC 2001 | NIST , 2002 .

[2] Rong Tang,et al. Towards the Identification of the Optimal Number of Relevance Categories , 1999, J. Am. Soc. Inf. Sci..

[3] Peter Ingwersen,et al. Dimensions of relevance , 2000, Inf. Process. Manag..

[4] Linda Schamber. Relevance and Information Behavior. , 1994 .

[5] Stephen P. Harter,et al. Variations in Relevance Assessments and the Measurement of Retrieval Effectiveness , 1996, J. Am. Soc. Inf. Sci..

[6] Rong Tang,et al. Towards the Identification of the Optimal Number of Relevance Categories , 1999, J. Am. Soc. Inf. Sci..

[7] Pia Borlund,et al. The concept of relevance in IR , 2003, J. Assoc. Inf. Sci. Technol..

[8] Ellen M. Voorhees,et al. Evaluation by highly relevant documents , 2001, SIGIR '01.

[9] Jaana Kekäläinen,et al. Using graded relevance assessments in IR evaluation , 2002, J. Assoc. Inf. Sci. Technol..

[10] Stefano Mizzaro,et al. Relevance: The Whole History , 1997, J. Am. Soc. Inf. Sci..

[11] Pertti Vakkari,et al. Changes in relevance criteria and problem stages in task performance , 2000, J. Documentation.

[12] Jaana Kekäläinen,et al. IR evaluation methods for retrieving highly relevant documents , 2000, SIGIR '00.

[13] Ellen M. Voorhees,et al. Variations in relevance judgments and the measurement of retrieval effectiveness , 1998, SIGIR '98.