Cardinal, Nominal or Ordinal Similarity Measures in Comparative Evaluation of Information Retrieval Process

Similarity measures are used to quantify the resemblance of two sets. Simplest ones are calculated by ratios of the document's number of the compared sets. These measures are simple and usually employed in first steps of evaluation studies, they are called cardinal measures. Others measures compare sets upon the number of common documents they have. They are usually employed in quantitative information retrieval evaluations, some examples are Jaccard, Cosine, Recall or Precision. These measures are called nominal ones. There are more or less adapted in function of the richness of the information system's answer. Indeed, in the past, they were sufficient because answers given by systems were only composed by an unordered set of documents. But usual systems improve the quality or the visibility of there answers by using a relevant ranking or a clustering presentation of documents. In this case, similarity measures aren't adapted. In this paper we present some solu tions in the case of totally ordered and partially ordered answer.

[1]  Martin Frické Jean Tague-Sutcliffe on Measuring Information , 1998, Inf. Process. Manag..

[2]  David Ellis The Dilemma of Measurement in Information Retrieval Research , 1996, J. Am. Soc. Inf. Sci..

[3]  Robert M. Losee,et al.  The Science of Information: Measurement and Applications , 1990 .

[4]  Peter Ingwersen,et al.  Measures of relative relevance and ranked half-life: performance indicators for interactive IR , 1998, SIGIR '98.

[5]  Stephen P. Harter,et al.  Variations in Relevance Assessments and the Measurement of Retrieval Effectiveness , 1996, J. Am. Soc. Inf. Sci..

[6]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[7]  Oren Etzioni,et al.  Grouper: A Dynamic Clustering Interface to Web Search Results , 1999, Comput. Networks.

[8]  Ellen M. Voorhees,et al.  Overview of the seventh text retrieval conference (trec-7) [on-line] , 1999 .

[9]  Donald H. Kraft,et al.  Measurement in Information Science , 1994 .

[10]  Ellen M. Voorhees,et al.  Overview of the Seventh Text REtrieval Conference , 1998 .

[11]  David Ellis,et al.  The Dilemma of Measurement in Information Retrieval Research , 1996, J. Am. Soc. Inf. Sci..

[12]  Lynette Hirschman,et al.  Overview of evaluation in speech and natural language processing , 1997 .

[13]  Jean Tague-Sutcliffe,et al.  Some Perspectives on the Evaluation of Information Retrieval Systems , 1996, J. Am. Soc. Inf. Sci..

[14]  Christine Michel,et al.  Diagnostic evaluation of a personalized filtering information retrieval system: methodology and experimental results , 2000 .

[15]  Stephen P. Harter,et al.  Evaluation of information retrieval systems : Approaches, issues, and methods , 1997 .

[16]  David Dubin Measurement in information science , 1997 .