A Proposal to Extend and Enrich the Scientific Data Curation of Evaluation Campaigns

This paper examines the current way of keeping the data produced during an evaluation campaign of Information Retrieval Systems (IRSs) and highlights some shortenings of it. In particular, the Cranfield methodology has been designed for creating comparable experiments and evaluating the performances of IRS rather than modeling and managing the scientific data produced during an evaluation campaign. The data produced during an evaluation campaign of IRSs are valuable scientific data, and as a consequence, their lineage should be tracked since it allows us to judge the quality and applicability of information for a given use; those data should be enriched progressively adding further analyses and interpretations on them; it should be possibile to cite them and their further elaboration, since this is an effective way forexplicitly mentioningandmakingreferencesto useful information, for improving the cooperation among researchers and to facilitate the transfer of scientific and innovative results from research groups to the industrial sector.