An Evaluation Framework for Aggregated Temporal Information Extraction

This paper focusses on the representation and evaluation of temporal information about a certain event or entity. In particular, we consider temporal information that can be normalized to specic dates. This task requires the aggregation of temporal relations between events and dates extracted from multiple texts. Given that the resulting temporal information can be vague, it is necessary that an evaluation framework captures and compares the temporal uncertainty of system outputs and human assessed gold-standard data. Current representation models and measures are not suitable for this scenario. In this paper, we propose a novel representation model and assess its properties and limitations. In order to compare extracted information against a gold standard, we dene an evaluation metric based on a set of formal constraints. Finally, we present experiments that show the behavior of the proposed metric. The task setting and the evaluation measure presented here have been introduced in the TAC 2011 Knowledge Base Population evaluation for the Temporal Slot Filling task.