Algorithms for Scoring Coreference Chains

Scoring the performance of a system is an extremely important aspect of coreference algorithm performance. The score for a particular run is the single strongest measure of how well the system is performing and it can strongly determine directions for further improvements. In this paper, we present several diierent scoring algorithms and detail their respective strengths and weaknesses for varying classes of processing. We also demonstrate that tasks like information extraction have very diierent needs from information retrieval in terms of how to score the performance of coreference annotation.