This paper describes the development of a ground truth dataset of culturally diverse Romanized names in which approximately 70,000 names are matched against a subset of 700. We ran the subset as queries against the complete list using several matchers, created adjudication pools, adjudicated the results, and compiled two versions of ground truth based on different sets of adjudication guidelines and methods for resolving adjudicator conflicts. The name list, drawn from publicly available sources, was manually seeded with over 1500 name variants. These names include transliteration variation, database fielding errors, segmentation differences, incomplete names, titles, initials, abbreviations, nicknames, typos, OCR errors, and truncated data. These diverse types of matches, along with the coincidental name similarities already in the list, make possible a comprehensive evaluation of name matching systems. We have used the dataset to evaluate several open source and commercial algorithms and provide some of those results.
[1]
J. L. Harrison,et al.
The Government Printing Office
,
1968,
American Journal of Pharmaceutical Education.
[2]
Ellen M. Voorhees,et al.
The Philosophy of Information Retrieval Evaluation
,
2001,
CLEF.
[3]
Donna K. Harman,et al.
Overview of the Eighth Text REtrieval Conference (TREC-8)
,
1999,
TREC.
[4]
Matthew A. Jaro,et al.
Advances in Record-Linkage Methodology as Applied to Matching the 1985 Census of Tampa, Florida
,
1989
.
[5]
L. Philips,et al.
Hanging on the metaphone
,
1990
.
[6]
William E. Winkler,et al.
String Comparator Metrics and Enhanced Decision Rules in the Fellegi-Sunter Model of Record Linkage.
,
1990
.
[7]
Vladimir I. Levenshtein,et al.
Binary codes capable of correcting deletions, insertions, and reversals
,
1965
.