Record linkage refers to the task of finding same entity across different databases. We propose a machine learning based record linkage algorithm for financial entity databases. Record linkage on financial databases are essential for information integration on certain financial entity, since those databases do not have common unified identifier. Our algorithm works in two steps to determine if a pair of record is same entity or not. First we check with proposed rules if the record pair can be exactly matched after cleaning the entity name and address. Second, inspired by earlier work on author name disambiguation, we train a binary Random Forest classifier to decide the linkage. To reduce and scale the computation, this process is done only for candidate pairs within a proposed heuristic. Initial evaluation for precision, recall and F1 measures on two different linking tasks in the Financial Entity Identification and Information Integration (FEIII) Challenge show promising results.
[1]
C. Lee Giles,et al.
Disambiguating authors in academic publications using random forests
,
2009,
JCDL '09.
[2]
Madian Khabsa,et al.
Online Person Name Disambiguation with Constraints
,
2015,
JCDL.
[3]
References
,
1971
.
[4]
P. Jaccard,et al.
Etude comparative de la distribution florale dans une portion des Alpes et des Jura
,
1901
.
[5]
Leo Breiman,et al.
Random Forests
,
2001,
Machine Learning.
[6]
Madian Khabsa,et al.
Inventor name disambiguation for a patent database using a random forest and DBSCAN
,
2016,
2016 IEEE/ACM Joint Conference on Digital Libraries (JCDL).
[7]
William E. Winkler,et al.
String Comparator Metrics and Enhanced Decision Rules in the Fellegi-Sunter Model of Record Linkage.
,
1990
.