CLEF eHealth 2018 Multilingual Information Extraction Task Overview: ICD10 Coding of Death Certificates in French, Hungarian and Italian

This paper reports on Task 1 of the 2018 CLEF eHealth evaluation lab which extended the previous information extraction tasks of ShARe/CLEF eHealth evaluation labs. The task continued with coding of death certificates, as introduced in CLEF eHealth 2016. This largescale classification task consisted of extracting causes of death as coded in the International Classification of Diseases, tenth revision (ICD10). The languages offered for the task this year were French, Hungarian and Italian. Participant systems were evaluated against a blind reference standard of 11,932 death certificates in the French dataset 21,176 certificates in the Hungarian dataset and 3,618 certificates in the Italian dataset using Precision, Recall and F-measure. In total, fourteen teams participated: 14 teams submitted runs for the French dataset, 5 submitted runs for the Hungarian dataset and 6 for the Italian dataset. For death certificate coding, the highest performance was 0.838 F-measure for French, 0.9627 for Hungarian and 0.9524 for Italian.

[1]  Prakash M. Nadkarni,et al.  Overcoming barriers to NLP for clinical text: the role of shared tasks and the need for additional creative solutions , 2011, J. Am. Medical Informatics Assoc..

[2]  Gayo Diallo,et al.  SITIS-ISPED in CLEF eHealth 2018 Task 1 : ICD10 coding using Deep Learning , 2018, CLEF.

[3]  Zhiyong Lu,et al.  Community challenges in biomedical text mining over 10 years: success, failure and the future , 2016, Briefings Bioinform..

[4]  Pierre Zweigenbaum,et al.  Multiple Methods for Multi-class, Multi-label ICD-10 Coding of Multi-granularity, Multilingual Death Certificates , 2017, CLEF.

[5]  Nerea Ezeiza,et al.  IxaMed at CLEF eHealth 2018 Task 1: ICD10 Coding with a Sequence-to-Sequence Approach , 2018, CLEF.

[6]  Mario Almagro,et al.  MAMTRA-MED at CLEF eHealth 2018: A Combination of Information Retrieval Techniques and Neural Networks for ICD-10 Coding of Death Certificates , 2018, CLEF.

[7]  Ulf Leser,et al.  WBI at CLEF eHealth 2018 Task 1: Language-independent ICD-10 Coding using Multi-lingual Embeddings and Recurrent Neural Networks , 2018, CLEF.

[8]  Guido Zuccon,et al.  Overview of the CLEF eHealth Evaluation Lab 2015 , 2015, CLEF.

[9]  K. Bretonnel Cohen,et al.  CLEF eHealth 2017 Multilingual Information Extraction task Overview: ICD10 Coding of Death Certificates in English and French , 2017, CLEF.

[10]  Gérard Pavillon,et al.  IRIS: A language-independent coding system based onthe NCHS system MMDS , 2005 .

[11]  Fleur Mougin,et al.  IAM at CLEF eHealth 2018 : Concept Annotation and Coding in French Death Certificates , 2018, CLEF.

[12]  Sumithra Velupillai,et al.  KCL-Health-NLP@CLEF eHealth 2018 Task 1: ICD-10 Coding of French and Italian Death Certificates with Character-Level Convolutional Neural Networks , 2018, CLEF.

[13]  Patrick Ruch,et al.  Instance-based Learning for ICD10 Categorization , 2018, CLEF.

[14]  Luis Alfonso Ureña López,et al.  Machine Learning to Detect ICD10 Codes in Causes of Death , 2018, CLEF.

[15]  Cong Xu,et al.  ECNU at 2018 eHealth Task1 Multilingual Information Extraction , 2018, CLEF.

[16]  Giorgio Maria Di Nunzio.  Classification of ICD10 Codes with no Resources but Reproducible Code. IMS Unipd at CLEF eHealth Task 1 , 2018, CLEF.

[17]  Rémi Flicoteaux.  ECSTRA-APHP @ CLEF eHealth2018-task 1: ICD10 Code Extraction from Death Certificates , 2018, CLEF.

[18]  Mohammed El Amine Abderrahim,et al.  Tlemcen University at CELF eHealth 2018 Team Techno: Multilingual Information Extraction - ICD10 coding , 2018, CLEF.

[19]  Guido Zuccon,et al.  Overview of the CLEF eHealth Evaluation Lab 2018 , 2018, CLEF.

[20]  Pierre Zweigenbaum,et al.  A Dataset for ICD-10 Coding of Death Certificates: Creation and Usage , 2016, BioTxtM@COLING 2016.

[21]  Guido Zuccon,et al.  CLEF 2017 eHealth Evaluation Lab Overview , 2017, CLEF.

[22]  Graeme Hirst,et al.  TorontoCL at the CLEF 2018 eHealth Challenge Task 1 , 2018 .

[23]  Sanna Salanterä,et al.  Overview of the ShARe/CLEF eHealth Evaluation Lab 2013 , 2013, CLEF.