Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference

There has been a steady need in the medical community to precisely extract the temporal relations between clinical events. In particular, temporal information can facilitate a variety of downstream applications such as case report retrieval and medical question answering. Existing methods either require expensive feature engineering or are incapable of modeling the global relational dependencies among the events. In this paper, we propose a novel method, Clinical Temporal ReLation Exaction with Probabilistic Soft Logic Regularization and Global Inference (CTRL-PG) to tackle the problem at the document level. Extensive experiments on two benchmark datasets, I2B2-2012 and TB-Dense, demonstrate that CTRL-PG significantly outperforms baseline methods for temporal relation extraction.

[1]  Russ B. Altman,et al.  OrderRex: clinical order decision support and outcome predictions by data-mining electronic medical records , 2016, J. Am. Medical Informatics Assoc..

[2]  Anna Rumshisky,et al.  Context-Aware Neural Model for Temporal Information Extraction , 2018, ACL.

[3]  Timothy Miller,et al.  A BERT-based Universal Model for Both Within- and Cross-sentence Clinical Temporal Relation Extraction , 2019, Proceedings of the 2nd Clinical Natural Language Processing Workshop.

[4]  Nanyun Peng,et al.  Deep Structured Neural Network for Event Temporal Relation Extraction , 2019, CoNLL.

[5]  Johannes Dellert,et al.  Exploring Probabilistic Soft Logic as a framework for integrating top-down and bottom-up processing of language in a task context , 2020, ArXiv.

[6]  Steven Bethard,et al.  A BERT-based One-Pass Multi-Task Model for Clinical Temporal Relation Extraction , 2020, BIONLP.

[7]  Goran Nenadic,et al.  Extraction of temporal relations from clinical free text: A systematic review of current approaches , 2020, J. Biomed. Informatics.

[8]  Jun'ichi Tsujii,et al.  An end-to-end system to identify temporal relation in discharge summaries: 2012 i2b2 challenge , 2013, J. Am. Medical Informatics Assoc..

[9]  Estela Saquete Boró,et al.  TIPSem (English and Spanish): Evaluating CRFs and Semantic Roles in TempEval-2 , 2010, *SEMEVAL.

[10]  Taylor Cassidy,et al.  Dense Event Ordering with a Multi-Pass Architecture , 2014, TACL.

[11]  Yizhou Sun,et al.  A Comprehensive Typing System for Information Extraction from Clinical Narratives , 2019 .

[12]  Hua Xu,et al.  A hybrid system for temporal information extraction from clinical text , 2013, J. Am. Medical Informatics Assoc..

[13]  Yizhou Sun,et al.  A reference set of curated biomedical data and metadata from clinical case reports , 2018, Scientific Data.

[14]  Yusuke Miyao,et al.  Classifying Temporal Relations by Bidirectional LSTM over Dependency Paths , 2017, ACL.

[15]  Nanyun Peng,et al.  Domain Knowledge Empowered Structured Neural Net for End-to-End Event Temporal Relation Extraction , 2020, EMNLP.

[16]  Lise Getoor,et al.  A Declarative Approach to Fairness in Relational Domains , 2019, IEEE Data Eng. Bull..

[17]  James Pustejovsky,et al.  SemEval-2017 Task 12: Clinical TempEval , 2017, *SEMEVAL.

[18]  Naoaki Okazaki,et al.  Investigating the Challenges of Temporal Relation Extraction from Clinical Text , 2018, Louhi@EMNLP.

[19]  L. Getoor,et al.  Tractable Probabilistic Reasoning Through Effective Grounding , 2019 .

[20]  Chen Lin,et al.  Neural Temporal Relation Extraction , 2017, EACL.

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Anna Rumshisky,et al.  Evaluating temporal relations in clinical text: 2012 i2b2 Challenge , 2013, J. Am. Medical Informatics Assoc..

[23]  Veera Raghavendra Chikka.  CDE-IIITH at SemEval-2016 Task 12: Extraction of Temporal Information from Clinical documents using Machine Learning techniques , 2016, *SEMEVAL.

[24]  Janyce Wiebe,et al.  Joint Prediction for Entity/Event-Level Sentiment Analysis using Probabilistic Soft Logic Models , 2015, EMNLP.

[25]  Lenhart K. Schubert,et al.  Time revisited 1 , 1990, Comput. Intell..

[26]  Yizhou Sun,et al.  Embedding Uncertain Knowledge Graphs , 2018, AAAI.

[27]  Vipin Chaudhary,et al.  Attention Neural Model for Temporal Relation Extraction , 2019, Proceedings of the 2nd Clinical Natural Language Processing Workshop.

[28]  James Pustejovsky,et al.  SemEval-2015 Task 6: Clinical TempEval , 2015, *SEMEVAL.

[29]  Mourad Gridach,et al.  A framework based on (probabilistic) soft logic and neural network for NLP , 2020, Appl. Soft Comput..

[30]  Dan Roth,et al.  A Structured Learning Approach to Temporal Relation Extraction , 2017, EMNLP.

[31]  Graciela Gonzalez-Hernandez,et al.  Towards generating a patient's timeline: Extracting temporal relationships from clinical notes , 2013, J. Biomed. Informatics.

[32]  Hongfang Liu,et al.  CLAMP – a toolkit for efficiently building customized clinical natural language processing pipelines , 2017, J. Am. Medical Informatics Assoc..

[33]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[34]  Alan R. Aronson,et al.  An overview of MetaMap: historical perspective and recent advances , 2010, J. Am. Medical Informatics Assoc..

[35]  Hua Xu,et al.  Robustly Pre-Trained Neural Model for Direct Temporal Relation Extraction , 2020, 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI).

[36]  Chen Lin,et al.  Representations of Time Expressions for Temporal Relation Extraction with Convolutional Neural Networks , 2017, BioNLP.

[37]  Nanyun Peng,et al.  Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction , 2019, EMNLP.

[38]  Marie-Francine Moens,et al.  Structured Learning for Temporal Relation Extraction from Clinical Records , 2017, EACL.

[39]  Taylor Cassidy,et al.  An Annotation Framework for Dense Event Ordering , 2014, ACL.

[40]  Chen Lin,et al.  Self-training improves Recurrent Neural Networks performance for Temporal Relation Extraction , 2018, Louhi@EMNLP.

[41]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[42]  Olivier Ferret,et al.  Neural Architecture for Temporal Relation Extraction: A Bi-LSTM Approach for Detecting Narrative Containers , 2017, ACL.

[43]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[44]  Kentaro Inui,et al.  Empirical Exploration of the Challenges in Temporal Relation Extraction from Clinical Text , 2020, Journal of Natural Language Processing.

[45]  Yaoyun Zhang,et al.  UTHealth at SemEval-2016 Task 12: an End-to-End System for Temporal Information Extraction from Clinical Notes , 2016, *SEMEVAL.

[46]  Eric P. Xing,et al.  Harnessing Deep Neural Networks with Logic Rules , 2016, ACL.

[47]  Lise Getoor,et al.  Operating System Support for Data Management on Modern Hardware. , 2019 .

[48]  Sunghwan Sohn,et al.  Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications , 2010, J. Am. Medical Informatics Assoc..

[49]  GetoorLise,et al.  Hinge-loss Markov random fields and probabilistic soft logic , 2017 .

[50]  A. Caban-Martinez,et al.  Advancing medicine one research note at a time: the educational value in clinical case reports , 2012, BMC Research Notes.

[51]  Lise Getoor,et al.  Knowledge Graph Identification , 2013, SEMWEB.

[52]  Dimitar P. Filev,et al.  Fuzzy SETS AND FUZZY LOGIC , 1996 .