Twenty-five years of information extraction

Abstract Information extraction is the process of converting unstructured text into a structured data base containing selected information from the text. It is an essential step in making the information content of the text usable for further processing. In this paper, we describe how information extraction has changed over the past 25 years, moving from hand-coded rules to neural networks, with a few stops on the way. We connect these changes to research advances in NLP and to the evaluations organized by the US Government.

[1]  Hongyu Guo,et al.  The Unreasonable Effectiveness of Word Representations for Twitter Named Entity Recognition , 2015, NAACL.

[2]  Mark A. Przybocki,et al.  The Automatic Content Extraction (ACE) Program – Tasks, Data, and Evaluation , 2004, LREC.

[3]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[4]  Oren Etzioni,et al.  Named Entity Recognition in Tweets: An Experimental Study , 2011, EMNLP.

[5]  Jing Lu,et al.  Improving Event Coreference Resolution by Learning Argument Compatibility from Unlabeled Data , 2019, NAACL.

[6]  Yoram Singer,et al.  Unsupervised Models for Named Entity Classification , 1999, EMNLP.

[7]  Alan Ritter,et al.  Results of the WNUT16 Named Entity Recognition Shared Task , 2016, NUT@COLING.

[8]  Nanda Kambhatla,et al.  Combining Lexical, Syntactic, and Semantic Features with Maximum Entropy Models for Information Extraction , 2004, ACL.

[9]  Michael Collins,et al.  A New Statistical Parser Based on Bigram Lexical Dependencies , 1996, ACL.

[10]  Ralph Grishman,et al.  Compensating for Annotation Errors in Training a Relation Extractor , 2012, EACL.

[11]  Vincent Ng,et al.  Machine Learning for Entity Coreference Resolution: A Retrospective Look at Two Decades of Research , 2017, AAAI.

[12]  Yue Zhang,et al.  Deep Learning for Event-Driven Stock Prediction , 2015, IJCAI.

[13]  Satoshi Sekine,et al.  A survey of named entity recognition and classification , 2007 .

[14]  Hongfang Liu,et al.  Journal of Biomedical Informatics , 2022 .

[15]  Christopher D. Manning,et al.  Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling , 2005, ACL.

[16]  C. Ré,et al.  A Machine Reading System for Assembling Synthetic Paleontological Databases , 2014, PloS one.

[17]  Lifu Huang,et al.  Zero-Shot Transfer Learning for Event Extraction , 2017, ACL.

[18]  Appendix H: Text and Answer Key Templates for TST1-MUC3-0099 , 2022, MUC.

[19]  Ralph Grishman,et al.  Message Understanding Conference- 6: A Brief History , 1996, COLING.

[20]  Ralph Grishman,et al.  Extracting Relations with Integrated Information Using Kernel Methods , 2005, ACL.

[21]  Heng Ji,et al.  Knowledge Base Population: Successful Approaches and Challenges , 2011, ACL.

[22]  Ralph Grishman,et al.  Automatic Acquisition of Domain Knowledge for Information Extraction , 2000, COLING.

[23]  Ralph Grishman,et al.  Joint Event Extraction via Recurrent Neural Networks , 2016, NAACL.

[24]  Heng Ji,et al.  Joint Event Extraction via Structured Prediction with Global Features , 2013, ACL.

[25]  Ralph Grishman,et al.  Infusion of Labeled Data into Distant Supervision for Relation Extraction , 2014, ACL.

[26]  Dan Klein,et al.  Named Entity Recognition with Character-Level Models , 2003, CoNLL.

[27]  Benjamin Van Durme,et al.  A Comparison of the Events and Relations Across ACE, ERE, TAC-KBP, and FrameNet Annotation Standards , 2014, EVENTS@ACL.

[28]  Kenneth Ward Church.  A Stochastic Parts Program and Noun Phrase Parser for Unrestricted Text , 1988, ANLP.

[29]  Vasudeva Varma,et al.  Structured Information Extraction from Natural Disaster Events on Twitter , 2014, Web-KR '14.

[30]  Ellen Riloff,et al.  Automatically Generating Extraction Patterns from Untagged Text , 1996, AAAI/IAAI, Vol. 2.

[31]  Mary Ellen Okurowski.  Information Extraction Overview , 1993, TIPSTER.

[32]  Ming Zhou,et al.  Recognizing Named Entities in Tweets , 2011, ACL.

[33]  Thien Huu Nguyen,et al.  Similar but not the Same: Word Sense Disambiguation Improves Event Detection via Neural Representation Matching , 2018, EMNLP.

[34]  Jing Lu,et al.  Event Coreference Resolution: A Survey of Two Decades of Research , 2018, IJCAI.

[35]  David Ahn,et al.  The stages of event extraction , 2006 .

[36]  ChengXiang Zhai,et al.  A Systematic Exploration of the Feature Space for Relation Extraction , 2007, NAACL.

[37]  Luis Gravano,et al.  Snowball: extracting relations from large plain-text collections , 2000, DL '00.

[38]  Richard M. Schwartz,et al.  BBN: Description of the SIFT System as Used for MUC-7 , 1998, MUC.

[39]  Daniel Jurafsky,et al.  Distant supervision for relation extraction without labeled data , 2009, ACL.

[40]  Xiaoqiang Luo,et al.  On Coreference Resolution Performance Metrics , 2005, HLT.

[41]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[42]  Jerry R. Hobbs.  The Generic Information Extraction System , 1993, MUC.

[43]  Ralph Grishman,et al.  Relation Extraction: Perspective from Convolutional Neural Networks , 2015, VS@HLT-NAACL.

[44]  Roland Vollgraf,et al.  Pooled Contextualized Embeddings for Named Entity Recognition , 2019, NAACL.

[45]  Beth M. Sundheim.  The Message Understanding Conferences , 1996, TIPSTER.

[46]  Wenpeng Yin,et al.  Comparative Study of CNN and RNN for Natural Language Processing , 2017, ArXiv.

[47]  Omer Levy,et al.  Zero-Shot Relation Extraction via Reading Comprehension , 2017, CoNLL.

[48]  Lynette Hirschman,et al.  Evaluating Message Understanding Systems: An Analysis of the Third Message Understanding Conference (MUC-3) , 1993, CL.

[49]  Ramesh Nallapati,et al.  Multi-instance Multi-label Learning for Relation Extraction , 2012, EMNLP.