Learning from Language Description: Low-shot Named Entity Recognition via Decomposed Framework

In this work, we study the problem of named entity recognition (NER) in a low resource scenario, focusing on few-shot and zero-shot settings. Built upon large-scale pre-trained language models, we propose a novel NER framework, namely SpanNER, which learns from natural language supervision and enables the identification of never-seen entity classes without using in-domain labeled data. We perform extensive experiments on 5 benchmark datasets and evaluate the proposed method in the few-shot learning, domain transfer and zero-shot learning settings. The experimental results show that the proposed method can bring 10%, 23% and 26% improvements in average over the best baselines in few-shot learning, domain transfer and zero-shot learning settings respectively.

[1]  Zhihan Zhou,et al.  Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network , 2020, ACL.

[2]  Jianfeng Gao,et al.  SOLOIST: Few-shot Task-Oriented Dialog with A Single Pre-trained Auto-regressive Model , 2020, ArXiv.

[3]  Diego Mollá Aliod,et al.  Named Entity Recognition for Question Answering , 2006, ALTA.

[4]  Daniel S. Weld,et al.  Fine-Grained Entity Recognition , 2012, AAAI.

[5]  Oren Etzioni,et al.  Open domain event extraction from twitter , 2012, KDD.

[6]  Leon Derczynski,et al.  Results of the WNUT2017 Shared Task on Novel and Emerging Entity Recognition , 2017, NUT@EMNLP.

[7]  Ali Farhadi,et al.  Bidirectional Attention Flow for Machine Comprehension , 2016, ICLR.

[8]  Morteza Ziyadi,et al.  Example-Based Named Entity Recognition , 2020, ArXiv.

[9]  Karl Stratos,et al.  Label-Agnostic Sequence Labeling by Copying Nearest Neighbors , 2019, ACL.

[10]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[11]  Satoshi Sekine,et al.  A survey of named entity recognition and classification , 2007 .

[12]  Jiwei Li,et al.  A Unified MRC Framework for Named Entity Recognition , 2019, ACL.

[13]  Hang Li,et al.  Named entity recognition in query , 2009, SIGIR.

[14]  Varvara Logacheva,et al.  Few-shot classification in named entity recognition task , 2018, SAC.

[15]  Wen-tau Yih,et al.  Efficient One-Pass End-to-End Entity Linking for Questions , 2020, EMNLP.

[16]  Ming-Wei Chang,et al.  Zero-Shot Entity Linking by Reading Entity Descriptions , 2019, ACL.

[17]  Ahmed Hassan Awadallah,et al.  Adaptive Self-training for Few-shot Neural Sequence Labeling , 2020, ArXiv.

[18]  Andrey Kormilitzin,et al.  Few-shot Learning for Named Entity Recognition in Medical Text , 2018, ArXiv.

[19]  Dan Roth,et al.  Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach , 2019, EMNLP.

[20]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[21]  Luke Zettlemoyer,et al.  Zero-shot Entity Linking with Dense Entity Retrieval , 2019, ArXiv.

[22]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[23]  Omer Levy,et al.  Zero-Shot Relation Extraction via Reading Comprehension , 2017, CoNLL.

[24]  Partha Pratim Talukdar,et al.  Zero-shot Word Sense Disambiguation using Sense Definition Embeddings , 2019, ACL.

[25]  Jianfeng Gao,et al.  Few-Shot Named Entity Recognition: A Comprehensive Study , 2020, ArXiv.

[26]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[27]  James R. Glass,et al.  Asgard: A portable architecture for multilingual dialogue systems , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[28]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[29]  Anders Søgaard,et al.  Zero-Shot Sequence Labeling: Transferring Knowledge from Sentences to Tokens , 2018, NAACL.

[30]  Dan Roth,et al.  Zero-Shot Open Entity Typing as Type-Compatible Grounding , 2019, EMNLP.

[31]  Erik F. Tjong Kim Sang,et al.  Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition , 2003, CoNLL.

[32]  Jason Weston,et al.  Reading Wikipedia to Answer Open-Domain Questions , 2017, ACL.