Semantic Parsing in Task-Oriented Dialog with Recursive Insertion-based Encoder

We introduce Recursive INsertion-based Encoder (RINE), a novel approach for semantic parsing in task-oriented dialog. Our model consists of an encoder network that incrementally builds the semantic parse tree by predicting the non-terminal label and its positions in the linearized tree. At the generation time, the model constructs the semantic parse tree by recursively inserting the predicted non-terminal labels at the predicted positions until termination. RINE achieves state-ofthe-art exact match accuracy on lowand high-resource versions of the conversational semantic parsing benchmark TOP (Gupta et al. 2018; Chen et al. 2020), outperforming strong sequence-to-sequence models and transition-based parsers. We also show that our model design is applicable to nested named entity recognition task, where it performs on par with state-of-the-art approach designed for that task. Finally, we demonstrate that our approach is 2 − 3.5× faster than the sequence-to-sequence model at inference time.

[1]  Jacob Andreas,et al.  Task-Oriented Dialogue as Dataflow Synthesis , 2020, Transactions of the Association for Computational Linguistics.

[2]  Asish Ghoshal,et al.  Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing , 2020, EMNLP.

[3]  Claire Cardie,et al.  Nested Named Entity Recognition Revisited , 2018, NAACL.

[4]  Saleh Soltan,et al.  Don’t Parse, Insert: Multilingual Semantic Parsing with Insertion Based Decoding , 2020, CONLL.

[5]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[6]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[7]  David R. Traum,et al.  Dialogue-AMR: Abstract Meaning Representation for Dialogue , 2020, LREC.

[8]  Myle Ott,et al.  fairseq: A Fast, Extensible Toolkit for Sequence Modeling , 2019, NAACL.

[9]  Jan Hajic,et al.  Neural Architectures for Nested NER through Linearization , 2019, ACL.

[10]  Stefano Soatto,et al.  Structured Prediction as Translation between Augmented Natural Languages , 2021, ICLR.

[11]  Omer Levy,et al.  GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding , 2018, BlackboxNLP@EMNLP.

[12]  David J. Weir,et al.  Characterizing Structural Descriptions Produced by Various Grammatical Formalisms , 1987, ACL.

[13]  Luke Zettlemoyer,et al.  Span-based Hierarchical Semantic Parsing for Task-Oriented Dialog , 2019, EMNLP.

[14]  Sonal Gupta,et al.  Semantic Parsing for Task Oriented Dialog using Hierarchical Representations , 2018, EMNLP.

[15]  Shrey Desai,et al.  Low-Resource Task-Oriented Semantic Parsing via Intrinsic Modeling , 2021, ArXiv.

[16]  Joakim Nivre,et al.  Parsing Discontinuous Phrase Structure with Grammatical Functions , 2008, GoTAL.

[17]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[18]  Bing Liu,et al.  Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling , 2016, INTERSPEECH.

[19]  Dain Kaplan,et al.  Conversational Semantic Parsing for Dialog State Tracking , 2020, EMNLP.

[20]  Bing Xiang,et al.  Augmented Natural Language for Generative Sequence Labeling , 2020, EMNLP.

[21]  Jiwei Li,et al.  A Unified MRC Framework for Named Entity Recognition , 2019, ACL.

[22]  Armen Aghajanyan,et al.  Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog , 2021, NAACL.

[23]  Andreas Vlachos,et al.  Merge and Label: A novel neural network architecture for nested NER , 2019, ACL.

[24]  Yoshua Bengio,et al.  Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding , 2013, INTERSPEECH.

[25]  George R. Doddington,et al.  The ATIS Spoken Language Systems Pilot Corpus , 1990, HLT.

[26]  Eduard Hovy,et al.  Nested Named Entity Recognition via Second-best Sequence Learning and Decoding , 2019, Transactions of the Association for Computational Linguistics.

[27]  Navdeep Jaitly,et al.  Pointer Networks , 2015, NIPS.

[28]  Jakob Uszkoreit,et al.  Insertion Transformer: Flexible Sequence Generation via Insertion Operations , 2019, ICML.

[29]  Philipp Koehn,et al.  Abstract Meaning Representation for Sembanking , 2013, LAW@ACL.

[30]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[31]  Luke S. Zettlemoyer,et al.  Improving Semantic Parsing for Task Oriented Dialog , 2019, ArXiv.

[32]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[33]  Noah A. Smith,et al.  Recurrent Neural Network Grammars , 2016, NAACL.

[34]  Andrew Gordon Wilson,et al.  Averaging Weights Leads to Wider Optima and Better Generalization , 2018, UAI.

[35]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[36]  Yaojie Lu,et al.  Sequence-to-Nuggets: Nested Entity Mention Detection via Anchor-Region Networks , 2019, ACL.

[37]  Wen Wang,et al.  BERT for Joint Intent Classification and Slot Filling , 2019, ArXiv.

[38]  Shrey Desai,et al.  Span Pointer Networks for Non-Autoregressive Task-Oriented Semantic Parsing , 2021, EMNLP.

[39]  Emilio Monti,et al.  Don’t Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing , 2020, WWW.

[40]  Wei Lu,et al.  Neural Segmental Hypergraphs for Overlapping Mention Recognition , 2018, EMNLP.

[41]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[42]  Christopher D. Manning,et al.  Get To The Point: Summarization with Pointer-Generator Networks , 2017, ACL.

[43]  Haoran Li,et al.  Conversational Semantic Parsing , 2020, EMNLP.

[44]  Mari Ostendorf,et al.  A general framework for information extraction using dynamic span graphs , 2019, NAACL.

[45]  Dan Klein,et al.  A Minimal Span-Based Neural Constituency Parser , 2017, ACL.

[46]  Stefan Müller,et al.  Continuous or Discontinuous Constituents? A Comparison between Syntactic Analyses for Constituent Order and Their Processing Systems , 2004 .