SETNet: A Novel Semi-Supervised Approach for Semantic Parsing

In this work, we study on semi-supervised semantic parsing under a multi-task learning framework to alleviate limited performance caused by limited annotated data. Two novel strategies are proposed to leverage unlabeled natural language utterances. The first one takes entity predicate sequences as training targets to enhance representation learning. The second one extends Mean Teacher to seq2seq model and generates more target-side data to improve the generalizability of decoder network. Different from original Mean Teacher, our strategy produces hard targets for the student decoder and update the decoder weights instead of the whole model. Experiments demonstrate that our proposed methods significantly outperform the supervised baseline and achieve more impressive improvement than previous methods.

[1]  Luke S. Zettlemoyer,et al.  Online Learning of Relaxed CCG Grammars for Parsing to Logical Form , 2007, EMNLP.

[2]  Graham Neubig,et al.  StructVAE: Tree-structured Latent Variable Models for Semi-supervised Semantic Parsing , 2018, ACL.

[3]  Percy Liang,et al.  Data Recombination for Neural Semantic Parsing , 2016, ACL.

[4]  Ivan Skorokhodov,et al.  Semi-Supervised Neural Machine Translation with Language Models , 2018, LoResMT@AMTA.

[5]  Jiajun Zhang,et al.  Exploiting Source-side Monolingual Data in Neural Machine Translation , 2016, EMNLP.

[6]  Lu Wang,et al.  Semi-Supervised Learning for Neural Keyphrase Generation , 2018, EMNLP.

[7]  Maosong Sun,et al.  Semi-Supervised Learning for Neural Machine Translation , 2016, ACL.

[8]  Quoc V. Le,et al.  Semi-Supervised Sequence Modeling with Cross-View Training , 2018, EMNLP.

[9]  Mark Steedman,et al.  Inducing Probabilistic CCG Grammars from Logical Form with Higher-Order Unification , 2010, EMNLP.

[10]  Kai Zhao,et al.  Type-Driven Incremental Semantic Parsing with Polymorphism , 2014, NAACL.

[11]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[12]  Wei Lu,et al.  Neural Architectures for Multilingual Semantic Parsing , 2017, ACL.

[13]  Parag Jain,et al.  Unified Semantic Parsing with Weak Supervision , 2019, ACL.

[14]  Rico Sennrich,et al.  Improving Neural Machine Translation Models with Monolingual Data , 2015, ACL.

[15]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[16]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[17]  Yejin Choi,et al.  Neural AMR: Sequence-to-Sequence Models for Parsing and Generation , 2017, ACL.

[18]  Dan Klein,et al.  Abstract Syntax Networks for Code Generation and Semantic Parsing , 2017, ACL.

[19]  Xing Fan,et al.  Transfer Learning for Neural Semantic Parsing , 2017, Rep4NLP@ACL.

[20]  Quoc V. Le,et al.  Semi-supervised Sequence Learning , 2015, NIPS.

[21]  Harri Valpola,et al.  Weight-averaged consistency targets improve semi-supervised deep learning results , 2017, ArXiv.

[22]  Rohit J. Kate,et al.  Semi-Supervised Learning for Semantic Parsing using Support Vector Machines , 2007, NAACL.

[23]  Chris Dyer,et al.  Semantic Parsing with Semi-Supervised Sequential Autoencoders , 2016, EMNLP.

[24]  Kai Yu,et al.  Semantic Parsing with Dual Learning , 2019, ACL.

[25]  Mirella Lapata,et al.  Coarse-to-Fine Decoding for Neural Semantic Parsing , 2018, ACL.

[26]  Luke S. Zettlemoyer,et al.  Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars , 2005, UAI.

[27]  Mirella Lapata,et al.  Language to Logical Form with Neural Attention , 2016, ACL.

[28]  Muhua Zhu,et al.  Deep Cascade Multi-Task Learning for Slot Filling in Online Shopping Assistant , 2018, AAAI.