Pseudo Siamese Network for Few-shot Intent Generation

Few-shot intent detection is a challenging task due to the scare annotation problem. In this paper, we propose a Pseudo Siamese Network (PSN) to generate labeled data for few-shot intents and alleviate this problem. PSN consists of two identical subnetworks with the same structure but different weights: an action network and an object network. Each subnetwork is a transformer-based variational autoencoder that tries to model the latent distribution of different components in the sentence. The action network is learned to understand action tokens and the object network focuses on object-related expressions. It provides an interpretable framework for generating an utterance with an action and an object existing in a given intent. Experiments on two real-world datasets show that PSN achieves state-of-the-art performance for the generalized few shot intent detection task.

[1]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[2]  Philip S. Yu,et al.  Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer , 2021, SIGIR.

[3]  Samy Bengio,et al.  Generating Sentences from a Continuous Space , 2015, CoNLL.

[4]  Xiaodong Liu,et al.  Unified Language Model Pre-training for Natural Language Understanding and Generation , 2019, NeurIPS.

[5]  Verena Rieser,et al.  Benchmarking Natural Language Understanding Services for building Conversational Agents , 2019, IWSDS.

[6]  Maxine Eskénazi,et al.  Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders , 2017, ACL.

[7]  Ruhi Sarikaya,et al.  Convolutional neural network based triangular CRF for joint intent detection and slot filling , 2013, 2013 IEEE Workshop on Automatic Speech Recognition and Understanding.

[8]  Philip S. Yu,et al.  Open-world Learning and Application to Product Classification , 2018, WWW.

[9]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[10]  Eric P. Xing,et al.  Toward Controlled Generation of Text , 2017, ICML.

[11]  Sang-goo Lee,et al.  Data Augmentation for Spoken Language Understanding via Joint Variational Generation , 2018, AAAI.

[12]  Philip S. Yu,et al.  Zero-shot User Intent Detection via Capsule Neural Networks , 2018, EMNLP.

[13]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[14]  Kai Zou,et al.  EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks , 2019, EMNLP.

[15]  Philip S. Yu,et al.  Composed Variational Natural Language Generation for Few-shot Intents , 2020, FINDINGS.

[16]  Francesco Caltagirone,et al.  Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces , 2018, ArXiv.

[17]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[18]  Max Welling,et al.  Semi-supervised Learning with Deep Generative Models , 2014, NIPS.

[19]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[20]  Philip S. Yu,et al.  Low-shot Learning in Natural Language Processing , 2020, 2020 IEEE Second International Conference on Cognitive Machine Intelligence (CogMI).

[21]  Philip S. Yu,et al.  Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference , 2020, EMNLP.

[22]  Gökhan Tür,et al.  End-to-End Memory Networks with Knowledge Carryover for Multi-Turn Spoken Language Understanding , 2016, INTERSPEECH.

[23]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[24]  Wenpeng Yin,et al.  Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System , 2021, NAACL.

[25]  Angeliki Metallinou,et al.  Controlled Text Generation for Data Augmentation in Intelligent Artificial Agents , 2019, EMNLP.

[26]  Philip S. Yu,et al.  Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection , 2020, EMNLP.

[27]  Philip S. Yu,et al.  CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection , 2020, ArXiv.