Multi-domain spoken language understanding with transfer learning

This paper addresses the problem of multi-domain spoken language understanding (SLU) where domain detection and domain-dependent semantic tagging problems are combined. We present a transfer learning approach to the multi-domain SLU problem in which multiple domain-specific data sources can be incorporated. To implement multi-domain SLU with transfer learning, we introduce a triangular-chain structured model. This model effectively learns multiple domains in parallel, and allows use of domain-independent patterns among domains to create a better model for the target domain. We demonstrate that the proposed method outperforms baseline models on dialog data for multi-domain SLU problems.

[1]  Gökhan Tür,et al.  The AT&T spoken language understanding system , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[2]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[3]  Ben Taskar,et al.  Max-Margin Markov Networks , 2003, NIPS.

[4]  Fernando Pereira,et al.  Shallow Parsing with Conditional Random Fields , 2003, NAACL.

[5]  Allen L. Gorin,et al.  Knowledge collection for natural language spoken dialog systems , 1999, EUROSPEECH.

[6]  Koby Crammer,et al.  Confidence-weighted linear classification , 2008, ICML '08.

[7]  Giuseppe Riccardi,et al.  Generative and discriminative algorithms for spoken language understanding , 2007, INTERSPEECH.

[8]  Alessandro Moschitti,et al.  Spoken language understanding with kernels for syntactic/semantic structures , 2007, 2007 IEEE Workshop on Automatic Speech Recognition & Understanding (ASRU).

[9]  Hal Daumé,et al.  Frustratingly Easy Domain Adaptation , 2007, ACL.

[10]  Daniel Marcu,et al.  Learning as search optimization: approximate large margin methods for structured prediction , 2005, ICML.

[11]  Andrew McCallum,et al.  Piecewise pseudolikelihood for efficient training of conditional random fields , 2007, ICML '07.

[12]  Tetsuya Ogata,et al.  Managing out-of-grammar utterances by topic estimation with domain extensibility in multi-domain spoken dialogue systems , 2008, Speech Commun..

[13]  Gary Geunbae Lee,et al.  Exploiting Non-Local Features for Spoken Language Understanding , 2006, ACL.

[14]  Stephanie Seneff,et al.  Towards multi-domain speech understanding using a two-stage recognizer , 1999, EUROSPEECH.

[15]  Gökhan Tür.  Multitask Learning for Spoken Language Understanding , 2006, ICASSP.

[16]  Gregory A. Sanders,et al.  DARPA communicator: cross-system results for the 2001 evaluation , 2002, INTERSPEECH.

[17]  Ye-Yi Wang,et al.  Spoken language understanding , 2005, IEEE Signal Processing Magazine.

[18]  Jeremy Peckham,et al.  Speech Understanding and Dialogue over the telephone: an overview of the ESPRIT SUNDIAL project. , 1991, HLT.

[19]  Daniel Gildea,et al.  The Proposition Bank: An Annotated Corpus of Semantic Roles , 2005, CL.

[20]  Dilek Z. Hakkani-Tür,et al.  Spoken language understanding , 2008, IEEE Signal Processing Magazine.

[21]  Daniel Jurafsky,et al.  Automatic Labeling of Semantic Roles , 2002, CL.

[22]  Thomas Hofmann,et al.  Large Margin Methods for Structured and Interdependent Output Variables , 2005, J. Mach. Learn. Res..

[23]  Daniel Marcu,et al.  Domain Adaptation for Statistical Classifiers , 2006, J. Artif. Intell. Res..

[24]  Michael F. McTear.  SPOKEN LANGUAGE UNDERSTANDING FOR CONVERSATIONAL DIALOG SYSTEMS , 2006, 2006 IEEE Spoken Language Technology Workshop.

[25]  Stephen Cox,et al.  Some statistical issues in the comparison of speech recognition algorithms , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[26]  Koby Crammer,et al.  Online Passive-Aggressive Algorithms , 2003, J. Mach. Learn. Res..

[27]  Mitchell P. Marcus,et al.  Text Chunking using Transformation-Based Learning , 1995, VLC@ACL.

[28]  Gary Geunbae Lee,et al.  CHAT AND GOAL-ORIENTED DIALOG TOGETHER: A UNIFIED EXAMPLE-BASED ARCHITECTURE FOR MULTI-DOMAIN DIALOG MANAGEMENT , 2006, 2006 IEEE Spoken Language Technology Workshop.

[29]  Gary Geunbae Lee,et al.  Triangular-Chain Conditional Random Fields , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[30]  Alex Acero,et al.  Spoken Language Understanding "” An Introduction to the Statistical Framework , 2005 .

[31]  P. J. Price,et al.  Evaluation of Spoken Language Systems: the ATIS Domain , 1990, HLT.

[32]  Jeremy Peckham.  Speech understanding and dialouge over the telephone: an overview of the ESPRIT SUNDIAL , 1991 .

[33]  Michael Collins,et al.  Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms , 2002, EMNLP.

[34]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[35]  Trevor Cohn.  Efficient Inference in Large Conditional Random Fields , 2006, ECML.

[36]  Tomek Strzalkowski,et al.  The Amities system: Data-driven techniques for automated dialogue , 2006, Speech Commun..

[37]  Rich Caruana,et al.  Multitask Learning , 1997, Machine-mediated learning.