Jointly Modeling Inter-Slot Relations by Random Walk on Knowledge Graphs for Unsupervised Spoken Language Understanding

A key challenge of designing coherent semantic ontology for spoken language understanding is to consider inter-slot relations. In practice, however, it is difficult for domain experts and professional annotators to define a coherent slot set, while considering various lexical, syntactic, and semantic dependencies. In this paper, we exploit the typed syntactic dependency theory for unsupervised induction and filling of semantics slots in spoken dialogue systems. More specifically, we build two knowledge graphs: a slot-based semantic graph, and a word-based lexical graph. To jointly consider word-to-word, word-toslot, and slot-to-slot relations, we use a random walk inference algorithm to combine the two knowledge graphs, guided by dependency grammars. The experiments show that considering inter-slot relations is crucial for generating a more coherent and compete slot set, resulting in a better spoken language understanding model, while enhancing the interpretability of semantic slots.

[1]  Graeme Hirst,et al.  Building and Using a Lexical Knowledge Base of Near-Synonym Differences , 2006, Computational Linguistics.

[2]  Christopher D. Manning,et al.  The Stanford Typed Dependencies Representation , 2008, CF+CDPE@COLING.

[3]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[4]  Geoffrey Zweig,et al.  Linguistic Regularities in Continuous Space Word Representations , 2013, NAACL.

[5]  Geoffrey Zweig,et al.  Probabilistic enrichment of knowledge graph entities for relation detection in conversational understanding , 2014, INTERSPEECH.

[6]  Alexander I. Rudnicky,et al.  An empirical investigation of sparse log-linear models for improved dialogue act classification , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  John B. Lowe,et al.  The Berkeley FrameNet Project , 1998, ACL.

[8]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[9]  Ni Lao,et al.  Reading The Web with Learned Syntactic-Semantic Inference Rules , 2012, EMNLP.

[10]  Alexander I. Rudnicky,et al.  Dynamically supporting unexplored domains in conversational interactions by enriching semantics with neural word embeddings , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[11]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[12]  Xindong Wu,et al.  Assessing sparse information extraction using semantic contexts , 2013, CIKM.

[13]  Matthew Henderson,et al.  Discriminative spoken language understanding using word confusion networks , 2012, 2012 IEEE Spoken Language Technology Workshop (SLT).

[14]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Alexander I. Rudnicky,et al.  Leveraging frame semantics and distributional semantics for unsupervised semantic slot induction in spoken dialogue systems , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[16]  Jason Weston,et al.  Learning Structured Embeddings of Knowledge Bases , 2011, AAAI.

[17]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[18]  Noah A. Smith,et al.  Probabilistic Frame-Semantic Parsing , 2010, NAACL.

[19]  Gökhan Tür,et al.  Deriving local relational surface forms from dependency-based entity embeddings for unsupervised spoken language understanding , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[20]  Florian Metze,et al.  Two-layer mutually reinforced random walk for improved multi-party meeting summarization , 2012, 2012 IEEE Spoken Language Technology Workshop (SLT).

[21]  Florian Metze,et al.  Multi-layer mutually reinforced random walk with hidden parameters for improved multi-party meeting summarization , 2013, INTERSPEECH.

[22]  Alexander I. Rudnicky,et al.  Unsupervised induction and filling of semantic slots for spoken dialogue systems using frame-semantic parsing , 2013, 2013 IEEE Workshop on Automatic Speech Recognition and Understanding.

[23]  Omer Levy,et al.  Dependency-Based Word Embeddings , 2014, ACL.

[24]  Noah A. Smith,et al.  Frame-Semantic Parsing , 2014, CL.

[25]  Christopher Meek,et al.  Semantic Parsing for Single-Relation Question Answering , 2014, ACL.

[26]  Zhoujun Li,et al.  Concept-based Short Text Classification and Ranking , 2014, CIKM.

[27]  Tom M. Mitchell,et al.  Random Walk Inference and Learning in A Large Scale Knowledge Base , 2011, EMNLP.

[28]  Gökhan Tür,et al.  Beyond ASR 1-best: Using word confusion networks in spoken language understanding , 2006, Comput. Speech Lang..

[29]  Xindong Wu,et al.  Computing term similarity by large probabilistic isA knowledge , 2013, CIKM.

[30]  Andrew Y. Ng,et al.  Parsing with Compositional Vector Grammars , 2013, ACL.

[31]  Gökhan Tür,et al.  Using a knowledge graph and query click logs for unsupervised learning of relation detection , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[32]  Gökhan Tür,et al.  Leveraging knowledge graphs for web-scale unsupervised semantic parsing , 2013, INTERSPEECH.

[33]  C. Fillmore.  FRAME SEMANTICS AND THE NATURE OF LANGUAGE * , 1976 .

[34]  Haixun Wang,et al.  Short Text Conceptualization Using a Probabilistic Knowledgebase , 2011, IJCAI.

[35]  Gökhan Tür,et al.  Extending domain coverage of language understanding systems via intent transfer between domains using knowledge graphs and search query click logs , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[36]  Farrell Ackerman,et al.  Charles J. Fillmore , 2014 .

[37]  Amy Nicole Langville,et al.  A Survey of Eigenvector Methods for Web Information Retrieval , 2005, SIAM Rev..