Situated incremental natural language understanding using Markov Logic Networks

We present work on understanding natural language in a situated domain in an incremental, word-by-word fashion. We explore a set of models specified as Markov Logic Networks and show that a model that has access to information about the visual context during an utterance, its discourse context, the words of the utterance, as well as the linguistic structure of the utterance performs best and is robust to noisy speech input. We explore the incremental properties of the models and offer some analysis. We conclude that mlns provide a promising framework for specifying such models in a general, possibly domain-independent way.

[1]  David Schlangen,et al.  Incremental Reference Resolution: The Task, Metrics for Evaluation, and a Bayesian Filtering Model that is Sensitive to Disfluencies , 2009, SIGDIAL Conference.

[2]  Ann Copestake.  Semantic Composition with (Robust) Minimal Recursion Semantics , 2007, ACL 2007.

[3]  Raquel Fernández,et al.  Referring under Restricted Interactivity Conditions , 2007, SIGDIAL.

[4]  David Schlangen,et al.  A Simple Method for Resolution of Definite Reference in a Shared Visual Context , 2008, SIGDIAL Workshop.

[5]  Hermann Ney,et al.  Comparing Stochastic Approaches to Spoken Language Understanding in Multiple Languages , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[6]  Rajeev Rastogi,et al.  Web information extraction using Markov logic networks , 2011, WWW.

[7]  Gabriel Skantze,et al.  A General, Abstract Model of Incremental Dialogue Processing , 2011 .

[8]  Nathan Schneider,et al.  Association for Computational Linguistics: Human Language Technologies , 2011 .

[9]  Renato De Mori,et al.  Spoken language interpretation: On the use of dynamic Bayesian networks for semantic composition , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[10]  Meng Joo Er,et al.  A hybrid computational model for spoken language understanding , 2010, 2010 11th International Conference on Control Automation Robotics & Vision.

[11]  David Schlangen,et al.  Markov Logic Networks for Situated Incremental Natural Language Understanding , 2012, SIGDIAL Conference.

[12]  David Schlangen,et al.  Comparing Local and Sequential Models for Statistical Incremental Natural Language Understanding , 2010, SIGDIAL Conference.

[13]  David Schlangen,et al.  Assessing and Improving the Performance of Speech Recognition for Incremental Systems , 2009, NAACL.

[14]  Iván V. Meza,et al.  Jointly Identifying Predicates, Arguments and Senses using Markov Logic , 2009, NAACL.

[15]  Pedro M. Domingos,et al.  Discriminative Training of Markov Logic Networks , 2005, AAAI.

[16]  Luke S. Zettlemoyer,et al.  Learning Context-Dependent Mappings from Sentences to Logical Form , 2009, ACL.

[17]  David Schlangen,et al.  The InproTK 2012 release , 2012, SDCTD@NAACL-HLT.

[18]  R. Mitkov.  ANAPHORA RESOLUTION: THE STATE OF THE ART , 2007 .

[19]  Luke S. Zettlemoyer,et al.  Online Learning of Relaxed CCG Grammars for Parsing to Logical Form , 2007, EMNLP.

[20]  Collin F. Baker,et al.  Frame semantics for text understanding , 2001 .

[21]  David DeVault,et al.  Can I Finish? Learning When to Respond to Incremental Interpretation Results in Interactive Dialogue , 2009, SIGDIAL Conference.

[22]  Gabriel Skantze,et al.  Towards Incremental Speech Generation in Dialogue Systems , 2010, SIGDIAL Conference.

[23]  David Schlangen,et al.  Collaborating on Utterances with a Spoken Dialogue System Using an ISU-based Approach to Incremental Dialogue Management , 2010, SIGDIAL Conference.

[24]  Gabriel Skantze,et al.  Incremental Dialogue Processing in a Micro-Domain , 2009, EACL.

[25]  Ellen Campana,et al.  Incremental understanding in human-computer dialogue and experimental evidence for advantages over nonincremental methods , 2007 .

[26]  Dan Klein,et al.  Learning Dependency-Based Compositional Semantics , 2011, CL.

[27]  Hermann Ney,et al.  A Comparison of Various Methods for Concept Tagging for Spoken Language Understanding , 2008, LREC.

[28]  Ellen Campana,et al.  Software architectures for incremental understanding of human speech , 2006, INTERSPEECH.

[29]  Stefan Kopp,et al.  Combining Incremental Language Generation and Incremental Speech Synthesis for Adaptive Information Presentation , 2012, SIGDIAL Conference.

[30]  James F. Allen,et al.  Software architectures for incremental u , 2006 .

[31]  David DeVault,et al.  Incremental interpretation and prediction of utterance meaning for interactive dialogue , 2011, Dialogue Discourse.

[32]  Pedro M. Domingos,et al.  Unifying Logical and Statistical AI , 2006, 2016 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[33]  Okko Buß,et al.  DIUM – An Incremental Dialogue Manager That Can Produce Self-Corrections , 2011 .

[34]  Dan Roth,et al.  On the Hardness of Approximate Reasoning , 1993, IJCAI.

[35]  Jason D. Williams,et al.  Integrating Incremental Speech Recognition and POMDP-Based Dialogue Systems , 2012, SIGDIAL Conference.

[36]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[37]  D. Roy.  Grounding words in perception and action: computational insights , 2005, Trends in Cognitive Sciences.

[38]  David Schlangen,et al.  Joint Satisfaction of Syntactic and Pragmatic Constraints Improves Incremental Spoken Language Understanding , 2012, EACL.

[39]  Renato De Mori,et al.  Markov Logic Networks for Spoken Language Interpretation , 2008 .