Markov Logic Networks for Situated Incremental Natural Language Understanding

We present work on understanding natural language in a situated domain, that is, language that possibly refers to visually present entities, in an incremental, word-by-word fashion. Such type of understanding is required in conversational systems that need to act immediately on language input, such as multi-modal systems or dialogue systems for robots. We explore a set of models specified as Markov Logic Networks, and show that a model that has access to information about the visual context of an utterance, its discourse context, as well as the linguistic structure of the utterance performs best. We explore its incremental properties, and also its use in a joint parsing and understanding module. We conclude that mlns offer a promising framework for specifying such models in a general, possibly domain-independent way.

[1]  Rajeev Rastogi,et al.  Web information extraction using markov logic networks , 2011, KDD.

[2]  Gabriel Skantze,et al.  A General, Abstract Model of Incremental Dialogue Processing , 2009, EACL.

[3]  Raquel Fernández,et al.  Referring under Restricted Interactivity Conditions , 2007, SIGDIAL.

[4]  David Schlangen,et al.  Collaborating on Utterances with a Spoken Dialogue System Using an ISU-based Approach to Incremental Dialogue Management , 2010, SIGDIAL Conference.

[5]  Michael Spranger,et al.  Open-ended Grounded Semantics , 2010, ECAI.

[6]  Luke S. Zettlemoyer,et al.  Learning Context-Dependent Mappings from Sentences to Logical Form , 2009, ACL.

[7]  David Schlangen,et al.  Joint Satisfaction of Syntactic and Pragmatic Constraints Improves Incremental Spoken Language Understanding , 2012, EACL.

[8]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[9]  D. Roy.  Grounding words in perception and action: computational insights , 2005, Trends in Cognitive Sciences.

[10]  David DeVault,et al.  Can I Finish? Learning When to Respond to Incremental Interpretation Results in Interactive Dialogue , 2009, SIGDIAL Conference.

[11]  Pedro M. Domingos,et al.  Discriminative Training of Markov Logic Networks , 2005, AAAI.

[12]  J. Laurie Snell,et al.  Markov Random Fields and Their Applications , 1980 .

[13]  David Schlangen,et al.  A Simple Method for Resolution of Definite Reference in a Shared Visual Context , 2008, SIGDIAL Workshop.

[14]  Ellen Campana,et al.  Incremental understanding in human-computer dialogue and experimental evidence for advantages over nonincremental methods , 2007 .

[15]  Dan Klein,et al.  Learning Dependency-Based Compositional Semantics , 2011, CL.

[16]  Renato De Mori,et al.  Markov Logic Networks for Spoken Language Interpretation , 2008 .

[17]  Ann Copestake.  Semantic Composition with (Robust) Minimal Recursion Semantics , 2007, ACL 2007.

[18]  Matthew Crosby,et al.  Association for the Advancement of Artificial Intelligence , 2014 .

[19]  Iván V. Meza,et al.  Jointly Identifying Predicates, Arguments and Senses using Markov Logic , 2009, NAACL.

[20]  David Schlangen,et al.  Comparing Local and Sequential Models for Statistical Incremental Natural Language Understanding , 2010, SIGDIAL Conference.

[21]  David DeVault,et al.  Incremental interpretation and prediction of utterance meaning for interactive dialogue , 2011, Dialogue Discourse.

[22]  David Schlangen,et al.  Incremental Reference Resolution: The Task, Metrics for Evaluation, and a Bayesian Filtering Model that is Sensitive to Disfluencies , 2009, SIGDIAL Conference.