Learning Content Selection Rules for Generating Object Descriptions in Dialogue

A fundamental requirement of any task-oriented dialogue system is the ability to generate object descriptions that refer to objects in the task domain. The subproblem of content selection for object descriptions in task-oriented dialogue has been the focus of much previous work and a large number of models have been proposed. In this paper, we use the annotated COCONUT corpus of task-oriented design dialogues to develop feature sets based on Dale and Reiter's (1995) incremental model, Brennan and Clark's (1996) conceptual pact model, and Jordan's (2000b) intentional influences model, and use these feature sets in a machine learning experiment to automatically learn a model of content selection for object descriptions. Since Dale and Reiter's model requires a representation of discourse structure, the corpus annotations are used to derive a representation based on Grosz and Sidner's (1986) theory of the intentional structure of discourse, as well as two very simple representations of discourse structure based purely on recency. We then apply the rule-induction program RIPPER to train and test the content selection component of an object description generator on a set of 393 object descriptions from the corpus. To our knowledge, this is the first reported experiment of a trainable content selection component for object description generation in dialogue. Three separate content selection models that are based on the three theoretical models, all independently achieve accuracies significantly above the MAJORITY CLASS baseline (17%) on unseen test data, with the intentional influences model (42.4%) performing significantly better than either the incremental model (30.4%) or the conceptual pact model (28.9%). But the best performing models combine all the feature sets, achieving accuracies near 60%. Surprisingly, a simple recency-based representation of discourse structure does as well as one based on intentional structure. To our knowledge, this is also the first empirical comparison of a representation of Grosz and Sidner's model of discourse structure with a simpler model for any generation task.

[1]  Johanna D. Moore,et al.  The agreement process: an empirical investigation of human-human computer-mediated collaborative dialogs , 2000, Int. J. Hum. Comput. Stud..

[2]  Johanna D. Moore,et al.  Learning Features that Predict Cue Usage , 1997, ACL.

[3]  M. Pickering,et al.  Toward a mechanistic psychology of dialogue , 2004, Behavioral and Brain Sciences.

[4]  Ellen F. Prince,et al.  Toward a taxonomy of given-new information , 1981 .

[5]  Marilyn A. Walker,et al.  The Effect of Resource Limits and Task Complexity on Collaborative Planning in Dialogue , 1995, Artif. Intell..

[6]  A WalkerMarilyn.  Limited attention and discourse structure , 1996 .

[7]  Graeme Hirst,et al.  Collaborating on Referring Expressions , 1991, CL.

[8]  Marilyn A. Walker,et al.  Learning Attribute Selections for Non-Pronominal Expressions , 2000, ACL.

[9]  Karen E. Lochbaum,et al.  The Use of Knowledge Preconditions in Language Processing , 1995, IJCAI.

[10]  Alexander I. Rudnicky,et al.  Stochastic natural language generation for spoken dialog systems , 2002, Comput. Speech Lang..

[11]  H. H. Clark,et al.  Conceptual pacts and lexical choice in conversation. , 1996, Journal of experimental psychology. Learning, memory, and cognition.

[12]  Massimo Poesio,et al.  Annotating a Corpus to Develop and Evaluate Discourse Entity Realization Algorithms: Issues and Preliminary Results , 2000, LREC.

[13]  Pamela W. Jordan.  Influences on Attribute Selection in Redescriptions: A Corpus Study , 2000 .

[14]  Chris Mellish,et al.  Experiments Using Stochastic Search for Text Planning , 1998, INLG.

[15]  Dragomir R. Radev.  Learning Correlations between Linguistic Indicators and Semantic Constraints: Reuse of Context-Dependent Descriptions of Entities , 1998, ACL.

[16]  H. H. Clark,et al.  Referring as a collaborative process , 1986, Cognition.

[17]  Kevin Knight,et al.  Generation that Exploits Corpus-Based Statistical Knowledge , 1998, ACL.

[18]  Kees van Deemter.  Generating Referring Expressions: Boolean Extensions of the Incremental Algorithm , 2002, CL.

[19]  Adwait Ratnaparkhi,et al.  Trainable approaches to surface natural language generation and their application to conversational dialog systems , 2002, Comput. Speech Lang..

[20]  Rob Malouf,et al.  The Order of Prenominal Adjectives in Natural Language Generation , 2000, ACL.

[21]  Marilyn A. Walker,et al.  Training a sentence planner for spoken dialogue using boosting , 2002, Comput. Speech Lang..

[22]  Chris Mellish,et al.  An Empirical Study on the Generation of Anaphora in Chinese , 1997, Comput. Linguistics.

[23]  Bonnie Webber,et al.  Pragmatic overloading in Natural Language instructions , 1996 .

[24]  Jon Oberlander.  Do the Right Thing ... but Expect the Unexpected , 1998, Comput. Linguistics.

[25]  Lauri Karttunen,et al.  Discourse Referents , 1969, COLING.

[26]  Pamela W. Jordan,et al.  Intentional influences on object redescriptions in dialogue: evidence from an empirical study , 2000 .

[27]  Stuart J. Russell,et al.  Do the right thing , 1991 .

[28]  Siobhan Chapman.  Logic and Conversation , 2005 .

[29]  Robert Dale,et al.  Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions , 1995, Cogn. Sci..

[30]  Kathleen McKeown,et al.  Empirically Estimating Order Constraints for Content Planning in Generation , 2001, ACL.

[31]  Amichai Kronfeld,et al.  Donnellan's Distinction and a Computational Model of Reference , 1986, ACL.

[32]  Rebecca J. Passonneau,et al.  Integrating Gricean and Attentional Constraints , 1995, IJCAI.

[33]  Donia Scott,et al.  Book Reviews: Generating Referring Expressions , 1994, CL.

[34]  Deb K. Roy,et al.  Learning visually grounded words and syntax for a scene description task , 2002, Comput. Speech Lang..

[35]  James F. Allen,et al.  Draft of DAMSL Dialog Act Markup in Several Layers , 2007 .

[36]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[37]  Martha E. Pollack,et al.  Overloading Intentions for Efficient Practical Reasoning , 1991 .

[38]  Walter Daelemans,et al.  Evaluation of Machine Learning Methods for Natural Language Processing Tasks , 2002, LREC.

[39]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[40]  Claire Gardent,et al.  Generating Minimal Definite Descriptions , 2002, ACL.

[41]  Douglas E. Appelt,et al.  Some Pragmatic Issues in the Planning of Definite & Indefinite Noun Phrases , 1985, ACL.

[42]  Emiel Krahmer,et al.  Graph-Based Generation of Referring Expressions , 2003, CL.

[43]  Douglas E. Appelt,et al.  Planning English Sentences , 1988, Cogn. Sci..

[44]  Ehud Reiter,et al.  Should Corpora Texts Be Gold Standards for NLG? , 2002, INLG.

[45]  Casimir A. Kulikowski,et al.  Computer Systems That Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning and Expert Systems , 1990 .

[46]  Uwe Reyle,et al.  From Discourse to Logic - Introduction to Modeltheoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory , 1993, Studies in linguistics and philosophy.

[47]  Srinivas Bangalore,et al.  Exploiting a Probabilistic Hierarchical Model for Generation , 2000, COLING.

[48]  Julia Hirschberg,et al.  Pitch Accent in Context: Predicting Intonational Prominence from Text , 1993, Artif. Intell..

[49]  Barbara J. Grosz,et al.  Generating appropriate natural language object descriptions , 1990 .

[50]  R. Passonneau.  Using Centering to Relax Gricean Informational Constraints on Discourse Anaphoric Noun Phrases , 1996 .

[51]  William W. Cohen.  Learning Trees and Rules with Set-Valued Features , 1996, AAAI/IAAI, Vol. 1.

[52]  Marilyn A. Walker,et al.  Limited Attention and Discourse Structure , 1995, CL.

[53]  Matthew Stone,et al.  Textual Economy Through Close Coupling of Syntax and Semantics , 1998, INLG.