Learning Adaptive Referring Expression Generation Policies for Spoken Dialogue Systems

Adaptive generation of referring expressions in dialogues is beneficial in terms of grounding between the dialogue partners. However, handcoding adaptive REG policies is hard. We present a reinforcement learning framework to automatically learn an adaptive referring expression generation policy for spoken dialogue systems.

[1]  Kees van Deemter.  Generating Referring Expressions: Boolean Extensions of the Incremental Algorithm , 2002, CL.

[2]  Tatsuya Kawahara,et al.  User Modeling in Spoken Dialogue Systems to Generate Flexible Guidance , 2004, User Modeling and User-Adapted Interaction.

[3]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[4]  Oliver Lemon,et al.  Learning Lexical Alignment Policies for Generating Referring Expressions for Spoken Dialogue Systems , 2009, ENLG.

[5]  Kallirroi Georgila,et al.  Learning user simulations for information state update dialogue systems , 2005, INTERSPEECH.

[6]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[7]  Oliver Lemon,et al.  Adaptive natural language generation in dialogue using reinforcement learning , 2008 .

[8]  Albert Gatt,et al.  Attribute Selection for Referring Expression Generation: New Algorithms and Evaluation Methods , 2008, INLG.

[9]  Oliver Lemon,et al.  Learning Effective Multimodal Dialogue Strategies from Wizard-of-Oz Data: Bootstrapping and Evaluation , 2008, ACL.

[10]  Robert Dale,et al.  Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions , 1995, Cogn. Sci..

[11]  Pat Langley,et al.  Separating Skills from Preference: Using Learning to Program by Reward , 2002, ICML.

[12]  Oliver Lemon,et al.  A Two-Tier User Simulation Model for Reinforcement Learning of Adaptive Referring Expression Generation Policies , 2009, SIGDIAL Conference.

[13]  Anja Belz,et al.  Generation of repeated references to discourse entities , 2007, ENLG.

[14]  Jason Williams,et al.  Applying POMDPs to Dialog Systems in the Troubleshooting Domain , 2007, HLT-NAACL 2007.

[15]  Steve J. Young,et al.  A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies , 2006, The Knowledge Engineering Review.

[16]  Stefan Kopp,et al.  Modelling and Evaluation of Lexical and Syntactic Alignment with a Priming-Based Microplanner , 2010, Empirical Methods in Natural Language Generation.

[17]  H. H. Clark,et al.  References in Conversation Between Experts and Novices , 1987 .

[18]  M. Pickering,et al.  Toward a mechanistic psychology of dialogue , 2004, Behavioral and Brain Sciences.

[19]  Robert Dale,et al.  Cooking Up Referring Expressions , 1989, ACL.

[20]  H. H. Clark,et al.  Audience Design in Meaning and Reference , 1982 .

[21]  Oliver Lemon,et al.  User simulations for online adaptation and knowledge-alignment in troubleshooting dialogue systems , 2008 .

[22]  Kees van Deemter.  What Game Theory Can Do for NLG: The Case of Vague Language (Invited Talk) , 2009, ENLG.

[23]  Hui Ye,et al.  Agenda-Based User Simulation for Bootstrapping a POMDP Dialogue System , 2007, NAACL.

[24]  Oliver Lemon,et al.  A Wizard-of-Oz Environment to Study Referring Expression Generation in a Situated Spoken Dialogue Task , 2009, ENLG.

[25]  Johan Boye.  Dialogue Management for Automatic Troubleshooting and other Problem-solving Applications , 2007, SIGdial.

[26]  Oliver Lemon,et al.  Natural Language Generation as Planning Under Uncertainty for Spoken Dialogue Systems , 2009, EACL.

[27]  Kathleen McKeown,et al.  Tailoring Lexical Choice to the User's Vocabulary in Multimedia Explanation Generation , 1993, ACL.

[28]  Laura Stoia,et al.  Noun Phrase Generation for Situated Dialogs , 2006, INLG.

[29]  A. Bell.  Language style as audience design , 1984, Language in Society.

[30]  Robin L. Hill,et al.  PRE-CogSci 2009 , 2009 .

[31]  Jakob Nielsen,et al.  Improving a human-computer dialogue , 1990, CACM.

[32]  Rainer Bromme,et al.  How to refer to ‘diabetes’? Language in online health advice , 2005 .

[33]  Tatsuya Kawahara,et al.  Flexible Guidance Generation Using User Model in Spoken Dialogue Systems , 2003, ACL.

[34]  Advaith Siddharthan,et al.  Generating Referring Expressions in Open Domains , 2004, ACL.

[35]  SUSAN E. BRENNAN,et al.  Conversation with and through computers , 1991, User Modeling and User-Adapted Interaction.

[36]  Pamela J. Hinds,et al.  The curse of expertise: The effects of expertise and debiasing methods on prediction of novice performance. , 1999 .

[37]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[38]  David Schlangen,et al.  Causes and Strategies for Requesting Clarification in Dialogue , 2004, SIGDIAL Workshop.

[39]  M. Pickering,et al.  Linguistic alignment between people and computers , 2010 .

[40]  W. Kintsch,et al.  Language and comprehension , 1982 .

[41]  Emiel Krahmer,et al.  Graph-Based Generation of Referring Expressions , 2003, CL.

[42]  Roberto Pieraccini,et al.  Learning dialogue strategies within the Markov decision process framework , 1997, 1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings.