Natural Language Generation as Planning Under Uncertainty for Spoken Dialogue Systems

We present and evaluate a new model for Natural Language Generation (NLG) in Spoken Dialogue Systems, based on statistical planning, given noisy feedback from the current generation context (e.g. a user and a surface realiser). We study its use in a standard NLG problem: how to present information (in this case a set of search results) to users, given the complex tradeoffs between utterance length, amount of information conveyed, and cognitive load. We set these trade-offs by analysing existing match data. We then train a NLG policy using Reinforcement Learning (RL), which adapts its behaviour to noisy feedback from the current generation context. This policy is compared to several baselines derived from previous work in this area. The learned policy significantly outperforms all the prior approaches.

[1]  Oliver Lemon,et al.  Learning what to say and how to say it: Joint optimisation of spoken dialogue management and natural language generation , 2011, Comput. Speech Lang..

[2]  Johanna D. Moore,et al.  Fish or Fowl:A Wizard of Oz Evaluation of Dialogue Strategies in the Restaurant Domain , 2002, LREC.

[3]  Eric Horvitz,et al.  Conversation as Action Under Uncertainty , 2000, UAI.

[4]  A. Baddeley.  Working memory and language: an overview. , 2003, Journal of communication disorders.

[5]  Wolfgang Wahlster,et al.  Plan-based integration of natural language and graphics generation , 1994 .

[6]  Marilyn A. Walker,et al.  Trainable Sentence Planning for Complex Information Presentations in Spoken Dialog Systems , 2004, ACL.

[7]  Marilyn A. Walker,et al.  Towards developing general models of usability with PARADISE , 2000, Natural Language Engineering.

[8]  Marilyn A. Walker,et al.  User tailored generation in the match multimodal dialogue system , 2004 .

[9]  Matthew Stone,et al.  Sentence generation as a planning problem , 2007, ACL.

[10]  Oliver Lemon,et al.  Adaptive natural language generation in dialogue using reinforcement learning , 2008 .

[11]  Johanna D. Moore,et al.  The influence of user tailoring and cognitive load on user performance in spoken dialogue systems , 2007, INTERSPEECH.

[12]  Oliver Lemon,et al.  Mixture Model POMDPs for Efficient Handling of Uncertainty in Dialogue Management , 2008, ACL.

[13]  Marilyn A. Walker,et al.  Should i tell all?: an experiment on conciseness in spoken dialogue , 2003, INTERSPEECH.

[14]  Marilyn A. Walker,et al.  Generation and evaluation of user tailored responses in multimodal dialogue , 2004 .

[15]  Johanna D. Moore,et al.  Generating Tailored, Comparative Descriptions in Spoken Dialogue , 2004, FLAIRS Conference.

[16]  Oliver Lemon,et al.  Predicting how it sounds: re-ranking dialogue prompts based on TTS quality for adaptive spoken dialogue systems , 2009, INTERSPEECH.

[17]  Milica Gasic,et al.  Training and Evaluation of the HIS POMDP Dialogue System in Noise , 2008, SIGDIAL Workshop.

[18]  Ronald P. A. Petrick,et al.  EXPERIENCES WITH PLANNING FOR NATURAL LANGUAGE GENERATION , 2011, Comput. Intell..

[19]  Michael White,et al.  Learning to Say It Well: Reranking Realizations by Predicted Synthesis Quality , 2006, ACL.

[20]  Oliver Lemon,et al.  Learning Effective Multimodal Dialogue Strategies from Wizard-of-Oz Data: Bootstrapping and Evaluation , 2008, ACL.

[21]  Oliver Lemon,et al.  Does this list contain what you were searching for? Learning adaptive dialogue strategies for interactive question answering , 2009, Natural Language Engineering.

[22]  Marilyn A. Walker,et al.  User-tailored generation for spoken dialogue: an experiment , 2002, INTERSPEECH.

[23]  Kallirroi Georgila,et al.  Hybrid Reinforcement/Supervised Learning of Dialogue Policies from Fixed Data Sets , 2008, CL.

[24]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[25]  Oliver Lemon,et al.  A Wizard-of-Oz interface to study information presentation strategies for spoken dialogue systems , 2009 .

[26]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[27]  Hui Ye,et al.  The Hidden Information State Approach to Dialog Management , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[28]  Luke S. Zettlemoyer,et al.  Reinforcement Learning for Mapping Instructions to Actions , 2009, ACL.

[29]  Marilyn A. Walker,et al.  Individual and Domain Adaptation in Sentence Planning for Dialogue , 2007, J. Artif. Intell. Res..

[30]  Johanna D. Moore,et al.  Information Presentation in Spoken Dialogue Systems , 2006, EACL.

[31]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[32]  Oliver Lemon,et al.  Learning Adaptive Referring Expression Generation Policies for Spoken Dialogue Systems , 2010, Empirical Methods in Natural Language Generation.

[33]  Alexander I. Rudnicky,et al.  Stochastic natural language generation for spoken dialog systems , 2002, Comput. Speech Lang..

[34]  Joseph Polifroni,et al.  Intensional Summaries as Cooperative Responses in Dialogue: Automation and Evaluation , 2008, ACL.

[35]  Oliver Lemon,et al.  User simulations for online adaptation and knowledge-alignment in troubleshooting dialogue systems , 2008 .

[36]  Kees van Deemter.  What Game Theory Can Do for NLG: The Case of Vague Language (Invited Talk) , 2009, ENLG.

[37]  S. Singh,et al.  Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System , 2011, J. Artif. Intell. Res..