论文信息 - REINFORCEMENT LEARNING WITH SIMULATED USER FOR AUTOMATIC DIALOG STRATEGY OPTIMIZATION

REINFORCEMENT LEARNING WITH SIMULATED USER FOR AUTOMATIC DIALOG STRATEGY OPTIMIZATION

In this paper, we propose a solution to the problem of formulating strategies for a spoken dialog system. Our approach is based on reinforcement learning with the help of a simulated user in order to identify an optimal dialog strategy. Our method considers the Markov decision process to be a framework for representation of speech dialog in which the states represent history and discourse context, the actions are dialog acts and the transition strategies are decisions on actions to take between states. We present our reinforcement learning architecture with a novel objective function that is based on dialog quality rather than its duration.

Jean-Guy Meunier | Philip H. P. Nguyen | Minh-Quang Nguyen | Tho-Hau Nguyen | Douglas O’Shaughnessy

[1] Michael English,et al. Learning Mixed Initiative Dialog Strategies By Using Reinforcement Learning On Both Conversants , 2005, HLT.

[2] Steve Young,et al. Simulation of human-machine dialogues , 1999 .

[3] H. Cuayahuitl,et al. Human-computer dialogue simulation using hidden Markov models , 2005, IEEE Workshop on Automatic Speech Recognition and Understanding, 2005..

[4] J. Schatztnann,et al. Effects of the user model on simulation-based learning of dialogue strategies , 2005, IEEE Workshop on Automatic Speech Recognition and Understanding, 2005..

[5] Steve J. Young,et al. A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies , 2006, The Knowledge Engineering Review.

[6] Kallirroi Georgila,et al. Hybrid reinforcement/supervised learning for dialogue policies from COMMUNICATOR data , 2005 .

[7] Olivier Pietquin,et al. A Framework for Unsupervised Learning of Dialogue Strategies , 2004 .

[8] Richard S. Sutton,et al. Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[9] Roberto Pieraccini,et al. A stochastic model of human-machine interaction for learning dialog strategies , 2000, IEEE Trans. Speech Audio Process..