论文信息 - A Quality-Focused Spoken Dialog System With Reinforcement Learning And Simulated User

A Quality-Focused Spoken Dialog System With Reinforcement Learning And Simulated User

In this paper, we propose a solution to the problem of formulating strategies for a spoken dialog system. Our approach is based on reinforcement learning (RL) with the help of a simulated user (SU), involving unsupervised learning and trialsand-errors with a return value (negative or positive) for each decision, in order to identify an optimal dialog strategy. Our method considers the Markov decision process (MPD) to be a framework for representation of speech dialog in which the states represent history and discourse context, the actions are dialog acts and the transition strategies are decisions on actions to take between states. We present our reinforcement learning approach with a novel objective function that is based on dialog quality as well as other quantitative factors. KeywordsLearning control systems; Unsupervised learning; Markov processes; Artificial intelligence; Intelligent systems

Jean-Guy Meunier | Philip H. P. Nguyen | Minh-Quang Nguyen | Douglas O’Shaughnessy

[1] H. Cuayahuitl,et al. Human-computer dialogue simulation using hidden Markov models , 2005, IEEE Workshop on Automatic Speech Recognition and Understanding, 2005..

[2] Olivier Pietquin,et al. A Framework for Unsupervised Learning of Dialogue Strategies , 2004 .

[3] David Elkind,et al. Learning: An Introduction , 1968 .

[4] Richard S. Sutton,et al. Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[5] J. Schatztnann,et al. Effects of the user model on simulation-based learning of dialogue strategies , 2005, IEEE Workshop on Automatic Speech Recognition and Understanding, 2005..

[6] Roberto Pieraccini,et al. A stochastic model of human-machine interaction for learning dialog strategies , 2000, IEEE Trans. Speech Audio Process..

[7] Michael English,et al. Learning Mixed Initiative Dialog Strategies By Using Reinforcement Learning On Both Conversants , 2005, HLT.

[8] Steve Young,et al. Simulation of human-machine dialogues , 1999 .

[9] Kallirroi Georgila,et al. Hybrid reinforcement/supervised learning for dialogue policies from COMMUNICATOR data , 2005 .

[10] Steve J. Young,et al. A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies , 2006, The Knowledge Engineering Review.