A novel factored POMDP model for affective dialogue management

Partially observable Markov decision process (POMDP) model has been demonstrated many times to be suited for robust spoken dialogue management. Recently, some factored representations of POMDP model are designed for specific dialogue tasks. This paper proposes a novel factored POMDP model to describe a new application of affective dialogue management. Different from existing models, the user's state space and the system's observation space are both divided into two distinct components: goal and emotion. Moreover, the system's action space is for the first time factored into two parts, i.e., goal response and emotion response, and the reward function is accordingly updated by weighted sum of the two-part rewards. An example of intelligent music player is given to explain how to apply the new model to build an affective dialogue system. Four experiments are designed to reveal the influence of key parameters on the system performance. The simulation results demonstrate the rationality and feasibility of the proposed model.

[1]  Fuji Ren,et al.  Affective Information Processing and Recognizing Human Emotion , 2006, MFCSIT.

[2]  Pascal Poupart,et al.  Factored partially observable Markov decision processes for dialogue management , 2005 .

[3]  Rosalind W. Picard.  Affective computing: (526112012-054) , 1997 .

[4]  Milica Gasic,et al.  POMDP-Based Statistical Spoken Dialog Systems: A Review , 2013, Proceedings of the IEEE.

[5]  Kai-Tai Song,et al.  Robotic Emotional Expression Generation Based on Mood Transition and Personality Model , 2013, IEEE Transactions on Cybernetics.

[6]  Steve J. Young,et al.  Partially observable Markov decision processes for spoken dialog systems , 2007, Comput. Speech Lang..

[7]  Steve J. Young,et al.  Reinforcement learning for parameter estimation in statistical spoken dialogue systems , 2012, Comput. Speech Lang..

[8]  Mahmoud Neji,et al.  The Affective Tutoring System , 2010, Expert Syst. Appl..

[9]  Rosalind W. Picard.  Affective Computing , 1997 .

[10]  Fuji Ren,et al.  Predicting User-Topic Opinions in Twitter with Social and Topical Context , 2013, IEEE Transactions on Affective Computing.

[11]  Ioannis Arapakis,et al.  Theories, methods and current research on emotions in library and information science, information retrieval and human-computer interaction , 2011, Inf. Process. Manag..

[12]  Gregory A. Sanders,et al.  Darpa Communicator Evaluation: Progress from 2000 to 2001 Darpa Communicator Evaluation: Progress from 2000 to 2001 , 2022 .

[13]  Anton Nijholt,et al.  A tractable DDN-POMDP Approach to Affective Dialogue Modeling for General Probabilistic Frame-based Dialogue Systems , 2007 .

[14]  Joseph Mariani,et al.  An Overview of EU Programs Related to Conversational/Interactive Systems , 2007 .

[15]  Arvid Kappas,et al.  Affect and Social Processes in Online Communication--Experiments with an Affective Dialog System , 2013, IEEE Transactions on Affective Computing.

[16]  Joelle Pineau,et al.  Spoken Dialogue Management Using Probabilistic Reasoning , 2000, ACL.

[17]  Milos Hauskrecht,et al.  Value-Function Approximations for Partially Observable Markov Decision Processes , 2000, J. Artif. Intell. Res..

[18]  Gary Geunbae Lee,et al.  Recent Approaches to Dialog Management for Spoken Dialog Systems , 2010, J. Comput. Sci. Eng..

[19]  Anton Nijholt,et al.  Affective Dialogue Management Using Factored POMDPs , 2010, Interactive Collaborative Information Systems.

[20]  Milica Gasic,et al.  The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management , 2010, Comput. Speech Lang..

[21]  Kee-Eung Kim,et al.  Robust Performance Evaluation of POMDP-Based Dialogue Systems , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[22]  Oliver Lemon,et al.  Real user evaluation of a POMDP spoken dialogue system using automatic belief compression , 2014, Comput. Speech Lang..

[23]  S. Young,et al.  Scaling POMDPs for Spoken Dialog Management , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[24]  Steve J. Young,et al.  USING POMDPS FOR DIALOG MANAGEMENT , 2006, 2006 IEEE Spoken Language Technology Workshop.

[25]  Weihong Zhang,et al.  Speeding Up the Convergence of Value Iteration in Partially Observable Markov Decision Processes , 2011, J. Artif. Intell. Res..

[26]  Nikos A. Vlassis,et al.  Perseus: Randomized Point-based Value Iteration for POMDPs , 2005, J. Artif. Intell. Res..

[27]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[28]  Xin Kang,et al.  Employing hierarchical Bayesian networks in simple and complex emotion topic analysis , 2013, Comput. Speech Lang..

[29]  Wolfgang Minker,et al.  Emotion recognition and adaptation in spoken dialogue systems , 2010, Int. J. Speech Technol..

[30]  Dana Kulic,et al.  Affective State Estimation for Human–Robot Interaction , 2007, IEEE Transactions on Robotics.

[31]  Ramón López-Cózar,et al.  Enhancement of emotion detection in spoken dialogue systems by combining several information sources , 2011, Speech Commun..

[32]  Anton Nijholt,et al.  A tractable hybrid DDN–POMDP approach to affective dialogue modeling for probabilistic frame-based dialogue systems , 2008, Natural Language Engineering.