Simulating heterogeneous user behaviors to interact with conversational interfaces

Research in techniques to simulate users has a long history within the fields of language processing, speech technologies and conversational interfaces. In this paper, we describe a technique to develop heterogeneous user models that are able to interact with this kind of interfaces. By means of simulated users, it is possible not only to automatically evaluate the overall operation of a conversational interface, but also to assess the impact of the user responses on the decisions that are selected by the system. The selection of the user responses by the simulated user are based on a statistical model that considers the complete history of the interaction to carry out this selection. We describe this technique and its practical application to measure the influence of the most important user's features characteristics that affect the interaction of the simulated user with the a conversational interface.

[1]  David Griol,et al.  Modeling Users Emotional State for an Enhanced Human-Machine Interaction , 2015, HAIS.

[2]  Steve J. Young,et al.  A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies , 2006, The Knowledge Engineering Review.

[3]  Lin-Shan Lee,et al.  Computer-aided analysis and design for spoken dialogue systems based on quantitative simulations , 2001, IEEE Trans. Speech Audio Process..

[4]  Kallirroi Georgila,et al.  Learning user simulations for information state update dialogue systems , 2005, INTERSPEECH.

[5]  Grace Chung,et al.  Developing a Flexible Spoken Dialog System Using Simulation , 2004, ACL.

[6]  Roberto Pieraccini,et al.  User modeling for spoken dialogue system evaluation , 1997, 1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings.

[7]  Steve Young,et al.  Automatic learning of dialogue strategy using dialogue simulation and reinforcement learning , 2002 .

[8]  Ramón López-Cózar,et al.  A domain-independent statistical methodology for dialog management in spoken dialog systems , 2014, Comput. Speech Lang..

[9]  Steve J. Young,et al.  Error simulation for training statistical dialogue systems , 2007, 2007 IEEE Workshop on Automatic Speech Recognition & Understanding (ASRU).

[10]  Steve J. Young,et al.  Partially observable Markov decision processes for spoken dialog systems , 2007, Comput. Speech Lang..

[11]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[12]  John Mourjopoulos,et al.  Automatic speech recognition performance in different room acoustic environments with and without dereverberation preprocessing , 2013, Comput. Speech Lang..

[13]  Steve Young,et al.  Statistical User Simulation with a Hidden Agenda , 2007, SIGDIAL.

[14]  Feng Gao,et al.  Spoken language understanding using weakly supervised learning , 2010, Comput. Speech Lang..

[15]  Ramón López-Cózar,et al.  Assessment of dialogue systems by means of a new simulation technique , 2003, Speech Commun..

[16]  Encarna Segarra,et al.  The Incorporation of Confidence Measures to Language Understanding , 2003, TSD.

[17]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[18]  Sebastian Möller,et al.  Memo: towards automatic usability evaluation of spoken dialogue services by user error simulations , 2006, INTERSPEECH.

[19]  Gary Geunbae Lee,et al.  Hybrid user intention modeling to diversify dialog simulations , 2011, Comput. Speech Lang..

[20]  H. Cuayahuitl,et al.  Human-computer dialogue simulation using hidden Markov models , 2005, IEEE Workshop on Automatic Speech Recognition and Understanding, 2005..

[21]  Thierry Dutoit,et al.  A probabilistic framework for dialog simulation and optimal strategy learning , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[22]  Roberto Pieraccini,et al.  A stochastic model of human-machine interaction for learning dialog strategies , 2000, IEEE Trans. Speech Audio Process..

[23]  Hui Ye,et al.  Agenda-Based User Simulation for Bootstrapping a POMDP Dialogue System , 2007, NAACL.

[24]  Oliver Lemon,et al.  Learning what to say and how to say it: Joint optimisation of spoken dialogue management and natural language generation , 2011, Comput. Speech Lang..

[25]  Eric Horvitz,et al.  Conversation as Action Under Uncertainty , 2000, UAI.

[26]  Chris Mellish,et al.  Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04) , 2004, ACL 2004.

[27]  Klaus-Peter Engelbrecht.  Estimating Spoken Dialog System Quality with User Models , 2012 .